Allow Google's AI models
User-agent: Google-Extended
Disallow:

Allow OpenAI's models (ChatGPT)
User-agent: GPTBot
Disallow:

Allow Anthropic's models (Claude)
User-agent: anthropic-ai
Disallow:

Allow Perplexity's AI search
User-agent: PerplexityBot
Disallow:

Allow Common Crawl (used for training many open models)
User-agent: CCBot
Disallow: