The three OpenAI crawlers and what each controls
Confusing OpenAI's user agents is the single most common self-inflicted GEO wound. GPTBot performs broad crawling associated with model improvement. OAI-SearchBot builds the index used to answer search-backed questions inside ChatGPT. ChatGPT-User fetches a specific page live when a user or a tool asks for it. These are governed independently in robots.txt.
The practical implication: a site can allow GPTBot, feel confident it is AI-friendly, and still be entirely absent from ChatGPT search citations because OAI-SearchBot is disallowed by an inherited wildcard rule. Audit each agent explicitly rather than assuming a blanket allow covers them, and verify the live file rather than the template that was supposed to generate it.
There is a legitimate reason to restrict training-scale crawling while permitting search: some publishers want attribution and referral traffic without contributing to model training. That is a coherent position, and robots.txt supports it precisely because the agents are separate.
Prompt-shaped content beats keyword-shaped content
Keyword research produces fragments: 'best seo tool', 'seo tool pricing'. Conversational search produces sentences with constraints: 'what is the best seo tool for a two-person agency that already uses search console'. The constraint is the interesting part, because it is exactly what a generic page fails to address and what a targeted section answers completely.
Build a prompt inventory the same way you build a keyword list, but collect full sentences from sales calls, support tickets, and community threads. Group them by the constraint they carry - team size, budget, industry, existing stack, region - and give each constraint cluster its own heading and its own direct answer. This is inexpensive to produce and it maps almost one-to-one onto the way people actually query assistants.
Comparison framing deserves particular attention. A very large share of assistant queries are implicitly comparative, and pages that name alternatives honestly, including where a competitor is the better fit, are cited noticeably more often than pages that pretend no alternatives exist.