AI assistants cite websites through two different routes, and each has its own requirements. Google’s AI features draw on its ordinary search index. ChatGPT search uses a separate crawler run by OpenAI. Neither needs a special file.
Google’s AI features
Google’s own documentation says a page needs to be indexed and eligible to appear in Google Search with a snippet. It also says there are no additional requirements to appear in AI Overviews or AI Mode, and that no new machine-readable files, AI text files, or special markup are needed.
In practice, a page that Google can crawl, index, and show as a search result is already eligible. A page that is not indexed cannot be cited, however good it is.
Check indexing first. In Google Search Console, the URL Inspection tool shows whether a page is indexed and what Googlebot received when it fetched the page.
Check what limits snippets. A nosnippet or max-snippet rule, or a noindex tag, limits or removes the page from search and from the AI features built on it. These are sometimes left in place from a site’s staging period.
Keep important content as text. Text in images, or loaded only after a script runs, may not be read the way text in the HTML is.
Make structured data match the page. Structured data is not required, but where it exists it should describe what is visible on the page.
ChatGPT search
ChatGPT search relies on OpenAI’s own crawler, OAI-SearchBot. OpenAI’s documentation says sites that opt out of it will not be shown in ChatGPT search answers, though they can still appear as links. It recommends allowing OAI-SearchBot in robots.txt and allowing requests from its published IP ranges.
The second part is where sites fail without knowing. A firewall or bot-protection rule can block the crawler even when robots.txt allows it, and nothing on the site shows it happening. Check the firewall or CDN settings, as well as robots.txt.
GPTBot is a different crawler. OpenAI uses GPTBot to collect content for training its models, and controls it separately. A site can allow OAI-SearchBot and disallow GPTBot. Blocking one does not block the other.
Changes are not instant. OpenAI says it can take about 24 hours for its systems to adjust after a robots.txt update.
What is left after the technical checks
Once a page can be crawled, indexed, and fetched, whether it is cited depends on the same things as ranking: whether it answers the question being asked, and whether it does so more clearly than the other pages that could. Nothing in either company’s documentation describes a shortcut, and a service that sells one is selling something the documentation says is not needed.
When to ask for help
Most of this can be checked in an afternoon with Search Console, your robots.txt file, and your firewall settings. If you would rather have it checked and fixed, our SEO & AI Visibility service starts with a written audit that covers these points, and we can make the fixes on our own hosting or on any platform where we are given an editor account.