Google, Microsoft and OpenAI plan to support a single robots.txt line that opts a site out of AI training
On October 30, 2026 Kevin Kelley of Anthropic submitted an IETF draft that lets a site state one AI training preference in robots.txt through a policy token named __usage_ai-training. Google, Microsoft and OpenAI announced plans to support it the same day. None honors it yet, and the draft is not a ratified standard.
Three companies plan to back one shared training preference
Kevin Kelley of Anthropic submitted an Internet-Draft titled "Machine-Readable Training Preference Signal" to the IETF's AI Preferences working group, dated October 30, 2026. Rather than writing separate rules for every AI company, a site would state its training preference once in robots.txt through a single policy token. The same day, plans to take the shared line into account appeared on Google's Search Central blog, on OpenAI's own site and on Microsoft's Bing blog.
Search Engine Journal writes that, judging by the timelines given, none of the three supports the line today. For Anthropic the picture rests on secondary reports: Search Engine Journal and The Next Web relay a message Kelley posted to the working group's mailing list, saying Anthropic has started development, that the support will complement its existing crawler controls and that rolling it out will take time. Search Engine Journal notes that as of October 30 Anthropic's newsroom carried no post on the signal, so the company's role is read through the draft's authorship and those two reports.
How the token is written, and each company's timetable
Per the draft, a site owner writes an ordinary robots.txt group and puts the __usage_ai-training token in its User-Agent line. A Disallow rule expresses a preference against AI training and an Allow rule a preference in favor. The draft's example closes the whole site with Disallow: / and reopens one folder with Allow: /press-kit/. No new directive or syntax is added. Generic groups written for User-Agent: * do not count as matching the token, and a file with no group naming it expresses no preference. The token is not a crawling command either: crawl rules written for a crawler's own product token keep working independently, and the draft says private agreements between a site and a company can override these preferences.
Google's post is by John Mueller. Google builds on Google-Extended, the control it has offered since 2023: if a site uses the token to opt out of training and does not explicitly specify Google-Extended, Google plans to read that as a Google-Extended block. Company-specific controls keep working next to the shared signal. Google wrote: "We aim to support this new control along with other AI providers in 2027". According to Search Engine Journal, Google's crawler documentation puts it at early 2027.
Microsoft's post is by Krishna Madhavan of Bing. Bing will take the shared training preference into account as it crawls from February 1, 2027, and adds an AI Grounding preference that is set only at page level, with Allow, DisallowExtended and Disallow values. A page-level training setting overrides robots.txt for that page. The new preferences replace NoArchive and NoCache: after March 1, 2027 those tags will no longer be read as a training or grounding preference. For sites still using them on December 15, 2026, a temporary grounding preference opens in Bing Webmaster Tools that day and applies per page until recrawl or September 30, 2027, whichever comes first. OpenAI says it is targeting the months ahead for implementation, will publish a further update on how it works, and that its current robots.txt controls remain in force: "This new signal will not replace the controls we already provide through robots.txt."
One line is easier to write, but it does not close the same things everywhere
The shared line helps by passing a training preference to several providers in one move. The controls each company ties to it differ, though. According to Search Engine Journal, at Google the same Google-Extended control covers both training and grounding in Gemini Apps, and the crawler documentation describes no way to stay out of Gemini training while remaining open to grounding. The same report, citing Google's documentation, says Google-Extended does not affect inclusion in Google Search, and it points out that AI Overviews and AI Mode have their own separate opt-out in Search Console. At Microsoft, training and grounding are two independent settings.
Bing's post warns that restricting grounding sharply reduces visibility in AI-generated experiences, without changing eligibility for Bing search results, and that neither a training nor a grounding preference guarantees inclusion, traffic or payment. Status matters too: the draft is an individual Internet-Draft, not endorsed by the IETF, with no formal standing in its standards process, and it expires on May 3, 2027. The Next Web reports that Meta's Andy Stone posted on X in favor of a common signal, without saying whether Meta would honor it. Search Engine Journal adds that neither OpenAI's post nor its crawler documentation describes what happens when a shared-line rule conflicts with a GPTBot rule.
The sources do not mention Türkiye, so this section is commentary
None of the sources we read describes a separate implementation or regulation for Türkiye, for sites in Türkiye or for Turkish-language content. This section is therefore UNALSOFT commentary, not a fact from the reporting.
Our reading is that robots.txt files are read the same way everywhere, so a site in Türkiye can use the same token. Since no company reads the line yet, though, a line added today has no concrete effect. What remains unsettled: a firm day for Google beyond early 2027, for Anthropic only the fact that development has started, for OpenAI only a general timeframe of coming months in place of a date, whether Meta will take part, and whether anything differs for Türkiye. We covered a setting that separates refusing AI training from search visibility on September 16 on the Cloudflare side; this story is about a shared preference line inside robots.txt itself.
The UNALSOFT view
We read this as a timetable and a decision list. Adding the line is easy; the real work is deciding which content should stay out of training and which should remain visible in AI answers. Because training and Gemini grounding travel together at Google but are set separately at Microsoft, one line may not produce the same result at every company. For sites using NoArchive or NoCache on Bing, the date that matters is March 1, 2027. That is why in web design work we treat robots.txt, meta tags and page headers together, as a deliberate choice. Since the draft is not a ratified standard, watching each company's own documentation before writing new rules is the sensible course for now.
Sources
Google Search Central Blog, shared AI training control · IETF Internet-Draft, training preference signal · OpenAI, expressing AI training preferences · IETF Datatracker, draft page · Bing Blog, training and grounding preferences · OpenAI, announcement page · Search Engine Journal, shared AI training opt-out · The Next Web, four companies back one opt-out · Search Engine Roundtable, Bing controls · PPC Land, one robots.txt rule from Google and Bing · Search Engine Roundtable, Google training control · Search Engine Roundtable, Bing announcement
Are your site's AI preferences a deliberate choice or a default?
A short conversation is enough to review your robots.txt, meta tags and page headers together.