__usage_ai-training<\/code>, that tells every participating company at once whether your content may be used for AI training. Google, OpenAI and Microsoft said the same week that they would support it. The draft’s example looks like this:<\/p>\nUser-Agent: __usage_ai-training\nDisallow: \/\nAllow: \/press-kit\/<\/code><\/pre>\nThat means “don’t use my site for AI training, except the press kit”. Important details:<\/p>\n
\n- It isn’t live yet.<\/strong> Google’s crawler documentation says it plans to support the signal in early 2027; Microsoft has said it will honour it from 1 February 2027; OpenAI says “in the coming months”; Anthropic says work has started. Until then, keep the per-bot rules above.<\/li>\n
- It is a draft, not a finished standard.<\/strong> It is an individual Internet-Draft and its details may change.<\/li>\n
- It covers training, not crawling or search.<\/strong> It doesn’t block crawling and doesn’t change search rankings. Google says that if you opt out with the shared token and don’t mention Google-Extended, it plans to treat that as a Google-Extended block.<\/li>\n
- Adding it now is harmless.<\/strong> You can add the group alongside your existing rules so it takes effect as each company switches it on.<\/li>\n<\/ul>\n
<\/span>FAQ<\/span><\/h2>\n<\/span>Does blocking AI crawlers hurt SEO?<\/span><\/h3>\nBlocking AI training bots such as GPTBot, ClaudeBot, Google-Extended and Applebot-Extended doesn’t affect Google or Bing search rankings. Blocking AI search bots (OAI-SearchBot, PerplexityBot, Claude-SearchBot) can remove your site from those AI search answers.<\/p>\n
<\/span>What is GPTBot?<\/span><\/h3>\nGPTBot is OpenAI’s crawler that collects public web content that may be used to train its AI models. OpenAI says disallowing GPTBot tells it your content shouldn’t be used for training.<\/p>\n
<\/span>How do I check which AI bots visit my site?<\/span><\/h3>\nLook for their user-agent names in your server access logs, your host’s statistics, or Cloudflare’s analytics. Many security plugins also show bot visits.<\/p>\n
<\/span>How long does a robots.txt change take to work?<\/span><\/h3>\nCrawlers cache robots.txt, so changes can take up to a day or so to be picked up. Meta, for example, says up to 24 hours.<\/p>\n","protected":false},"excerpt":{"rendered":"
How to block AI crawlers like GPTBot, ClaudeBot and Google-Extended with robots.txt, WordPress and Cloudflare, what each bot does, and the new shared AI training opt-out token.<\/p>\n","protected":false},"author":4,"featured_media":69280,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","footnotes":""},"categories":[6],"tags":[],"class_list":["post-69281","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blogging"],"_links":{"self":[{"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/posts\/69281","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/comments?post=69281"}],"version-history":[{"count":1,"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/posts\/69281\/revisions"}],"predecessor-version":[{"id":69318,"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/posts\/69281\/revisions\/69318"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/media\/69280"}],"wp:attachment":[{"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/media?parent=69281"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/categories?post=69281"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.quickonlinetips.com\/archives\/wp-json\/wp\/v2\/tags?post=69281"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
![\"Red](\"https:\/\/www.quickonlinetips.com\/archives\/wp-content\/uploads\/block-ai-crawlers-no-entry-sign.jpg\")
<\/p>\n