NewFirst month of Starter or Pro for $1Your first month of Starter or Pro for just $1, then cancel anytimeSee pricing
Ranken logoRanken.io

GPTBot: what it is and how to block it

What is GPTBot?

GPTBot is OpenAI's crawler for collecting public web pages that may be used to train its foundation models. Blocking it tells OpenAI not to use your content for training. It does not decide what ChatGPT search shows, which is the job of OAI-SearchBot.

Operator
OpenAI
Used for
Model training
robots.txt token
GPTBot
robots.txt
Opt out in robots.txt

Checked against OpenAI's documentation

robots.txtGPTBot
User-agent: *
Allow: /
 
User-agent: GPTBot
Disallow: /
With this rule
OpenAI model trainingBlocked
ChatGPT search answersNot affected

What is the GPTBot user agent string?

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot

OpenAI publishes this as an example. The version number can change.

How do I block GPTBot in robots.txt?

Add these two lines to the robots.txt file at the root of your domain. OpenAI documents the GPTBot token in robots.txt as the way to signal that your content should not be used for training.

User-agent: GPTBot
Disallow: /

To allow it everywhere instead, use:

User-agent: GPTBot
Allow: /

What happens if you block GPTBot?

Your content is not used to train OpenAI models. ChatGPT search can still find and show your pages.

Should you block GPTBot?

Block it if you do not want your content used to train OpenAI models. It has no effect on ChatGPT search, so a common setup is to block GPTBot and allow OAI-SearchBot.

GPTBot questions

Does blocking GPTBot remove my site from ChatGPT?

No. GPTBot only collects training data. ChatGPT search uses OAI-SearchBot, so keep that one allowed if you want ChatGPT to cite and link your pages.

How do I check that a visit is really GPTBot?

Compare the visitor's IP address with the GPTBot IP list OpenAI publishes. A request that only claims to be GPTBot in its user agent can come from anyone.

Does your robots.txt block GPTBot?The free AI Visibility Checker reads your robots.txt and shows which AI search and training crawlers can reach a page.Check a URL

See the full AI crawler list or read the official OpenAI docs

A blog AI search can read

Every Ranken blog ships a robots.txt that welcomes AI search crawlers, plus server-rendered pages, schema and llms.txt.