OpenAI · search
OAI-SearchBot: user agent and how to block it
Everything on this page is taken from OpenAI's own documentation, quoted and linked, and was checked on 4 October 2026. Where the documentation is silent the page says so instead of filling the gap.
What it is
Operator: OpenAI. Kind, as listed here: search.
“OAI-SearchBot is for search. OAI-SearchBot is used to surface websites in search results in ChatGPT’s search features.”
Source: OpenAI: crawlers and user agents, checked 4 October 2026.
User-agent string
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
OpenAI labels this an example string and says the version number may change. It also says it may add a robots.txt marker to the string when fetching robots.txt files.
Source: OpenAI: crawlers and user agents. The robots.txt token to match is OAI-SearchBot.
Does it obey robots.txt?
“OpenAI uses OAI-SearchBot and GPTBot robots.txt tags to enable webmasters to manage how their sites and content work with AI.”
Source: OpenAI: crawlers and user agents, checked 4 October 2026.
Does it affect search?
“Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links.”
Source: OpenAI: crawlers and user agents, checked 4 October 2026.
What else OpenAI says
“When fetching robots.txt files, we may use a user-agent string with an additional robots.txt marker.”
Source: OpenAI: crawlers and user agents, checked 4 October 2026.
robots.txt snippets
Two snippets that show the syntax for this token. They are examples, not advice: whether to allow or refuse OAI-SearchBot is your decision, and the effects above are the vendor's description of them.
Allow OAI-SearchBot
User-agent: OAI-SearchBot Allow: /
Disallow OAI-SearchBot
User-agent: OAI-SearchBot Disallow: /
How to verify requests
OpenAI publishes the IP addresses for OAI-SearchBot as a JSON file.
https://openai.com/searchbot.json
Source: OpenAI: crawlers and user agents.
What we measure
OAI-SearchBot is one of the four agents the Cite Files scanner requests with when it scans a site. Run the AI crawler checker on your own address to see how your site answers it, or read the open record of how scanned sites treat it.
Questions
What is OAI-SearchBot?
OpenAI describes OAI-SearchBot like this: “OAI-SearchBot is for search. OAI-SearchBot is used to surface websites in search results in ChatGPT’s search features.”
Does OAI-SearchBot obey robots.txt?
The OpenAI documentation says: “OpenAI uses OAI-SearchBot and GPTBot robots.txt tags to enable webmasters to manage how their sites and content work with AI.”
How do I block OAI-SearchBot in robots.txt?
Add a group to robots.txt with the line “User-agent: OAI-SearchBot” followed by “Disallow: /”. That is an example of the syntax, not a recommendation. On search, OpenAI says: “Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links.”
Also see the robots.txt tester for AI crawlers and the other entries in the AI crawler reference.
Last reviewed . Maintained by Cite Files — corrections to [email protected].