Meta · training
meta-externalagent: user agent and how to block it
Everything on this page is taken from Meta's own documentation, quoted and linked, and was checked on 4 October 2026. Where the documentation is silent the page says so instead of filling the gap.
What it is
Operator: Meta. Kind, as listed here: training. Meta’s page says it crawls the web “for use cases such as training foundation AI models or improving products by indexing content directly”, so “training” is one use it names, not the only one, and the kind shown here is our shorthand.
“The Meta-ExternalAgent crawler crawls the web for use cases such as training foundation AI models or improving products by indexing content directly.”
Source: Meta web crawlers, checked 4 October 2026.
User-agent string
meta-externalagent/1.1 (+/documentation/sharing/webmasters/web-crawlers)
Meta says the string in your logs will be similar to this one or to the bare meta-externalagent/1.1. The link inside the string is relative, exactly as Meta prints it.
Source: Meta web crawlers. The robots.txt token to match is meta-externalagent.
Does it obey robots.txt?
“By configuring the robots.txt file on your website, you can specify to the Meta web crawlers how you would prefer them to interact with your site.”
Source: Meta web crawlers, checked 4 October 2026.
Does it affect search?
The Meta page we checked does not say.
What else Meta says
“The specific UA string that you will see in your log files will be similar to one of the following:”
Source: Meta web crawlers, checked 4 October 2026.
“Please allow up to 24 hours for changes to robots.txt to take effect because crawlers may cache the contents of robots.txt for up to 24 hours.”
Source: Meta web crawlers, checked 4 October 2026.
robots.txt snippets
Two snippets that show the syntax for this token. They are examples, not advice: whether to allow or refuse meta-externalagent is your decision, and the effects above are the vendor's description of them.
Allow meta-externalagent
User-agent: meta-externalagent Allow: /
Disallow meta-externalagent
User-agent: meta-externalagent Disallow: /
How to verify requests
Meta gives no IP-range file or DNS method for this token on the page we checked.
Questions
What is meta-externalagent?
Meta describes meta-externalagent like this: “The Meta-ExternalAgent crawler crawls the web for use cases such as training foundation AI models or improving products by indexing content directly.”
Does meta-externalagent obey robots.txt?
The Meta documentation says: “By configuring the robots.txt file on your website, you can specify to the Meta web crawlers how you would prefer them to interact with your site.”
How do I block meta-externalagent in robots.txt?
Add a group to robots.txt with the line “User-agent: meta-externalagent” followed by “Disallow: /”. That is an example of the syntax, not a recommendation.
Also see the robots.txt tester for AI crawlers and the other entries in the AI crawler reference.
Last reviewed . Maintained by Cite Files — corrections to [email protected].