ChatGPT-User: a crawler run by OpenAI that opens pages on request
ChatGPT-User is a user crawler. It opens a page when a person asks this product to read it: ChatGPT. Operator: OpenAI.
At a glance
- Operator
- OpenAI
- robots.txt token
ChatGPT-User- Purpose
- User request. User crawlers open one page when a person asks an assistant to read it. A page they cannot read cannot be opened in that conversation.
- What it feeds
- ChatGPT
What blocking it changes
If you keep ChatGPT-User out, this product cannot open your page when a person asks it to: ChatGPT.
User agent
The full user agent, from the operator’s documentation. A request from ChatGPT-User carries this text, and a server log shows it.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/botrobots.txt lines for ChatGPT-User
robots.txt is a plain text file at the root of a site (yoursite.com/robots.txt). A group starts with one or more User-agent lines that name crawlers, followed by Allow and Disallow lines for paths. A crawler named in a group follows the groups that name it and ignores the group for every crawler (User-agent: *).
To let it in
User-agent: ChatGPT-User
Allow: /To keep it out
User-agent: ChatGPT-User
Disallow: /Does your site let it in?
The free crawler check reads your robots.txt for ChatGPT-User and requests your page with its user agent, so it shows whether a robots.txt rule or a firewall turns it away.