cohere-ai: a legacy robots.txt preference
Cohere currently lists no active crawler for training models. The cohere-ai token is retained as a legacy robots.txt check, without a live crawler request.
At a glance
- Operator
- Cohere
- robots.txt token
cohere-ai- Purpose
- legacy robots.txt preference
- What it feeds
- legacy robots.txt preference
What blocking it changes
An entry for cohere-ai records a preference for that token. Cohere does not currently list it as an active training crawler.
robots.txt lines for cohere-ai
robots.txt is a plain text file at the root of a site (yoursite.com/robots.txt). A group starts with one or more User-agent lines that name crawlers, followed by Allow and Disallow lines for paths. A crawler named in a group follows the groups that name it and ignores the group for every crawler (User-agent: *).
To let it in
User-agent: cohere-ai
Allow: /To keep it out
User-agent: cohere-ai
Disallow: /Does your site let it in?
cohere-ai is a name for robots.txt rules only. The crawler check reads robots.txt for it and sends no request as it.