cohere-ai: a legacy robots.txt preference

Cohere currently lists no active crawler for training models. The cohere-ai token is retained as a legacy robots.txt check, without a live crawler request.

At a glance

Operator
Cohere
robots.txt token
cohere-ai
Purpose
legacy robots.txt preference
What it feeds
legacy robots.txt preference

What blocking it changes

An entry for cohere-ai records a preference for that token. Cohere does not currently list it as an active training crawler.

robots.txt lines for cohere-ai

robots.txt is a plain text file at the root of a site (yoursite.com/robots.txt). A group starts with one or more User-agent lines that name crawlers, followed by Allow and Disallow lines for paths. A crawler named in a group follows the groups that name it and ignores the group for every crawler (User-agent: *).

To let it in

User-agent: cohere-ai
Allow: /

To keep it out

User-agent: cohere-ai
Disallow: /

Does your site let it in?

cohere-ai is a name for robots.txt rules only. The crawler check reads robots.txt for it and sends no request as it.

Check whether your site lets cohere-ai in

Questions about cohere-ai

Cohere currently lists no active crawler for training models. The cohere-ai token is retained as a legacy robots.txt check, without a live crawler request.

An entry for cohere-ai records a preference for that token. Cohere does not currently list it as an active training crawler.

Add a group with its name and a rule for every path: “User-agent: cohere-ai” followed by “Disallow: /”. To let it in, write “Allow: /” instead.