Google-Extended: a robots.txt control for Google

Google-Extended is a robots.txt control for Gemini training and grounding. It has no separate HTTP user agent and does not control Google Search rankings.

At a glance

Operator
Google
robots.txt token
Google-Extended
Purpose
Training. Training crawlers collect pages for future models. Keeping them out does not stop the crawlers that fetch pages for answers, which have names of their own.
What it feeds
Gemini training and grounding

What blocking it changes

Disallowing Google-Extended limits use of your content for Gemini training and grounding. It does not stop Googlebot from crawling for Google Search.

robots.txt lines for Google-Extended

robots.txt is a plain text file at the root of a site (yoursite.com/robots.txt). A group starts with one or more User-agent lines that name crawlers, followed by Allow and Disallow lines for paths. A crawler named in a group follows the groups that name it and ignores the group for every crawler (User-agent: *).

To let it in

User-agent: Google-Extended
Allow: /

To keep it out

User-agent: Google-Extended
Disallow: /

Does your site let it in?

Google-Extended is a name for robots.txt rules only. The crawler check reads robots.txt for it and sends no request as it.

Check whether your site lets Google-Extended in

Questions about Google-Extended

Google-Extended is a robots.txt control for Gemini training and grounding. It has no separate HTTP user agent and does not control Google Search rankings.

Disallowing Google-Extended limits use of your content for Gemini training and grounding. It does not stop Googlebot from crawling for Google Search.

Add a group with its name and a rule for every path: “User-agent: Google-Extended” followed by “Disallow: /”. To let it in, write “Allow: /” instead.