Quick answers

AIGrow provides AI visibility monitoring, a business assistant, blog publishing and scoped automation services. Start with the free scan to inspect the output before paying.

Run the free scan

The scan is free and monitoring plans start at $29 a month. The operations audit is $290, the visibility audit is $490, and custom builds receive a written price before work starts.

See the plans

Run the free scan. It reads your site, asks several AI assistants what your customers ask, and points you at one thing. No signup required, and it will tell you if you do not need us.

Start now

Yes. Use the contact form for product, billing, support or project questions. Describe the goal and the system involved so the first reply can be specific.

Contact AIGrow
AI crawlers

Googlebot: a search crawler run by Google

Googlebot is a search crawler. It reads pages for the index behind this product: Google Search and AI Overviews. Operator: Google.

At a glance

Operator
Google
robots.txt token
Googlebot
Purpose
Search. Search crawlers read pages to build the index an assistant searches when it answers. A page they cannot read cannot be found or cited in those answers.
What it feeds
Google Search and AI Overviews

What blocking it changes

If you keep Googlebot out, your pages cannot be found and cited in the search answers of this product: Google Search and AI Overviews.

robots.txt lines for Googlebot

robots.txt is a plain text file at the root of a site (yoursite.com/robots.txt). A group starts with one or more User-agent lines that name crawlers, followed by Allow and Disallow lines for paths. A crawler named in a group follows the groups that name it and ignores the group for every crawler (User-agent: *).

To let it in

User-agent: Googlebot
Allow: /

To keep it out

User-agent: Googlebot
Disallow: /

Does your site let it in?

The crawler check reads only robots.txt for Googlebot and sends no request as it.

Check whether your site lets Googlebot in

Other crawlers run by Google

Google-Extended
Training: Gemini training and grounding

See all 22 AI crawlers

Questions about Googlebot

Googlebot is a search crawler. It reads pages for the index behind this product: Google Search and AI Overviews. Operator: Google.

Only if you do not want your pages in the search answers of this product: Google Search and AI Overviews. Blocking it changes nothing for crawlers with other names.

Add a group with its name and a rule for every path: “User-agent: Googlebot” followed by “Disallow: /”. To let it in, write “Allow: /” instead.