Quick answers

AIGrow provides AI visibility monitoring, a business assistant, blog publishing and scoped automation services. Start with the free scan to inspect the output before paying.

Run the free scan

The scan is free and monitoring plans start at $29 a month. The operations audit is $290, the visibility audit is $490, and custom builds receive a written price before work starts.

See the plans

Run the free scan. It reads your site, asks several AI assistants what your customers ask, and points you at one thing. No signup required, and it will tell you if you do not need us.

Start now

Yes. Use the contact form for product, billing, support or project questions. Describe the goal and the system involved so the first reply can be specific.

Contact AIGrow
AI crawler check

Can AI assistants read your site?

Type the address of a page. The check reads its robots.txt and requests the page as each AI crawler, then shows which crawlers can read it and which are turned away.

Free, with no account and no email. The check takes several seconds.

A homepage or any other page. Only that page is checked.

The AI crawler check reads a page’s robots.txt and requests the page once as each AI crawler, such as OAI-SearchBot, ChatGPT-User, PerplexityBot and Claude-User. It shows which crawlers may read the page and which a firewall or CDN turns away. The check is free, needs no account and takes several seconds.

How this check works

robots.txt
The site’s robots.txt is read once and each crawler’s rules for the page are applied as RFC 9309 describes: the group that names the crawler wins over the group for every crawler.
A request as each crawler
The page is requested once with each crawler’s published user agent. A page served counts as readable; a refusal or a challenge page counts as turned away.
Control request
The page is also requested as an ordinary browser. When that request is refused too, the crawler requests say nothing about crawlers in particular, and only robots.txt is judged.
The limit
Our requests come from our address. A firewall that checks a crawler’s published IP addresses can refuse our copy and still admit the real crawler, so Googlebot and Bingbot are judged by robots.txt only.

Every crawler in the table has its own page in the AI crawler list.

Questions about AI crawlers

Search crawlers, such as OAI-SearchBot, build the index an assistant cites. User crawlers, such as ChatGPT-User, fetch one page when a person asks about it. Training crawlers, such as GPTBot, collect text for future models. The crawlers that fetch pages for answers are separate, so blocking a training crawler costs no answer today; blocking a search or user crawler does.

A CDN, a firewall or a bot protection service can refuse requests by user agent before robots.txt is ever read. Some have a setting for AI crawlers. The site still opens in a browser, so the block goes unnoticed.

No. llms.txt is a proposed file that lists a site’s main pages for language models. The crawlers in this check need robots.txt to allow them and the server to serve the page; an llms.txt file changes neither.

No. Allowing them only means the assistants can read the page. Whether a business is named depends on what the page and other sites say about it.

See what the assistants say about you

The free scan reads your site, checks the same crawlers and asks assistants the questions your buyers ask.

Run the free AI scan