PerplexityBot: a search crawler run by Perplexity

PerplexityBot is a search crawler. It reads pages for the index behind this product: Perplexity. Operator: Perplexity.

At a glance

Operator
Perplexity
robots.txt token
PerplexityBot
Purpose
Search. Search crawlers read pages to build the index an assistant searches when it answers. A page they cannot read cannot be found or cited in those answers.
What it feeds
Perplexity

What blocking it changes

If you keep PerplexityBot out, your pages cannot be found and cited in the search answers of this product: Perplexity.

User agent

A representative user agent for PerplexityBot. We use operator documentation where a full string is published. Versions and browser details in real server logs can differ.

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)

robots.txt lines for PerplexityBot

robots.txt is a plain text file at the root of a site (yoursite.com/robots.txt). A group starts with one or more User-agent lines that name crawlers, followed by Allow and Disallow lines for paths. A crawler named in a group follows the groups that name it and ignores the group for every crawler (User-agent: *).

To let it in

User-agent: PerplexityBot
Allow: /

To keep it out

User-agent: PerplexityBot
Disallow: /

Does your site let it in?

The free crawler check reads your robots.txt for PerplexityBot and requests your page with its user agent, so it shows whether a robots.txt rule or a firewall turns it away.

Check whether your site lets PerplexityBot in

Questions about PerplexityBot

PerplexityBot is a search crawler. It reads pages for the index behind this product: Perplexity. Operator: Perplexity.

Only if you do not want your pages in the search answers of this product: Perplexity. Blocking it changes nothing for crawlers with other names.

Add a group with its name and a rule for every path: “User-agent: PerplexityBot” followed by “Disallow: /”. To let it in, write “Allow: /” instead.