Botservatory

← Catalogue

Google

Googlebot

Also feeds AI Overviews and AI Mode. Blocking it removes you from Google Search, and there is no separate crawler to block for AI features.
Class
Search index
robots.txt
Says it honours robots.txt
Evidence
Confirmed from operator documentation
Source

Proving it is real

  • Check the IP against Google's Common crawlers file.
  • Run a reverse DNS lookup, confirm the name ends in .googlebot.com, then confirm it resolves back to the same IP.

Check an address now.

Blocking it

robots.txt
User-agent: Googlebot
Disallow: /
Cloudflare custom rule expression
(http.user_agent contains "Googlebot")
nginx
if ($http_user_agent ~* "Googlebot") {
    return 403;
}

Matching a name only blocks bots that tell the truth about it. Before you do this, read what each decision costs.

Last reviewed Nov 2, 2026