# Googlebot

## In short

Googlebot is an AI crawler operated by Google. Its robots.txt token is Googlebot, and the vendor documentation we fetched on 2026-09-06 is the source for every statement on this page.

| Fact | Value |
| --- | --- |
| Robots token | Googlebot |
| Vendor | Google |
| Purpose | search index |
| Documented verification | published IP ranges and reverse DNS |
| Vendor documentation last verified | 2026-09-06 |
| Share of read domains disallowing it for / | 0.8% of 128 |
| Last verified | 2026-09-06 |

## Does Googlebot obey robots.txt?

The vendor documentation at https://developers.google.com/search/docs/crawling-indexing/googlebot states that Googlebot respects robots.txt. We fetched that page and it answered our checker on 2026-09-06. This records what the vendor documents, not what any individual request did.

## How do I allow Googlebot in robots.txt?

Add this group to the robots.txt at the root of the host: User-agent: Googlebot
Allow: /. The token is matched case insensitively as a substring of the user agent by RFC 9309, and the longest matching rule wins.

## How do I disallow Googlebot in robots.txt?

Add this group to the robots.txt at the root of the host: User-agent: Googlebot
Disallow: /. A robots.txt rule is a request that a well behaved crawler honours; it is not an access control.

## How do I verify a request really came from Googlebot?

The vendor publishes both an IP range file (https://developers.google.com/static/search/apis/ipranges/googlebot.json) and a reverse DNS convention. Either check is documented; the address is what proves the request, never the user agent string.

## Source

- https://developers.google.com/search/docs/crawling-indexing/googlebot (HTTP 200, Verified 2026-09-06)
- https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers (HTTP 200, Verified 2026-09-06)

AEO Watch is an independent, factual monitor. It is not affiliated with, endorsed by or speaking for any crawler vendor. Every statement is an observation with the date it was made and the raw evidence behind it: a robots.txt line, an HTTP status code, a header. There are no scores, no grades and no verdicts here, and nothing on this page is advice. A site opts out at any time and the opt out is honoured automatically and permanently.