# AEO Watch

## In short

AEO Watch reads the robots.txt of 22044 seeded domains and resolves 21 documented AI crawler tokens against it, then records what it saw with the evidence and the date.

| Fact | Value |
| --- | --- |
| Domains seeded | 22044 |
| Domains with a recorded observation | 20763 |
| Observations held | 375486 |
| AI crawler tokens in the registry | 21 |
| Recorded changes in the last 7 days | 2281 |
| Newest observation | 2026-10-22 |
| Last verified | 2026-10-22 |

## The crawler guides

- All 21 documented crawlers on one page: https://bikoosh.com/aeo/crawlers
- GPTBot (OpenAI): https://bikoosh.com/aeo/crawler/gptbot
- ClaudeBot (Anthropic): https://bikoosh.com/aeo/crawler/claudebot
- PerplexityBot (Perplexity): https://bikoosh.com/aeo/crawler/perplexitybot
- OAI-SearchBot (OpenAI): https://bikoosh.com/aeo/crawler/oai-searchbot
- CCBot (Common Crawl Foundation): https://bikoosh.com/aeo/crawler/ccbot
- Bytespider (ByteDance): https://bikoosh.com/aeo/crawler/bytespider
- Googlebot (Google): https://bikoosh.com/aeo/crawler/googlebot
- ChatGPT-User (OpenAI): https://bikoosh.com/aeo/crawler/chatgpt-user
- Claude-User (Anthropic): https://bikoosh.com/aeo/crawler/claude-user
- Claude-SearchBot (Anthropic): https://bikoosh.com/aeo/crawler/claude-searchbot
- Perplexity-User (Perplexity): https://bikoosh.com/aeo/crawler/perplexity-user
- anthropic-ai (Anthropic): https://bikoosh.com/aeo/crawler/anthropic-ai
- Bingbot (Microsoft): https://bikoosh.com/aeo/crawler/bingbot
- Applebot (Apple): https://bikoosh.com/aeo/crawler/applebot
- Amazonbot (Amazon): https://bikoosh.com/aeo/crawler/amazonbot
- meta-externalagent (Meta): https://bikoosh.com/aeo/crawler/meta-externalagent
- DuckAssistBot (DuckDuckGo): https://bikoosh.com/aeo/crawler/duckassistbot
- Amzn-SearchBot (Amazon): https://bikoosh.com/aeo/crawler/amzn-searchbot
- Amzn-User (Amazon): https://bikoosh.com/aeo/crawler/amzn-user
- Applebot-Extended (Apple): https://bikoosh.com/aeo/crawler/applebot-extended
- Google-Extended (Google): https://bikoosh.com/aeo/crawler/google-extended

## Aggregate pages

- Weekly changes: https://bikoosh.com/aeo/changes
- AI crawler access index: https://bikoosh.com/aeo/index
- Industry reports: https://bikoosh.com/aeo/industry

## What you get

- Free monitor: free, one domain, a weekly recheck, and an email only when something changed.
- Pro monitoring: 9 USD a month, up to 10 domains, a daily recheck, an alert on any recorded change, and a weekly digest. Checkout: https://gumroad.com/l/bikooshwatch?wanted=true&watch=aeo
- Agency monitoring: 79 USD a month, up to 100 domains, a daily recheck, an alert on any recorded change, and a weekly digest. Checkout: https://gumroad.com/l/bikooshwatchgroup?wanted=true&watch=aeo

The checkout is titled Bikoosh Watch: one membership shared across Bikoosh products. Activate with the licence key on the receipt or the email it was paid with, and add domains, at https://bikoosh.com/aeo/activate. The free check on https://bikoosh.com/aeo needs no payment and no address.

AEO Watch is an independent, factual monitor. It is not affiliated with, endorsed by or speaking for any crawler vendor. Every statement is an observation with the date it was made and the raw evidence behind it: a robots.txt line, an HTTP status code, a header. There are no scores, no grades and no verdicts here, and nothing on this page is advice. A site opts out at any time and the opt out is honoured automatically and permanently.