# AI crawler access in software and developer tools

## In short

107 seeded domains sit under .app, .dev, .software, and we have read the robots.txt of 1 of them. This page reports counts and shares for that set only.

| Fact | Value |
| --- | --- |
| Industry | Software and developer tools |
| Top level domains | .app, .dev, .software |
| Eligibility | stated purpose |
| Registry | Google Registry (.app, .dev) and the .software registry |
| Domains in the set | 107 |
| Domains with a robots.txt reading | 1 |
| Month | 2026-09 |
| Last verified | 2026-09-06 |

The rule, in full: a seeded domain is placed in an industry only when its top level domain is one whose registry publishes that purpose. A top level domain marked verified is one where the registry checks that the registrant qualifies; one marked stated purpose is open to register and the registry states an intended use. Country code top level domains are never read as industries here, because a country code says where a registrant is, not what it does: .ai, .io, .co, .tv and .me are country codes and are excluded on purpose. Every other domain is unclassified and appears on no industry page. Nothing is inferred from a name, a word list or a model.

| Crawler | Domains read | Disallowed | Share |
|---|---|---|---|
| Amazonbot | 1 | 0 | 0.0% |
| anthropic-ai | 1 | 0 | 0.0% |
| Applebot | 1 | 0 | 0.0% |
| Applebot-Extended | 1 | 0 | 0.0% |
| Bingbot | 1 | 0 | 0.0% |
| Bytespider | 1 | 0 | 0.0% |
| CCBot | 1 | 0 | 0.0% |
| ChatGPT-User | 1 | 0 | 0.0% |
| Claude-SearchBot | 1 | 0 | 0.0% |
| Claude-User | 1 | 0 | 0.0% |
| ClaudeBot | 1 | 0 | 0.0% |
| DuckAssistBot | 1 | 0 | 0.0% |
| Google-Extended | 1 | 0 | 0.0% |
| Googlebot | 1 | 0 | 0.0% |
| GPTBot | 1 | 0 | 0.0% |
| meta-externalagent | 1 | 0 | 0.0% |
| OAI-SearchBot | 1 | 0 | 0.0% |
| Perplexity-User | 1 | 0 | 0.0% |
| PerplexityBot | 1 | 0 | 0.0% |

AEO Watch is an independent, factual monitor. It is not affiliated with, endorsed by or speaking for any crawler vendor. Every statement is an observation with the date it was made and the raw evidence behind it: a robots.txt line, an HTTP status code, a header. There are no scores, no grades and no verdicts here, and nothing on this page is advice. A site opts out at any time and the opt out is honoured automatically and permanently.