Applebot: robots token, verification and how to allow or disallow it
Applebot is an AI crawler operated by Apple. Its robots.txt token is Applebot, and the vendor documentation we fetched on 2026-09-06 is the source for every statement on this page.
| Robots token | Applebot |
|---|---|
| Vendor | Apple |
| Purpose | search index, user-initiated fetch |
| Documented verification | reverse DNS |
| Vendor documentation last verified | 2026-09-06 |
| Share of read domains disallowing it for / | 3.9% of 128 |
| Last verified |
Identity
| Robots token | Applebot |
|---|---|
| Matched as | applebot |
| Vendor | Apple |
| Purpose | search index, user-initiated fetch |
| Documentation | https://support.apple.com/en-us/119829 |
| Documented verification | reverse DNS (.applebot.apple.com) |
The crawler behind Siri and Spotlight suggestions. Vendor example on the fetched page: 'User-agent: Applebot Allow: / Disallow: /private/'. The same page notes a separate iTMS user agent that does not follow robots.txt because it is not a general search crawler.
Allow or disallow it
User-agent: Applebot Allow: /
User-agent: Applebot Disallow: /
A robots.txt rule is a request that a well behaved crawler honours. It is not an access control, and this page does not tell anyone what to choose.
Questions
Does Applebot obey robots.txt?
The vendor documentation at https://support.apple.com/en-us/119829 states that Applebot respects robots.txt. We fetched that page and it answered our checker on 2026-09-06. This records what the vendor documents, not what any individual request did.
How do I allow Applebot in robots.txt?
Add this group to the robots.txt at the root of the host: User-agent: Applebot Allow: /. The token is matched case insensitively as a substring of the user agent by RFC 9309, and the longest matching rule wins.
How do I disallow Applebot in robots.txt?
Add this group to the robots.txt at the root of the host: User-agent: Applebot Disallow: /. A robots.txt rule is a request that a well behaved crawler honours; it is not an access control.
How do I verify a request really came from Applebot?
The vendor documents reverse DNS verification: look up the address of the request, check the name ends in .applebot.apple.com, then look that name up again and confirm it resolves back to the same address.
What we measure
Of the 128 seeded domains whose robots.txt we have read, 5 disallow Applebot for / and 123 allow it, which is 3.9% disallowed, week 2026-W37. Counts only: no domain is named.
The weekly AI crawler access index
Sources
| Source | Type | HTTP | Verified | Note |
|---|---|---|---|---|
| https://support.apple.com/en-us/119829 | vendor | 200 | 2026-09-06 | 75 occurrences of Applebot in the fetched body |
Free data
Check a domain against every token
AEO Watch is an independent, factual monitor. It is not affiliated with, endorsed by or speaking for any crawler vendor. Every statement is an observation with the date it was made and the raw evidence behind it: a robots.txt line, an HTTP status code, a header. There are no scores, no grades and no verdicts here, and nothing on this page is advice. A site opts out at any time and the opt out is honoured automatically and permanently.