Anthropic documents three separate bots so site owners can make different choices for model development, search, and user-directed retrieval. Blocking one should not be described as blocking every Claude surface.
The three documented bots
| Bot | Documented purpose | Merchant decision |
|---|---|---|
ClaudeBot |
collects public web content that may contribute to model development | training-related crawl policy |
Claude-User |
retrieves content when a Claude user directs a request | user-triggered access policy |
Claude-SearchBot |
navigates the web to improve search result quality | search discovery policy |
Anthropic says these bots respect robots directives and anti-circumvention controls. It currently does not publish stable IP ranges, according to its official crawler guidance.
Five-store response record
On July 27, 2026, we sent spoofable user-agent test requests to five public commerce pages:
| Store | ClaudeBot | Claude-User | Claude-SearchBot | Final behavior |
|---|---|---|---|---|
| The Woobles | 200 | 200 | 200 | redirected to Canadian product path |
| Rhode | 200 | 200 | 200 | remained on product path |
| Ring Australia | 200 | 200 | 200 | remained on Australian path |
| HexClad UK | 200 | 200 | 200 | remained on UK path |
| MyFonts | 200 | 200 | 200 | remained on Axiforma family path |
These were our requests, not verified Anthropic traffic. The result means the pages did not reject those strings at the status level.
Robots policy example
Anthropic documents separate rules such as:
User-agent: ClaudeBot
Disallow: /
A merchant can make a different decision for Claude-SearchBot and Claude-User. Evaluate every subdomain separately.
The observed Ring robots group included older/general Anthropic-related names alongside other AI bots, allowed /, and disallowed private commerce paths. Current Anthropic documentation should take precedence over copied historical rules.
Test Shopify access
url='https://example.com/products/example-product'
for agent in ClaudeBot Claude-User Claude-SearchBot; do
curl -sSL -A "$agent" -D "$agent.headers" \
-o "$agent.html" \
-w "$agent -> %{http_code} %{url_effective} %{size_download}\n" \
"$url"
done
Compare:
- final status and URL
- content type and body size
- password or challenge signatures
- canonical URL
- product title and decisive facts
- Product and Offer output
Product evidence still controls accuracy
All three Anthropic-string requests returned 200 for The Woobles, but the page’s visible sold-out state conflicted with structured InStock offers. Crawler reachability cannot decide which stock statement Claude would use.
Similarly, MyFonts returned 200 while the initial JSON-LD response contained no detected Product entity. Digital license semantics may need to be recovered from visible text rather than conventional physical-product markup.
Run an honest Claude observation
Use fixed buyer prompts and record:
- exact prompt
- date, timezone, language, and market
- displayed Claude mode when available
- completed, refused, timeout, or error state
- exact brand/product mention
- visible source URLs
- expected versus asserted facts
Do not count a 200 crawler test as a Claude mention. Do not count a provider error as a non-mention.
Example prompt set
Discovery:
Which beginner crochet kits support left-handed learners and include video lessons?
Compatibility:
Which Australian battery doorbells support 2.4 GHz Wi-Fi without existing wiring?
Verification:
What is included in the HexClad UK six-piece pot set? Cite the official page.
Refresh time-sensitive expected facts immediately before each run.
Acceptance criteria
- each Anthropic bot has an intentional policy
- public product pages return usable evidence for allowed traffic
- private account and checkout paths remain protected
- user-agent strings are not treated as identity proof
- Claude answer observations retain prompts, sources, and timestamps
- crawler access, training preference, search visibility, and recommendation are reported separately