| Route | Purpose | Does not prove |
|---|---|---|
/robots.txt |
crawler path policy | WAF allows legitimate traffic |
/sitemap.xml |
published discovery URLs | indexing or citation |
/agents.md |
Shopify agent guidance | correct product facts |
/llms.txt |
alternate/custom guidance | recommendation |
/llms-full.txt |
optional fuller guidance | catalog freshness |
Observed stores: The Woobles, Rhode, and HexClad returned 200 for agent routes; Ring returned 404; MyFonts used a separate versioned llms.txt. Ring’s product remained publicly reachable, proving an agent file is supplementary.
The Woobles, Rhode, and HexClad target products were present in product sitemaps. Multi-market URLs must be interpreted as alternates, not duplicate products.
for path in robots.txt sitemap.xml agents.md llms.txt llms-full.txt; do
curl -sSIL "https://example.com/$path" | sed -n '1,8p'
done
Validate content type and body, not status only. Preserve Shopify-generated robots defaults when customizing.