Sampling Shopify AI Visibility Without Inventing a Ranking

An AI visibility study should begin with a fixed observation protocol, not a leaderboard. The objective is to learn whether a defined store appears for a defined buying question under recorded conditions.

This note includes a real five-store technical preflight performed July 27, 2026. It does not invent ChatGPT or Perplexity answer results that were not run.

Research question

A valid question has a population and unit:

For 25 fixed English-language buyer prompts in the Canadian market, how often does the approved storefront domain appear in completed answers on each tested platform during a four-week window?

The unit is one provider, prompt, locale, timestamp, and completed answer.

Store entity registry

Each merchant record needs:

merchant_id: store-001
name: The Woobles
approved_domains:
  - thewoobles.com
approved_brand_strings:
  - The Woobles
market: US
category: beginner crochet kits

Internal category labels are calibration notes, not merchant claims.

Five-store public storefront transport preflight

SkuWatch AI Visibility checked The Woobles, Rhode, Ring Australia, HexClad UK, and MyFonts from SkuWatch AI Visibility infrastructure. The crawler-associated User-Agent values were substituted request headers, not verified provider traffic. The table preserves transport and storefront observations; it is not an answer-engine response record.

Store	Product access	/agents.md	/llms.txt	Notable evidence condition
The Woobles	200	200	200	visible stock conflicted with offers
Rhode	200	200	200	two size entities shared observed SKU
Ring Australia	200	404	404	explicit public-vs-private robots rules
HexClad UK	200	200	200	Product name leaked Default Title
MyFonts	200	200	200	separate versioned llms.txt; no initial Product entity

The historical preflight recorded final statuses and selected storefront fields, but it did not retain enough provider-authentication evidence to identify the requester as an official crawler. Future collection must retain final URL, redirect chain, content type, body hash, expected-title check, canonical, product-entity count, market, and timestamp.

This preflight can identify storefront-readiness issues. It cannot produce a mention rate, citation rate, index status, or recommendation claim. Those require separately collected provider answers and citations.

Prompt portfolio

Use balanced intent groups:

Intent	Example	Count in 25-prompt set
category discovery	“Which beginner crochet kits include video lessons?”	5
constrained recommendation	“Battery doorbell for 2.4 GHz Wi-Fi without existing wiring”	8
comparison	“Compare two pot sets by capacities, induction support, and warranty”	5
brand verification	“What sizes and prices are listed for Glazing Milk?”	4
risk or exclusion	“Which features require a subscription?”	3

Freeze exact wording before the first run. New prompts create a new portfolio version.

Observation record

{
  "portfolio_version": "2026-07-a",
  "query_id": "doorbell-au-02",
  "provider": "provider-name",
  "locale": "en-AU",
  "scheduled_at": "2026-07-27T12:00:00Z",
  "state": "success",
  "answer_text": "retained provider output",
  "matched_brands": [],
  "visible_sources": [],
  "fact_checks": []
}

Store raw output where provider terms permit. Derived labels must be reproducible from that record.

Completion and visibility denominators

Assume 100 scheduled runs:

70 successful answers
15 rate limits
10 timeouts
5 parser errors
14 successful answers mention the brand
8 successful answers cite the official domain

Report:

completion rate = 70 / 100 = 70%
mention rate among completed = 14 / 70 = 20%
official-domain citation rate = 8 / 70 = 11.4%

Do not report 14% visibility. That would count 30 unobserved answers as non-mentions.

Fact accuracy

A mention is not automatically useful. Attach an expected fact sheet:

Fact	Expected source	Example failure
availability	selected product offer	visible sold out repeated as in stock
size	selected variant	starting size treated as only size
currency	observed market	USD copied into Canadian answer
compatibility	specification table	subscription feature presented as included

Score only facts the answer actually asserts. Unknown should remain unknown.

Repeat schedule

• run priority prompts weekly
• run the full portfolio monthly
• preserve provider and mode labels
• use the same locale and language
• record storefront deployments separately
• compare rolling windows, not individual screenshots

Limitations to publish

• providers can change models and retrieval systems
• personalization may not be controllable
• user-agent-string fetches do not prove provider infrastructure access
• a source may be proxied or hidden
• visibility samples do not represent all private users
• a storefront repair and later mention are correlated events, not proven causation

Acceptance criteria

A study is publishable when the query portfolio, entity registry, raw completed answers, excluded failures, source URLs, fact checks, dates, and limitations are all available to a reviewer.

If those inputs are absent, publish a technical readiness report instead of inventing an AI ranking.