Required columns
run_id, query_id, portfolio_version, provider, mode, timestamp,
timezone, locale, prompt, state, brand_mentioned, product_mentioned,
source_url, source_class, expected_fact, asserted_fact, fact_result,
competitors, notes
Use one row per answer-source-fact combination or separate Runs, Sources, and Fact checks sheets. Do not put several URLs into an unparseable cell.
State values
success, refused, rate_limited, timeout, provider_error, parser_error
Only success rows enter mention denominators.
Worked calculation
If 40 runs are scheduled, 34 complete, nine mention the brand, and five cite the official domain:
completion = 34 / 40 = 85%
mention rate = 9 / 34 = 26.5%
official citation rate = 5 / 34 = 14.7%
A timeout is blank evidence, not a zero mention.
Fact sheet
For the Ring battery-doorbell prompt, expected facts include 2.4 GHz Wi-Fi, rechargeable battery, optional wiring limits, and qualified subscription features. Score only claims made by the answer.
Controls
- freeze prompt text under a portfolio version
- preserve original timestamp
- use approved domain list
- distinguish retailer and official storefront sources
- never overwrite an old run
Acceptance criteria
A second reviewer can reproduce every label from the saved answer and source URL. Charts show their denominator and provider completion rate.
Community discussion
Add to the article
Ask a technical question, share a storefront result, or challenge a conclusion with evidence.