Median score 77 out of 100. The most common gap is that product descriptions contain no specifications at all.
When someone asks an assistant to find them a product, the assistant reads the store's public product feed. It does not see the storefront, the photography, or the brand. If the feed cannot answer the shopper's question, the product does not enter the candidate set.
I wanted to know how many real catalogues actually carry that data, so I scored 255 live Shopify storefronts.
Median 77/100, mean 77.4. The distribution is wide: the bottom decile sits at 42 and the top at 94. Exactly one store scored 100.
| Band | Stores | Share | |
|---|---|---|---|
| Severe (under 60) | 28 | 11.0% | |
| Weak (60–75) | 81 | 31.8% | |
| Adequate (75–90) | 98 | 38.4% | |
| Strong (90+) | 48 | 18.8% |
Roughly four stores in five scored below 90.
The single most useful finding is that the checks split cleanly into two groups.
| Check | Median pass | Stores below 80% |
|---|---|---|
| Product titles are descriptive | 100% | 0 |
| Price present on every variant | 100% | 0 |
| Stock availability exposed | 100% | 0 |
| Brand / vendor populated | 100% | 1 |
| Product category is set | 96% | 92 (36%) |
| Descriptions detailed enough to answer questions | 88% | 108 (42%) |
| Shipping weight set on items that ship | 80% | 124 (49%) |
| Specs appear in the description | 28% | 214 (84%) |
The platform handles the basics; the gaps are everything above them. Titles, prices, stock and brand are essentially universal, because Shopify's own data model makes them hard to omit. Every bit of variation between stores lives in the four fields a merchant has to fill in deliberately: category, description depth, specifications, and shipping weight.
This also means about 40 of the 100 points are close to free for any Shopify store. Scores are compressed upward, and a store at 60 is failing most of what actually varies.
84% of stores fail this, and the median store has spec language in only 28% of its products. This is not sloppiness. It is well-written marketing copy aimed at a human who can see the photograph.
One store in the panel describes a product this way:
“The same best-selling swim trunks you love, now with a shorter inseam.”
Shorter than what? A shopper asking an assistant for a 7-inch inseam gets nothing back, because the number is not in the feed. The copy is doing its job for one audience and failing completely for the other.
92 stores set a product category on under 80% of products, and 28 stores set it on none at all. Category is typically the first filter applied before anything else is read. It is a dropdown.
More instructive than the results. Three checks I built turned out to measure nothing, and all three would have shipped inside a paid report telling merchants about problems that do not exist.
My first scorer flagged missing barcodes almost everywhere. The field is simply not exposed in Shopify's public products.json. It was reading an absent column as a universal defect.
Across 120 products on three live stores, every single Default Title belonged to a legitimate one-variant product, which is correct behaviour. The real defect — multiple variants all named Default Title — occurred zero times.
updated_at — passed 255 of 255Every store showed a median of zero days since last update. Shopify bumps updated_at on any inventory change, so it always reads as today. Freshness is not measurable from public data, and pretending otherwise handed every store four free points on a check that could never fail.
Three different failure shapes, one lesson: a check whose output does not vary carries no information, in either direction. One failed everyone, one falsely flagged most people, one passed everyone. Any check that fires on nearly every subject is an instrument fault until proven otherwise.
products.json only. No admin access, no app install, no authenticated endpoints.robots.txt checked and honoured before any fetch; 1.05 seconds between requests.It does not show that any particular assistant excludes any particular product. That is not observable from outside, and nobody who tells you otherwise can demonstrate it. The narrow, defensible claim is the only one worth making: these are the questions a store's feed currently cannot answer.
The score is a directional rubric, not a standard. Data held only in metafields or a private PIM is invisible here and is counted neither for nor against a store. The specifications check is a text heuristic and carries the joint-heaviest weight, which makes it the softest part of the score; it was validated by hand against real failures, but it is the first place to look if a score seems wrong.