Home › Research › How AI decides who to recommend
SEMPITE Research · Measurement Study · September 2026
Half the Stores an AI Agent Can Read Aren’t Ready to Sell to One.
AI shopping agents are starting to discover, compare, and buy on shoppers’ behalf. We checked 1,600 Shopify storefronts for whether an agent can actually read and act on them. Most of the ones it can reach show it a blank product, hide their reviews, or aren’t transaction-ready.
What an agent needs, and how far we got
An AI shopping agent has to clear four gates on your store: crawl it (robots.txt allows its bot), read the catalog (a machine-readable product source), parse each product (server-side Product and Offer schema with price and availability, not JavaScript-only), and trust it (agent-readable reviews). We measured all four.
Of 1,600 sampled Shopify storefronts, 888 (55.5%) were reachable and 729 exposed a machine-readable catalog we could read a product from. Everything below is measured on those 729 — the stores an agent can already read — because that is the fair denominator. The stores it can’t even reach are a worse problem we don’t count against this bar.
A third of readable stores show the agent a blank page
Here is what the 729 readable storefronts actually expose in their served HTML, before any JavaScript runs, which is all a non-rendering agent gets.
35.0% (95% CI 31.6–38.5) serve no server-side Product schema at all — their product data is injected by JavaScript, so an agent that doesn’t render the page sees no product, no price, no stock. Even among those that do, only about half expose structured price and availability.
The store looks fine to you in a browser, which runs the JavaScript. The agent isn’t a browser. It reads the HTML the server sends, and on a third of these stores that HTML says nothing about the product.
Almost nobody gives the agent a reason to trust them
Reviews are the single strongest signal an AI shopping assistant leans on when it decides what to recommend. 94.5% of readable storefronts (95% CI 92.6–95.9) expose no AggregateRating or Review data an agent can read. The star ratings are on the page for humans, rendered in a widget, invisible to the machine doing the recommending.
1 in 11 blocks the agent at the door, probably by accident
Before any of that matters, the agent’s crawler has to be allowed in. 8.7% of reachable storefronts (95% CI 7.0–10.7) disallow at least one core AI crawler in robots.txt. The tell that it’s not a considered choice: the block rate is nearly identical across every major bot.
A blanket Disallow copied from a template takes you out of ChatGPT, Claude, Perplexity, Google’s AI, Amazon’s Rufus, and Apple’s intelligence in one line, for stores that almost certainly didn’t mean to opt out of the channel their shoppers are moving to.
Put together: only half are transaction-ready
Combine the gates — allows the core crawlers and serves Product schema and price and availability — and 52.4% of readable storefronts clear the bar (95% CI 48.8–56.0). The other 47.6% are visible to a human and functionally invisible, or unreadable, to an agent trying to act.
Home & kitchen (n=129) edged out apparel & accessories (n=375), but both carry the same two holes: roughly a third serve JavaScript-only product data, and over 90% expose no agent-readable reviews.
Frequently asked questions
What does it mean for a store to be “ready” for an AI shopping agent?
The agent has to crawl it (robots.txt allows its bot), read the catalog (a machine-readable product source), parse each product (server-side Product and Offer schema with price and availability, not JavaScript-only), and weigh trust (agent-readable reviews). Of the 729 stores an agent could read, only 52.4% cleared the transaction-ready bar.
What is the single biggest gap?
Two. 35.0% of readable stores serve product data only in JavaScript, so a non-rendering agent sees a blank product. And 94.5% expose no agent-readable review data at all, even though agents lean heavily on reviews. Structured price and availability were present on only about 53%.
Are stores blocking AI shopping agents on purpose?
Some: 8.7% disallow at least one core AI crawler in robots.txt, and the near-identical block rate across GPTBot, ClaudeBot, Google-Extended, Amazonbot, and Applebot-Extended (~8.6%) suggests a copied blanket rule, not a considered choice.
Does having an llms.txt help?
Not here. 99.8% served an llms.txt, but that is Shopify auto-injecting a commerce template on most stores, not a curated file. It is a checkout manifest, not an agent-readiness signal, so it didn’t distinguish ready stores from unready ones.
Which categories were most and least ready?
Home & kitchen was more transaction-ready (58.1%) than apparel & accessories (49.9%), but both had the same core problems: about a third JavaScript-only, over 90% with no agent-readable reviews.
Method, in short
Seeded random sample (seed 613) of 1,600 custom-domain Shopify storefronts drawn from the frame we built for our Shopify llms.txt study, re-crawled 15 September 2026. For each store we fetched robots.txt and parsed the agent-bot policy; fetched /products.json to confirm a machine-readable catalog, classify the vertical from product types and tags, and pull one product handle; fetched that product page and extracted every JSON-LD block, looking for Product, Offer (price, availability), and AggregateRating/Review; and fetched /llms.txt. Readiness metrics are computed on the 729 storefronts that exposed a readable catalog, so schema is measured only where we actually fetched a product page. All proportions carry Wilson score intervals.
Known limits, stated plainly: a static fetch cannot see JavaScript-rendered schema — which is the point, because non-rendering agents can’t either, though some agents do render and would see more than we credit here; we read one product page per store, not the whole catalog; stores that disable /products.json are excluded from the schema metrics rather than counted as failing; the frame is Shopify custom-domain stores from a list built five weeks earlier, so 44.5% were unreachable on re-crawl and the readable set skews to live, trafficked stores; and llms.txt presence is near-universal only because Shopify auto-serves it. Raw per-store data and the crawler ship with the study.
We run this readiness check against individual stores, then fix what an agent can’t read.
Agentic-Commerce Readiness