Executive summary
In under a year, AI commerce has grown its first real protocol layer: discovery files at the site root, structured commerce attributes in the feed, agent-facing endpoints, and crawler-permission tokens. Platforms have moved from ignoring these to shipping them by default. This guide maps the layer as it stands in July 2026 — what each protocol does, what the defaults leave empty, and the boundary that matters most: protocols decide whether an agent can find and transact with you; they say nothing about whether you’re the answer it gives.
1. Why protocols suddenly matter
Consumer behaviour moved first. By April 2026, 77% of US consumers had used AI when shopping in the past six months, and 44% start purchase journeys inside an assistant (Exploding Topics, reported by Search Engine Land). Yet only 2% complete the purchase inside the AI — the assistant is the shop window, not the till. The industry’s plumbing has been racing to catch up with exactly that split: standards for being read by agents have matured fastest, standards for transacting through them are still finding their footing — Walmart’s public step back from checkout-inside-chat, after conversion ran at roughly a third of its own site’s rate, is the defining data point.
For a merchant, that resolves the priority question. The protocols worth adopting today are the ones that make you legible at the discovery layer, where the behaviour already is.
2. The discovery files
Four files at the root of a store now do the work of routing AI crawlers and agents. Each serves a different reader; all must agree.
robots.txt — the oldest and still the gatekeeper. It advertises the sitemap and sets crawler permissions (see §4). Every AI fetcher checks it first; a misconfigured robots.txt silently undoes everything below.
llms.txt — proposed in late 2024 as a short, model-facing summary of what a site contains: the most important pages, top products, essential context. Its long-form sibling, llms-full.txt, carries enough detail that a model can answer basic questions about the store without crawling further.
agents.md — instructions for AI agents rather than models reading passively: what the store is, what’s allowed, where to look, how to interact. The significant development of 2026 is that Shopify now ships this family natively: production stores serve llms.txt, llms-full.txt and agents.md at the root by default, with agents.md describing itself as the canonical agent-facing file and llms.txt as a mirror. A /.well-known/ucp endpoint sits alongside them.
The default is the opportunity. Pull the default agents.md from any set of Shopify stores and you find the same short generic template — a handful of fixed sections with only the store name and domain substituted in, its largest block routing shopping agents through platform checkout. No product authority. No brand voice. No curated answers about what the brand sells or what an agent should say when a shopper asks. A clothing brand, a supplements brand and a homewares brand ship the same file on day one. The platform has made the file canonical; filling it with substance — an identity statement an agent could quote, canonical product and category URLs, real answers in the brand’s voice — remains entirely the merchant’s job, and almost no merchant has done it. (Override mechanics are theme-level; check current Shopify documentation for the supported route.)
3. The commerce protocols
Universal Commerce Protocol (UCP) — announced at NRF in January 2026 and developed with ecosystem partners including Shopify, UCP is Google’s framework for plugging brands into AI shopping conversations across Search, Gemini and AI Mode: conversational brand agents on the search surface, offers surfaced at the moment of stated intent, and discovery endpoints a store exposes. Its significance is less any single feature than the admission it encodes: on Google’s own surfaces, the shopping journey is expected to happen inside conversations.
Merchant Center conversational attributes — the structured-feed side of the same shift. New attribute sets feed Google’s conversational layer with the machine-readable signal it needs to answer confidently. Worth adopting with one caution: the feed powers the Shopping graph — carousels and shopping modules — which is not the same retrieval path as conversational citations in chat answers. Feed quality lifts one; it doesn’t purchase the other.
Checkout rails — instant-checkout integrations inside assistants exist and continue to evolve, but the consumer-trust evidence (31% of US consumers say they wouldn’t authorise an AI agent to spend on their behalf) and the Walmart result argue against making them the priority. Discovery on the assistant; transaction on your own surface, where the account, saved payments and returns flow live.
4. Crawler permissions: two decisions, not one
The most common protocol mistake we see is treating “AI bots” as one category in robots.txt. There are two separate decisions:
The training decision. Crawlers like GPTBot, CCBot and Bytespider collect content for model training. Blocking them is a defensible intellectual-property posture and does not remove you from AI answers today. Two tokens are widely misunderstood here: Google-Extended and Applebot-Extended are training opt-out signals, not fetchers that serve answers — Google states explicitly that Google-Extended does not affect a site’s inclusion in Search.
The retrieval decision. Fetchers such as PerplexityBot and OAI-SearchBot read pages live, at the moment a user asks, so the engine can cite them. Blocking these makes you invisible to the answer surfaces you most likely want traffic from. Audit your robots.txt — and any CDN-level bot management — against both lists separately, and re-check after September 2026, when Cloudflare’s announced split of crawler controls into Search, Agent and Training categories lands and existing blanket settings inherit new semantics.
5. The job no protocol does
Here is the boundary this guide exists to draw. Every protocol above answers a findability question: can an agent locate your store, read it, and (eventually) transact with it. Adopt them all, perfectly, and one question remains entirely open: when a shopper asks for the best product in your category, does the assistant name you?
That is decided by what the agent finds when the protocols deliver it to your content — whether your product information is specific, structured, current, and above all consistent with itself across every surface the model reads. A perfect agents.md pointing at contradictory product pages is a well-signposted road to a shop the assistant won’t recommend. The evidence base for that claim — including the controlled studies showing structure without substance moves nothing — is assembled in our companion report, The Coherence Report.
Protocols make you findable. What makes you chosen is the coherence of what they find.
6. Adoption checklist
In priority order, for a store starting today: confirm robots.txt and CDN bot settings implement the training/retrieval split deliberately (§4); check what your platform already serves at /llms.txt, /llms-full.txt and /agents.md, and replace default substance with real substance; keep all discovery files consistent with each other and with the visible site; adopt the Merchant Center conversational attributes if you’re on Google surfaces; treat checkout-in-chat as an experiment, not a priority; and put your effort where the protocols can’t help — the consistency and specificity of the product information itself.
Conclusion
Protocol layers reward early, correct adoption and punish blanket defaults. In 2026 the discovery files are canonical, the commerce protocols are converging, and the defaults are empty. The merchants who fill them — and whose underlying catalogue deserves the citation the plumbing makes possible — are competing for answer slots most of their category doesn’t yet know exist.
Geoffy builds coherence infrastructure for AI commerce: discovery files, structured data and intent-tier content engineered on the merchant’s own domain and kept in sync as the catalogue changes. Live today on Shopify and WordPress / WooCommerce.
References
- Exploding Topics survey (n=1,009 US consumers), reported by Search Engine Land, April 2026.
- Walmart EVP product remarks on ChatGPT checkout performance, reported March 2026.
- Shopify production-store discovery files: first-party probes, May–July 2026.
- Google, Universal Commerce Protocol announcement, NRF, January 2026.
- Google Search documentation on Google-Extended.
- Cloudflare, crawler-control category changes effective 15 September 2026.