What Anthropic's Duet Sees When It Shops. And the Fence We Built Ourselves.

Third agent, same Vermont lighting showroom. Anthropic’s Claude Duet found the store without being told, scored it 8 out of 10, and traced the biggest problem to a fence we built ourselves. We had it down by the afternoon.

Neural TL;DR

• Asked for kitchen pendants near the showroom’s own town, Duet fanned out across nine retailers and found our store from local listings, with no hint from us. That’s a first for this series.
• It rated the store 8 out of 10 for agent-readiness: “one of the most agent-ready storefronts I hit today, and the bar is low.”
• Its top finding was our own robots.txt rule. Written to stop crawlers from churning through filter combinations, it also refused 169 of the 218 shop links on the homepage, so a rule-following agent could see “Pendants” in the menu and wasn’t allowed to open it. We moved categories into the URL path the same day. Now it’s 2 of 218.
• With Stripe’s Link connected, Duet built a $109.39 spend request that said in plain text it came from Claude, and stopped for our approval. We cancelled it.

Muse only reached our store because we named it. The dot only reached our catalog because we built the door and handed it the address. Both posts ended on the same open question: who finds you when nobody tells them where you are?

Anthropic opened Claude Duet as a research preview on October 21: a proactive agent with its own virtual machine and its own browser, which can buy things through Stripe’s Link wallet with a one-time virtual card once you approve the spend. On November 4 we gave it a shopping list and the same storefront the other two shopped: an independent lighting showroom in Vermont, 130,000-plus products, not on Shopify, built and run on our platform. Every session described here was ours.

Cold: it found the store on its own

We said nothing about the store. The prompt was the kind a homeowner sends: kitchen pendant lights near me, in the showroom’s own town, on sale under $400.

Duet planned it like a person would (find local stores, search the sales, check each product page, rank a handful) and browsed nine retailers in its own browser, Rejuvenation and West Elm among them. About ten minutes later, our store was on the list. Its own account: it “found [the store] from local listings with no hints, checked every price on the product page, and caught data errors instead of repeating them.”

Then a fresh Duet browser agent took a narrower ask, a light over a kitchen island, and came back in under three minutes with three fixtures from the store’s catalog at $499, $514.80 and $790. No bot checks, no blocks.

Why a local showroom made the cut is the boring part: a local listing pointed there, and the homepage is server-rendered, so a plain fetch reads it. On that most basic test, being readable, the independent beat the national names. Lumens, YLighting and Build.com, by Duet’s report, blocked it outright.

Then it hit our fence

Duet came at the store three ways: a crawler that follows robots.txt, a plain page fetch, and a full browser. The one that follows the rules is the one that got stuck.

The rule-following pass found the Pendants link in our mega menu, saw it carried two query parameters (?category=Ceiling&subcategories=Pendants), and read our robots.txt, which tells ClaudeBot, PerplexityBot and every other polite crawler to skip any shop URL with more than one. It reached one fixture only because that product was linked from the homepage. Ceiling, then “Linear & Island”, the only route to an island light, hit the same wall.

We checked before we touched anything. On the homepage HTML, 169 of 218 unique shop links matched the skip rule. The menu is template code, so every storefront we run had the same problem. Duet guessed that too.

The rule wasn’t an accident. On October 10, 1,204 of the last 1,208 shop requests from Meta were stacked-filter URLs, the same listing sliced every way a crawler can slice it. Loosening the rule would have invited that back. Duet said as much, unprompted: the rule “is probably there to stop crawlers from piling up filter combos.” Its recommendation was to fix the URLs, not the rule.

This is the part robots.txt advice misses. SEO guides say disallow faceted URLs to save crawl budget. AI guides argue about blocking AI bots at all. Nobody connects the two: if your navigation spells a category as a filter combination, your crawl-budget rule quietly locks rule-following agents out of your own menu. A browser agent ignores robots.txt and walks straight in. The polite one obeys and leaves.

The fix is a naming problem. A category isn’t a filter combination. It’s one node in a tree, but our menu spelled it as two parameters. So the node moved into the path, /shop/c/lighting-ceiling/pendants, and the query string went back to doing what it’s for: real filters. The robots rule didn’t change.

It went live across our client storefronts that afternoon (one shared server pool rolls on the next deploy). Refused homepage links: 169 to 2. All 183 category paths answer 200 with products and a self-pointing canonical. A new test walks every link the menu emits and fails the build if robots.txt would refuse one; it failed on the old code. The 2 holdouts are homepage tiles the client authored in our site editor, a content edit.

It read the body, not the head

The best moment of the day was Duet getting something wrong.

It told us the store’s pages had no link tag pointing agents to our UCP profile or our llms.txt. We checked. Every page had both, in the head. Duet’s reply: “Fair, I got that one wrong.” Then it explained why, and the explanation is the finding. Its fetcher turns HTML into markdown and drops head link tags completely. It could never have seen them.

Then it went further. The fresh browser agent, which could see the head, walked straight past those tags to the search box. Its conclusion: rel=alternate and llms.txt help agents built to look for them, and general shopping agents don’t read them. That’s the dot and Muse lesson again, from a third lab. If the way to your catalog lives only in metadata, a whole class of agent tooling can’t see it. Put the directions in the visible page.

Same story with our UCP profile, the live catalog endpoint we shipped after the dot test. Asked about it, Duet gave the cleanest line of the series: “Use it, yes, easily. Find it on my own, no.” It would take the MCP endpoint over scraping any day, and that route skips the robots wall entirely. It just never stumbled onto it. Three agents from three labs now, and store discovery and endpoint discovery turn out to be different problems. Duet solved the first. Nobody has solved the second.

The ceiling is the data

Once the doors are open, the next thing an agent trips on is the catalog itself. The fresh browser agent found the fixtures fast and then hit:

• A product card that said Light Brass and linked to Gunmetal.
• A linear island fixture listed at 60” long and 60” wide, named “LED” with a light source of three E26 bulbs.
• A fixture with no dimensions block at all, and breadcrumbs that didn’t match the menu.

We reproduced the 60-by-60, LED-versus-bulbs and category conflicts on the live product page. They came in with the catalog data and nothing checked them on the way, and they’re exactly what wrecks an agent’s comparison. A person shrugs at a typo. An agent ranks on it.

Duet’s path from 8 to 10: open the category pages, validate specs at ingest, put a plain search form in the HTML, add dimensions and finish to the product structured data, add checkout to the UCP profile, and get found (redirect the old domain, add hours and coordinates to the store schema, link the Google Business Profile). “The first three are a day of work and get you to 9.”

As of November 4: category pages are live. The search form and the structured-data additions are merged and roll out next. A gate that stops contradictory specs before they publish is filed with our data team. Checkout over UCP isn’t built, and the old domain’s redirect waits on the client. We’ll rerun Duet after the rollout and ask for the score again.

The first request that said “agent”

Duet’s own read on the series was blunt: ahead of Muse on finding and accuracy, level with the dot on discovery, “behind both on getting to checkout,” because our brief had kept it read-only. So we lifted that.

We connected Stripe’s Link, gave Duet a product URL and told it to stop before payment. It built the order (a Savoy House Avon wall sconce in warm brass, store pickup at the showroom, Vermont sales tax included) and sent a spend request to Link. In Link it read: “Claude Duet is requesting to spend $109.39. Requested by Claude on your behalf.” We cancelled it.

Look back at the dot’s checkout: a real Chrome, a real person’s name and card, a person approving the total, and nothing anywhere to say an agent picked the pendant. This time the payment rail knew. The request named the agent, described what it was buying and why, and would have paid with a one-time card that works for that transaction only. The same afternoon we asked ChatGPT whether it could hook up to Link or Shop Pay. It said it didn’t have a direct connection to either wallet.

That’s the agent’s side. What our own logs saw is a separate question, and we haven’t reconciled it yet.

What we’re not claiming

• The 8 is Duet’s opinion, not a benchmark. One agent, one store, one day. Fixtures and prices are as Duet reported them on November 4.
• “Found it on its own” is Duet’s account. We saw the store come up in its cold run. We can’t see which listing it used, and by Duet’s read the domain still doesn’t rank for its own name.
• No order was placed. We stopped at approval, same as the dot and Muse runs. Duet built the order from a URL we gave it, not from the UCP catalog.
• Disclosure: we’re in the Claude Startup Program, and we build on Claude every day. Weigh the praise accordingly. The defects were ours.

What to do this week

• Test your own menu against your robots.txt. Run every link your navigation emits through your disallow rules as ClaudeBot. If most of them fail, you have our bug.
• Spell categories as paths, filters as queries. One URL, one meaning. Keep the crawl-budget rule. Fix the URLs it was catching.
• Put the directions in the body. A plain-HTML search form and visible catalog links beat head tags for every agent that strips them.
• Validate specs before they publish. Length equal to width on a linear fixture, LED plus a bulb count, a finish that links to another finish. Agents compare on these fields.
• Watch the payment layer. An agent wallet was the first place in this series where the agent identified itself in the transaction.

The part worth remembering

The new protocols didn’t trip Duet up. Its problems came from three old surfaces: a robots.txt rule, a page’s HTML, and a product spec sheet. We already owned all three. The rule had been written to keep bad crawlers out, and it worked so well it kept the good ones out too.

Duet was the first agent in this series to find the store without being told, and the first to arrive at payment announcing what it was. Discovery of the store moved. Discovery of the endpoint didn’t. Next time: the rerun, and whether a 9 is a day of work.

---

Neural Partners is an AI-native marketing and technology agency founded and based in Vermont. We build on frontier AI platforms every day, as practitioners, not spectators.