What share of Claude's brand mentions only ever show up when search is switched on

Every brand mention Claude produces in this study falls into one of three buckets: named only from memory (web search off), named only with web search (on), or named under both conditions. The largest of the three, by a clear margin, is the search-only bucket.

Across 33,734 total brand mentions gathered from Claude Sonnet 5's answers to buyer questions spread over 82 categories, 49% of mentions (16,525 of 33,734) appeared only in the answers where web search was switched on. That compares with 35.1% (11,825 of 33,734) for memory-only mentions, and just 16% (5,384 of 33,734) for brands named under both conditions. Search-only mentions are the biggest single class, bigger than memory-only and bigger than the overlap class, and in fact bigger than the other two combined.

The setup behind this number: Claude was asked the same buyer question four times per question, twice with web search off and twice on, holding the model, question wording, and length limit fixed across all four runs. Claude Haiku then extracted every brand name from every answer against a fixed rubric. Pooling the two search-on answers per question against the two search-off answers, each resulting brand mention is tagged with exactly one of the three classes above.

Why would search produce so many brands that memory alone never surfaces? The most straightforward explanation is that a live web search pulls in material that simply is not available to the model purely from training: a recent "best of 2026" listicle, a comparison page, a review aggregator, a forum thread about a niche product. Some of these sources will mention brands that are small, new, regionally specific, or otherwise underrepresented in whatever text the model was trained on. Search doesn't just confirm what the model already "knows," it introduces a materially different candidate pool.

A second, less flattering explanation is also worth naming: search results can introduce brands that are highly visible in search (through SEO, sponsored placement, or simply being discussed more in recent web content) regardless of whether they are the best answer to the buyer's question. This study measures what gets named, not what should get named, so it cannot distinguish "search surfaced a genuinely better or newer answer" from "search surfaced whatever ranks well in a search engine, independent of merit." Telling those apart would require comparing the search-only brands against some independent quality or satisfaction measure, which this dataset does not contain.

What we can say with the data in hand: the imbalance between the search-only share (49%) and the memory-only share (35.1%) is large enough that it is unlikely to be explained by run-to-run sampling noise alone. If search and memory answers were just two independent random draws from the same underlying distribution of plausible brands, the two only-in-one-condition buckets would tend to be closer in size. Instead, the search-only bucket is meaningfully larger, consistent with search genuinely expanding, not just reshuffling, the set of brands in play.

For a company trying to understand its own visibility, this finding points to a specific question: do you appear in the kind of content a search engine would surface for this buyer question right now, independent of whatever Claude might already associate with your category from training? That is a different, and more actionable, question than asking whether the model has merely heard of you. It can change by publishing or earning visibility in current web content, on a timescale of weeks, in a way that training-data presence cannot.