Why We Built LLM Metrix

It Started with a Spreadsheet

About eighteen months ago, I was helping a friend who runs a B2B SaaS company figure out why their new customer acquisition had slowed down. Organic search traffic was flat, paid channels were performing normally, and the product hadn’t changed. But the number of demos booked per month had dropped noticeably over about a quarter.

While talking through it, we noticed something odd: a few new customers mentioned they’d heard about the company from “asking ChatGPT.” We started asking more new customers how they’d first heard about the product. The answers were all over the place — still mostly traditional channels — but a meaningful and growing percentage mentioned AI engines. ChatGPT specifically, but also Perplexity and, increasingly, Gemini.

Then we asked: what were they told when they asked?

Nobody knew. There was no way to know. You could go ask ChatGPT right now, but that was a single data point. What was the typical response? How was the brand described relative to competitors? Was the description accurate? Was it improving or degrading over time?

We built a spreadsheet. Every Monday morning, we manually queried ChatGPT, Claude, and Perplexity with fifteen category-relevant questions and tracked the responses. It took about two hours a week. After a month, we had patterns. After three months, we had enough data to act on.

The results of acting on that data were unambiguous. Targeted content changes — rewriting the product description to be more specific, adding comparison content for the top two competitors, updating the G2 page — produced measurable improvements in how often and how favorably the product was described in AI responses. New customer acquisition started recovering.

We shared what we were doing in a post on a founder forum. The thread got hundreds of replies. Half of them were asking for the spreadsheet template. The other half were asking if we’d built a tool for this yet.

We hadn’t. But it was obvious we should.

The Problem We’re Solving

There are a lot of ways to frame the problem LLM Metrix addresses. The simplest: AI engines are now a primary channel for brand discovery, and nobody is tracking them.

This isn’t a future problem. Right now, across every B2B software category, product comparison, and consumer buying decision, people are asking AI engines for recommendations. The AI engine they ask is synthesizing a response from whatever it knows about the brands in the category and delivering a ranking. If your brand isn’t in that ranking, or is described inaccurately, or is mentioned fifth instead of first — you’re losing deals and not knowing it.

Traditional analytics tell you nothing about this. Google Search Console doesn’t track AI engine queries. GA4 doesn’t have an “AI referral” dimension. SEMrush doesn’t rank you on ChatGPT. There is a massive and growing channel that is completely invisible to most marketing teams.

What We Built

LLM Metrix does three things.

First, it measures. We run thousands of category-relevant queries against seven major AI engines every week and analyze the responses: does your brand appear, in what position, with what sentiment, and how does that compare to competitors. We turn this into a Unified Visibility Score — one number that tells you, at a glance, how visible your brand is in AI search.

Second, it explains. A score without context is useless. We show you exactly which queries your brand appears in and which it doesn’t, how each engine describes your brand, what competitors are doing differently, and where the specific gaps are in your AI visibility.

Third, it alerts. When your score drops, when a competitor overtakes you, when a new AI engine starts representing your brand inaccurately — you hear about it in real time, not when you remember to check a dashboard.

Where We’re Headed

We’re still early, and we know it. The AI search landscape is changing fast — new engines launching, existing engines changing their retrieval patterns, the line between search and AI assistant continuing to blur. We’re building LLM Metrix to stay ahead of those changes rather than reactively catching up to them.

Our near-term roadmap is focused on three areas: expanding engine coverage (we’re adding support for two new engines this quarter), deepening the query intelligence (helping users understand which queries are highest-value to win, not just whether they’re winning them), and building the feedback loop between LLM Metrix data and the content changes that move the needle.

The longer-term bet: AI search is going to become as measurable and optimizable as traditional search eventually, and the companies that build the measurement layer will help define how that optimization works. We want to be the platform that marketing and content teams use to understand and improve their AI search presence the same way they use SEMrush or Ahrefs for traditional search.

If you’re reading this because you’re trying to figure out how your brand shows up in AI search, you’re thinking about the right problem at the right time. We’d love to help you figure it out.