Discussion about this post

User's avatar
Chris McElroy's avatar

You left out the biggest question of all. Do you think that currently, you can accurately and reliably check brand mentions and track them over time? My answer is no.

These three biggest problems.

The “Segment of One”: Modern AI uses “dynamic profiles” that update in real-time based on the user’s browsing behavior, scroll depth, and even device type. This makes a single “rank” or “mention score” for a brand technically impossible to standardize. This is not like SEO or ranking in Google where there is actually a database to draw from. The difference between measuring AI visibility and Google rankings could not be further apart.

Synthetic Data Problem: Tools like Semrush and Nightwatch mostly rely on synthetic queries.These are automated bots asking a fixed set of questions via API. These bots do not have the user’s context, history, hobbies, or professional, which is why their “visibility scores” mostly differ from what real users see.

The “Transparency” Gap: OpenAI, Google, Anthropic & Perplexity AI do not provide analytics APIs for brand mentions. Until they do, any tool claiming to “track” mentions is effectively using “directional surveillance” or simulations rather than hard data.

So a lot of my questions on that survey are applying to where we are right now. No I'm not going to pay a lot for a tool that can't reliably track brand mentions. We don't have that data yet.

My problem isn't that people are trying to build it and who are informing people the limits of that data they are providing. But those that are selling it as an absolute that you can track brand mentions, I believe are either being misleading or don't actually understand the tool they created.

3 more comments...

No posts

Ready for more?