Why AI gets your own business numbers wrong

When you point an AI tool at your own systems, ask it a plain question, and get back a number that does not match the one in your head, the model is usually not what went wrong. The question had more than one correct answer, nobody had ever written down which one you meant, and the tool picked one silently.

The test is simple. If two people in your business would answer the question differently, your AI has no chance.

What counts as an active client?

Try that question on your own business. Active could mean anyone who has paid you in the last twelve months. It could mean anyone with work in progress right now. It could mean anyone still on the books who has not formally told you they are leaving.

All three are defensible. Most businesses use all three, depending on who is talking and what they are trying to work out. The sales conversation uses one, the invoicing conversation uses another, and the how-are-we-going conversation uses a third.

Between people, that gap almost never surfaces. Someone says active, someone else nods, and the conversation moves on. The ambiguity gets absorbed by context and tone and shared history.

A machine cannot absorb it. It has to choose one meaning, it chooses without telling you, and then it gives you a confident answer built on a definition you never agreed to.

Is this a data quality problem or something else?

It is related to data quality, but it is not the same thing, and the difference matters because the fixes are different. Data quality is about whether the records are accurate and complete. This is about whether the words describing those records mean anything specific.

The venture firm Andreessen Horowitz made this argument at length in March 2026, writing that data and analytics agents are essentially useless without the right context because they cannot tease apart vague questions, decipher business definitions, and reason across disparate data effectively. Their worked example is a large company asking what revenue growth was last quarter, and the agent having no way to know which definition of revenue or which fiscal calendar the asker meant.

Worth being clear about the source: a16z invests in this category, so read it as a well-argued position rather than as measurement. But the argument holds, and it holds harder at small scale, not less. A big company at least has a semantic layer to be out of date. Most small businesses have never written the definitions down at all.

Why this is good news

Almost every article about AI readiness ends with something you have to buy. This one does not. The fix is a business owner deciding what a handful of words actually mean. Active client. Won job. Complete. Overdue. Quoted. Write down one sentence for each.

That is an afternoon, not a project. There is no software in it, no vendor, and no integration.

First, you should do it whether or not you ever touch AI. Right now your team is quietly disagreeing about these words, and the cost shows up as rework, as two reports that do not reconcile, and as the meeting where everyone realises halfway through they have been discussing different things.

Second, it tells you which vendor claims to discount. Any tool promising to answer questions about your business straight out of the box is promising something it cannot deliver, because it does not know what you mean by active. Nobody has told it. And it will not warn you that it guessed.

That last point is the practical reason this matters more than it sounds. The failure is silent. A tool that returns nothing tells you it failed. A tool that returns a plausible wrong number does not.

How to actually do it

Keep it small enough that it gets finished. Take the number you quote most often, the one that comes up every time someone asks how the business is going. Ask two people to define it separately, in writing, not in a meeting where the first answer anchors the second. If the definitions do not match, write the real one down in one sentence and put it somewhere the whole team can see, not in your head. Then repeat for the next four words, and stop at five. You can always add more later, and a list of five that people actually use beats a glossary of forty that nobody opens.

Once those definitions exist, they become the thing you hand an AI tool, and they are also the thing that makes any automation you build behave predictably when the business changes. We wrote about that durability problem separately in AI automation that adapts, and about giving a tool the wider picture of how your business works in giving AI your business context.

It is also worth being honest about what you are building. If the job is answering a defined question the same way every time, that is a workflow, not an agent, and the distinction changes what you should build. We covered it in AI agent or workflow. This is genuinely where a lot of our engagements start, and it surprises people how unglamorous the first step is. Some examples of where it leads are on our use cases page.

Ready to find out where you stand?

If you want to know whether your business is actually ready to point AI at its own numbers, our free AI Readiness Check covers it in about five minutes. No cost, no pitch.