The cheapest AI quote usually leaves out the part that tells you whether it works

You have three quotes for the same AI project. Two land within a few thousand pounds of each other. The third is 40% cheaper, and it comes from a supplier who seems to understand the problem better than the other two. The demo was good. The timeline is shorter. It is very tempting.

Look for the line in each quote that says how anyone will know the system works.

In the cheap one, it usually is not there.

We have been on both sides of this. We have written quotes for AI work and bought it, and the gap between those two views is almost always the same thing: the cost of proving that the AI gives the right answer often enough to be trusted with real work. Sellers leave it out because it makes them uncompetitive. Buyers do not ask for it because nobody told them it exists.

What does testing an AI system actually cost?

More than most quotes allow for, by a wide margin. Hamel Husain and Shreya Shankar, who have trained thousands of engineers in AI evaluation, say that on their own projects 60 to 80% of development time goes on error analysis and evaluation. That is not a rounding figure added at the end. It is most of the work.

A traditional software quote does not look like that, because traditional software is mostly right or wrong in ways a tester can check once. An AI system is right some of the time, and the whole question is how much of the time, on which kinds of input, and what happens to the rest. You cannot answer that by clicking through a demo. You answer it by building a set of real examples with known correct answers, running the system against them, reading the failures one by one, fixing things, and running it again. Then again when the model underneath changes.

That is the expensive part. It is also the part that tells you whether you have bought anything.

The industry is mostly guessing

If careful teams spend most of their time on evaluation, you might expect the rest of the market to be doing at least some. The figures are not reassuring.

In Applause's 2026 research, covering more than 1,000 developers and QA professionals, 46% said human sentiment and usability were the main factors in deciding whether an AI feature was ready for production. Only 33% used an automated method such as LLM-as-judge to score outputs, and 52% said fewer than half of their AI projects made it from proof of concept to production.

Read the first number again. Nearly half of teams decide an AI feature is ready largely because it feels right to the people using it. That is the demo, extended by a few weeks.

The cost of skipping the work shows up later. S&P Global's research found the share of companies abandoning most of their AI initiatives before production rose from 17% to 42% in a year. Not every one of those died for lack of testing. But a project with no agreed measure of "good enough" has no way to show it has arrived, and it tends to drift until somebody stops paying for it.

Why the cheap quote leaves it out

This is the uncomfortable bit, and it is not usually dishonesty.

Evaluation looks like padding. On a quote, "build evaluation set, 12 days" reads like a supplier inventing work. A buyer comparing three quotes line by line will cut it, so experienced sellers learn to leave it out and absorb it later, or not do it at all.

The demo already looks like proof. A good demo answers ten hand-picked questions correctly. It says nothing about the hundredth question, or the one phrased the way your customers actually write. But it feels like evidence, and it is what the buyer remembers.

Nobody owns "correct". A test set needs someone in your organisation to say what the right answer is for each example. That is your staff's time, not the supplier's, and a quote that names it is a quote that asks you for something. It is easier to sell one that does not.

It moves the risk to you. If nobody agreed what accuracy the system needed, nobody can say it failed. The supplier delivered what was specified. The specification just never said whether it had to work.

From the selling side, we have watched a lower quote win by doing exactly this. From the buying side, we have inherited the result: a system that worked in the demo, nobody could say how often it was wrong, and fixing that cost more than doing it properly at the start would have.

How to read an AI quote

You do not need to understand the technology to check this. You need to ask questions a good supplier will be pleased to answer and a weak one will find awkward.

• Where is the evaluation work, and how many days is it? If it is not a named line, it is not in the price. If it is a day or two on a project of several weeks, it is a gesture.
• What accuracy are we aiming for, and measured how? "High accuracy" is not an answer. "At least 92% of invoices coded correctly on a set of 300 real ones you provide" is.
• Who builds the test set, and how much of our time does it need? A supplier who says "none" either plans to make up the examples or plans not to have any.
• What happens when the model provider updates the model? If the answer is not "we rerun the test set", you will find out about changes from your customers. We wrote about what that looks like when it happens.
• Do we keep the test set when you leave? It is the most valuable thing the project produces, and it should belong to you.

Our post on testing an AI vendor's accuracy claims goes further into the last two.

What a fair price looks like

Expect evaluation to be one of the largest lines on the quote, not the smallest. On a short proof of concept, a fair split puts more effort into proving the thing works than into building it, because building something that answers the demo questions is the easy part.

That makes honest quotes look expensive next to dishonest ones. It is also why we price our four-week proof of concept as a fixed fee with the test set and an agreed pass mark inside it, rather than as a build with testing to follow. If it does not hit the number you agreed, you know before you have committed to the full project, and you keep the evidence either way. We set out the wider reasoning in what a proof of concept should cost.

The cheapest quote is often the one that has left out the question you most need answered. Before you sign it, ask where the answer is.