How to evaluate a data or AI proposal before you sign it

There's a proposal on your desk from a data or AI vendor — maybe ours — and you have to decide whether to sign it. You don't need to be technical to evaluate it well; you need to ask the right questions. And the ones that matter are almost never about the technical content — which you can rarely judge from outside, and the vendor knows it — but about what happens around it and afterwards: what stays in your company when the work is done, what dependencies it creates, how you'll know it worked, and what happens if it stops. A serious vendor answers these questions gladly. One who gets uncomfortable is already answering.

What stays inside when the vendor finishes

The deliverable isn't just the working system. It's everything that lets you live with it without calling anyone:

• Documentation your team understands. Not a courtesy technical manual, but enough for someone inside to know what the system does, where its data comes from, and what to touch — and not touch — when something changes.
• Access in the company's name. Accounts, credentials and admin rights on the platforms involved, held by you, not by the vendor. It's one of those things nobody checks at signing and everybody regrets at the first dispute.
• Knowledge transferred. Someone on your team able to operate the basics and explain to a third party how it's put together.

If, when the work ends, only the vendor knows how it works, you haven't bought a system — you've rented one. The question that exposes it is simple: what will my team know how to do at the end that it doesn't know today?

What dependencies it creates

Every proposal creates dependencies; that's unavoidable and not bad in itself. The question is which ones, whether you choose them with your eyes open, and what it costs to get out.

• On the platform. Can the data be exported in a usable format? Does what's being built only work inside that environment, or does the underlying work — definitions, processes, clean data — survive a change of tool?
• On the vendor. Could someone else continue the work from what's handed over, or does it all live in their heads and their accounts? The documentation from the previous section is exactly this.
• On one person. Sometimes the dependency isn't on the vendor company but on the specific person who built it. Ask who else knows it.

A reasonable dependency is one chosen knowingly, with the exit door in sight. The uncomfortable question that sums it up: if we want to leave in a while, what do we take with us and how much does it hurt?

How you'll know it worked

Be wary of proposals whose definition of success is the delivery itself: "system implemented and running". That measures the vendor's activity, not your outcome. Something being delivered and something working for the business are different things.

The success conditions should be written down before the work starts, in business terms: which task stops being done by hand, which decision gets made on better data, which error stops happening. And with two surnames attached: who will measure it, and when — because a success condition with no review date is decoration.

If the vendor resists concrete success conditions, there's a reason, and it's worth listening to carefully: it may be honest caution — data projects depend on the state of the data — or it may be that they're selling activity. You can tell the difference by whether they offer alternatives: someone who can't promise the final outcome can at least commit to verifiable milestones along the way.

What happens if it stops

Nobody signs planning to stop, which is why this is the least-asked question. It has two versions.

If it stops halfway. What's left that's usable? A well-structured proposal has intermediate milestones with standalone value: if the project halts after the first phase, that phase is useful on its own. If the design is all-or-nothing — nothing works until everything works — you carry the entire risk.

If it stops afterwards. Anything built on living systems needs maintenance: the ERP gets updated, a data source changes, something stops adding up. Who responds then, under what commitment and on what terms? The proposal doesn't need every detail closed, but the "afterwards" has to exist in it. Discovering at the first breakdown that maintenance "wasn't included" is an avoidable classic.

What a good proposal doesn't hide

A serious proposal also states what it doesn't include, what it assumes, and what it needs from you. That it assumes your data is in a certain state. That it needs hours from your team — because without your knowledge of the business, no data project lands; a proposal that asks nothing of you is suspicious, not convenient. That some things are out of scope, and which ones.

Explicit limits are not vendor weakness: they're the sign that someone has genuinely thought the project through. Whoever promises outcomes with no conditions, without having seen your data or your systems, has seen nothing resembling your company — or hasn't wanted to look.

Five questions for any vendor (ourselves included)

1. What will my team know how to do when you finish that it doesn't know today?
2. What dependencies does this create, and how do we get out of them?
3. How will we know it worked, in business terms, and who will measure it?
4. What's left usable if this stops halfway — and who maintains it afterwards?
5. What do you need from us for this to succeed?

Any serious vendor should come out of this interrogation stronger, and that includes us: if you want to see where our own answers come from, this is how we work. If a proposal doesn't survive these five questions, the meeting didn't fail. It worked.