When someone asks me to "add AI," they almost never have a model problem. They have a which problem is even worth it problem. An AI audit is how you find out — before you spend six months and a budget line on the wrong one.

I've spent twelve years building ML and AI in production: healthcare integration platforms, clinical tooling, edge systems, generative products. Along the way I've watched a lot of AI initiatives stall. The ones that failed almost never failed because the model wasn't good enough. They failed earlier than that, in ways an audit catches cheaply. Here's what a real audit actually surfaces.

1. Half your ideas shouldn't be built

The single most valuable output of an audit is usually a "no." Not every problem is an AI problem, and plenty of the ones that are have a $20/month tool that already solves them. A good audit ranks your list of ideas by leverage — cost of the problem versus cost of the solution — and is honest about which ones are a database query, a form, or an existing SaaS in disguise.

If an audit only ever says "yes, build everything," it isn't an audit. It's a sales pitch.

2. The problem is written wrong

Most AI briefs are written as solutions — "we want a chatbot," "we want to use LLMs." A workable brief is written as an outcome: hours reclaimed, revenue moved, risk reduced, a specific decision made faster. The audit's job is to rewrite each idea as an outcome a non-engineer would sign off on, with a way to measure it. If you can't measure success, you can't ship responsibly, and you certainly can't tell whether the thing worked.

3. The data isn't where you think it is

This is where audits find the most surprises. The model needs data it can reach — and again and again, that data is trapped in a PDF, a legacy system, one person's spreadsheet, or a vendor you don't control. Or it exists, but nobody knows how often it's wrong. The audit answers three questions bluntly: does the data exist, can the model actually get to it, and is it good enough to trust. In regulated spaces — healthcare especially — it also draws the line the model must never cross around PII/PHI.

Most of the time, the honest finding is: your AI project is a data project wearing a costume.

4. There's no way to know if it works

A demo that works once is not a system. The gap between the two is evaluation. An audit checks whether there's a task-specific eval that catches regressions before your users do, whether the failure modes are known, and whether the worst case is contained. Teams that skip this ship on vibes and find out in production. Teams that build it treat quality as a number, not a feeling.

5. The unit economics don't survive real usage

The prototype ran fine on ten requests. Nobody checked what it costs at ten thousand, or what the latency does when it sits in the middle of a real workflow. An audit puts real numbers on per-request cost, latency, and the operational reality: monitoring, drift, what happens when the model or vendor changes under you, and — the question everyone forgets — who owns this after launch.

What you walk away with

A real audit isn't a slide deck of AI trends. It's a prioritized map: what to build, what to buy, what to skip, and in what order — plus a plan you can execute with or without the person who wrote it. Two weeks of clarity is a lot cheaper than six months of building the wrong thing.

If you want to pressure-test your own thinking first, I put the same framework into a free AI Audit checklist — no signup, just the questions I actually ask. And if you'd rather have it run on your business, that's the audit: a couple of weeks, an honest map, from $5k.

The goal isn't more AI. It's knowing which AI is worth building — and having the nerve to skip the rest.