How we evaluate an AI feature before it ships 

A pilot that impresses a boardroom is not evidence. Here is the harness we build first.

Demos select for the happy path. Production selects for the twelve documents a week that are photographs of a photocopy. The gap between those two populations is where most AI projects quietly die.

Build the harness before the feature

Our first deliverable on any AI engagement is not a model call. It is a labelled evaluation set drawn from the client's real, ugly data, and a script that scores any candidate pipeline against it.

const results = await Promise.all(
  cases.map(async (c) => {
    const out = await pipeline(c.input);
    return {
      id: c.id,
      f1: scoreFields(out.fields, c.expected),
      confident: out.confidence >= THRESHOLD,
      costPaise: out.usage.costPaise,
      ms: out.latencyMs,
    };
  }),
);

assert(mean(results, "f1") >= BASELINE_F1, "extraction regressed");
assert(p95(results, "costPaise") <= COST_CEILING, "cost ceiling breached");

Three assertions, run in CI on every prompt, model or retrieval change: quality does not regress, cost stays under the ceiling, and latency stays inside the budget. A prompt tweak that improves one and quietly wrecks another cannot merge.

Confidence is a routing decision

• High confidence, high score in eval: straight through.
• Low confidence: human review queue, and the correction becomes a new eval case.
• Escalation to a larger model must log a reason, so cost is explainable after the fact.

The interesting number is not accuracy. It is the share of work the system can complete without a human, at a cost you can defend.

On Atlas, the documents the system is unsure about are routed to a person rather than guessed at. The harness is what lets us say that with a straight face.