Confidence and review queues

The person-in-the-loop lesson sent a whole item to review when a check failed. In document extraction you can do better, field by field: give each extracted value a confidence, send only the doubtful fields to a person, and let the rest through. A reviewer who confirms two fields is much faster than one who re-keys twenty.

The obvious signal is to ask the model how sure it is, or to read token probabilities where your model's API offers them (check your model's docs). Both are weak on their own. Models can be confidently wrong, and a self-reported "high" often isn't right much more often than a "medium". Treat it as one signal, never the only one.

Signals you compute yourself are usually stronger. Validation: the value doesn't parse, is out of range, or the totals don't reconcile. Disagreement: run two extraction passes (two prompts, two models, or the same model on two text extractions) and compare; where they differ, at least one is wrong. OCR quality: OCR engines report a confidence per word, and a smudged scan gives low scores. Grounding: the quoted source text can't be found on the page it cites.

fields = [
    {"name": "invoice_no", "pass_a": "INV-2291", "pass_b": "INV-2291", "ocr": 0.98, "valid": True},
    {"name": "total", "pass_a": "1240.00", "pass_b": "1210.00", "ocr": 0.95, "valid": True},
    {"name": "due_date", "pass_a": "2024-11-30", "pass_b": "2024-11-30", "ocr": 0.62, "valid": True},
    {"name": "tax", "pass_a": "-12.00", "pass_b": "-12.00", "ocr": 0.97, "valid": False},
]
OCR_MIN = 0.80

for f in fields:
    reasons = []
    if f["pass_a"] != f["pass_b"]:
        reasons.append("passes disagree")
    if f["ocr"] < OCR_MIN:
        reasons.append("low OCR")
    if not f["valid"]:
        reasons.append("failed validation")
    print(f["name"], "->", "review: " + ", ".join(reasons) if reasons else "auto")

Set thresholds per field, by what an error costs. A wrong payment total or interest rate is expensive, so it gets strict rules (any disagreement goes to review); a contact phone number can be looser. Some fields always go to a person, whatever the score, because the stakes are too high.

Design the review screen around the field. Show the proposed value, the source snippet highlighted on the page image, the page number with a link to jump there, why it was flagged ("the two passes read 1,240.00 and 1,210.00"), and buttons to accept, correct, or mark it as not in the document. A bare score with no evidence makes people either rubber-stamp or start from scratch.

Log every decision: the proposed value, its signals and score, and what the reviewer kept or changed. That log tunes the thresholds. For each candidate threshold, count how many fields would go straight through and how many of those were wrong.

# (score, was the value right?) for fields a person checked
log = [(0.97, True), (0.95, True), (0.91, True), (0.88, False),
       (0.85, True), (0.83, True), (0.72, True), (0.64, False)]

for t in (0.80, 0.85, 0.90):
    auto = [right for score, right in log if score >= t]
    print(t, "auto:", len(auto), "wrong:", auto.count(False))

The corrected fields are also your best test cases. Add them to the eval set (the evaluation lesson in the LLMs course), and re-run it whenever you change a prompt, a model or a threshold.

Two numbers tell you whether the system works. Straight-through rate: the share of documents (or fields) that went through with no person touching them. Error rate: the share of those automatic values that were wrong, measured by having a person check a random sample. They pull against each other: loosen the thresholds and the straight-through rate rises, and so, usually, does the error rate. Report both together, every time.

docs = 2_000
straight_through = 1_540   # no person touched them
audited = 200              # a random sample of those, checked by a person
wrong_in_audit = 3

print(f"straight-through {straight_through / docs:.0%}, error rate {wrong_in_audit / audited:.1%}")