Extracting data from documents

A common job for AI systems is reading documents that people used to key in by hand: invoices, bank statements, receipts, forms. The model is one step in a pipeline, and most of the reliability comes from the other steps.

Get the text first. A PDF made by software usually has real text in it that can be pulled out directly, often with its layout. A scanned or photographed page is just an image, so it needs OCR (optical character recognition) or a model that can read images. Tables are where extraction goes wrong most often, so look at the output.

Know what kind of document it is. An invoice, a statement and a tax form need different fields and checks, so classify first and route each type to its own extraction prompt and schema.

Extract into a fixed schema: supplier, invoice number, dates, currency, line items, subtotal, tax, total. Ask for null when a field isn't there, and for the page and a short quote for each value so a reviewer can find it.

Then check, because the document usually contains its own answers. On an invoice, the line items should add up to the subtotal, and subtotal plus tax to the total. On a bank statement, the opening balance plus the transactions should equal the closing balance. Dates should parse and be in a sensible range; an IBAN or tax number has a known format.

statement = {
    "opening": 1250.00,
    "transactions": [-89.99, -1200.00, 3100.00, -45.25],
    "closing": 3014.76,
}
expected = round(statement["opening"] + sum(statement["transactions"]), 2)
print(expected, expected == statement["closing"])

Items that pass every check can be loaded automatically; anything that fails goes to review with the failed check shown (the last lesson). Keep the original file, the extracted text, the model's output and the final values together, so any number in your database can be traced back to the page it came from.

Money needs care in code: store amounts as integer cents or as decimals, not floating-point numbers, so 0.1 + 0.2 really is 0.3. The example above rounds to cents before comparing for the same reason.