LLM invoice extraction vs. template OCR: cost, accuracy, and where each breaks

Let me start where most vendor comparisons will not: template OCR is better than a language model on the vendors it has been configured for. It is faster, an order of magnitude cheaper, deterministic, and it never invents anything. If you process documents from eleven issuers and that list genuinely does not change, buy a template engine and stop reading.

That is a real scenario and it is rarer than people expect. Most accounts-payable document mixes have hundreds of issuers, a steady trickle of new ones, and layouts that change without telling you.

The two failure modes are not equivalent

This is the whole argument, so it is worth being precise about it.

Template OCR fails silently. When a vendor moves their total two centimetres to the left, the template still matches something in that region — a tax figure, a page number, a subtotal. It returns a number with full confidence. You find out at reconciliation, or at audit, or when a supplier calls about an underpayment.

LLM extraction fails loudly, if you build it to. A language model is also capable of producing a confidently wrong figure. The difference is that its uncertainty is measurable — and, more importantly, the value can be required to be traceable to a specific region of the page. A field the model returns that cannot be located in the document’s text-and-layout representation is not an extraction, it is a guess, and that distinction is computable.

Neither approach is trustworthy on its own. The question is which one lets you detect being wrong.

What accuracy numbers actually mean

Any vendor quoting “99.2% accuracy” without stating the document mix it was measured on is selling you a number, not information. Accuracy on clean, text-layer PDFs from known vendors is high for both approaches. Accuracy on a phone photograph of a crumpled thermal receipt is poor for both.

The useful decomposition looks roughly like this:

Document class	Template OCR	LLM extraction
Known vendor, text-layer PDF, unchanged layout	Excellent	Excellent
Known vendor, layout changed	Silently wrong	Good
New vendor, clean PDF	Fails or needs configuration	Good
Scanned document, decent quality	Fair	Good
Phone photo, poor lighting	Poor	Fair
Multi-page with continuation tables	Usually needs bespoke handling	Good
Line items with merged cells	Poor	Fair to good

The row that decides most real deployments is the second one, and it never appears in vendor comparison tables because it is the one where the incumbent looks worst.

The number to optimise is not accuracy

It is straight-through processing rate: the proportion of documents that clear every confidence threshold and every validation rule without a human touching them.

Accuracy is a property of a model on a corpus. Straight-through rate converts directly into cost per document and headcount, which is what you are actually buying. It is also the number that improves over the first few weeks of a deployment as the vendor corpus fills in, which makes it a useful signal that the system is working rather than a static claim.

If a vendor cannot tell you their straight-through rate on your documents after a two-week pilot, that is the answer to your question.

Cost per document, honestly

Model spend per document is small and it is not zero. For a typical one-to-two page invoice with a text layer, you are looking at a fraction of a cent to a few cents depending on model choice, page count, and whether OCR was needed. Add storage and compute and it stays in the same order of magnitude.

Compare that to a person doing manual entry at any wage, and the model cost is not the interesting variable. The interesting variable is the straight-through rate, because a document that goes to review costs you the model spend plus the human time — which is worse than manual entry alone.

This is why the economics only work above a certain volume and a certain automation rate, and why anyone who tells you the ROI is obvious at 50 documents a month has not done the arithmetic.

Two cost-engineering notes that matter more than model choice:

• Do not OCR documents that already have a text layer. Most PDFs do. Running OCR over them adds cost and introduces errors that were not there.
• Cache aggressively on the prompt prefix. Extraction prompts are long and mostly static; the schema and instructions do not change between documents. Getting this right cuts input cost substantially.

The architecture that makes either one safe

Whichever engine you pick, the pipeline around it is where the reliability lives:

1. Text layer first, OCR only as fallback. Cheaper and more accurate.
2. Layout-aware parsing. Reading order and table structure preserved, not a flat string.
3. Schema-constrained output. Typed fields, required keys, enumerated currencies. Reject and retry rather than parsing leniently.
4. Grounding. Every value must be locatable in the source. Unlocatable values score low and go to review. This is the single most useful signal in the system.
5. Independent arithmetic validation. Line items sum to subtotal, subtotal plus tax equals total, dates ordered sensibly — checked by ordinary code that has no knowledge of what the model produced.
6. Risk-ranked review queue. Low confidence and high value first.
7. Append-only audit trail. Every value a field has ever held, with actor and reason.

Step 5 catches a class of error that steps 3 and 4 cannot: the document where every field extracted with high confidence and the totals still do not close. That is more suspicious than a couple of low-confidence fields, and a queue that ranks by confidence alone will bury it.

Why we do not fine-tune

The obvious way to improve accuracy on a client’s specific vendors is to fine-tune on their corrections. We deliberately do not, for three reasons:

• It couples accuracy improvements to a training cycle measured in days, when the correction a reviewer made this morning could be helping this afternoon.
• Per-tenant data isolation becomes much harder to guarantee, and in a BPO context that is not a theoretical concern.
• Any individual extraction becomes impossible to explain after the fact.

Retrieval gets you most of the benefit with none of that. Fingerprint the document by layout and issuer, retrieve the nearest accepted exemplars from that tenant, supply them as few-shot context. The improvement on the second and subsequent documents from a vendor is substantial, it takes effect immediately, and the tenant boundary is a WHERE clause rather than a promise about a model’s weights.

How to decide

Run both on a hundred of your real documents, sampled to reflect your actual mix rather than your best-case one. Measure straight-through rate, not accuracy. Include at least ten documents from vendors you have never processed before, because that is the case that separates the two approaches.

If template OCR wins on your corpus, use it — and put a monitor on it that alerts when a known vendor’s extraction pattern changes, because that is the failure you cannot see.