Data · AI · Engineering
Turning thousands of complex financial documents into trusted, machine-readable data
An engagement where the data isn't a report someone reads — it feeds automated decision systems directly. That raised the bar from "mostly accurate" to "trustworthy without human checking," and shaped an approach built on three pillars: extraction, intelligence, and cross-validation.
The problem shape
A capital-markets client depends on data drawn from a large, ever-changing library of complex published documents — dense tabular schedules with multi-column layouts, footnote-driven exceptions, mixed currencies, and revisions that land without warning. The output feeds automated systems, so the accuracy requirement is existential: a single mislabeled value can misinform a downstream decision. And because the client independently diffs our data against the original source, the extraction has to be faithful to the source and defensible line by line.
Off-the-shelf OCR or a naïve "LLM reads the PDF" approach fails this bar — it produces plausible output with no guarantee of correctness, no measure of confidence, and no way to catch the one row in ten thousand that's wrong before it ships.
The three pillars we built
A versioned pipeline that reads heterogeneous, complex tabular documents into clean structured records — resolving multi-column layouts, footnote relationships, mixed currencies and units, and tracking every revision so changes are detectable over time.
Raw extracted values are normalised into typed, enumerated fields (a signed, currency-aware numeric value with an explicit direction and unit — not free text). Every record is scored for correctness and completeness on a 0–100 scale, so consumers know how much to trust each row.
An independent validation layer re-derives the answer from the original source and flags any disagreement as a defect candidate. A pre-delivery audit runs before every hand-off; when one defect class is found, the whole dataset is swept for its siblings.
The operating principle: a mistake must be caught by our own audit before the client sees it. Quality is measured as client-found defects per delivery, with a target of zero — the difference between "usually right" and "nothing wrong ships."
How the intelligence + cross-validation work together
- Extract & normalise. Each document version is parsed and every value decomposed into typed fields — the economic meaning made explicit and machine-checkable, while the original text is preserved for audit.
- Score independently. A separate scoring pass grades each record against its own source chunk for correctness, and each section for completeness (did we miss anything the document stated?). Scores are stored on the record and delivered alongside it.
- Cross-validate & gate. An audit re-checks the data against source and flags contradictions — for example, a value whose sign or label disagrees with what the document actually states. High-confidence, clean data ships automatically; anything ambiguous is held for human review, never shipped blind.
- Sweep for siblings. Whenever a new defect pattern is identified, the entire back-catalogue is swept for the same pattern, so the second instance is found by us — not the client.
Most extraction projects stop at "we got the data out." The value here is the layer on top: a confidence signal on every record and an independent cross-check against source that turns a best-effort extraction into a delivery you can gate on. The consumer isn't asked to trust a black box — they're handed a measured, verifiable data product.
Outcomes
| Dimension | Result |
|---|---|
| Scale & coverage | 160k+ records across 45+ sources, multi-currency, with rolling historical revisions |
| Record-level quality | ~95% average correctness, ~99% completeness — measured, not asserted |
| Trust signal | Every delivered record carries correctness + completeness scores the consumer can gate on |
| Defect prevention | Independent audit surfaces contradictions and sweeps for siblings before delivery |
| Fidelity | Values traceable line-by-line to the original source; survives the client's own diff checks |
What transfers to a new engagement. The same three-pillar method — faithful extraction, an intelligence layer that scores and normalises, and independent cross-validation that catches defects before the consumer does — applies to any problem where documents must become trusted structured data: contracts, regulatory filings, specifications, catalogues, rate cards. The domain changes; the discipline of "measured, verifiable, prevention-first" is the reusable asset.