How to automate document reading and classification with AI

Reading, classification and extraction are different tasks

OCR recognises characters in an image. Classification suggests the document type. Extraction assigns values to fields. Validation checks them against rules and context. Accuracy at one stage does not ensure accuracy at the next.

A digital PDF may already contain text, but its reading order can still be wrong. A structured electronic invoice may allow direct integration. Do not rasterise reliable structured data just to reinterpret it with a model.

A complete flow from email to review queue

Restrict file formats and size, and apply security controls. A document is untrusted input. Use message and attachment IDs to trace origin without copying unnecessary data across systems.

A hash finds identical files; supplier, invoice number and period can flag a resent invoice with a new filename. Review possible duplicates so legitimate documents are not silently blocked.

Define the output before choosing a model

The ERP needs a data contract: fields, types, required values and statuses. Valid JSON proves structure, not truth. An absent value should remain explicit, not become zero or today’s date.

{
  "document": "example-001",
  "type": "invoice",
  "number": null,
  "total": null,
  "status": "pending_review",
  "reasons": ["number_not_found"]
}

Keep page, region or passage for each proposed field so reviewers can inspect its evidence. Give each ERP operation a stable identifier; after a timeout, check whether the draft already exists before retrying. Otherwise a retry can duplicate records.

Business rules and model confidence

Validate formats, required fields, totals, currency, duplicates and matches to master data. A changed bank account or unknown supplier needs a specific review even if extraction confidence is high.

Microsoft’s documentation explains confidence scores in document systems. They are not universal guarantees or directly comparable across fields and models. Set thresholds from observed errors and the consequences of action.

Use deterministic calculations for totals and tax. The model may locate candidate fields; rules calculate and compare. Handle rounding differences explicitly rather than silently changing values.

Design review as part of the product

Group cases by reason: missing field, ambiguous supplier, possible duplicate or integration failure. Reviewers need to correct, reject or return a document to intake and understand each action’s effect.

Keep proportionate change history. Human corrections should not automatically become training data without a defined purpose and permissions. Monitor queue age and review capacity; rapid extraction is no gain if unresolved work accumulates.

Test difficult documents in the pilot

1. Separate tuning material from evaluation samples.
2. Include real formats, suppliers and languages, skewed pages, multiple attachments and missing fields.
3. Measure field accuracy and critical document errors, not only an average.
4. Count review time, retries, duplicates and lost records.
5. Test ERP outages, revoked permissions and new formats.

Acceptance criteria should match risk. Start alongside the existing process without creating final records, then expand permissions after evaluation. A parser for a common format or a better scan may help more than a larger model.

What to prepare for an assessment

A process map, document types, destination fields and a list of exceptions are enough for an initial discussion. Agree a secure channel before sharing real confidential samples; do not put them into a public contact form.

The ten-process guide compares other opportunities. If your main problem is finding answers in filed documents, read the RAG guide. Our document intelligence service addresses the integration.