Project 8: Invoice & KYC Document Extraction Pipeline

Document intelligence is one of the most deployed GenAI use cases in banking, insurance and operations. You'll turn messy PDFs and scans into validated, structured JSON, measure field-level accuracy, and route low-confidence documents to humans.

What you'll build

• Upload invoices (or bank statements / KYC forms; use synthetic or public samples only).
• OCR + layout parsing, then an LLM extracts fields into a strict schema: vendor, invoice number, date, line items, tax, total.
• Validation rules: totals add up, dates are valid, GSTIN format, duplicate detection.
• A review queue for documents failing validation or with low confidence.

Architecture

PDF / image → (scanned?) OCR (Tesseract / PaddleOCR / cloud OCR) → text + layout blocks
   → LLM structured extraction (JSON schema) → validators (sum check, regex, cross-field rules)
   → pass → export (CSV / ERP API)        fail → human review UI → corrections → evaluation set

Tech stack

pdfplumber or PyMuPDF, Tesseract or PaddleOCR, an LLM (or vision LLM) with JSON schema output, Pydantic validators, FastAPI, Streamlit.

Step by step

1. Collect 30–50 sample documents (public invoice datasets or ones you create) and label the target fields.
2. Text path: digital PDFs go through pdfplumber with table extraction; scans go through OCR.
3. Extraction prompt: the schema, field definitions, "return null if missing", never guess.
4. Pydantic validation: types, formats (dates, GSTIN, amounts), sum(line_items) + tax == total within tolerance.
5. Confidence: validation-failure flags, plus self-consistency (extract twice and compare) to route to review.
6. Evaluate: field-level precision and recall, document-level exact match, straight-through-processing rate.
7. Review UI: side-by-side view of the document and the editable fields; save corrections for the evaluation set.

Starter code

from pydantic import BaseModel, field_validator, model_validator
from datetime import date
import re
class Line(BaseModel): desc: str; qty: float; unit_price: float; amount: float
class Invoice(BaseModel):
    vendor: str; invoice_no: str; invoice_date: date; gstin: str | None = None
    lines: list[Line]; tax: float; total: float
    @field_validator("gstin")
    @classmethod
    def gst_fmt(cls, v):
        if v and not re.fullmatch(r"\d{2}[A-Z]{5}\d{4}[A-Z][1-9A-Z]Z[0-9A-Z]", v): raise ValueError("bad GSTIN")
        return v
    @model_validator(mode="after")
    def totals(self):
        if abs(sum(l.amount for l in self.lines) + self.tax - self.total) > 1: raise ValueError("totals do not add up")
        return self
def extract(text):
    raw = llm_json(f"Extract the invoice into this JSON schema. Use null if missing; never guess.\n{Invoice.model_json_schema()}\n\n{text}")
    try: return Invoice.model_validate(raw), "auto"
    except Exception as e: return raw, f"review: {e}"

Deploy it

A Streamlit app on Hugging Face Spaces with sample documents. Add a metrics table (field F1, STP rate) to the README.

Put it on your resume

• Built a document extraction pipeline (OCR + LLM structured output + Pydantic validation) achieving 96% field-level F1 on 50 invoices and 78% straight-through processing.
• Implemented cross-field validation (sum checks, GSTIN format) and confidence-based routing to a human review queue.

Interview questions you'll get

• How do you make extraction reliable and avoid hallucinated values?
• OCR vs vision LLMs: trade-offs in cost, accuracy and privacy?
• How do you measure accuracy, and decide what goes to human review?

Stretch goals

• A layout-aware model (e.g. LayoutLM-style) for tables.
• Duplicate-invoice and fraud checks.
• Batch processing with a queue and an ERP export.