Lab 6: Invoice Data Extractor

Build a document-processing app: create a Content Understanding analyzer for invoices, extract invoice number, date, vendor, line items and totals from sample PDFs, run the extraction from Python and save it as JSON and CSV, then try the same idea on an image and a short audio or video clip.

Time: 2 hours · Project: #5 Invoice Data Extractor · You need: your Foundry resource (in a region that supports Content Understanding), its endpoint and key, and 3–5 sample invoices (PDF or photos).

Outcome. A document-processing app that turns PDFs into clean structured data.

Step 1: Connect Content Understanding to your Foundry resource

1. Open Content Understanding Studio (from the Foundry portal’s tools area, or contentunderstanding.ai.azure.com).
2. Go to Settings → + Add resource, select your Foundry resource → Next → Save.
3. Make sure Enable autodeployment for required models if no defaults are available is ticked. Content Understanding uses default model deployments (a chat model and an embedding model) behind the scenes.

Step 2: Create an invoice analyzer

1. In Studio, create a new project / analyzer from the document template, and upload one sample invoice.
2. Start from the invoice template (or the prebuilt invoice analyzer) and review the suggested fields.
3. Define (or confirm) the schema:

4. Select Test / Run analysis on 2–3 invoices. Check each field’s value, confidence and highlighted source on the page. Refine the field descriptions where values are wrong (for example “TotalAmount: the grand total payable, including GST”).
5. Build / save the analyzer and note its analyzer ID, for example contoso-invoice.

Step 3: Run the extraction from Python

Host your sample PDFs where the service can read them: an Azure Blob Storage container with a SAS URL (or a public sample URL to start with). Then create extract.py:

import os, time, json, csv, requests
from dotenv import load_dotenv

load_dotenv()
ENDPOINT = os.environ["FOUNDRY_RESOURCE_ENDPOINT"].rstrip("/")
KEY = os.environ["FOUNDRY_KEY"]
ANALYZER = os.environ.get("CU_ANALYZER_ID", "prebuilt-invoice")
API = "2025-11-01"

def analyze(url):
    r = requests.post(f"{ENDPOINT}/contentunderstanding/analyzers/{ANALYZER}:analyze?api-version={API}",
                      headers={"Ocp-Apim-Subscription-Key": KEY, "Content-Type": "application/json"},
                      json={"inputs": [{"url": url}]})
    r.raise_for_status()
    op = r.headers["Operation-Location"]
    while (res := requests.get(op, headers={"Ocp-Apim-Subscription-Key": KEY}).json())["status"] not in ("Succeeded", "Failed"):
        time.sleep(2)
    if res["status"] == "Failed":
        raise RuntimeError(json.dumps(res)[:500])
    return res["result"]["contents"][0]["fields"]

def val(field):
    """Return the typed value of a Content Understanding field (string, number, date, object, array)."""
    if field is None:
        return None
    t = field.get("type")
    if t == "array":
        return [val(x) for x in field.get("valueArray", [])]
    if t == "object":
        return {k: val(v) for k, v in field.get("valueObject", {}).items()}
    return next((v for k, v in field.items() if k.startswith("value")), None)

urls = [line.strip() for line in open("invoices.txt") if line.strip()]    # one URL per line
records = []
for url in urls:
    f = analyze(url)
    rec = {name: val(f.get(name)) for name in f}
    rec["_source"] = url
    rec["_low_confidence"] = [n for n, x in f.items() if isinstance(x, dict) and x.get("confidence", 1) < 0.7]
    records.append(rec)
    print("✔", url.split("/")[-1], "| fields:", len(f), "| low confidence:", rec["_low_confidence"])

json.dump(records, open("invoices.json", "w"), indent=2, default=str)

# flat CSV: one row per invoice (line items as a JSON string)
cols = sorted({k for r in records for k in r})
with open("invoices.csv", "w", newline="", encoding="utf-8") as out:
    w = csv.DictWriter(out, fieldnames=cols)
    w.writeheader()
    for r in records:
        w.writerow({k: json.dumps(v, default=str) if isinstance(v, (list, dict)) else v for k, v in r.items()})
print("Saved invoices.json and invoices.csv")

Set CU_ANALYZER_ID=contoso-invoice in .env to use your custom analyzer; leave it unset to use prebuilt-invoice.

Step 4: Validate the output

• Does TotalAmount equal the sum of line-item amounts plus TaxAmount? Add a check and flag mismatches.
• Send rows with fields in _low_confidence to a review queue (a separate CSV). This is the human-in-the-loop pattern that keeps the solution reliable and accountable.

Step 5: Beyond documents

1. Image: create an image analyzer with fields such as ProductName, PriceShown, ShelfIsFull: boolean and test it on a shop-shelf photo.
2. Audio: create an audio analyzer for a 1-minute mock support call with Summary, CustomerSentiment (classify) and RefundRequested: boolean.
3. Video: try a short clip and inspect the segments, key frames and transcript in the result JSON.

Step 6: Portfolio

Push to GitHub with a README (problem, architecture PDF → Content Understanding → JSON/CSV → review queue, schema table, sample output, screenshots). Never commit real customer invoices; use samples.

Resume line: Built an invoice-extraction pipeline with Azure Content Understanding (custom schema, confidence-based human review) that converts PDF invoices into validated JSON and CSV.

Check yourself