GenAI Course · Module 9 · Session 32

Document Extraction

⏱ 12 min read🎯 Intermediate🖼 2 diagrams🧪 1 live demo🐍 Python code

RAG is only as good as the text you feed it. Real company documents are messy: scanned forms, two-column PDFs, tables, headers and footers. In this session, let us learn which tool to use for which document, so your chunks start clean.

In this session

Why extraction quality decides RAG quality
Digital PDFs: PyPDF and PyMuPDF
Mixed formats: Unstructured
Scanned documents, forms and tables: OCR, AWS Textract, Azure Document Intelligence
A decision helper for picking the right tool

01Garbage in, garbage out

If extraction mixes up a table’s columns or drops a page header, the retriever will find wrong text and the LLM will confidently answer from it. Common problems:

Scanned pages have no text layer at all, only an image.
Tables become a jumble of numbers when read line by line.
Two-column layouts get read across both columns.
Headers, footers and page numbers repeat on every page and pollute chunks.

The same FD rate table, extracted badly and well. The right side is what you want in your chunks.

02The extraction pipeline

Every document goes through the same five steps, whatever tool you use for step 3.

03Digital PDFs: PyPDF and PyMuPDF

If the PDF was created digitally (you can select the text), a text loader is fast and free.

Library	Strengths	Watch out
PyPDF (pypdf)	pure Python, simple, page by page	weak on complex layouts and tables
PyMuPDF (pymupdf)	very fast, keeps layout better, can extract images and find tables	AGPL licence: check before commercial use

pymupdf_extract.py

# pip install pymupdf
import pymupdf

doc = pymupdf.open("fd_policy.pdf")
for page in doc:
    text = page.get_text()                        # plain text of the page
    tables = page.find_tables()                   # detect tables on the page
    for t in tables.tables:
        print(t.to_markdown())                    # keep table structure as Markdown

04Mixed formats: Unstructured

Unstructured handles PDFs, Word, PowerPoint, HTML, emails and images through one interface. It splits a document into typed elements (Title, NarrativeText, ListItem, Table…), which is very useful for structure-aware chunking in the next session.

unstructured_extract.py

# pip install "unstructured[pdf]"
from unstructured.partition.auto import partition

elements = partition(filename="annual_report.pdf", strategy="hi_res", infer_table_structure=True)
for el in elements[:5]:
    print(el.category, "|", str(el)[:60])

Title | NovaBank Annual Report 2025–26 NarrativeText | This year our digital transactions grew by … Table | Segment Revenue … (sample output)

05Scanned documents, forms and tables

For scanned pages, photos of documents, and forms, you need OCR (optical character recognition), ideally layout-aware.

Tool	What it gives you	Good for
Tesseract (open source)	plain text from images	simple scans, offline use
AWS Textract	text, tables, key–value form fields, queries	KYC forms, invoices, bank statements on AWS
Azure Document Intelligence	layout, tables, and prebuilt models (invoice, receipt, ID document)	enterprise docs on Azure; Indian invoices and IDs
Vision LLMs	describe charts, diagrams and complex pages in text	pages where meaning lives in images

textract_tables.py

import boto3

textract = boto3.client("textract", region_name="ap-south-1")
with open("kyc_form.png", "rb") as f:
    resp = textract.analyze_document(Document={"Bytes": f.read()}, FeatureTypes=["TABLES", "FORMS"])

blocks = resp["Blocks"]
print(sum(b["BlockType"] == "TABLE" for b in blocks), "tables,",
      sum(b["BlockType"] == "KEY_VALUE_SET" for b in blocks), "form fields")

Privacy first: KYC forms and statements contain personal data. Check where a cloud service processes data (for example an Indian region like ap-south-1) and your compliance rules before sending documents.

06Picking the right tool

Demo · What kind of document do you have?

Always clean after extracting: remove repeated headers and footers, fix broken hyphenation, keep page numbers in metadata (not in the text), and store tables as Markdown.

07Summary

Extraction quality decides RAG quality.
Use PyPDF/PyMuPDF for digital PDFs, Unstructured for mixed formats.
Use OCR, Textract or Azure Document Intelligence for scans, forms and tables.
Keep tables as Markdown, and page/section info as metadata.

08Quick quiz

1. A scanned PDF has no selectable text. What do you need?

2. Which tool returns typed elements like Title and Table across many file types?

3. Best format to keep a table inside a chunk?

4. Extracting key–value fields from a KYC form fits…

5. Where should page numbers go?

09References

Created by Ajeetkumar