Module 9 cheat sheet: LangChain + RAG + LangGraph

RAG answers from your own documents: extract and chunk them well, retrieve the best chunks, generate with citations, and measure everything.

All modules๐Ÿ“– Glossary

The RAG pipeline

Offline: load โ†’ extract โ†’ chunk โ†’ embed โ†’ store
Per question: embed query โ†’ retrieve โ†’ re-rank โ†’ prompt โ†’ answer + citations
  • Fine-tune for behaviour; RAG for knowledge; long context for one-off documents. S31 โ†’
  • Extraction quality decides RAG quality; keep tables as Markdown. S32 โ†’
  • Start with 300โ€“800 token chunks plus overlap; recursive splitting is the safe default. S33 โ†’
  • Use the same embedding model for documents and queries. S34 โ†’

Pick a tool

NeedTool
Digital PDFPyPDF, PyMuPDF
Mixed formats (Word, HTML, PPT)Unstructured
Scans, forms, tablesOCR, AWS Textract, Azure Document Intelligence
Learning / prototype DBChroma
Managed production DBPinecone
Fast library, self-managedFAISS
Already on PostgreSQLpgvector

Better retrieval

TechniqueFixes
Multi-queryVague or short questions
Self-queryAdds metadata filters (year, type)
Parent-documentSearch small chunks, return the full section
Hybrid (BM25 + vectors)Exact codes, names, IDs
Cross-encoder re-rankWrong order in the top results
RRF score = ฮฃ 1 / (60 + rank)

S35 โ†’

Vector index types

IndexHowSpeed (our test)
FlatCompare with every vector (exact)3.90 ms / query
IVFSearch only the nearest clusters0.21 ms
HNSWHop through a graph of neighbours0.06 ms (recall 0.97)

FAISS on a small clustered dataset; your numbers will differ. S34 โ†’

Evaluate before you ship

StageMetricQuestion it answers
RetrievalHit rate @kIs the right chunk in the top k?
RetrievalMRRHow high is the first right chunk?
GenerationFaithfulnessIs every claim supported by the chunks?
GenerationAnswer relevancyDoes it answer the question asked?

Automate with RAGAS or DeepEval on a golden dataset. S36 โ†’

LangGraph and production

  • LangGraph = state + nodes (steps) + edges (arrows), with conditional edges and cycles. S37 โ†’
  • Agentic RAG: grade the retrieved chunks, rewrite the query and retry, then check the answer. S37 โ†’
  • Index incrementally: add, update and delete only what changed. S38 โ†’
  • Make citations mandatory; refuse when nothing relevant is found. S38 โ†’
  • Apply access control at retrieval, mask PII, keep audit logs. S38 โ†’

Code: Reciprocal Rank Fusion (no libraries)

def rrf(rankings, k=60):
    scores = {}
    for ranking in rankings:
        for rank, doc in enumerate(ranking, start=1):
            scores[doc] = scores.get(doc, 0) + 1 / (k + rank)
    return sorted(scores.items(), key=lambda x: -x[1])

bm25  = ["FD_rules_p4", "Savings_p2", "FD_faq_p1"]    # keyword ranking
dense = ["FD_faq_p1", "FD_rules_p4", "Loan_p9"]       # vector ranking
print(rrf([bm25, dense]))     # FD_rules_p4 and FD_faq_p1 come out on top

Common mistakes

  • Different embedding models for documents and queries.
  • Chunks with no metadata: no filters, no citations, no access control.
  • Checking permissions after generation instead of at retrieval.
  • Shipping without a golden test set, so you cannot tell if a change helped.
  • Stuffing 20 chunks into the prompt: higher cost, and the answer often gets worse.

Created by Ajeetkumar ยท EasyLearnSpace