GenAI Course · Module 9 · Session 37

LangGraph & Agentic RAG

⏱ 14 min read🎯 Advanced🖼 2 diagrams🧪 2 live demos🐍 Python code

Simple RAG is a straight line: retrieve, then answer. But what if the retrieved chunks are useless? A smart system should notice, rewrite the question and try again. That needs loops and decisions, which is exactly what LangGraph is built for.

In this session

LangGraph in four words: state, nodes, edges, cycles
Agentic RAG: grade the documents, rewrite and retry
A runnable LangGraph example (real output)
Multi-modal RAG for images, charts and tables
RAG with memory for follow-up questions

01LangGraph basics

Concept	Meaning	In our example
State	a shared dictionary every step can read and update	question, docs, tries, answer
Node	a Python function that does one job	retrieve, rewrite, generate
Edge	which node runs next	retrieve → grade
Conditional edge	choose the next node based on the state	docs found? → generate, else → rewrite
Cycle	an edge that goes back to an earlier node	rewrite → retrieve again

🗺️

Like a flowchart that can loop. A chain is a straight road. A LangGraph graph is a city map with junctions and roundabouts: at each junction, the state decides which road to take, and you can come back around if needed.

02Agentic RAG

Agentic RAG adds decisions around retrieval:

If the documents are not relevant, rewrite the question and retrieve again (a cycle). If the answer is not grounded, regenerate or refuse.

Here is a complete, runnable version without any API key: the “retriever” only understands the words “fixed deposit”, so the graph must rewrite “FD” and loop back:

agentic_rag_graph.py

from typing import TypedDict
from langgraph.graph import StateGraph, START, END

class State(TypedDict):
    question: str
    docs: list
    tries: int
    answer: str

def retrieve(s):
    found = ["FD_Policy p.4: early withdrawal costs 1%"] if "fixed deposit" in s["question"] else []
    return {"docs": found, "tries": s["tries"] + 1}

def grade(s):                                  # decide where to go next
    return "generate" if s["docs"] else ("rewrite" if s["tries"] < 2 else "give_up")

def rewrite(s):
    return {"question": s["question"].replace("FD", "fixed deposit")}

def generate(s):
    return {"answer": f"Answer based on: {s['docs'][0]}"}

def give_up(s):
    return {"answer": "Sorry, I could not find this in our documents."}

g = StateGraph(State)
for name, fn in [("retrieve", retrieve), ("rewrite", rewrite), ("generate", generate), ("give_up", give_up)]:
    g.add_node(name, fn)
g.add_edge(START, "retrieve")
g.add_conditional_edges("retrieve", grade)     # route by the grade() result
g.add_edge("rewrite", "retrieve")              # a cycle: try again with a better query
g.add_edge("generate", END)
g.add_edge("give_up", END)
app = g.compile()

out = app.invoke({"question": "FD penalty?", "docs": [], "tries": 0, "answer": ""})
print(out["tries"], "|", out["question"], "|", out["answer"])

2 | fixed deposit penalty? | Answer based on: FD_Policy p.4: early withdrawal costs 1%

Demo · What happened inside the graph

In a real system the grade step is an LLM call (“Is this chunk relevant to the question? yes/no”), and you always cap the number of loops so the graph cannot run forever.

03Multi-modal RAG

Many documents carry meaning in images: charts in annual reports, scanned tables, product photos. Two common approaches:

Most document RAG systems start with “summarise, then embed”.

At answer time, the original image can be passed to a vision-capable LLM along with the text chunks, so it can read the chart itself.

04RAG with memory

Users ask follow-ups: “What about senior citizens?”. Searching for that sentence alone finds nothing useful. The fix is to condense the follow-up into a standalone question using the chat history:

Demo · Condensing a follow-up question

memory_graph.py

from langgraph.checkpoint.memory import MemorySaver

app = g.compile(checkpointer=MemorySaver())          # g is the StateGraph from above
cfg = {"configurable": {"thread_id": "employee-42"}}  # one thread per conversation
app.invoke({"question": "FD penalty?", "docs": [], "tries": 0, "answer": ""}, cfg)
# later calls with the same thread_id continue from the saved state

Go deeper: the LangGraph complete notes cover reducers, checkpointers, time travel, human approval, Send, Command handoffs and a ReAct agent.

05Summary

LangGraph models workflows as state + nodes + edges, including cycles.
Agentic RAG grades retrieval, rewrites and retries, and checks the answer.
Multi-modal RAG usually summarises images into text, then embeds.
Memory condenses follow-ups into standalone questions; checkpointers save conversation state.

06Quick quiz

1. In LangGraph, what decides the next node based on the state?

2. Why does agentic RAG need cycles?

3. What must you always add to a loop?

4. “What about senior citizens?” should be…

5. A common first approach for charts in PDFs is…

07References

LangGraph documentation
Self-RAG (Asai et al., 2023)
Corrective RAG (CRAG) (Yan et al., 2024)

Created by Ajeetkumar