Conversational Chains & Tracing

Real apps work with many documents and long conversations. A model’s context window is limited, so we need patterns to combine documents, ways to remember what the user said earlier, and a way to see what is happening inside every chain. That is this session.

01Four ways to work with many documents

Suppose you must summarise 50 customer-call transcripts, or answer a question from 20 retrieved chunks. Here are the classic patterns:

Older LangChain versions had ready-made load_summarize_chain and load_qa_chain helpers for these. Today it is simpler to write them yourself with LCEL:

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

map_chain = ChatPromptTemplate.from_template("Summarise this support call in one line:\n{chunk}") | llm | StrOutputParser()
reduce_chain = ChatPromptTemplate.from_template("Combine these summaries into the top issues:\n{summaries}") | llm | StrOutputParser()

partials = map_chain.batch([{"chunk": c} for c in call_transcripts])     # map, in parallel
print(reduce_chain.invoke({"summaries": "\n".join(partials)}))         # reduce

Output from a test run with a fake model on langchain-core.

02Managing conversation memory

LLMs are stateless: each call knows nothing about earlier calls. “Memory” simply means sending the relevant past messages again with each new question. The art is choosing which messages to send.

The modern way: LangGraph persistence

In current LangChain, the old memory classes (like ConversationBufferMemory and RunnableWithMessageHistory) are deprecated. The recommended approach is a small LangGraph app with a checkpointer that saves each conversation thread. We will learn LangGraph properly in Session 37.

# pip install langgraph langchain-openai
from langgraph.graph import StateGraph, MessagesState, START, END
from langgraph.checkpoint.memory import MemorySaver
from langchain_core.messages import SystemMessage, trim_messages

def chatbot(state: MessagesState):
    history = trim_messages(state["messages"], max_tokens=2000, token_counter=llm, strategy="last")
    reply = llm.invoke([SystemMessage("You are a bank support assistant.")] + history)
    return {"messages": [reply]}

graph = StateGraph(MessagesState)
graph.add_node("chatbot", chatbot)
graph.add_edge(START, "chatbot")
graph.add_edge("chatbot", END)
app = graph.compile(checkpointer=MemorySaver())       # use a database checkpointer in production

cfg = {"configurable": {"thread_id": "customer-42"}}
app.invoke({"messages": [("user", "Hi, my name is Priya.")]}, cfg)
out = app.invoke({"messages": [("user", "What is my name?")]}, cfg)
print(out["messages"][-1].content, "|", len(out["messages"]), "messages in thread")

03Tracing with LangSmith

When a chain gives a bad answer, you need to see which step went wrong. LangSmith records every run as a tree: each prompt, model call and parser, with inputs, outputs, tokens, cost and time.

Turning it on needs only environment variables; your chain code does not change:

LANGSMITH_TRACING=true
LANGSMITH_API_KEY=your-key-here
LANGSMITH_PROJECT=bank-support-bot

What to use traces for:

• Debugging: see the exact filled prompt and the raw model output.
• Cost and latency: find the slow or expensive step.
• Datasets: save good and bad runs as test cases for evaluation (Session 36).
• Monitoring: watch error rates and user feedback in production.

Open-source alternatives include Langfuse, Arize Phoenix and OpenTelemetry-based tools.

04Summary

• Stuff (one prompt), Map-Reduce (parallel then combine), Refine (step by step) and Map-Rerank (best single source) handle many documents.
• LLMs are stateless; memory means re-sending chosen history: window, summary or token-trimmed.
• Use LangGraph with a checkpointer for conversation memory in modern LangChain.
• LangSmith traces every step for debugging, cost tracking and evaluation.

05Quick quiz

06References

• LangChain: short-term memory
• LangGraph persistence
• LangSmith documentation