Deloitte: GenAI Engineer Reports (2025)

• RAG: retrieve relevant chunks from a knowledge base and put them in the prompt, so the LLM answers from your data with citations. The flow is fixed: retrieve → augment → generate.
• Agent: the LLM runs a loop: think → pick a tool → observe the result → repeat, until the goal is met. Retrieval can be one of its tools, alongside APIs, calculators, SQL and so on.

• Choose RAG for "answer questions from our policies".
• Choose an agent for "check the order status, apply the refund rule, email the customer".
• Many production systems are agentic RAG: an agent that decides when and where to retrieve.

• Shared state object: in LangGraph, a typed state (messages, plan, findings) is passed between nodes and persisted by a checkpointer, so every agent reads and writes the same state.
• Blackboard / shared store: agents write findings to a shared database, cache or vector store that others query.
• Message passing: a supervisor routes messages and each agent receives only what it needs, which saves tokens.
• Design concerns:
• Scope memory per user or session.
• Summarise to control context size.
• Prevent conflicting writes (one owner per field).
• Never put secrets in shared memory.

Answer (example: an expense-claim agent):

1. Goal and scope: validate claims against policy and approve small ones automatically.
2. Tools: - read_receipt (OCR or a vision LLM) - get_policy (RAG) - get_employee_grade (HR API) - submit_decision
3. Flow (LangGraph): extract → validate → decide. If the amount exceeds the limit or confidence is low, route to human approval.
4. Guardrails: typed tool arguments, read-only permissions except for submit_decision, a maximum of 8 steps, and an audit log.
5. Evaluation: 200 historical claims as a test set; measure decision accuracy, false approvals and time saved.
6. Rollout: run in shadow mode first, then auto-approve claims under ₹2,000.

• Check MTEB leaderboard scores for retrieval, then test on your own documents: build 50 to 100 question→document pairs and measure recall@5.
• Consider:
• Language support (multilingual for Hindi or other Indian languages)
• Domain (legal or medical text)
• Dimension size (cost and memory)
• Maximum input tokens
• Hosted API vs self-hosted (privacy)
• Examples: OpenAI text-embedding-3-small/large, Cohere embed-v3, BGE / E5 / GTE (open models), and all-MiniLM for quick prototypes.
• Re-embed everything if you switch models, because vectors from different models are not comparable.

• Index design: ANN (HNSW or IVF-PQ), with the index sharded or partitioned by business unit or tenant.
• Pre-filter with metadata (department, year, document type) so search runs only over a small slice.
• Two-stage retrieval: a fast hybrid search returns the top 100, then a cross-encoder re-ranks them to the top 5.
• Hierarchical retrieval: search document summaries first, then the chunks inside the best documents.
• Operations: incremental indexing, caching of frequent queries, and monitoring of recall and latency. Use a managed service (Azure AI Search, Pinecone, Elasticsearch) for scale.

from collections import Counter
import re

def word_freq(text):
    words = re.findall(r"[a-z0-9']+", text.lower())
    return Counter(words).most_common()

print(word_freq("RAG or agents? RAG first."))

• Counter is O(n). Mention normalising case and punctuation, and most_common(k) for the top k.

Answer (keep it plain):

• "It only answers from your approved documents, and every answer shows the source so your team can check it."
• "If it isn't sure, it says so and hands over to a person. It doesn't guess."
• "We tested it on 300 real questions from your team before launch. It was right 92% of the time, and every miss was logged and fixed."
• "Your data stays in your cloud tenant, access follows your existing permissions, and we monitor it every week."
• Finish with the business value: hours saved, faster responses, and a clear next step such as a pilot with one team.