GenAI in Banking: Use Cases, Architecture & Guardrails

Generative AI is moving from pilots to production in banks: assistants for employees, document processing, and carefully controlled customer chat. This lesson covers the high-value use cases, a reference architecture, and the guardrails that make GenAI acceptable to risk and compliance teams.

High-value use cases (most to least common)

1. Employee knowledge assistants: RAG over policies, product manuals and SOPs, with citations.
2. Document intelligence: extract fields from KYC documents, bank statements, invoices and trade documents into structured JSON.
3. Contact-centre copilots: live call summaries, suggested replies, after-call notes and complaint classification.
4. Developer productivity: code assistants, test generation, legacy code explanation.
5. Risk and compliance support: AML case summaries, regulatory change analysis, policy gap checks.
6. Customer-facing chat: account FAQs and guided journeys. This is the most guarded tier: no free-form financial advice, and strict escalation.

Reference architecture

User (employee / customer) → Auth (SSO, entitlements) → GenAI gateway
   → Guardrails in (PII masking, injection & jailbreak filters, topic policy)
   → Orchestrator (RAG / tools / agent with allow-listed actions)
        → Retrieval: hybrid search over approved docs, filtered by user entitlements
        → Tools: read-only APIs (balance, transactions); write actions need confirmation
        → LLM: private endpoint / self-hosted model, pinned version
   → Guardrails out (grounding check, citation required, policy and tone filters)
   → Response + citations → Logging (prompt, sources, output, user feedback) → Monitoring

Guardrails that matter in finance

• Data protection: PII redaction before the LLM, no training on customer data without consent, data residency, encryption, short retention.
• Grounding: answers only from retrieved, approved sources. Respond "I don't know" if retrieval confidence is low, and require citations.
• Numbers: never let the LLM do arithmetic for balances, EMIs or interest. Call deterministic calculators or APIs instead.
• Actions: least-privilege tools, human confirmation for anything that moves money or changes records, full audit trail.
• Security: prompt-injection defences (treat documents as data, not instructions), output encoding, rate limits.
• Conduct: no personalised investment advice unless the system is licensed and approved for it, with clear disclaimers.

Evaluating a banking GenAI app

Build a tiny compliant RAG answerer

SYSTEM = """You are the bank's internal policy assistant.
Answer ONLY from the provided context and cite sources as [doc:section].
If the context does not contain the answer, reply: "I can't find this in approved policy documents."
Never give investment advice. Never perform calculations; say a calculator tool is required."""
def answer(question, user):
    docs = search(question, filters={"business_unit": user.bu, "status": "approved"}, k=5)
    if not docs or docs[0].score < 0.35:
        return "I can't find this in approved policy documents."
    ctx = "\n\n".join(f"[{d.id}:{d.section}] {d.text}" for d in docs)
    return llm(system=SYSTEM, user=f"Context:\n{ctx}\n\nQuestion: {question}", temperature=0)

Check yourself