Accenture: GenAI & AI/ML Engineer (2025)

• Input: tokens are turned into embeddings, and positional encodings are added because attention alone has no sense of order.
• Self-attention: each token creates a Query, Key and Value. Attention weights are softmax(QKᵀ/√d) and are applied to V, so every token can look at every other token in one step.
• Multi-head: several attention heads run in parallel, each learning a different relationship (syntax, coreference, and so on). Their outputs are concatenated and projected.
• Each block: attention → add & layer-norm → feed-forward network → add & layer-norm. Encoders (BERT) see the whole sentence; decoders (GPT) use a causal mask.
• Why it beats RNNs: there is no step-by-step recurrence, so training is fully parallel on GPUs. Long-range dependencies are one hop away instead of N hops, which avoids vanishing gradients. It also scales cleanly with data and compute.

• Building blocks:
• Models: chat-model wrappers such as ChatOpenAI or Azure OpenAI.
• Prompts: ChatPromptTemplate with variables.
• Output parsers / structured output: Pydantic schemas.
• Retrievers: a vector store via .as_retriever().
• Tools: decorated Python functions.
• LCEL: chains such as prompt | llm | parser with invoke, batch and stream.
• A typical project answer: "I built a RAG assistant over policy PDFs. A loader feeds a recursive splitter (800 tokens, 100 overlap), then embeddings go into FAISS or Chroma. A retriever with k=4 feeds a prompt that tells the model to answer only from the context and cite sources. Streaming responses are served through FastAPI. For agent steps I moved to LangGraph, because it gives explicit state, loops and human approval."
• Always say why you chose it (speed of integration, swapping providers) and what you would change, such as fewer abstraction layers for latency-critical paths.

• Packaging: wrap the model or chain in a FastAPI service with /health and /predict (or /chat) endpoints and Pydantic request/response models, then build a Docker image.
• Hosting: use a managed endpoint (Azure ML, SageMaker or Vertex AI), or Kubernetes / AKS for scale, behind an API gateway that handles authentication and rate limits.
• CI/CD: GitHub Actions or Azure DevOps runs tests and an evaluation suite (a golden question set for LLM apps), builds the image, and deploys blue-green or canary.
• Operations:
• Structured logs with request ID, latency and token count.
• Tracing with LangSmith or OpenTelemetry.
• Dashboards for p95 latency, error rate and cost.
• Alerts on drift or a drop in quality.
• Secrets live in Key Vault or Secrets Manager, never in code.

• The loop: the LLM is the reasoning engine. It reads the goal, chooses a tool, reads the tool result and repeats until it is done (the ReAct pattern).
• Tools: typed functions such as search_kb(query), get_order(id) or create_ticket(...). Each has a clear description and a JSON schema, and only what the role needs is exposed.
• Memory:
• Short-term: the conversation state in the graph or checkpointer.
• Long-term: a vector store or database of user facts and past resolutions.
• Summary memory: compresses long chats into a summary.
• Safety: a maximum number of iterations, timeouts, human approval for write actions, and validation of tool arguments.
• Example: "A support agent in LangGraph. It classifies the intent, retrieves policy, calls the order API, drafts a reply, and escalates to a human when confidence is low or a refund exceeds ₹5,000."

• Retrieval quality:
• Use semantic or structure-aware chunking with metadata.
• Use hybrid search (BM25 + vectors) and a cross-encoder re-ranker.
• Filter on metadata (department, date).
• Grounding:
• Instruct the model to answer only from the context, and to say "I don't know" when it can't.
• Require citations.
• Run a faithfulness check (RAGAS or an LLM judge) on samples.
• Latency:
• Cache embeddings and answers (including a semantic cache).
• Use ANN indexes such as HNSW.
• Retrieve a small top-k after re-ranking.
• Stream tokens.
• Route easy questions to smaller models.
• Run async parallel retrieval.
• Scale: shard or partition the index by tenant, update it incrementally, and monitor recall@k and p95 latency.

• Traditional: a single pass of input → (retrieve) → generate. The flow is fixed by the developer and easy to test, cheap and predictable.
• Agentic: the model plans and chooses actions. It calls tools, loops, reflects and may coordinate several agents. It handles open-ended, multi-step tasks.
• Trade-offs: agents cost more and are slower and less predictable, so they need guardrails, step limits, tracing and evaluation of whole trajectories.
• Rule of thumb: use a deterministic workflow when the steps are known, and an agent only when the path truly depends on intermediate results.

• What it is: MCP is an open protocol for connecting LLM applications to tools and data in a standard way. A server exposes tools, resources and prompts, and any MCP-capable client (an IDE, a chat app or an agent) can use them.
• Why it helps:
• Write an integration once and reuse it across many models and apps, instead of writing custom function-calling glue for each one.
• It has clear capability discovery and keeps permissions and transport separate (stdio or HTTP).
• Enterprise view: put authentication, least-privilege scopes and audit logs on the server side, and treat tool output as untrusted input.

• Separate instructions from data: retrieved documents and tool outputs are untrusted. Wrap them in delimiters and tell the model never to follow instructions found inside them.
• Least privilege:
• Tools get only the permissions they need.
• Document-level access control is applied before retrieval, so users only retrieve what they are allowed to see.
• Filters:
• On input, detect jailbreaks and injections.
• On output, mask PII and secrets and enforce policy (for example Azure AI Content Safety or Guardrails).
• Human approval for sensitive actions (payments, emails, deletes).
• Testing: red-team prompt sets in CI, plus logging and monitoring for anomalies.

• Use the provider's structured output or JSON schema mode, or function calling, with a Pydantic model.
• Validate the response. If validation fails, send the validation error back for a single retry. If it still fails, fall back to a safe default.
• Keep the schema small and give field descriptions and enums, with temperature at 0 to 0.2.
• In LangChain use llm.with_structured_output(Model); parsers such as PydanticOutputParser work with any model.

Answer (STAR structure):

• Situation: "Our RAG bot's answer accuracy dropped from about 85% to 60% after a document refresh."
• Task: find the root cause within a day.
• Action: "I checked traces in LangSmith and saw that the new PDFs were scanned images, so the extracted text was empty and retrieval was returning irrelevant chunks. I added OCR, a check that rejects documents with too little text, and an alert on mean retrieval score."
• Result: accuracy came back to 86%, and the evaluation set now runs on every ingestion.
• Interviewers want to see observability, root-cause analysis and prevention, not just the fix.