TCS Gen AI interview questions & answers

Sure. It is a policy assistant which answers employee questions from internal company documents. The flow has five steps. First, ingestion — we read PDFs, Word files and Markdown, and clean the extra noise. Second, chunking and embedding — we split documents into small chunks and convert each chunk into a vector using an embedding model like text-embedding-3-small or bge-large. Third, we store these vectors in a vector database like Pinecone, Qdrant or FAISS, along with metadata like file name, page number and access permission. Fourth, retrieval — the user question is also converted into a vector, we do a top-K similarity search, and then a reranker like Cohere Rerank picks the best 3 to 5 chunks. Fifth, generation — we put those chunks into a structured prompt and send it to an LLM like GPT-4o or Claude, and the answer comes with citations.

See, a normal LLM does not know our private company data, so it will hallucinate when we ask about internal policies. Compared to fine-tuning, RAG has three benefits. One, knowledge updates instantly — we just upload a new document, no GPU retraining needed. Two, we can show citations, because we know exactly which chunk the answer came from. Fine-tuned weights cannot tell you the source. Three, access control — with RAG we can filter chunks based on user role. Fine-tuning cannot do user-level security.

There are four main ones. Fixed-size chunking, like 500 tokens with 50 tokens overlap — simple, but it can cut a sentence in the middle. Recursive chunking — it splits first by paragraph, then by line, then by space, so paragraphs stay complete. Semantic chunking — it checks the embedding similarity between sentences, and when the topic changes, it makes a new chunk. And parent-child chunking — we search using small child chunks of 100–150 tokens for accuracy, but we send the bigger parent block, around 1000 tokens, to the LLM for full context. In my project, recursive plus parent-child worked best.

Precision at K means — out of the top K chunks we retrieved, how many are actually relevant. So if K is 5 and 3 are relevant, precision is 3 by 5. Recall at K means — out of all the relevant chunks that exist in the database, how many we managed to get in the top K. MRR, Mean Reciprocal Rank, checks where the first correct chunk appears. If it comes at position 1, score is 1. At position 2, score is half. We take the average over many test questions. Higher MRR means the right answer comes on top.

System prompt sets the overall behaviour — role, tone, rules and safety limits. For example, "You are an internal support assistant. Only answer from the given context." It has higher priority. User prompt is the actual question the user asks in that turn, like "What is our parental leave policy?"

Temperature controls randomness. Low temperature, like 0 to 0.2, gives factual and consistent answers — good for RAG and code. High temperature, 0.7 to 1, gives more creative answers. Top-K means the model picks only from the top K most likely next words. Top-P, also called nucleus sampling, picks from the smallest group of words whose total probability reaches P, like 90 percent.

Top-K is fixed — always K options. Top-P is adaptive. When the model is very sure, the pool becomes small automatically. When many words are possible, like in creative writing, the pool becomes bigger. So for factual extraction or code we use low P like 0.1 to 0.3, and for brainstorming or copywriting we use higher P like 0.8 to 0.95.

Four things. First, grounding — in the system prompt we clearly say, "Answer only from the context. If the answer is not there, say you don't have enough information." Second, a verification step — a small second LLM checks whether every statement in the answer is supported by the context. Third, guardrails and structured output — using Pydantic parsing or NeMo Guardrails to reject ungrounded answers. Fourth, keep temperature at 0.

Normal Gen AI is single step — you give a prompt, you get one output, like drafting an email. Agentic AI can plan multiple steps, choose tools, interact with systems, and correct itself to reach a bigger goal, without a human guiding every step.

I will use the ReAct pattern — Reasoning plus Acting. Step one, Thought — the agent thinks what is needed and which tool to use. Step two, Action — it gives a structured tool call, like calling a REST API or running a SQL query. Step three, Observation — the tool runs and the result comes back into the context. Step four, it checks the result and either takes the next action or gives the final answer.

Three safety checks. One, max iteration limit, like max 5 steps. Two, action tracking — if the agent calls the same tool with the same inputs twice in a row, we stop it. Three, timeout and fallback — if it takes too long, we hand over to a human or show a proper error message.

In multithreading, many threads run inside one process and share memory. But because of the GIL, they cannot run CPU-heavy code in parallel. So threading is good for I/O tasks, like API calls or vector DB queries. Multiprocessing creates separate processes, each with its own memory and own Python interpreter, so it bypasses the GIL and uses multiple CPU cores. Good for CPU-heavy work like tokenization, chunking or local model inference.

GIL is Global Interpreter Lock. It is a lock in CPython that allows only one thread to run Python code at a time. It keeps memory safe for reference counting, but it stops threads from using multiple cores for heavy computation.

First, I will add end-to-end tracing using LangSmith, Arize Phoenix or OpenTelemetry. Total time is embedding time plus vector search plus reranking plus time to first token plus generation time. Tracing shows which step is slow. If vector search is slow, I will use an approximate index like HNSW instead of exact search, or cache frequent embeddings. If generation is slow, I will enable streaming, use a smaller or quantized model, or use prompt caching.

Three groups. System metrics — latency, time to first token, error rate, GPU usage, tokens per second. Quality metrics — context relevance, groundedness or faithfulness, and user feedback like thumbs up, thumbs down. Cost and operations — daily token usage, cost per query, and how often guardrails block things like prompt injection.

Transformer came from the paper "Attention Is All You Need." It uses self-attention, where every token looks at every other token in the sentence at the same time. RNNs read word by word, one after another, so training is slow. Transformers process the full sequence in parallel on GPUs. Also, RNNs forget information in long text because of vanishing gradients. In transformers, any two tokens are directly connected through attention, however far apart they are.

Decoder-only models, like GPT-4 and Llama, can only look at previous tokens while generating. They are best for text generation, chat, reasoning and code. Encoder-decoder models, like T5 and BART, first read the full input in both directions with the encoder, then the decoder generates output while looking at that encoded input. They are good for translation and summarization.

Hallucination is when the model gives an answer that sounds correct but is actually wrong or made up. It happens because the model just predicts the next likely word — it has no built-in way to check truth. Also because of missing context, noisy training data, or questions outside what it learned. High temperature, high Top-P and long unstructured prompts increase hallucination.

Main metric is faithfulness or groundedness — what percent of claims in the answer can be found in the retrieved context. We also check context recall and context precision — did we retrieve the right text, and did we avoid noise. For tools, we use RAGAS or DeepEval. They use LLM-as-a-judge and NLI models to score hallucination before going live.

We should never depend on the prompt alone for security. During ingestion, every chunk gets security tags like role, department and access level. During search, we filter the vector query based on the logged-in user's rights. Public data is open for everyone, but restricted chunks come back only if the user's token matches the tags.

Because the prompt can be bypassed by prompt injection. If the restricted chunk never reaches the LLM, it cannot leak. So filtering must happen before retrieval.

FAISS is Facebook AI Similarity Search — an open-source library for fast similarity search on vectors, with GPU support and indexes like IVF-PQ and HNSW. PEFT is Parameter-Efficient Fine-Tuning. Methods like LoRA and QLoRA freeze the base model and train only small adapter layers, so we need much less GPU memory and compute.

Four things. One, use a serving engine like vLLM or TGI, which use PagedAttention and continuous batching. Two, quantization — load the model in 4-bit or 8-bit using AWQ or GPTQ to save memory. Three, autoscaling — multiple replicas behind a load balancer, scaling based on GPU usage and queue length. Four, caching and speculative decoding — semantic caching in Redis for repeated questions, and a small draft model to speed up generation.

A decorator is a function that wraps another function to add extra behaviour without changing its code, like @app.get in FastAPI or a timing decorator. A generator uses yield instead of return, so it gives values one by one and remembers its state. In LLM streaming, tokens come chunk by chunk, so the backend uses a generator or async generator to send them to the frontend as server-sent events, without holding the full response in memory.

First, stream the data — use generators, ijson, or pandas with chunksize, so we process batch by batch, not all at once. Second, use memory-efficient tools like numpy, Polars or PyArrow instead of normal Python objects. Third, if one machine is not enough, use distributed processing like PySpark or Dask.

chunk_size — the size limit. chunk_overlap — how much text repeats between chunks so context is not lost. separators — the order of splitting, like paragraph, then line, then space. And length_function — whether we count by characters or by tokens.

We give the LLM a list of tools with their JSON schemas. When the user question needs outside data or an action — like checking order status or querying a database — the model returns structured JSON arguments for that tool instead of plain text. It triggers when the model understands it cannot answer from its own knowledge.

MCP is Model Context Protocol. It is an open standard to connect AI applications with tools, databases and data sources in one common way. Earlier, for every tool and every model provider, we had to write a custom integration. MCP gives a standard client-server protocol for tool discovery, resources and prompts — so build once, use anywhere.

Okay. For the server, I will write an async generator that loops over some tokens, wraps each one in a JSON payload, adds the "data:" prefix, and yields it as bytes with a small sleep to simulate network delay. At the end, it yields "data: DONE". To test properly, I will also cut each message into two halves on purpose, so the client gets broken pieces like a real network. For the client, I will note the start time, read chunks in a loop, keep a buffer, and only process when I get a full line. For each "data:" line, I parse the JSON and print the content. When the first token comes, I record time to first token. At the end, tokens per second is total tokens divided by total time.

Hindi and emojis take multiple bytes in UTF-8. If I directly decode a broken byte piece, Python throws UnicodeDecodeError. So I use the codecs incremental decoder — it holds the incomplete bytes and waits for the next chunk to complete the character. And the line buffer makes sure I parse JSON only after a full newline arrives.

LLM streams usually cannot resume like a file download. So I catch the exception, keep whatever text was already received, and show it to the user with a note like "Response incomplete due to network issue."

Using asyncio.wait_for, with two separate timeouts. One for the first token — for example, if nothing comes in 2 seconds, fail. Second is an idle timeout between chunks — if there is a gap of more than 5 seconds, we assume the LLM is stuck and stop.

Then the server keeps generating to a dead connection and we waste tokens and money. In FastAPI, I check request.is_disconnected. Once true, I cancel the task and also cancel the upstream LLM call so generation stops.

I will create an asyncio Semaphore of 5, so only 5 calls run at the same time. Each prompt goes into a retry function. If the call fails with a rate limit error, I wait and retry, up to 4 attempts. The wait doubles each time — 1 second, 2, 4 — that is exponential backoff. And I add full jitter, meaning the actual wait is a random value between zero and that delay. Finally, I run all tasks with asyncio.gather with return_exceptions true, so one failure doesn't crash the whole batch, and I mark failed prompts in the result.

Without jitter, if 50 workers get 429 at the same time, all of them wait exactly 2 seconds and retry together. That creates another spike and crashes the API again. This is called the thundering herd problem. Random jitter spreads the retries over time, so the load becomes smooth.

Retry for temporary errors — 429, 502, 503, 504 and network drops. Don't retry for client errors — 400 bad request, 401 unauthorized, 403 forbidden, 422 invalid input. Retrying those will give the same error every time.

No, they are different. Concurrency limit, like our semaphore of 5, controls how many requests run at the same moment. Rate limit controls how many requests or tokens we send in a time window, like 60 requests per minute or 40,000 tokens per minute. Even with 5 concurrent calls, if they finish fast and keep going, we can still cross the per-minute limit. So in production we need a token bucket or leaky bucket rate limiter along with the semaphore.