EY Gen AI interview questions & answers

First, I will find where the failure is — retrieval or generation. I will check the logs for the failing queries — the similarity scores and the exact chunks that went to the LLM. If the correct chunks are not in the top-K, it is a retrieval problem, and I will fix chunking, embeddings or search. If the correct chunks are there but the model still adds its own facts, it is a generation problem. For that, immediately I will tighten the system prompt — "Answer only from the given context. If the answer is not there, say I do not have this information." Then I will add a fast groundedness check — a small classifier or small model that compares the answer with the retrieved chunks before showing it to the user. And if the groundedness score is below a threshold, like 0.85, we don't show that answer. We show a safe fallback message like "I cannot find this in the official documents."

Yes sir. For a client-facing solution, it is much better to give a safe fallback than to give wrong information confidently.

Four things. One, metadata pre-filtering — we don't search all 5 million vectors. We first filter by things like tenant, department, document type or date range, so the search space becomes small. Two, tune the ANN index — for HNSW, parameters like M and efSearch — so we lose a very small amount of recall but get a big speed gain. Three, two-stage retrieval. Stage one, a cheap keyword search like BM25 to bring 5 million down to the top 500 candidates. Stage two, dense vector search and cross-encoder reranking only on those 500. Four, move to a scalable, distributed vector database like Pinecone, Weaviate, Qdrant or sharded pgvector, and add a semantic cache in Redis, so repeated questions don't hit the vector DB or the LLM at all.

First, model routing. We should not use a big frontier model for every small task. Simple classification, extraction or summarization can go to smaller, cheaper models like GPT-4o-mini or Claude Haiku. Only complex reasoning goes to the big model. Second, reduce the prompt size — check the logs and make sure we are not sending full documents when three small chunks are enough. Third, caching — use the provider's prompt caching, and semantic caching on our side, so duplicate calls are removed. Fourth, batch processing — non-real-time jobs like nightly reports or summaries can go through the batch API, which is usually around 50 percent cheaper.

Okay sir. I will explain in four layers. First, ingestion. We chunk the PDFs and attach department metadata to every chunk — like department ID and access level. And for the weekly updates, we don't re-process all 3,000 documents. We do delta indexing — we compare a file hash, like SHA-256, and only re-embed the new or changed files.

That is the most critical part. We use pre-retrieval filtering. The user logs in, and their identity comes as a token, like a JWT, at the API gateway. From that we get their department. Then the vector search itself is filtered to only that department's chunks. We should never depend on the prompt to hide data — like telling the LLM "don't show Legal documents to HR." If the user doesn't have permission, that chunk should never even enter the prompt.

In generation, we pass only the permission-checked chunks to the LLM, and the prompt forces the model to give citations — document name and page number — so employees can verify the answer from the original PDF. And the fourth layer is evaluation and compliance. We run a test suite for every department, regularly — checking answer accuracy with something like RAGAS, and also trying cross-department questions to make sure nothing leaks. Any access bug is treated as a critical compliance issue.

Instead of storing the chunk twice, we store a list of allowed departments in the metadata — like departments equal to Legal and HR. Then in the search filter, instead of an exact match like department equals user department, we use an "in" condition — user department in the allowed departments list. So one copy of the data, and both teams can see it.

I will not use technical words like probability or neural networks. I will talk in terms of risk management. I will say — no system, not even a team of human experts, is 100 percent error-free. That's why even human specialists have review processes. In the same way, our focus is not to promise perfection, but to build guardrails around the system. When the question is unclear or the answer is not in the documents, the system will not guess — it will give a safe message or send it to a human. And for high-stakes decisions, there is always a human-in-the-loop.

I will first protect the core deliverables. Then I will discuss the trade-offs with stakeholders, document the change properly through change control, and re-plan the timeline transparently, so nobody is surprised later.

I will try to settle it with data, not opinion. We can do a small proof of concept for both approaches and compare the trade-offs — latency, accuracy and cost — and then decide based on the common project goal.

I follow research papers and major open-source releases, I build small prototypes to try new things hands-on, and I check standard benchmarks before trusting any new tool.