Tiger Analytics: GenAI Engineer

• Temperature scales the logits before softmax. Low values (0 to 0.3) make output deterministic and focused; high values (above 0.8) make it more creative and random.
• Top-k: sample only from the k most likely tokens, for example k=40. It is a fixed-size cut regardless of how confident the model is.
• Top-p (nucleus): sample from the smallest set of tokens whose probabilities add up to p, for example 0.9. The set adapts: small when the model is confident, large when it isn't.
• In practice:
• Extraction, SQL or JSON: temperature 0.
• Chat: temperature around 0.7 with top-p 0.9.
• Avoid tuning top-k and top-p heavily at the same time.

1. Define the task and success metric with the business, and collect 30 to 50 real examples.
2. Structure the prompt: - A role - The task - The context in delimiters - Rules (cite sources, never invent numbers) - The output format, as a JSON schema - 2 or 3 few-shot examples
3. Iterate against an evaluation set, not by eye. Version prompts in git.
4. Add guardrails: PII masking, refusal rules, a check on maximum length.
5. Optimise cost: a shorter system prompt, prompt caching, a smaller model where it is good enough.

• Ingestion:
• Connectors (SharePoint, S3, databases)
• Parsing (PDF, tables, OCR)
• Cleaning and deduplication
• Chunking with metadata
• Embedding
• Indexing
• A scheduled or incremental refresh
• Retrieval: query understanding or rewriting → hybrid search → filters → re-ranker.
• Generation: prompt assembly, the LLM, citations, and output validation.
• Platform: access control, caching, monitoring, an evaluation harness, and a feedback loop.

• Risks:
• Hallucination
• Bias and unfair outcomes
• Toxic content
• Privacy leaks (PII in training data or prompts)
• Copyright
• Over-reliance by users
• Prompt injection
• Testing for bias:
• Build counterfactual pairs: the same input with only a protected attribute changed (name, gender, region), then compare outputs and scores.
• Measure differences in refusal or approval rates and in sentiment.
• Use benchmarks such as BBQ or StereoSet for a general check.
• Mitigations:
• Remove protected attributes from decision prompts.
• Add policy prompts and output filters.
• Keep a human in the loop for consequential decisions.
• Document everything in a model card, and audit regularly.

• Ingestion: audio → speech-to-text (Whisper or Azure Speech with speaker diarisation) → transcript stored with call metadata.
• Processing: a queue (Kafka or Service Bus) feeding workers. Long calls use map-reduce: summarise chunks, then combine. A JSON schema captures the summary, intent, sentiment, action items and compliance flags.
• Model choice: a small, cheap model for most calls; a large model for escalations. Use the batch API for overnight jobs, which costs about 50% less.
• Quality: a sampled human review, LLM-judge faithfulness checks, and PII redaction before storage.
• Output: written to a warehouse and dashboards; searchable through RAG for supervisors.

• Recommendation pattern: start with an API plus RAG. Fine-tune (LoRA) only when you need a specific style or format, low latency or cost at scale, or must run on-premises. Prove it with an evaluation set first.

• Latency:
• Streaming
• Response and semantic caching
• Smaller models for routing
• Parallel tool and retrieval calls
• Shorter prompts
• Prompt caching
• Region close to users
• Availability:
• Multiple deployments or providers with fallback
• Retries with exponential backoff
• Timeouts and circuit breakers
• Rate-limit and quota management
• Autoscaling stateless services
• Health checks
• Monitor p50/p95 latency, error rate, tokens and cost, with alerts against SLOs.

import pandas as pd

df = pd.read_csv("tickets.csv", parse_dates=["created_at"])
recent = df[(df.created_at >= "2025-01-01") & (df.status != "spam")]
pivot = (recent.assign(month=recent.created_at.dt.to_period("M"))
               .pivot_table(index="month", columns="category",
                            values="ticket_id", aggfunc="count", fill_value=0))
long = pivot.reset_index().melt(id_vars="month", var_name="category", value_name="tickets")

• Explain boolean masks, pivot_table vs groupby().unstack(), and melt to go back from wide to long format.

import re, html, unicodedata

def clean(t: str) -> str:
    t = html.unescape(t)
    t = unicodedata.normalize("NFKC", t)
    t = re.sub(r"<[^>]+>", " ", t)              # HTML tags
    t = re.sub(r"https?://\S+", "<URL>", t)      # URLs
    t = re.sub(r"\S+@\S+", "<EMAIL>", t)         # PII-ish
    t = re.sub(r"\s+", " ", t).strip()
    return t

• For Transformers, don't stem, remove stop words or lowercase blindly, because the tokenizer and model rely on natural text. Do deduplicate, drop boilerplate, fix encoding and filter by language.

• Baseline first: average handling time, cost per ticket, error rate, all measured before launch.
• After launch: A/B or phased rollout, then compare. Example: "Agents using the assistant cut handling time from 9.5 to 6.8 minutes (−28%) across 40k tickets a month, about ₹38 lakh a year saved, against ₹6 lakh a year of LLM and infrastructure cost."
• Include quality (CSAT and accuracy) and adoption (weekly active users), not just cost.