Fractal: Sr Data Scientist, GenAI (2025, Offer)

• Batch GD uses the whole dataset per step. SGD / mini-batch uses small batches, which is noisier but faster and generalises well.
• Momentum: keeps a moving average of gradients (v = βv + g) to speed through ravines and damp oscillation.
• RMSProp: scales the learning rate per parameter by a running average of squared gradients, which suits non-stationary problems.
• Adam: Momentum + RMSProp (first and second moments) with bias correction; the default for deep learning. AdamW decouples weight decay and is the standard for Transformers.
• Learning-rate schedule: warm-up then cosine decay for LLM training or fine-tuning.

• Preprocessing:
• Handle missing values (impute plus a missing-indicator flag) and outliers.
• Encode categoricals (one-hot, target encoding inside the CV folds).
• Scale numeric features for distance-based or linear models.
• Engineer features from domain knowledge.
• Avoid leakage: fit transformers only on the training fold, and use Pipeline + ColumnTransformer. For time-based data, use a time-based split.
• Evaluation: stratified k-fold, a metric that matches the business (F1 or PR-AUC for imbalance, RMSE or MAPE for forecasting), a baseline to compare against, and error analysis by segment.

• What they are: BERT-style encoders fine-tuned (often in a Siamese setup with contrastive or MultipleNegativesRanking loss) so that one vector represents a whole sentence, with cosine similarity reflecting meaning. Examples: all-MiniLM-L6-v2, bge, e5.
• Uses:
• Semantic search and RAG retrieval
• Clustering and topic discovery
• Deduplication
• Semantic textual similarity
• Zero-shot classification by similarity to label descriptions
• Bi-encoder vs cross-encoder: a bi-encoder embeds the query and document separately (fast, indexable); a cross-encoder reads the pair together (slower, more accurate), so it is used for re-ranking.

• Word2Vec:
• Skip-gram predicts the context words from the centre word; CBOW predicts the centre word from its context.
• It is trained with negative sampling.
• Words that appear in similar contexts end up close together.
• GloVe: factorises global co-occurrence statistics. FastText: adds sub-word n-grams, so it handles rare and misspelt words.
• Limitation: one vector per word, so "bank" (river) = "bank" (money).
• Contextual (ELMo, BERT, GPT): the vector for a token depends on the whole sentence, produced by deep bidirectional or causal Transformers, so different senses get different vectors.

• Why: self-attention is permutation-invariant, so without position information, "dog bites man" = "man bites dog".
• Sinusoidal (original Transformer): fixed sin and cos at different frequencies, added to the embeddings. It can extrapolate somewhat.
• Learned absolute: a trainable vector per position (BERT, GPT-2), limited to the maximum length seen in training.
• Relative / rotary (RoPE): rotates Q and K by position so attention depends on relative distance. Used in Llama, Mistral and Qwen, and extendable with scaling (NTK, YaRN) for long context.
• ALiBi: adds a distance-based bias to the attention scores.

• Ingestion: document parsing that keeps layout and tables → semantic or heading-based chunks with metadata → embeddings → vector DB, refreshed incrementally.
• Retrieval:
• Query rewriting or multi-query
• Hybrid BM25 + dense search
• Metadata filters for access control
• Cross-encoder re-rank to the top 5
• Generation: a grounded prompt with citations and an "insufficient context" fallback. JSON output if a downstream system consumes it.
• Evaluation and operations:
• A RAGAS evaluation set
• Tracing and logging
• Feedback loop
• Cost and latency dashboards
• Guardrails for PII

• RAGAS evaluates RAG systems, mostly using an LLM as the judge:
• Faithfulness: are the claims supported by the context?
• Answer relevancy
• Context precision / recall: did retrieval bring the right chunks?
• Answer correctness against ground truth.
• It can also generate synthetic test sets.
• DSPy programs LLM pipelines instead of hand-writing prompts:
• You declare signatures (input → output) and modules such as ChainOfThought or Retrieve.
• An optimiser (BootstrapFewShot, MIPRO) automatically tunes prompts and few-shot examples against your metric.
• Together: DSPy optimises the pipeline, and RAGAS-style metrics act as the objective and the regression tests.

• Public benchmarks for shortlisting:
• MMLU and GPQA (knowledge)
• GSM8K and MATH (reasoning)
• HumanEval (code)
• MTEB (embeddings)
• LMSYS Arena (preference)
• Decide on your own benchmark:
• 200+ in-domain examples
• Fixed prompts
• Temperature 0
• Several runs
• Metrics: accuracy / F1, faithfulness, latency, cost per 1k requests
• Control variables: same context and same prompt format; watch for contamination. Report confidence intervals, and do a blind human review of a sample.

words = ["rag", "agent", "llm", "vector", "embedding"]
lengths = {w: len(w) for w in words if len(w) > 3}
squares = [n * n for n in range(1, 6)]      # list comprehension
print(lengths, squares)

• Comprehensions are concise and faster than equivalent for loops with append. Use a generator expression ( ) for large data to save memory.

Answer (an honest, specific shape):

• "I do my best work in teams that ship to real users and look at metrics. I like owning a problem end to end, from data to deployment."
• "I value candid feedback and knowledge sharing. In my current role I run a short weekly demo of what we learned."
• "I'm comfortable with ambiguity: I break a vague client ask into a small proof of concept within a week, then iterate."
• Fractal promotes a client-first, learning culture, so connect your answer to client impact and continuous learning.