Company Interview Questions · AI Labs & Startups

Cohere, Glean & Perplexity: ML Engineer

❓ 6 questions🏢 Cohere & Glean & Perplexity🆕 Recent reports

Cohere, Glean & Perplexity: ML Engineer Interview Formats (2025). Rounds, real reported questions and model answers, compiled from public candidate reports (sources at the end). Practice mode hides every answer: say your answer first, press Show answer, then rate yourself. Use 🔁 Review again to build your revision list and 🎲 Random question for a quick drill.

★ Must-know 145📝 Assignments
About this interview

Compiled from public candidate-thread summaries: Cohere MLE (system design, NumPy ML coding, research presentation, post-training expertise), Glean ML onsite in San Francisco (ML design, an AI assignment, coding) and a Perplexity MLE HR screen that unexpectedly included Transformer positional-encoding questions. The full thread details were behind logins, so the answers here target the reported formats. The questions are paraphrased, and the model answers are written by EasyLearnSpace.

Cohere Glean Perplexity
Role Machine Learning Engineer Machine Learning Engineer ML Engineer
Reported format System design · ML coding in NumPy · research presentation · post-training depth ML design · AI take-home / assignment · coding HR screen with technical questions (positional encoding)
Signal LLM training and post-training depth Enterprise search and RAG Transformer fundamentals even in early screens

01Rounds at a glance

02Questions and model answers

Q1
In NumPy, implement softmax cross-entropy with a numerically stable gradient.
import numpy as np
def softmax_xent(logits, y):            # logits (N, C), y (N,) int labels
    z = logits - logits.max(axis=1, keepdims=True)          # stability
    p = np.exp(z); p /= p.sum(axis=1, keepdims=True)
    N = len(y)
    loss = -np.log(p[np.arange(N), y] + 1e-12).mean()
    grad = p.copy(); grad[np.arange(N), y] -= 1; grad /= N  # dL/dlogits = p - onehot
    return loss, grad
  • Mention log-sum-exp, why subtracting the maximum doesn't change the softmax, and checking the gradient numerically.
Q2
Explain post-training: SFT → preference optimisation → RL. What can go wrong?
  • SFT: curated instruction–response data teaches format and helpfulness. Data quality beats quantity.
  • Preference optimisation: RLHF (reward model + PPO with KL) or DPO/IPO/KTO on chosen-vs-rejected pairs, aligning tone, safety and helpfulness.
  • RL with verifiable rewards: maths and code unit tests (GRPO-style) improve reasoning.
  • Failure modes:
  • Reward hacking.
  • Verbosity bias.
  • Over-refusal.
  • Mode collapse (lost diversity).
  • Catastrophic forgetting.
  • Benchmark contamination.
  • Mitigations: KL regularisation, length-controlled evaluation, diverse preference data, held-out human evaluations.
Q3
Perplexity screen: why do Transformers need positional encoding, and how does RoPE work?
  • Attention is permutation-invariant, so without position information token order is lost.
  • RoPE: rotate each 2-D pair of Q/K dimensions by an angle θ·position, so the dot product q_m · k_n depends on (m − n): relative position comes for free and extends to longer contexts (with NTK/YaRN scaling).
  • Alternatives: sinusoidal (fixed, added), learned absolute (bounded length), ALiBi (a linear bias by distance).
Q4
Glean ML design: design permission-aware enterprise search across Slack, Drive, Jira and email.
  • Connectors ingest content and ACLs incrementally (webhooks plus crawls); normalise into a unified document schema with metadata (owner, time, source, people graph).
  • Index: BM25 plus dense embeddings; a knowledge graph of people, documents and activity. Permission filtering at query time using ACL sets (pre-filtered search).
  • Ranking:
  • A learning-to-rank model with text relevance, freshness, personal signals (my team's docs), popularity, authority.
  • A cross-encoder for the top results.
  • Assistant: RAG over the permitted results with citations.
  • Evaluation: click models, NDCG on judged queries, answer faithfulness; strict tests that ACL leaks never occur.
Q5
Research presentation: how should you present your work to an LLM research team?
  • Contribution first: "We improved X by Y% at Z cost."
  • Setup: data, model size, baselines. Be precise about compute.
  • Results with ablations and error bars; failure cases.
  • Why it works (hypothesis plus evidence) and its limits.
  • Next steps and how it would scale.
  • Expect hard questions on confounders and evaluation validity. Welcome them.
Q6
AI take-home: build a small RAG evaluation harness. What would you include?
  • A dataset of about 50–100 questions with reference answers and gold document IDs.
  • Retrieval metrics: recall@k, MRR against the gold documents.
  • Generation metrics: faithfulness (an LLM judge plus claim-level checks), correctness against the reference, citation accuracy.
  • Reports: per-question failures grouped by type (retrieval miss vs generation error), latency and cost.
  • Engineering: a config-driven CLI, caching of LLM calls, deterministic seeds, a clear README with results and next steps.

03How to prepare

Sources

Created by Ajeetkumar