Qualcomm DS & Community Reports: LLM Engineer Rounds

• Minimise the loss L(θ). A first-order Taylor expansion gives L(θ + Δ) ≈ L(θ) + ∇L(θ)ᵀΔ.
• To decrease L fastest for a small step of fixed size, choose Δ in the direction opposite to the gradient: Δ = −η ∇L(θ).
• Update: θₜ₊₁ = θₜ − η ∇L(θₜ), where the learning rate η controls the step size. Too large and it diverges or oscillates; too small and it is slow.
• Example (MSE linear regression): L = (1/n)‖Xw − y‖² gives ∇L = (2/n)Xᵀ(Xw − y), so w ← w − η (2/n)Xᵀ(Xw − y).
• Convergence: for an L-smooth convex function, η ≤ 1/L guarantees a decrease.

import numpy as np

def pca(X, k):
    Xc = X - X.mean(axis=0)                     # 1. centre
    C = Xc.T @ Xc / (len(X) - 1)                # 2. covariance (d x d)
    vals, vecs = np.linalg.eigh(C)              # 3. eigen-decomposition (symmetric)
    order = np.argsort(vals)[::-1][:k]          # 4. top-k by variance
    W = vecs[:, order]
    return Xc @ W, vals[order] / vals.sum()     # projected data, explained-variance ratio

• By hand: centre → covariance → eigenvalues and eigenvectors → sort → project. Using the SVD of Xc is numerically more stable.

• Normalisation (standardise or min-max): puts features on similar scales, so gradient descent converges faster (a rounder loss surface), distance-based models work, and regularisation treats features fairly. Inside networks, Batch or Layer Norm stabilise training.
• Activations:
• ReLU: simple, no saturation for positive values, but can produce "dead" neurons.
• Leaky ReLU / GELU / SiLU: smooth, standard in Transformers.
• Sigmoid: binary output probabilities.
• Softmax: multi-class output.
• tanh: zero-centred, used in older RNNs.

• Outcomes with at least one 3: 6 + 6 − 1 = 11.
• Of those, the sum is 7 for (3,4) and (4,3): 2.
• P = 2/11 ≈ 0.18.
• Method: count the conditioning event first, then the favourable outcomes inside it. Bayes' rule P(A|B) = P(A∩B)/P(B) = (2/36)/(11/36).

• Split d_model into h heads of size d_k = d_model/h. Each head has its own W_Q, W_K, W_V and computes attention in parallel. The outputs are concatenated and projected with W_O.
• Why: different heads learn different relationships (syntax, coreference, position, delimiters) at the same compute cost as a single full-width head.
• Variants: Multi-Query and Grouped-Query Attention (MQA / GQA) share K and V across heads, which shrinks the KV cache and speeds up inference (used in Llama 2/3 and Mistral).

• Limitations:
• Hallucination.
• Knowledge cutoff.
• Context-length limits and "lost in the middle".
• Weak exact arithmetic.
• Sensitivity to prompts.
• Bias.
• Cost and latency.
• Vulnerability to prompt injection.
• Reducing hallucination:
• Ground answers with RAG and citations.
• Tell the model to say "I don't know".
• Lower the temperature.
• Use tools for facts and maths.
• Structured outputs.
• Self-check or verification passes (a second model checks the claims).
• Retrieval-score thresholds.
• Evaluate faithfulness continuously.

• Chunking:
• Recursive or structure-aware (headings, sections) at 300–1,000 tokens with 10–20% overlap.
• Semantic chunking: split where embedding similarity drops.
• Keep tables intact.
• Hierarchical / parent-child: retrieve small chunks for precision, but send the parent section to the LLM for context.
• Summaries: map-reduce or refine summarisation for "summarise the whole document"; a RAPTOR-style tree of summaries.
• Long-context models: even with 128k–1M tokens, retrieval is cheaper and often more accurate. Put key context at the start or end of the prompt.

• GIL: in CPython only one thread runs Python bytecode at a time.
• I/O-bound work (calling LLM APIs, vector-DB queries, HTTP): use threads or better asyncio (asyncio.gather), which releases the GIL while waiting. This gives large speedups for parallel API calls.
• CPU-bound work (parsing PDFs, tokenising, local embeddings on CPU): use multiprocessing or ProcessPoolExecutor, which bypasses the GIL across processes.
• Example: embed 10k documents through an API with asyncio, a semaphore (rate limit) and retries; parse PDFs with a process pool.

• Model: export to ONNX or TorchScript, quantize, batch requests dynamically.
• Serving: Triton, TorchServe or vLLM (for LLMs) on GPU nodes; autoscale on queue depth or GPU utilisation.
• Architecture: API gateway → stateless service → model server; feature store or cache; async queue for heavy jobs.
• Reliability: canary releases, health checks, timeouts, fallback to a smaller model.
• Monitoring: latency (p50/p95/p99), throughput, GPU memory, drift in inputs and outputs, and cost per 1,000 requests.