Cisco & Synopsys: ML / AI Engineer

1. Search first: grep -rn "reward" --include=*.py, plus terms like score, return, advantage.
2. Start from the entry point (training script or main) and follow the loop: environment step() → reward, or trajectory → reward model / verifier call.
3. Check configs (YAML) for reward function names, then jump to their definitions in the IDE.
4. Confirm by running a small test with logging or a breakpoint to watch where the reward value is created. - Explain your systematic navigation; interviewers care about the method as much as finding the line.

• Distillation: fine-tune on reasoning traces (chain of thought) from a larger teacher model; filter for correct answers (rejection sampling).
• RL with verifiable rewards: maths and code tasks with automatic checkers (GRPO/PPO), plus format rewards. Watch for reward hacking.
• Data quality: curriculum (easy → hard), deduplication, decontamination.
• Inference-time: self-consistency (majority vote), verifier re-ranking, tool use (a calculator or code interpreter), extra "thinking" tokens.
• Evaluate on GSM8K, MATH and held-out domain tasks; check the calibration of the final answers.

• Train a model to score responses so that r(chosen) > r(rejected) (Bradley–Terry loss) on human preference pairs; then optimise the policy against it with a KL penalty.
• Pitfalls:
• Reward hacking: exploiting quirks such as length or flattery.
• Distribution shift as the policy improves.
• Noisy or biased preference labels.
• Overoptimisation (Goodhart's law).
• Mitigations: ensembles, regular reward-model refreshes, length normalisation, human audits.

• Model-based: tree gain or split counts (biased towards high-cardinality features), coefficients for scaled linear models.
• Permutation importance: the drop in the validation metric when a feature is shuffled. Model-agnostic, but correlated features split the importance.
• SHAP: consistent per-prediction attributions; global importance is the mean |SHAP|.
• Caveat: importance is not causality. Check stability across folds and deal with correlated groups.

• Design-space exploration: RL or Bayesian optimisation to tune synthesis and place-and-route parameters for power, performance and area.
• Prediction: estimate timing, congestion and power early, from netlist and graph features (GNNs), to avoid slow full runs.
• Verification: prioritise test cases, cluster failures, analyse logs with LLMs.
• GenAI copilots: RTL code assistants, documentation Q&A, script generation for EDA tools.

import heapq
def kth_largest(nums, k):
    h = nums[:k]; heapq.heapify(h)
    for x in nums[k:]:
        if x > h[0]: heapq.heapreplace(h, x)
    return h[0]

• A min-heap of size k gives O(n log k). Quickselect averages O(n).