ML system design, modelling fundamentals and the metric conversation - practise the rounds ML engineer and data scientist loops actually run.
The ML system design round is the one that decides most ML engineer loops, and it is not a modelling round. Designing a recommendation feed or a fraud detector is mostly about defining the label, choosing the training data without leaking the future into it, picking an offline metric that actually tracks the business outcome, and planning the online evaluation.
Label leakage is the single most common failure. A candidate builds a churn model using a feature only populated after the customer churned, reports excellent offline metrics, and does not catch it. Interviewers plant these. Talking through how you would construct the training set point-in-time is worth more than naming a fancier architecture.
Data scientist loops shift the weight toward statistics and experiment design: choosing a test, handling multiple comparisons, sizing a sample, and explaining what a confidence interval does and does not mean. ML engineer loops shift it toward serving: latency budgets, feature stores, training/serving skew and retraining cadence.
60 min
Label definition, training data, features, metrics, serving and monitoring end to end.
45 min
Bias-variance, regularisation, class imbalance, metric choice, model selection.
45 min
A/B test design, power, significance, and common pitfalls. Heavier in data scientist loops.
45 min
Python plus pandas or NumPy, sometimes implementing a metric or a simple model by hand.
A clear problem framing before any modelling. Define the label and prediction horizon, describe how you build the training set point-in-time, choose an offline metric and say why it tracks the business outcome, then design serving and monitoring. Naming architectures early without that framing is the common miss.
Data scientist loops weight statistics, experiment design and communicating results to non-technical stakeholders. ML engineer loops weight production concerns: serving latency, feature stores, training/serving skew, retraining and monitoring. The system design round appears in both, framed differently.
Only for roles that are explicitly deep learning. For most ML engineer loops, understanding when a gradient-boosted tree beats a neural network, and being able to justify the simpler model, is a stronger signal than reciting transformer internals.
Design an ML system end to end and get challenged on leakage and metrics.