CareerByteCode
ByteBlogby CareerByteCode

MLOps Interview Questions Answered with Labs

What you build: 6 MLOps interview answers proved with real scikit-learn experiments: leakage, reproducibility, imbalance, skew, versioning, latency. Every step tested.

MLOps Interview Questions Answered with Labs

✍️ Translate this blog into Hindi, Tamil or Telugu

MLOps interviews test whether you have been burned by real machine learning problems: leakage that makes a model look great until production, results nobody can reproduce, accuracy that lies, models that behave differently when served, and unversioned data. In this post, 6 classic questions are answered and each answer is proved with a small, real experiment in scikit-learn on a CPU, with the real numbers.

Who it is for: data scientists, ML engineers and DevOps engineers moving into MLOps. Time: about 30 minutes. Cost: none beyond a small VM you already have (or run it in Cloud Shell).

Every command below was run for real by the CareerByteCode Team, and every output is the real result we got. We ran the experiments in a Cloud Shell-like Linux VM with Python 3.

🧪⭐ Recommended by CareerByteCode · ByteLabs project bundleCloud and DevOps Realtime Starterโ‚น499Build the projects →

The lab behind the answers

All six experiments live in one file, one function per question, using scikit-learn's built-in data generators (so you get the same numbers). Save it as proofs.py:

# proofs.py <question>: one small, real experiment per MLOps interview question (scikit-learn, CPU only).
import hashlib, pickle, sys, time
import numpy as np
from sklearn.datasets import make_classification
from sklearn.ensemble import RandomForestClassifier
from sklearn.feature_selection import SelectKBest, f_classif
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, precision_score, recall_score, roc_auc_score
from sklearn.model_selection import cross_val_score, train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

q = sys.argv[1]

if q == "leakage":
    # 200 rows, 5000 random features, labels are pure noise: an honest model must score about 50%
    rng = np.random.default_rng(0); X = rng.normal(size=(200, 5000)); y = rng.integers(0, 2, 200)
    picked = SelectKBest(f_classif, k=20).fit(X, y).transform(X)                     # WRONG: picks features using ALL rows
    leaky = cross_val_score(LogisticRegression(max_iter=1000), picked, y, cv=5).mean()
    honest = cross_val_score(make_pipeline(SelectKBest(f_classif, k=20), LogisticRegression(max_iter=1000)), X, y, cv=5).mean()
    print(f"labels are random noise. feature selection before the split: accuracy {100 * leaky:.0f}% | inside the pipeline: {100 * honest:.0f}%")

elif q == "seed":
    X, y = make_classification(n_samples=3000, n_features=20, random_state=1)
    for name, seed in (("no seed, run 1", None), ("no seed, run 2", None), ("seed 42, run 1", 42), ("seed 42, run 2", 42)):
        Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=seed)
        m = RandomForestClassifier(n_estimators=50, random_state=seed).fit(Xtr, ytr)
        print(f"{name}: AUC {roc_auc_score(yte, m.predict_proba(Xte)[:, 1]):.4f}")

elif q == "imbalance":
    X, y = make_classification(n_samples=10000, n_features=10, weights=[0.97], flip_y=0, random_state=3)
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=3, stratify=y)
    lazy = np.zeros_like(yte)                                                          # a "model" that always says "no fraud"
    m = LogisticRegression(max_iter=1000).fit(Xtr, ytr); p = m.predict_proba(Xte)[:, 1]
    print(f"positives in test data: {yte.sum()} of {len(yte)} ({100 * yte.mean():.1f}%)")
    print(f"always 'no': accuracy {100 * accuracy_score(yte, lazy):.1f}%, recall {100 * recall_score(yte, lazy):.0f}%")
    for t in (0.5, 0.2, 0.05):
        pred = (p >= t).astype(int)
        print(f"model, threshold {t}: accuracy {100 * accuracy_score(yte, pred):.1f}%, precision {100 * precision_score(yte, pred):.0f}%, recall {100 * recall_score(yte, pred):.0f}%")

elif q == "skew":
    # training scaled the features; the serving code "forgot" the scaler and sends raw numbers
    X, y = make_classification(n_samples=4000, n_features=8, random_state=5); X[:, 0] *= 1000
    Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=5)
    sc = StandardScaler().fit(Xtr); m = LogisticRegression(max_iter=1000).fit(sc.transform(Xtr), ytr)
    print(f"serving WITH the same scaler: accuracy {100 * accuracy_score(yte, m.predict(sc.transform(Xte))):.1f}%")
    print(f"serving WITHOUT it (raw input): accuracy {100 * accuracy_score(yte, m.predict(Xte)):.1f}%")
    pipe = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)).fit(Xtr, ytr)
    print(f"fix, one pipeline object (scaler inside the model): accuracy {100 * accuracy_score(yte, pipe.predict(Xte)):.1f}%")

elif q == "version":
    X, y = make_classification(n_samples=2000, n_features=10, random_state=7)
    m = RandomForestClassifier(n_estimators=30, random_state=7).fit(X, y)
    blob = pickle.dumps(m); data = X.tobytes()
    print(f"data sha256 {hashlib.sha256(data).hexdigest()[:16]} | model sha256 {hashlib.sha256(blob).hexdigest()[:16]} | model size {len(blob) / 1024:.0f} KB")
    X2 = X.copy(); X2[0, 0] += 0.001
    print(f"one value changed by 0.001 -> data sha256 {hashlib.sha256(X2.tobytes()).hexdigest()[:16]} (different: this is why you version data)")

elif q == "latency":
    X, y = make_classification(n_samples=20000, n_features=20, random_state=9)
    for name, m in (("logistic regression", LogisticRegression(max_iter=1000)), ("random forest 300 trees", RandomForestClassifier(n_estimators=300, random_state=9))):
        m.fit(X, y)
        t = time.perf_counter(); [m.predict(X[i:i + 1]) for i in range(200)]; one = (time.perf_counter() - t) / 200
        t = time.perf_counter(); m.predict(X); batch = (time.perf_counter() - t) / len(X)
        print(f"{name:<24} one row at a time: {1000 * one:6.2f} ms per row | batch of {len(X)}: {1000 * batch:6.4f} ms per row | size {len(pickle.dumps(m)) / 1e6:.1f} MB")
The lab
One Python file, six small experiments: leakage, seeds, imbalance, training-serving skew, versions and latency
# One Python environment for every proof (scikit-learn on CPU)
python3 -m venv .venv && . .venv/bin/activate && pip install -q "scikit-learn==1.*" 2>&1 | grep -v notice | tail -1; python3 -c "import sklearn; print('scikit-learn', sklearn.__version__)"
scikit-learn 1.9.1

Q1. What is data leakage?

Answer: information from the test data (or the future) gets into training, so the score looks better than the model really is. A classic case: choosing features, scaling or encoding using all the data before splitting. The fix is to fit every step inside the cross-validation, for example with a scikit-learn Pipeline.

Proof: 200 rows, 5,000 random features and labels that are pure noise, so an honest model must score about 50 percent:

# Q1. What is data leakage? Choose features using all rows, then cross-validate, on labels that are pure noise
. .venv/bin/activate; python3 proofs.py leakage
labels are random noise. feature selection before the split: accuracy 80% | inside the pipeline: 51%

Picking the 20 "best" features on all rows first made noise look 80 percent accurate. Inside the pipeline, the honest 51 percent. Any score that looks too good deserves this check.

Q2. How do you make training reproducible?

Answer: fix every source of randomness (data split, model initialisation, sampling) with a seed, and pin library versions. Then version the data and log the parameters.

Proof: the same random forest, twice without a seed, twice with seed 42:

# Q2. How do you make training reproducible? Train the same model twice without a seed, then twice with one
. .venv/bin/activate; python3 proofs.py seed
no seed, run 1: AUC 0.9898
no seed, run 2: AUC 0.9843
seed 42, run 1: AUC 0.9819
seed 42, run 2: AUC 0.9819

Without a seed, two runs gave different AUCs (0.9898 and 0.9843); with a seed, both gave 0.9819. If you cannot reproduce a number, you cannot tell whether a change made things better.

Q3. Why is accuracy a bad metric for fraud detection?

Answer: when positives are rare (3 percent here), a model that always says "no fraud" is 97 percent accurate and useless. Use precision (of what I flagged, how much was fraud) and recall (of all fraud, how much I caught), and choose the threshold for the business cost.

Proof:

# Q3. Why is accuracy a bad metric for fraud? 3 percent positives; compare a lazy "always no" with a real model
. .venv/bin/activate; python3 proofs.py imbalance
positives in test data: 90 of 3000 (3.0%)
always 'no': accuracy 97.0%, recall 0%
model, threshold 0.5: accuracy 96.6%, precision 31%, recall 10%
model, threshold 0.2: accuracy 95.7%, precision 34%, recall 43%
model, threshold 0.05: accuracy 91.0%, precision 24%, recall 92%

The lazy "always no" model wins on accuracy (97 percent) and catches 0 percent of fraud. The real model at the default threshold 0.5 catches only 10 percent; at 0.05 it catches 92 percent, with more false alarms (precision 24 percent). The right threshold is a business decision, not 0.5 by default.

Leakage, seeds, imbalance
Noise scored 80 percent with leakage and 51 percent without; seeds make runs equal; 97 percent accuracy with 0 percent recall

Q4. What is training-serving skew?

Answer: the model sees different data in production than in training: a missing preprocessing step, a different unit, another library version. The model does not crash; it just gets worse quietly.

Proof: training scaled the inputs; the "serving code" forgot the scaler:

# Q4. What is training-serving skew? Training scaled the inputs; the serving code forgets to
. .venv/bin/activate; python3 proofs.py skew
serving WITH the same scaler: accuracy 87.2%
serving WITHOUT it (raw input): accuracy 82.1%
fix, one pipeline object (scaler inside the model): accuracy 87.2%

5 points of accuracy lost silently. The fix: ship preprocessing inside the model object (one pipeline), so training and serving cannot drift apart.

📝⭐ Recommended by CareerByteCode · Certification examGoogle Analytics Certification (GA4) PracticeTake the examTake the exam →

Q5. Why version data and models, not only code?

Answer: the same code on different data gives a different model. Record a fingerprint (hash) of the training data and of the model file with every run (MLflow, DVC or your registry do this).

Proof:

# Q5. Why version data and models, not only code? Fingerprint both with sha256
. .venv/bin/activate; python3 proofs.py version
data sha256 8f1458e7ec9494f1 | model sha256 30ee2263c4cd7b8d | model size 318 KB
one value changed by 0.001 -> data sha256 ddf78e791a966429 (different: this is why you version data)

Changing one number by 0.001 changes the data fingerprint completely, while the code is identical. Without data versions, "which data trained the model in production?" has no answer.

Q6. Online or batch prediction?

Answer: online (one request at a time, low latency) for user-facing decisions; batch (many rows at once, high throughput, cheap) for reports and nightly scoring. Model size and type change the trade-off.

Proof: 200 single-row predictions against one batch of 20,000 rows, for a small and a big model:

# Q6. Online or batch prediction? Time one row at a time against one big batch, for a small and a big model
. .venv/bin/activate; python3 proofs.py latency
logistic regression      one row at a time:   0.36 ms per row | batch of 20000: 0.0001 ms per row | size 0.0 MB
random forest 300 trees  one row at a time:  27.33 ms per row | batch of 20000: 0.0399 ms per row | size 34.2 MB

The 300-tree forest takes 27 ms per single request (fine for some APIs, too slow for others) but 0.04 ms per row in a batch, and its file is 34 MB against almost nothing for logistic regression. Measure before you promise a latency.

Skew, versions, latency
Forgetting the scaler costs 5 points; one changed value changes the data hash; 27 ms online against 0.04 ms per row in batch

Clean up (delete everything)

# Clean up: remove the lab folder (the VM is shared and stays)
cd ~ && rm -rf ~/lab261 && echo "lab folder removed"
lab folder removed

Quick answers to remember

QuestionOne-line answerProved in
Data leakageFit every step inside the split; noise scored 80 percent otherwiseQ1
ReproducibilitySeeds, pinned versions, versioned dataQ2
Accuracy on rare classesUse precision, recall and a business thresholdQ3
Training-serving skewShip preprocessing inside the modelQ4
VersioningHash data and model with every runQ5
Online or batchLatency per row vs throughput; measure itQ6

What next

Prepared by the CareerByteCode Team from our tested lab, with AI help for the write-up.

Read next →MLOps Project for Your Resume on Google Cloud⏱ 12 min

⭐ Keep learning with CareerByteCode

Hand-picked to go deeper on this topic
News, guides and updates from the CareerByteCode team
🏅 Expert writer

The CareerByteCode team writes about what is new on careerbytecode.in: courses, labs, exams, events and the community, which is free to join for influencers and content creators in tech.

Get CareerByteCode Team's next blog by email

One email a week when they publish. Add the weekly Best of ByteBlog too. No spam, unsubscribe with one click.

We email you once to confirm. By subscribing you agree to the privacy policy.

💬 Comments (0)

Top · Newest

Sign in to comment

Related blogs

🧪⭐ Recommended by CareerByteCode · ByteLabs project bundleCloud and DevOps Realtime Starterโ‚น499Build the projects →