CareerByteCode
ByteBlogby CareerByteCode

MLOps Project for Your Resume on Google Cloud

What you build: an MLOps loop on Google Cloud: quality gate, model registry, batch scoring to BigQuery, drift checks and a retrain that must earn its place. Every step tested.

MLOps Project for Your Resume on Google Cloud

✍️ Translate this blog into Hindi, Tamil or Telugu

A notebook with 0.9 accuracy is not an MLOps project. An MLOps project shows the loop around the model: a quality gate that decides which model goes to production, a model registry with versions you can roll back to, batch scoring that writes results where the business uses them, drift monitoring that tells you when the world changed, and a retrain that has to earn its place. In this lab you build that loop on Google Cloud for a churn model (which customers will leave?), with Cloud Storage as the registry and BigQuery for the scores, and you watch the gate refuse a bad model.

Who it is for: data scientists, ML engineers and DevOps engineers who want one MLOps project they can explain end to end in an interview. Time: about 1 hour. Cost: a few files in Cloud Storage and a small BigQuery table, inside the free tier.

Every command below was run for real by the CareerByteCode Team, and every output is the real result we got. We typed the commands in a Cloud Shell-like Linux VM (Debian with gcloud, bq and Python 3); Cloud Shell in your browser has the same tools.

🧪⭐ Recommended by CareerByteCode · ByteLabs project bundleCloud and DevOps Realtime Starterโ‚น499Build the projects →

Part A: the loop in plain English

The MLOps loop
Monthly data, train, gate, promote to the registry, batch score to BigQuery, check drift, retrain when needed
  • Quality gate: an automatic check before promotion. Ours: the new model's AUC must be at least 0.70 and not worse than production (minus 0.01). AUC (area under the ROC curve) measures how well a model ranks risky customers above safe ones: 0.5 is guessing, 1.0 is perfect.
  • Model registry: where approved models live with a version number and metadata. Ours is a Cloud Storage folder per version (v1/, v2/) plus prod.json saying which one is in production. (Vertex AI Model Registry or MLflow do the same with more features.)
  • Batch scoring: score every customer once a month and write the results to a table the retention team uses.
  • Drift: the input data changes over time. PSI (population stability index) compares a feature's distribution with the training data: below 0.1 nothing, 0.1 to 0.2 watch, above 0.2 act.
  • Real-life example: a restaurant's head chef (the gate) tastes every new recipe before it goes on the menu (the registry), keeps the old recipe card in case the new one fails, and changes the menu when the season's ingredients change (drift).

Before you start

A Google Cloud project with billing linked (the free trial works) and Cloud Shell or a small VM with gcloud, bq and Python 3.

Part B: the lab

The pipeline in one file

Each stage is one command. Data is generated (same month = same data), so your numbers match ours. Save it as churn.py:

# churn.py: the whole MLOps loop for a churn model (which customers will leave?), one command per stage.
#   data <month> [--drift]     make that month's customer data (same seed = same data); --drift = prices went up
#   train <month>              train on that month, evaluate on a held-out part, save model + metrics locally
#   gate                       pass only if AUC >= 0.70 and not worse than the model in production
#   promote                    copy the model to the registry in Cloud Storage as the next version, mark it "prod"
#   score <month>              batch scoring: load "prod" from the registry, score every customer, write to BigQuery
#   drift <month>              compare a month's features with the training month (PSI); > 0.2 = act
import json, os, pickle, subprocess, sys
import numpy as np
import pandas as pd
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.metrics import roc_auc_score
from sklearn.model_selection import train_test_split

BUCKET = open(".bucket").read().strip()
FEATURES = ["tenure_months", "monthly_bill", "support_calls", "late_payments"]


def sh(cmd):
    return subprocess.run(cmd, shell=True, capture_output=True, text=True).stdout.strip()


def make_data(month, drift):
    rng = np.random.default_rng(int(month.replace("-", "")))
    n = 8000
    df = pd.DataFrame({"customer_id": [f"C{i:05d}" for i in range(n)],
                       "tenure_months": rng.integers(1, 72, n),
                       "monthly_bill": rng.normal(900 if not drift else 1250, 200, n).round(0),
                       "support_calls": rng.poisson(1.2 if not drift else 2.0, n),
                       "late_payments": rng.poisson(0.4, n)})
    risk = -2.2 - 0.04 * df.tenure_months + 0.002 * (df.monthly_bill - 900) + 0.6 * df.support_calls + 0.8 * df.late_payments
    df["churned"] = (rng.random(n) < 1 / (1 + np.exp(-risk))).astype(int)
    df.to_csv(f"customers_{month}.csv", index=False)
    print(f"customers_{month}.csv: {n} customers, churn rate {100 * df.churned.mean():.1f}%, average bill {df.monthly_bill.mean():.0f}")


def psi(a, b, bins=10):
    edges = np.unique(np.quantile(a, np.linspace(0, 1, bins + 1)))
    pa = np.histogram(a, edges)[0] / len(a) + 1e-6; pb = np.histogram(np.clip(b, edges[0], edges[-1]), edges)[0] / len(b) + 1e-6
    return float(np.sum((pa - pb) * np.log(pa / pb)))


cmd = sys.argv[1]
if cmd == "data":
    make_data(sys.argv[2], "--drift" in sys.argv)
elif cmd == "train":
    df = pd.read_csv(f"customers_{sys.argv[2]}.csv")
    X_tr, X_te, y_tr, y_te = train_test_split(df[FEATURES], df.churned, test_size=0.25, random_state=1, stratify=df.churned)
    model = GradientBoostingClassifier(random_state=1).fit(X_tr, y_tr)
    auc = roc_auc_score(y_te, model.predict_proba(X_te)[:, 1])
    pickle.dump(model, open("model.pkl", "wb"))
    json.dump({"trained_on": sys.argv[2], "auc": round(auc, 4), "rows": len(df)}, open("metrics.json", "w"))
    print(f"trained on {sys.argv[2]}: AUC {auc:.3f} on {len(X_te)} held-out customers")
elif cmd == "gate":
    new = json.load(open("metrics.json"))
    prod = sh(f"gcloud storage cat gs://{BUCKET}/registry/prod.json 2>/dev/null")
    prod_auc = json.loads(prod)["auc"] if prod else 0.0
    ok = new["auc"] >= 0.70 and new["auc"] >= prod_auc - 0.01
    print(f"gate: new AUC {new['auc']:.3f}, production AUC {prod_auc:.3f}, minimum 0.70 -> {'PASS' if ok else 'FAIL'}")
    sys.exit(0 if ok else 1)
elif cmd == "promote":
    versions = [l for l in sh(f"gcloud storage ls gs://{BUCKET}/registry/").splitlines() if "/v" in l]
    v = f"v{len(versions) + 1}"; meta = json.load(open("metrics.json")) | {"version": v}
    json.dump(meta, open("metrics.json", "w"))
    sh(f"gcloud storage cp model.pkl metrics.json gs://{BUCKET}/registry/{v}/ && gcloud storage cp metrics.json gs://{BUCKET}/registry/prod.json")
    print(f"promoted {v} (AUC {meta['auc']:.3f}, trained on {meta['trained_on']}) and marked it prod")
elif cmd == "score":
    meta = json.loads(sh(f"gcloud storage cat gs://{BUCKET}/registry/prod.json"))
    sh(f"gcloud storage cp gs://{BUCKET}/registry/{meta['version']}/model.pkl prod_model.pkl")
    model = pickle.load(open("prod_model.pkl", "rb"))
    df = pd.read_csv(f"customers_{sys.argv[2]}.csv")
    out = pd.DataFrame({"customer_id": df.customer_id, "churn_risk": model.predict_proba(df[FEATURES])[:, 1].round(4),
                        "model_version": meta["version"], "month": sys.argv[2]})
    out.to_csv("scores.csv", index=False)
    sh("bq load --quiet --replace --autodetect --source_format=CSV churn.scores scores.csv")
    print(f"scored {len(out)} customers for {sys.argv[2]} with model {meta['version']}; {int((out.churn_risk > 0.5).sum())} at high risk; written to BigQuery churn.scores")
elif cmd == "drift":
    meta = json.loads(sh(f"gcloud storage cat gs://{BUCKET}/registry/prod.json"))
    base, new = pd.read_csv(f"customers_{meta['trained_on']}.csv"), pd.read_csv(f"customers_{sys.argv[2]}.csv")
    worst = 0.0
    for f in FEATURES:
        p = psi(base[f].values, new[f].values); worst = max(worst, p)
        print(f"  {f:<15} PSI {p:.3f} {'DRIFT' if p > 0.2 else 'ok'}")
    print(f"drift check {sys.argv[2]} vs {meta['trained_on']}: {'RETRAIN' if worst > 0.2 else 'no action'}")

Step 1: the registry bucket and the scores dataset

# The project's Google Cloud parts: a bucket for the model registry and a BigQuery dataset for the scores
python3 -m venv .venv && . .venv/bin/activate && pip install -q "scikit-learn==1.*" "pandas==2.*" 2>&1 | grep -v notice | tail -1; echo "Python packages installed"
echo "mlops-$(echo -n $PROJECT_ID | md5sum | cut -c1-8)" > .bucket
gcloud storage buckets create gs://$(cat .bucket) --location=us-east1 --uniform-bucket-level-access --public-access-prevention > /dev/null && echo "registry bucket created"
bq --location=us-east1 mk --dataset churn > /dev/null && echo "dataset churn created"
Python packages installed
Creating gs://mlops-2c18be0d/...
registry bucket created
dataset churn created

Step 2: month 1: train, gate, promote

Why: nobody copies a model by hand. The gate decides; only then the model is copied to the registry as the next version and marked prod.

# Month 1: data, train, gate, promote. The gate decides; nobody copies a model by hand
. .venv/bin/activate
python3 churn.py data 2026-09 && python3 churn.py train 2026-09 && python3 churn.py gate && python3 churn.py promote
customers_2026-09.csv: 8000 customers, churn rate 11.0%, average bill 900
trained on 2026-09: AUC 0.739 on 2000 held-out customers
gate: new AUC 0.739, production AUC 0.000, minimum 0.70 -> PASS
promoted v1 (AUC 0.739, trained on 2026-09) and marked it prod

Step 3: batch scoring to BigQuery

Why: the production model (read from the registry, not from a local file) scores every customer for October, and the results go to churn.scores where analysts and dashboards read them:

# Batch scoring: the production model scores every customer, the results land in BigQuery for the retention team
. .venv/bin/activate; python3 churn.py data 2026-10 && python3 churn.py score 2026-10
bq query --quiet --use_legacy_sql=false --format=pretty "SELECT model_version, COUNT(*) AS customers, ROUND(AVG(churn_risk), 3) AS avg_risk, COUNTIF(churn_risk > 0.5) AS high_risk FROM churn.scores GROUP BY model_version"
customers_2026-10.csv: 8000 customers, churn rate 11.2%, average bill 900
scored 8000 customers for 2026-10 with model v1; 237 at high risk; written to BigQuery churn.scores
+---------------+-----------+----------+-----------+
| model_version | customers | avg_risk | high_risk |
+---------------+-----------+----------+-----------+
| v1            |      8000 |    0.111 |       237 |
+---------------+-----------+----------+-----------+

237 of 8,000 customers are high risk this month; the retention team now has a list.

Step 4: monitoring: is October still like September?

# Monitoring: is October's data still like the training data (September)? PSI per feature
. .venv/bin/activate; python3 churn.py drift 2026-10
  tenure_months   PSI 0.001 ok
  monthly_bill    PSI 0.004 ok
  support_calls   PSI 0.001 ok
  late_payments   PSI 0.000 ok
drift check 2026-10 vs 2026-09: no action

All PSI values near 0: no drift, no action.

Train, gate, score, monitor
v1 passes the gate (AUC 0.739), scores 8,000 customers into BigQuery, and October shows no drift

Break it: the world changes

In November prices went up and support got busier. The data drifts; the old model still happily scores it:

# Break it: in November prices went up and support got busier. The data drifts; the old model still scores it
. .venv/bin/activate; python3 churn.py data 2026-11 --drift && python3 churn.py drift 2026-11
customers_2026-11.csv: 8000 customers, churn rate 26.1%, average bill 1251
  tenure_months   PSI 0.001 ok
  monthly_bill    PSI 2.639 DRIFT
  support_calls   PSI 0.408 DRIFT
  late_payments   PSI 0.000 ok
drift check 2026-11 vs 2026-09: RETRAIN

monthly_bill (PSI 2.6) and support_calls (PSI 0.41) moved far beyond 0.2, and churn jumped from 11 percent to 26 percent. The drift check says RETRAIN.

Break it again: a rushed retrain

Under pressure, someone retrains on a tiny slice of November (300 rows) and tries to push it:

# Break it again: a rushed retrain on a tiny, wrong slice (only 300 rows). The gate must refuse it
. .venv/bin/activate; head -301 customers_2026-11.csv > customers_tiny.csv && python3 churn.py train tiny && python3 churn.py gate
trained on tiny: AUC 0.656 on 75 held-out customers
gate: new AUC 0.656, production AUC 0.739, minimum 0.70 -> FAIL

The gate refused it: AUC 0.656 is below the 0.70 minimum and worse than production. Without a gate, this model would now be scoring every customer.

Fix it: retrain properly, gate, promote v2

# Fix it: retrain on the full November data, pass the gate, promote v2, re-score. Old versions stay for rollback
. .venv/bin/activate; python3 churn.py train 2026-11 && python3 churn.py gate && python3 churn.py promote && python3 churn.py score 2026-11
gcloud storage ls gs://$(cat .bucket)/registry/ | sed "s#gs://[^/]*/#gs://<bucket>/#"
📝⭐ Recommended by CareerByteCode · Certification examGoogle Analytics Certification (GA4) PracticeTake the examTake the exam →
trained on 2026-11: AUC 0.788 on 2000 held-out customers
gate: new AUC 0.788, production AUC 0.739, minimum 0.70 -> PASS
promoted v2 (AUC 0.788, trained on 2026-11) and marked it prod
scored 8000 customers for 2026-11 with model v2; 1320 at high risk; written to BigQuery churn.scores
gs://<bucket>/registry/prod.json
gs://<bucket>/registry/v1/
gs://<bucket>/registry/v2/

Trained on the full November data, the new model reaches AUC 0.788, passes the gate, becomes v2 and scores November: 1,320 customers at high risk (churn really did rise). v1 stays in the registry for a one-line rollback (point prod.json back to it).

Drift, gate, v2
November drifts, the tiny retrain fails the gate at AUC 0.656, the full retrain passes at 0.788 and becomes v2

Step 5: your resume numbers, from the registry and BigQuery

# Your resume numbers, from the registry and BigQuery
for v in v1 v2; do gcloud storage cat gs://$(cat .bucket)/registry/$v/metrics.json; echo; done
bq query --quiet --use_legacy_sql=false --format=csv "SELECT COUNT(*) FROM churn.scores" | tail -1 | sed 's/^/rows in churn.scores now: /'
{"trained_on": "2026-09", "auc": 0.7392, "rows": 8000, "version": "v1"}
{"trained_on": "2026-11", "auc": 0.7879, "rows": 8000, "version": "v2"}
rows in churn.scores now: 8000

(The scores table holds the latest month only, because each run replaces it; keep a month partition if you need history.)

Your resume lines (with the numbers you measured)

  • Built an end-to-end MLOps loop for a churn model on Google Cloud: versioned model registry in Cloud Storage, monthly batch scoring of 8,000 customers into BigQuery.
  • Added an automatic quality gate (AUC at least 0.70, no regression) that blocked a bad retrain (AUC 0.656).
  • Added drift monitoring with PSI per feature; detected a price shift (PSI 2.6) and retrained, improving AUC from 0.739 to 0.788.

POC: prove it works

  • v1 passed the gate, was promoted and scored 8,000 customers into BigQuery (Steps 2 and 3).
  • October showed no drift; November drifted and triggered a retrain (Step 4, Break it).
  • The gate refused a tiny retrain and accepted the full one as v2, with v1 kept for rollback (Break it again, Fix it).

Clean up (delete everything)

# Clean up: the dataset and the registry bucket
bq rm -r -f -d churn > /dev/null && echo "dataset deleted"
gcloud storage rm -r gs://$(cat .bucket) --quiet > /dev/null 2>&1; echo "left: $(gcloud storage ls 2>/dev/null | grep -c mlops-) buckets"
dataset deleted
left: 0 buckets
Numbers and clean up
v1 and v2 metadata from the registry, 8,000 scored rows, then everything deleted

FAQ

Why AUC and not accuracy? With 11 percent churners, a model that says "nobody leaves" is 89 percent accurate and useless. AUC measures ranking, which is what a retention list needs.

Where would this run in production? Each stage as a step in Vertex AI Pipelines or Cloud Composer, on a monthly schedule, with the gate as a pipeline condition. We ran it by hand here.

Why keep v1? Rollback in seconds if v2 misbehaves on real data: point prod back to v1 and re-score.

Is PSI the only drift check? No: also watch prediction drift (the share of high-risk customers) and, when labels arrive, real performance. PSI is the classic first check on inputs.

What is the 0.01 tolerance in the gate? Small AUC differences are noise; the tolerance avoids refusing an equally good model because of randomness.

Troubleshooting

You seeWhyFix
The gate always failsThe minimum is set above what your data allowsSet it from a baseline you measured, as we did (0.70)
Not found: Dataset churnThe dataset was not created, or in another locationbq --location=us-east1 mk --dataset churn
Scores come from an old modelThe scorer loaded a local fileAlways load the model prod.json points to
PSI is huge for every featureComparing against the wrong monthCompare with the month the production model was trained on
A version folder is missingThe promote step failed after the gateCheck the gate output; re-run promote

What next

Prepared by the CareerByteCode Team from our tested lab, with AI help for the write-up.

Read next →MLflow on Google Cloud: Track Every Experiment⏱ 12 min

⭐ Keep learning with CareerByteCode

Hand-picked to go deeper on this topic
News, guides and updates from the CareerByteCode team
🏅 Expert writer

The CareerByteCode team writes about what is new on careerbytecode.in: courses, labs, exams, events and the community, which is free to join for influencers and content creators in tech.

Get CareerByteCode Team's next blog by email

One email a week when they publish. Add the weekly Best of ByteBlog too. No spam, unsubscribe with one click.

We email you once to confirm. By subscribing you agree to the privacy policy.

💬 Comments (0)

Top · Newest

Sign in to comment

Related blogs

🧪⭐ Recommended by CareerByteCode · ByteLabs project bundleCloud and DevOps Realtime Starterโ‚น499Build the projects →