AI Engineer Roadmap: From Python to Production LLM Apps
What you build: a 7-stage AI engineer roadmap from Python to production LLM apps, with a real lab: a Gemini call, schema-checked JSON, a golden-set evaluation (6/6) and a temperature test.

✍️ Translate this blog into Hindi, Tamil or Telugu
"AI engineer" is one of the fastest-growing job titles, and one of the most confusing to prepare for. It is not a data scientist who invents models, and not a prompt writer. An AI engineer builds reliable software around large language models (LLMs): calls them from code, makes them return data a program can trust, measures their quality, and runs them in production. This roadmap takes you from Python to production LLM apps in seven stages, and the lab in this post runs four of them for real on Google Cloud, including the two most often skipped: evaluation and consistency.
Who it is for: developers, data people and students who know a little Python and want to build with LLMs. Time: about 4 to 6 months at 6 to 8 hours a week for the whole roadmap; about 30 minutes for this lab. Cost: the lab makes about 20 small Gemini calls on Vertex AI, a tiny fraction of a cent.
The lab was run for real by the CareerByteCode Team on a fresh Google Cloud project, and every output is the real answer Gemini gave us.
⭐ Recommended by CareerByteCode · BytePlay courseNVIDIA GPU AI Factory: Architecture, Engineering & Operationsโน40,000Start learning →Part A: the roadmap

| Stage | What you learn | You can do it when... |
|---|---|---|
| 1. Python and data | virtual environments, packages, JSON, HTTP, pandas basics | you can call any web API and handle its JSON |
| 2. Calling a model | SDKs, sign-in without API keys (Vertex AI), tokens and cost | your code gets an answer and you know what it cost |
| 3. Prompts that return data | system instructions, JSON mode, schemas | the answer is valid data your program can use, every time |
| 4. Retrieval (RAG) | embeddings, vector search, grounding in your documents | answers cite your documents and say "I don't know" when they should |
| 5. Evaluation | golden sets, scores, regression tests for prompts | you can say "version B scores 9/10, version A 7/10" |
| 6. Reliability | temperature, retries, timeouts, caching, guardrails, cost limits | the same input gives a predictable answer, and failures are handled |
| 7. Production | APIs on Cloud Run, service accounts, logging, monitoring, CI/CD | real users use it, and you can see what it does |
Real-life example: a new chef. Stage 1 is knife skills, 2 is turning on the stove, 3 is following a recipe exactly, 4 is using the pantry's own ingredients, 5 is tasting before serving, 6 is getting the same taste every night, and 7 is running a real restaurant.
The two habits that separate AI engineers from hobbyists: they measure (stage 5) and they control randomness (stage 6). The lab shows both.
Part B: the lab, four stages for real
Before you start
- A Google Cloud account (the free trial works) and a project.
- Open Cloud Shell (the
>_icon at the top right of the console) and turn on Vertex AI:
export PROJECT_ID="your-project-id"
gcloud config set project $PROJECT_ID
gcloud services enable aiplatform.googleapis.com- Save this file as
stages.pyin a folder~/lab319. Each part is one roadmap stage. It usesgemini-3.5-flash-lite(fast and cheap) through Vertex AI, which signs in with your Cloud Shell login: no API key to create or leak.
# stages.py: the AI engineer roadmap in one small file. Each stage is a skill from the roadmap, run for real.
# python stages.py call | prompt | eval | consistency
import json, os, sys
from google import genai
from google.genai import types
PROJECT = os.environ.get("GOOGLE_CLOUD_PROJECT") or os.environ.get("PROJECT_ID")
MODEL = "gemini-3.5-flash-lite"
client = genai.Client(vertexai=True, project=PROJECT, location="global")
def ask(prompt, temperature=0.0, json_schema=None):
cfg = types.GenerateContentConfig(temperature=temperature)
if json_schema:
cfg = types.GenerateContentConfig(temperature=temperature, response_mime_type="application/json", response_schema=json_schema)
r = client.models.generate_content(model=MODEL, contents=prompt, config=cfg)
return r.text, r.usage_metadata
stage = sys.argv[1]
if stage == "call": # stage 2: call a model from code
text, u = ask("In one sentence, what does a load balancer do?")
print(text.strip()); print(f"tokens: {u.prompt_token_count} in, {u.candidates_token_count} out")
elif stage == "prompt": # stage 3: prompts that return data a program can use
schema = types.Schema(type="OBJECT", required=["service", "category", "free_tier"], properties={
"service": types.Schema(type="STRING"), "category": types.Schema(type="STRING", enum=["compute", "storage", "database", "ai", "networking"]),
"free_tier": types.Schema(type="BOOLEAN")})
for s in ["Cloud Run", "BigQuery", "Cloud DNS"]:
text, _ = ask(f"Classify the Google Cloud service '{s}'. Does it have an Always Free tier?", json_schema=schema)
print(json.dumps(json.loads(text)))
elif stage == "eval": # stage 5: measure quality on a small golden set, not by feel
gold = [("What port does HTTPS use by default?", ["443"]), ("Which Linux command lists files?", ["ls"]),
("What does DNS turn a name into?", ["ip", "address"]), ("Name the Kubernetes object that runs a group of identical pods.", ["deployment", "replicaset"]),
("What is 17 times 23?", ["391"]), ("Which file lists packages for pip?", ["requirements.txt"])]
score = 0
for q, keys in gold:
text, _ = ask(q + " Answer in at most 8 words.")
hit = any(k in text.lower() for k in keys); score += hit
print(("PASS " if hit else "FAIL ") + q[:55].ljust(56) + "| " + text.strip().replace("\n", " ")[:50])
print(f"score: {score}/{len(gold)}")
elif stage == "consistency": # stage 6: the same question 3 times, at temperature 0 and 1.5
q = "Suggest a short name for a cloud cost dashboard. Reply with the name only."
for t in (0.0, 1.5):
answers = [ask(q, temperature=t)[0].strip() for _ in range(3)]
print(f"temperature {t}: {len(set(answers))} different answers of 3 -> {answers}")Stage 1: Python and the SDK
Why: a virtual environment (.venv) keeps each project's packages separate. google-genai is Google's Gen AI SDK.
# Stage 1 (Python): a virtual environment with the Google Gen AI SDK. Vertex AI signs in with gcloud: no API key
python3 -m venv .venv && source .venv/bin/activate && pip install -q google-genai==1.* 2>&1 | grep -v "^\[notice\]" | tail -1
python -c "import google.genai; print('google-genai', google.genai.__version__)"google-genai 1.75.0Stage 2: call a model from code
Why: the first real call. Note the token counts: input and output tokens are what you pay for.
# Stage 2: call a model from code
source .venv/bin/activate && python stages.py callA load balancer distributes incoming network traffic across multiple servers to ensure reliability, high availability, and optimal performance.
tokens: 11 in, 21 out11 tokens in, 21 out: one sentence costs almost nothing, but a chat app with long histories can send thousands of tokens per message. Watching tokens is a cost skill.
Stage 3: prompts that return data
Why: programs need data, not paragraphs. A response schema forces the shape (service, category from a fixed list, free_tier as true or false), so json.loads always works.
# Stage 3: prompts that return data (JSON with a schema), not chat text
source .venv/bin/activate && python stages.py prompt{"service": "Cloud Run", "category": "compute", "free_tier": true}
{"service": "BigQuery", "category": "database", "free_tier": true}
{"service": "Cloud DNS", "category": "networking", "free_tier": false}
Three clean JSON objects, with category limited to the values we allowed. This is how an LLM becomes a component in a real system.
Stage 5: evaluate on a golden set
Why: "it looks good" is not a quality measure. A golden set is a small list of questions with known correct answers. Run it after every change to your prompt, model or settings, and compare scores.
# Stage 5: evaluate on a small golden set (6 questions with known answers) and get a score
source .venv/bin/activate && python stages.py evalPASS What port does HTTPS use by default? | HTTPS uses port 443 by default.
PASS Which Linux command lists files? | The `ls` command lists files.
PASS What does DNS turn a name into? | DNS turns a name into an IP address.
PASS Name the Kubernetes object that runs a group of identic | A Kubernetes Deployment.
PASS What is 17 times 23? | 17 times 23 is 391.
PASS Which file lists packages for pip? | requirements.txt (or pyproject.toml)
score: 6/6Score 6/6. Notice the last answer: requirements.txt (or pyproject.toml) still passes because our check looks for the key word. Writing good checks is part of the skill: too strict and correct answers fail, too loose and wrong answers pass. Real projects use 50 to 500 examples and several kinds of checks (exact match, contains, a second model as judge, human review).
Stage 6: consistency, temperature 0 vs 1.5
Why: the same question three times. Temperature controls randomness: 0 picks the most likely words, higher values allow more variety.
# Stage 6: the same question 3 times at temperature 0 and at 1.5. Production apps need predictable answers
source .venv/bin/activate && python stages.py consistency🎓⭐ Recommended by CareerByteCode · Live trainingAI Engineering and Agentic AI Programโน35,400See the training →temperature 0.0: 1 different answers of 3 -> ['CloudPulse', 'CloudPulse', 'CloudPulse']
temperature 1.5: 2 different answers of 3 -> ['CloudPulse', 'CloudSpend', 'CloudPulse']
At temperature 0 the three answers were identical (CloudPulse x3). At 1.5 we got two different answers. For extraction, classification and anything a program reads, use a low temperature; for brainstorming, higher is fine. (Even at 0, models are not guaranteed to be 100 percent identical across runs, which is another reason for evaluation.)
POC: prove it works
- Stage 1: a Python environment with the Gen AI SDK, signed in through Vertex AI with no API key.
- Stage 2: a real model call with its token counts (11 in, 21 out).
- Stage 3: three schema-checked JSON answers a program can read.
- Stage 5: a 6-question golden set scored 6/6.
- Stage 6: temperature 0 gave 1 distinct answer of 3; temperature 1.5 gave 2.
Quiz task: check what you learned
- Why use Vertex AI with your Google Cloud login instead of an API key? (Answer: no secret to store or leak; access is controlled by IAM roles.)
- Your app parses the model's answer and sometimes crashes. What do you change? (Answer: use JSON mode with a response schema (stage 3).)
- You changed the prompt. How do you know it is better? (Answer: run the golden set before and after and compare scores (stage 5).)
- Which temperature for extracting invoice numbers? (Answer: 0 or very low.)
- Which stage teaches the app to answer from your own documents? (Answer: stage 4, retrieval (RAG).)
Clean up
This lab created no cloud resources: only a local virtual environment, which the last step removes.
# Clean up: only local files were created (no cloud resources in this lab)
rm -rf .venv && echo "virtual environment removed"virtual environment removedFAQ
Do I need to learn machine learning maths first? No. AI engineering is mostly software engineering with LLMs. Basic statistics helps for evaluation (stage 5). Deep maths matters if you move toward ML research or model training.
Which model should I learn with? Any good hosted model works; the skills transfer. We use Gemini on Vertex AI because sign-in, quotas and billing live in your Google Cloud project. Start with a small, cheap model and move up only if your evaluation says you need it.
What about LangChain and other frameworks? Learn the raw SDK first (this lab), so you know what frameworks do for you. Then use a framework when it saves real work, for example for agents or complex retrieval.
What portfolio projects should I build? One per stage group: a structured-output tool (stage 3), a RAG chatbot over your own documents (stage 4) with an evaluation set (stage 5), and one deployed app with logging and a cost limit (stage 7).
Where do GPUs come in? For using hosted models, you never touch a GPU. GPUs matter when you train, fine-tune or self-host models, which is the AI infrastructure path.
How do I keep costs safe while learning? Small models, short prompts, a budget alert on the project, and max_output_tokens limits in your code.
Troubleshooting
| You see | Why | Fix |
|---|---|---|
403 PERMISSION_DENIED aiplatform | Your account lacks a Vertex AI role, or the API is off | Enable aiplatform.googleapis.com; grant roles/aiplatform.user |
404 ... Publisher Model ... not found | Wrong model name or location | Use location="global" for gemini-3.5-flash-lite |
json.loads fails on the answer | No JSON mode or schema | Add response_mime_type="application/json" and a schema |
| Answers change between runs | Temperature too high | Use 0 for programs; evaluate on several runs |
429 RESOURCE_EXHAUSTED | Too many calls per minute | Retry with backoff; spread calls out |
What next
- Stage 2 in depth, with an app on Cloud Run: Gemini API for Beginners: Your First AI App on Google Cloud.
- Give an AI assistant your own tools: Model Context Protocol (MCP) Server on Cloud Run.
- Stage 7 basics: CI/CD for Beginners: Build and Deploy on Every Git Push.
- More hands-on posts: ByteBlog. Practice in ByteLabs and see the courses.
Prepared by the CareerByteCode Team from our tested lab, with AI help for the write-up.
⭐ Keep learning with CareerByteCode
Hand-picked to go deeper on this topicGet CareerByteCode Team's next blog by email
One email a week when they publish. Add the weekly Best of ByteBlog too. No spam, unsubscribe with one click.
We email you once to confirm. By subscribing you agree to the privacy policy.
Related blogs
CareerByteCode TeamDevOps Engineer Roadmap 2027 with Hands-On Labs
What you build: a 2027 DevOps roadmap in 7 stages where every stage links to a tested lab, plus a real 15-minute lab that runs the whole loop: code, build, deploy, observe and roll back.
#beginners
CareerByteCode TeamCloud Engineer Fresher Roadmap: 12 Labs in 12 Weeks
What you build: a 12-week cloud engineer plan for freshers with one tested lab every week, plus a real Week 0 readiness check that proves billing, a budget, a free VM and a bucket work.
#beginners
CareerByteCode TeamKubernetes Roadmap: From Pods to Platform Engineering
What you build: an 8-level Kubernetes roadmap from pods to platform engineering, with a real script that checks every level on GKE (pod, service, config, probes, HPA, RBAC, Helm, as code).
#beginners
CareerByteCode TeamHow to Talk About Your Cloud Projects in Interviews
What you build: a STAR answer for cloud interviews from a real lab: a Cloud Run load test (0 failed, p95 64 ms), a broken release caught on its tag URL, and a script that uses only measured numbers.

💬 Comments (0)
Top · NewestSign in to comment