CareerByteCode
ByteBlogby CareerByteCode

AI Engineer Roadmap: From Python to Production LLM Apps

What you build: a 7-stage AI engineer roadmap from Python to production LLM apps, with a real lab: a Gemini call, schema-checked JSON, a golden-set evaluation (6/6) and a temperature test.

AI Engineer Roadmap: From Python to Production LLM Apps

✍️ Translate this blog into Hindi, Tamil or Telugu

"AI engineer" is one of the fastest-growing job titles, and one of the most confusing to prepare for. It is not a data scientist who invents models, and not a prompt writer. An AI engineer builds reliable software around large language models (LLMs): calls them from code, makes them return data a program can trust, measures their quality, and runs them in production. This roadmap takes you from Python to production LLM apps in seven stages, and the lab in this post runs four of them for real on Google Cloud, including the two most often skipped: evaluation and consistency.

Who it is for: developers, data people and students who know a little Python and want to build with LLMs. Time: about 4 to 6 months at 6 to 8 hours a week for the whole roadmap; about 30 minutes for this lab. Cost: the lab makes about 20 small Gemini calls on Vertex AI, a tiny fraction of a cent.

The lab was run for real by the CareerByteCode Team on a fresh Google Cloud project, and every output is the real answer Gemini gave us.

⭐ Recommended by CareerByteCode · BytePlay courseNVIDIA GPU AI Factory: Architecture, Engineering & Operationsโ‚น40,000Start learning →

Part A: the roadmap

AI engineer roadmap
Seven stages: Python, calling models, structured prompts, retrieval, evaluation, reliability, and production
StageWhat you learnYou can do it when...
1. Python and datavirtual environments, packages, JSON, HTTP, pandas basicsyou can call any web API and handle its JSON
2. Calling a modelSDKs, sign-in without API keys (Vertex AI), tokens and costyour code gets an answer and you know what it cost
3. Prompts that return datasystem instructions, JSON mode, schemasthe answer is valid data your program can use, every time
4. Retrieval (RAG)embeddings, vector search, grounding in your documentsanswers cite your documents and say "I don't know" when they should
5. Evaluationgolden sets, scores, regression tests for promptsyou can say "version B scores 9/10, version A 7/10"
6. Reliabilitytemperature, retries, timeouts, caching, guardrails, cost limitsthe same input gives a predictable answer, and failures are handled
7. ProductionAPIs on Cloud Run, service accounts, logging, monitoring, CI/CDreal users use it, and you can see what it does

Real-life example: a new chef. Stage 1 is knife skills, 2 is turning on the stove, 3 is following a recipe exactly, 4 is using the pantry's own ingredients, 5 is tasting before serving, 6 is getting the same taste every night, and 7 is running a real restaurant.

The two habits that separate AI engineers from hobbyists: they measure (stage 5) and they control randomness (stage 6). The lab shows both.

Part B: the lab, four stages for real

Before you start

  1. A Google Cloud account (the free trial works) and a project.
  2. Open Cloud Shell (the >_ icon at the top right of the console) and turn on Vertex AI:
export PROJECT_ID="your-project-id"
gcloud config set project $PROJECT_ID
gcloud services enable aiplatform.googleapis.com
  1. Save this file as stages.py in a folder ~/lab319. Each part is one roadmap stage. It uses gemini-3.5-flash-lite (fast and cheap) through Vertex AI, which signs in with your Cloud Shell login: no API key to create or leak.
# stages.py: the AI engineer roadmap in one small file. Each stage is a skill from the roadmap, run for real.
#   python stages.py call | prompt | eval | consistency
import json, os, sys
from google import genai
from google.genai import types

PROJECT = os.environ.get("GOOGLE_CLOUD_PROJECT") or os.environ.get("PROJECT_ID")
MODEL = "gemini-3.5-flash-lite"
client = genai.Client(vertexai=True, project=PROJECT, location="global")


def ask(prompt, temperature=0.0, json_schema=None):
    cfg = types.GenerateContentConfig(temperature=temperature)
    if json_schema:
        cfg = types.GenerateContentConfig(temperature=temperature, response_mime_type="application/json", response_schema=json_schema)
    r = client.models.generate_content(model=MODEL, contents=prompt, config=cfg)
    return r.text, r.usage_metadata


stage = sys.argv[1]
if stage == "call":                       # stage 2: call a model from code
    text, u = ask("In one sentence, what does a load balancer do?")
    print(text.strip()); print(f"tokens: {u.prompt_token_count} in, {u.candidates_token_count} out")
elif stage == "prompt":                   # stage 3: prompts that return data a program can use
    schema = types.Schema(type="OBJECT", required=["service", "category", "free_tier"], properties={
        "service": types.Schema(type="STRING"), "category": types.Schema(type="STRING", enum=["compute", "storage", "database", "ai", "networking"]),
        "free_tier": types.Schema(type="BOOLEAN")})
    for s in ["Cloud Run", "BigQuery", "Cloud DNS"]:
        text, _ = ask(f"Classify the Google Cloud service '{s}'. Does it have an Always Free tier?", json_schema=schema)
        print(json.dumps(json.loads(text)))
elif stage == "eval":                     # stage 5: measure quality on a small golden set, not by feel
    gold = [("What port does HTTPS use by default?", ["443"]), ("Which Linux command lists files?", ["ls"]),
            ("What does DNS turn a name into?", ["ip", "address"]), ("Name the Kubernetes object that runs a group of identical pods.", ["deployment", "replicaset"]),
            ("What is 17 times 23?", ["391"]), ("Which file lists packages for pip?", ["requirements.txt"])]
    score = 0
    for q, keys in gold:
        text, _ = ask(q + " Answer in at most 8 words.")
        hit = any(k in text.lower() for k in keys); score += hit
        print(("PASS " if hit else "FAIL ") + q[:55].ljust(56) + "| " + text.strip().replace("\n", " ")[:50])
    print(f"score: {score}/{len(gold)}")
elif stage == "consistency":              # stage 6: the same question 3 times, at temperature 0 and 1.5
    q = "Suggest a short name for a cloud cost dashboard. Reply with the name only."
    for t in (0.0, 1.5):
        answers = [ask(q, temperature=t)[0].strip() for _ in range(3)]
        print(f"temperature {t}: {len(set(answers))} different answers of 3 -> {answers}")

Stage 1: Python and the SDK

Why: a virtual environment (.venv) keeps each project's packages separate. google-genai is Google's Gen AI SDK.

# Stage 1 (Python): a virtual environment with the Google Gen AI SDK. Vertex AI signs in with gcloud: no API key
python3 -m venv .venv && source .venv/bin/activate && pip install -q google-genai==1.* 2>&1 | grep -v "^\[notice\]" | tail -1
python -c "import google.genai; print('google-genai', google.genai.__version__)"
google-genai 1.75.0

Stage 2: call a model from code

Why: the first real call. Note the token counts: input and output tokens are what you pay for.

# Stage 2: call a model from code
source .venv/bin/activate && python stages.py call
A load balancer distributes incoming network traffic across multiple servers to ensure reliability, high availability, and optimal performance.
tokens: 11 in, 21 out

11 tokens in, 21 out: one sentence costs almost nothing, but a chat app with long histories can send thousands of tokens per message. Watching tokens is a cost skill.

Stage 3: prompts that return data

Why: programs need data, not paragraphs. A response schema forces the shape (service, category from a fixed list, free_tier as true or false), so json.loads always works.

# Stage 3: prompts that return data (JSON with a schema), not chat text
source .venv/bin/activate && python stages.py prompt
{"service": "Cloud Run", "category": "compute", "free_tier": true}
{"service": "BigQuery", "category": "database", "free_tier": true}
{"service": "Cloud DNS", "category": "networking", "free_tier": false}
Stages 2 and 3
A first model call with its token counts, then three answers as schema-checked JSON

Three clean JSON objects, with category limited to the values we allowed. This is how an LLM becomes a component in a real system.

Stage 5: evaluate on a golden set

Why: "it looks good" is not a quality measure. A golden set is a small list of questions with known correct answers. Run it after every change to your prompt, model or settings, and compare scores.

# Stage 5: evaluate on a small golden set (6 questions with known answers) and get a score
source .venv/bin/activate && python stages.py eval
PASS What port does HTTPS use by default?                    | HTTPS uses port 443 by default.
PASS Which Linux command lists files?                        | The `ls` command lists files.
PASS What does DNS turn a name into?                         | DNS turns a name into an IP address.
PASS Name the Kubernetes object that runs a group of identic | A Kubernetes Deployment.
PASS What is 17 times 23?                                    | 17 times 23 is 391.
PASS Which file lists packages for pip?                      | requirements.txt (or pyproject.toml)
score: 6/6

Score 6/6. Notice the last answer: requirements.txt (or pyproject.toml) still passes because our check looks for the key word. Writing good checks is part of the skill: too strict and correct answers fail, too loose and wrong answers pass. Real projects use 50 to 500 examples and several kinds of checks (exact match, contains, a second model as judge, human review).

Stage 6: consistency, temperature 0 vs 1.5

Why: the same question three times. Temperature controls randomness: 0 picks the most likely words, higher values allow more variety.

# Stage 6: the same question 3 times at temperature 0 and at 1.5. Production apps need predictable answers
source .venv/bin/activate && python stages.py consistency
🎓⭐ Recommended by CareerByteCode · Live trainingAI Engineering and Agentic AI Programโ‚น35,400See the training →
temperature 0.0: 1 different answers of 3 -> ['CloudPulse', 'CloudPulse', 'CloudPulse']
temperature 1.5: 2 different answers of 3 -> ['CloudPulse', 'CloudSpend', 'CloudPulse']
Evaluation and consistency
The golden set scores 6/6; temperature 0 gives the same name 3 times, 1.5 gives 2 different names

At temperature 0 the three answers were identical (CloudPulse x3). At 1.5 we got two different answers. For extraction, classification and anything a program reads, use a low temperature; for brainstorming, higher is fine. (Even at 0, models are not guaranteed to be 100 percent identical across runs, which is another reason for evaluation.)

POC: prove it works

  • Stage 1: a Python environment with the Gen AI SDK, signed in through Vertex AI with no API key.
  • Stage 2: a real model call with its token counts (11 in, 21 out).
  • Stage 3: three schema-checked JSON answers a program can read.
  • Stage 5: a 6-question golden set scored 6/6.
  • Stage 6: temperature 0 gave 1 distinct answer of 3; temperature 1.5 gave 2.

Quiz task: check what you learned

  1. Why use Vertex AI with your Google Cloud login instead of an API key? (Answer: no secret to store or leak; access is controlled by IAM roles.)
  2. Your app parses the model's answer and sometimes crashes. What do you change? (Answer: use JSON mode with a response schema (stage 3).)
  3. You changed the prompt. How do you know it is better? (Answer: run the golden set before and after and compare scores (stage 5).)
  4. Which temperature for extracting invoice numbers? (Answer: 0 or very low.)
  5. Which stage teaches the app to answer from your own documents? (Answer: stage 4, retrieval (RAG).)

Clean up

This lab created no cloud resources: only a local virtual environment, which the last step removes.

# Clean up: only local files were created (no cloud resources in this lab)
rm -rf .venv && echo "virtual environment removed"
virtual environment removed

FAQ

Do I need to learn machine learning maths first? No. AI engineering is mostly software engineering with LLMs. Basic statistics helps for evaluation (stage 5). Deep maths matters if you move toward ML research or model training.

Which model should I learn with? Any good hosted model works; the skills transfer. We use Gemini on Vertex AI because sign-in, quotas and billing live in your Google Cloud project. Start with a small, cheap model and move up only if your evaluation says you need it.

What about LangChain and other frameworks? Learn the raw SDK first (this lab), so you know what frameworks do for you. Then use a framework when it saves real work, for example for agents or complex retrieval.

What portfolio projects should I build? One per stage group: a structured-output tool (stage 3), a RAG chatbot over your own documents (stage 4) with an evaluation set (stage 5), and one deployed app with logging and a cost limit (stage 7).

Where do GPUs come in? For using hosted models, you never touch a GPU. GPUs matter when you train, fine-tune or self-host models, which is the AI infrastructure path.

How do I keep costs safe while learning? Small models, short prompts, a budget alert on the project, and max_output_tokens limits in your code.

Troubleshooting

You seeWhyFix
403 PERMISSION_DENIED aiplatformYour account lacks a Vertex AI role, or the API is offEnable aiplatform.googleapis.com; grant roles/aiplatform.user
404 ... Publisher Model ... not foundWrong model name or locationUse location="global" for gemini-3.5-flash-lite
json.loads fails on the answerNo JSON mode or schemaAdd response_mime_type="application/json" and a schema
Answers change between runsTemperature too highUse 0 for programs; evaluate on several runs
429 RESOURCE_EXHAUSTEDToo many calls per minuteRetry with backoff; spread calls out

What next

Prepared by the CareerByteCode Team from our tested lab, with AI help for the write-up.

Read next →DevOps Engineer Roadmap 2027 with Hands-On Labs⏱ 9 min

⭐ Keep learning with CareerByteCode

Hand-picked to go deeper on this topic
News, guides and updates from the CareerByteCode team
🏅 Expert writer

The CareerByteCode team writes about what is new on careerbytecode.in: courses, labs, exams, events and the community, which is free to join for influencers and content creators in tech.

Get CareerByteCode Team's next blog by email

One email a week when they publish. Add the weekly Best of ByteBlog too. No spam, unsubscribe with one click.

We email you once to confirm. By subscribing you agree to the privacy policy.

💬 Comments (0)

Top · Newest

Sign in to comment

Related blogs

Readers of this blog also read

⭐ Recommended by CareerByteCode · BytePlay courseNVIDIA GPU AI Factory: Architecture, Engineering & Operationsโ‚น40,000Start learning →