Build a RAG Chatbot on Your Own PDFs with Vertex AI
What you build: a chatbot that answers from your own PDFs with sources, using Vertex AI embeddings and Gemini (no API key), deployed on Cloud Run, in your own Google Cloud account. Every step tested.

✍️ Translate this blog into Hindi, Tamil or Telugu
In this lab you build a RAG chatbot: a small Python app that answers questions from your own PDF files and shows which page each answer came from. It uses two Vertex AI models: an embedding model (text-embedding-005) to find the right pages, and Gemini (gemini-3.5-flash-lite) to write the answer from those pages only. Then you deploy it on Cloud Run with its own service account and sign-in required. There is no API key anywhere: everything signs in with your Google identity.
Who it is for: developers and cloud learners with basic Python who want to understand how "chat with your documents" really works, without a framework hiding the steps. Time: about 1.5 to 2 hours. Cost: a few paise. This whole lab made about 12 Gemini calls and 10 embedding calls (well under USD 0.10 at list prices) plus a few seconds of Cloud Run time, which the free trial credit covers. Delete everything at the end.
Every command below was run for real by the CareerByteCode Team on a fresh Google Cloud project, and every output and screenshot is the real result we got. We typed the commands in a Cloud Shell-like Linux VM (Debian with gcloud, Python 3.11, curl and jq); Cloud Shell in your browser has the same tools ready.
Part A: RAG in plain English
A large language model (LLM) like Gemini knows a lot about the world, but it has never read your company's PDFs. Ask it about your leave policy and it can only guess, or say it does not know. RAG (retrieval-augmented generation) fixes that in two moves: first retrieve the few pieces of your documents that match the question, then generate an answer with those pieces pasted into the prompt.

Words you will use:
- Chunk: a small piece of a document (here about 300 characters, one or two rules). Real-life example: cutting a long book into index cards, one idea per card.
- Embedding: a list of numbers (a vector) that captures the meaning of a text. Texts with similar meaning get similar numbers, even with different words.
text-embedding-005gives 768 numbers per text. - Similarity: how close two vectors point, from 0 (unrelated) to 1 (same meaning). We use cosine similarity. Real-life example: two people pointing in almost the same direction on a map.
- Index: all chunks with their vectors, saved in one file (
index.json). Big systems use a vector database (Vertex AI Vector Search, AlloyDB, Firestore vector search); for a few hundred chunks a file is enough. - Grounding: telling the model to answer only from the given text and to say "I could not find that" otherwise. It is what makes answers checkable.
- Service account: a Google identity for a program. The deployed app signs in to Vertex AI as
rag-bot, which holds only the Vertex AI User role.
Before you start
- A Google Cloud account (the free trial works) and a project with billing linked.
- Open Cloud Shell (the
>_icon at the top right of the Google Cloud console). - Set your project and turn on the APIs this lab uses. Replace the text in quotes with your own project id:
export PROJECT_ID="your-project-id"
gcloud config set project $PROJECT_ID
gcloud services enable aiplatform.googleapis.com run.googleapis.com cloudbuild.googleapis.com artifactregistry.googleapis.comIn Cloud Shell you are already signed in, so the Python SDK finds your login by itself (this is called Application Default Credentials). On your own laptop, run gcloud auth application-default login once instead. Make a folder for the lab and work inside it:
mkdir -p ~/rag-lab && cd ~/rag-lab
export GOOGLE_CLOUD_PROJECT=$PROJECT_IDPart B: the lab
Step 1: Python and the Google Gen AI SDK
Why: a virtual environment keeps this lab's Python packages apart from everything else. google-genai is Google's SDK for Gemini and the embedding models; pypdf reads PDFs; fpdf2 makes our sample PDFs; numpy does the vector maths.
# a Python virtual environment with the Google Gen AI SDK (no API key: it uses your gcloud sign-in)
python3 -m venv .venv && source .venv/bin/activate
pip install -q google-genai==1.* pypdf==5.* fpdf2==2.* numpy==2.* flask==3.1.* 2>&1 | grep -v "^\[notice\]" | tail -2
python -c "import google.genai, pypdf, numpy; print('google-genai', google.genai.__version__, '| pypdf', pypdf.__version__, '| numpy', numpy.__version__)"
gcloud services list --enabled --filter="name:aiplatform.googleapis.com" --format="value(name)"google-genai 1.75.0 | pypdf 5.9.0 | numpy 2.4.6
projects/PROJECT_NUMBER/services/aiplatform.googleapis.comStep 2: three sample PDFs
Why: you need documents to chat with. We make three short policy PDFs for a made-up company, Nimbus Tech (leave, travel, IT security; 2 pages each). With your own PDFs, just copy them into the folder and skip this step. Save this as make_pdfs.py (the full file has three documents; the shape is the same for each):
from fpdf import FPDF
DOCS = {
"leave-policy.pdf": [
("Nimbus Tech leave policy (2026)", [
"Every full-time employee gets 18 days of paid annual leave per calendar year. ...",
"Up to 6 unused days can be carried over to the next year. ...",
]),
("Asking for leave", ["For leave longer than 10 working days in a row, the department head must also approve.", "..."]),
],
# "travel-policy.pdf": [...], "it-security.pdf": [...]
}
for name, pages in DOCS.items():
pdf = FPDF()
for heading, paras in pages:
pdf.add_page(); pdf.set_font("Helvetica", "B", 16); pdf.multi_cell(0, 10, heading); pdf.ln(4)
pdf.set_font("Helvetica", size=12)
for p in paras:
pdf.multi_cell(0, 7, p); pdf.ln(3)
pdf.output(name)
print("made", name, "with", len(pages), "pages")# make 3 sample PDFs (a made-up company's HR, travel and IT rules), then read one back as text
source .venv/bin/activate
python make_pdfs.py
ls -l *.pdf | awk '{print $5, $9}'
python -c "from pypdf import PdfReader; print(PdfReader('travel-policy.pdf').pages[0].extract_text()[:260])"made leave-policy.pdf with 2 pages
made travel-policy.pdf with 2 pages
made it-security.pdf with 2 pages
2031 it-security.pdf
2021 leave-policy.pdf
1975 travel-policy.pdf
Nimbus Tech travel policy
Book all work travel through the travel desk. Flights are economy class for trips shorter than 6 hours.
The daily food allowance is 1,200 rupees in metro cities and 800 rupees in other cities. No receipts
are needed for the food allow
The last lines prove the text can be read back out of the PDF. Scanned PDFs (photos of paper) have no text layer; for those you would add OCR first, for example with Document AI.
Step 3: chunk the PDFs and embed every chunk
Why: this is the "retrieval" half. rag.py holds the shared code. Note the two clients: embeddings on us-central1 and Gemini on the global endpoint, both with vertexai=True, which means "sign in with Google, no key". Save it as rag.py:
# The RAG engine: embed text with a Vertex AI embedding model, find the closest chunks, ask Gemini with them.
# No API key: google-genai signs in with your gcloud login (or the Cloud Run service account).
import os
import numpy as np
from google import genai
from google.genai import types
PROJECT = os.environ["GOOGLE_CLOUD_PROJECT"]
EMBED_MODEL = os.environ.get("EMBED_MODEL", "text-embedding-005")
EMBED_DIM = int(os.environ.get("EMBED_DIM", "768"))
CHAT_MODEL = os.environ.get("CHAT_MODEL", "gemini-3.5-flash-lite")
embedder = genai.Client(vertexai=True, project=PROJECT, location="us-central1")
chat = genai.Client(vertexai=True, project=PROJECT, location="global")
def embed(texts, task):
r = embedder.models.embed_content(model=EMBED_MODEL, contents=texts,
config=types.EmbedContentConfig(task_type=task, output_dimensionality=EMBED_DIM))
return [e.values for e in r.embeddings]
def search(index, question, k=3):
q = np.array(embed([question], "RETRIEVAL_QUERY")[0])
m = np.array([c["vec"] for c in index["chunks"]])
scores = (m @ q) / (np.linalg.norm(m, axis=1) * np.linalg.norm(q))
best = np.argsort(-scores)[:k]
return [(index["chunks"][i], float(scores[i])) for i in best]
RULES = ("Answer the question using ONLY the context below. If the context does not contain the answer, reply exactly: "
"I could not find that in the documents. Keep it to 2 sentences and end with the sources in square brackets, "
"like [leave-policy.pdf p1]. Use plain punctuation, no long dashes.")
def answer(index, question):
hits = search(index, question)
context = "\n\n".join(f"[{c['source']} p{c['page']}] {c['text']}" for c, _ in hits)
r = chat.models.generate_content(model=CHAT_MODEL,
contents=f"Context:\n{context}\n\nQuestion: {question}",
config=types.GenerateContentConfig(system_instruction=RULES, temperature=0.1, max_output_tokens=200))
u = r.usage_metadata
return {"answer": r.text.strip(), "sources": [f"{c['source']} p{c['page']} ({s:.2f})" for c, s in hits],
"tokens": {"in": u.prompt_token_count, "out": u.candidates_token_count}}ingest.py reads each page, cuts it into chunks of about 300 characters at sentence ends, embeds all chunks in one call with task_type="RETRIEVAL_DOCUMENT", and saves index.json:
# Reads every PDF in this folder, splits the text into chunks, embeds them and saves index.json.
import glob, json, sys
from pypdf import PdfReader
import rag
out = sys.argv[1] if len(sys.argv) > 1 else "index.json"
chunks = []
for path in sorted(glob.glob("*.pdf")):
for n, page in enumerate(PdfReader(path).pages, start=1):
text = " ".join(page.extract_text().split())
# chunks of about 300 characters, cut at sentence ends, so one chunk = one or two rules
part = ""
for sentence in text.replace(". ", ".\n").split("\n"):
if len(part) + len(sentence) > 300 and part:
chunks.append({"source": path, "page": n, "text": part.strip()})
part = ""
part += sentence + " "
if part.strip():
chunks.append({"source": path, "page": n, "text": part.strip()})
vectors = rag.embed([c["text"] for c in chunks], "RETRIEVAL_DOCUMENT")
for c, v in zip(chunks, vectors):
c["vec"] = v
json.dump({"model": rag.EMBED_MODEL, "dim": rag.EMBED_DIM, "chunks": chunks}, open(out, "w"))
print(f"{len(chunks)} chunks from {len(set(c['source'] for c in chunks))} PDFs, {len(vectors[0])} numbers per vector, saved {out}")
print("first chunk:", chunks[0]["source"], "p" + str(chunks[0]["page"]), "->", chunks[0]["text"][:90], "...")
print("its vector starts:", [round(x, 4) for x in vectors[0][:5]])# split the PDFs into chunks and turn every chunk into a vector with text-embedding-005 (one API call for all chunks)
source .venv/bin/activate
export GOOGLE_CLOUD_PROJECT=$PROJECT_ID
python ingest.py
ls -l index.json | awk '{print "index.json", $5, "bytes"}'12 chunks from 3 PDFs, 768 numbers per vector, saved index.json
first chunk: it-security.pdf p1 -> Nimbus Tech IT security rules Lock your laptop screen whenever you leave your desk. Laptop ...
its vector starts: [-0.0418, -0.0119, -0.0391, 0.0153, -0.0335]
index.json 206845 bytes
Twelve chunks, each now a list of 768 numbers. You never read these numbers yourself; what matters is that chunks about "leave" sit close together and far from chunks about "passwords".
Step 4: ask questions and get answers with sources
Why: this is the "generation" half. For each question, search() embeds the question (with task_type="RETRIEVAL_QUERY"), scores it against all 12 chunks and keeps the top 3. answer() pastes those 3 chunks into the prompt with the rules: answer only from the context, 2 sentences, cite the source. Save ask.py:
# python ask.py "question" [index file] : prints the answer, the sources (with similarity) and the tokens used
import json, sys
import rag
index = json.load(open(sys.argv[2] if len(sys.argv) > 2 else "index.json"))
r = rag.answer(index, sys.argv[1])
print("Q:", sys.argv[1])
print("A:", r["answer"])
print(" sources:", "; ".join(r["sources"]), "| tokens", r["tokens"])# ask questions: the closest 3 chunks go to Gemini with the question; the answer cites its sources
source .venv/bin/activate
export GOOGLE_CLOUD_PROJECT=$PROJECT_ID
python ask.py "How many days of paid leave do I get, and how many can I carry over?"
python ask.py "What is the food allowance in Chennai?"
python ask.py "I lost my laptop. What should I do?"Q: How many days of paid leave do I get, and how many can I carry over?
A: You get 18 days of paid annual leave per calendar year, and up to 6 unused days can be carried over to the next year [leave-policy.pdf p1]. Any days above 6 are lost on 31 December [leave-policy.pdf p1].
sources: leave-policy.pdf p1 (0.66); leave-policy.pdf p2 (0.66); leave-policy.pdf p2 (0.63) | tokens {'in': 263, 'out': 59}
Q: What is the food allowance in Chennai?
A: The daily food allowance is 1,200 rupees in metro cities and 800 rupees in other cities. No receipts are needed for this allowance. [travel-policy.pdf p1]
sources: travel-policy.pdf p1 (0.64); travel-policy.pdf p1 (0.62); travel-policy.pdf p2 (0.58) | tokens {'in': 240, 'out': 42}
Q: I lost my laptop. What should I do?
A: Report a lost or stolen laptop by calling the IT helpdesk on extension 4040 within 1 hour [it-security.pdf p2].
sources: it-security.pdf p2 (0.61); it-security.pdf p1 (0.50); it-security.pdf p2 (0.48) | tokens {'in': 222, 'out': 32}
Look closely at the Chennai answer: Gemini gives both numbers (metro and other cities) instead of picking one. The PDF says "metro cities" but never says which cities are metros, so the model cannot know Chennai's rate from the documents. Grounding stopped it from guessing; the real fix is in the document (add the list of metro cities), not in the prompt. (In an earlier run of the same lab, Gemini even said in words that the documents do not name Chennai: answers can vary a little from run to run.) Each question used about 220 to 260 input tokens and 25 to 60 output tokens, because only 3 small chunks are sent, not the whole PDFs.
Step 5: a question the PDFs do not answer
Why: a good RAG app must say "I don't know" instead of inventing. We also ask Gemini the leave question without RAG, to see the difference.
# a question the PDFs do not answer, then the same company question to Gemini WITHOUT the documents
source .venv/bin/activate
export GOOGLE_CLOUD_PROJECT=$PROJECT_ID
python ask.py "Does Nimbus Tech give free lunch in the office?"
python - <<'PY'
import rag
r = rag.chat.models.generate_content(model=rag.CHAT_MODEL, contents="How many days of paid annual leave do Nimbus Tech employees get? Answer in 1 sentence.")
print("Without RAG:", r.text.strip())
PYQ: Does Nimbus Tech give free lunch in the office?
A: I could not find that in the documents.
[travel-policy.pdf p1, leave-policy.pdf p1, it-security.pdf p1]
sources: travel-policy.pdf p1 (0.75); leave-policy.pdf p1 (0.71); it-security.pdf p1 (0.62) | tokens {'in': 296, 'out': 36}
Without RAG: I am sorry, but I do not have access to specific information regarding the employee benefits or paid annual leave policies of Nimbus Tech.
The bot refused to guess about free lunch (it still listed the nearest pages as sources; you could hide sources when the answer is "not found"). Without the documents, Gemini honestly says it has no access to Nimbus Tech's policies. With RAG, the same question got "18 days, 6 can be carried over" with the page number.
Break it and fix it: vectors of different sizes
A teammate sets EMBED_DIM=256 "to save space", but the index was built with 768 numbers per vector. Now the question vector and the chunk vectors do not match.
# Break it: a teammate sets EMBED_DIM=256 "to save space", but the index was built with 768 numbers per vector
source .venv/bin/activate
export GOOGLE_CLOUD_PROJECT=$PROJECT_ID
EMBED_DIM=256 python ask.py "What is the hotel limit in a metro city?" 2>&1 | tail -3 scores = (m @ q) / (np.linalg.norm(m, axis=1) * np.linalg.norm(q))
~~^~~
ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0, with gufunc signature (n?,k),(k,m?)->(n?,m?) (size 256 is different from 768)
The maths simply cannot compare a 256-number vector with 768-number vectors, so numpy stops with size 256 is different from 768. A worse version of this bug gives no error: if someone switches to a different embedding model with the same size, the numbers mean different things and the search quietly returns wrong chunks. Fix: questions and chunks must use the same model and the same size. That is why ingest.py saves both in the index:
# Fix it: questions and chunks must use the SAME embedding model and size. Check what the index was built with, then ask again
source .venv/bin/activate
export GOOGLE_CLOUD_PROJECT=$PROJECT_ID
python -c "import json; i = json.load(open('index.json')); print('index built with', i['model'], 'dim', i['dim'])"
EMBED_DIM=768 python ask.py "What is the hotel limit in a metro city?"index built with text-embedding-005 dim 768
Q: What is the hotel limit in a metro city?
A: The hotel limit per night is 6,000 rupees in metro cities [travel-policy.pdf p1].
sources: travel-policy.pdf p1 (0.72); travel-policy.pdf p2 (0.54); travel-policy.pdf p1 (0.51) | tokens {'in': 215, 'out': 25}🎓⭐ Recommended by CareerByteCode · Live trainingLearn AI, Build 12+ AI Agentsโน10,620See the training →
Rule: if you change the embedding model or size, rebuild the whole index (run ingest.py again).
Step 6: deploy the chatbot to Cloud Run
Why: so other people and apps can use it. app.py is a small Flask web app with GET /ask?q=.... It runs as a new service account rag-bot that may only call Vertex AI (least privilege), and --no-allow-unauthenticated means callers must sign in. Save app.py, requirements.txt and a one-line Procfile (it tells Cloud Run how to start the app):
# The chatbot as a small web app for Cloud Run: GET /ask?q=your+question
import json
from flask import Flask, request, jsonify
import rag
app = Flask(__name__)
INDEX = json.load(open("index.json"))
@app.get("/")
def home():
return f"RAG chatbot is running with {len(INDEX['chunks'])} chunks. Try /ask?q=your+question\n"
@app.get("/ask")
def ask():
q = request.args.get("q", "").strip()
if not q:
return jsonify(error="add ?q=your question"), 400
return jsonify(question=q, **rag.answer(INDEX, q))# requirements.txt
flask==3.1.*
gunicorn==23.*
google-genai==1.*
numpy==2.*
# Procfile
web: gunicorn --bind :$PORT --workers 1 --threads 4 app:appgcloud run deploy --source . sends the folder to Cloud Build, which builds a container image for you (no Dockerfile needed), stores it in Artifact Registry and starts it on Cloud Run. The first deploy asks to create the cloud-run-source-deploy repository; --quiet says yes.
# deploy the chatbot to Cloud Run with its own service account (Vertex AI User only), sign-in required
gcloud iam service-accounts create rag-bot --display-name="RAG chatbot on Cloud Run" 2>&1 | tail -1
until gcloud iam service-accounts describe rag-bot@$PROJECT_ID.iam.gserviceaccount.com > /dev/null 2>&1; do sleep 5; done
gcloud projects add-iam-policy-binding $PROJECT_ID --member="serviceAccount:rag-bot@$PROJECT_ID.iam.gserviceaccount.com" \
--role=roles/aiplatform.user --condition=None --format="value(etag)" > /dev/null && echo "granted roles/aiplatform.user to rag-bot"
mkdir -p app && cp app.py rag.py index.json requirements.txt Procfile app/ && cd app
gcloud run deploy rag-bot --source . --region=us-east1 --no-allow-unauthenticated \
--service-account=rag-bot@$PROJECT_ID.iam.gserviceaccount.com \
--set-env-vars=GOOGLE_CLOUD_PROJECT=$PROJECT_ID --memory=512Mi --max-instances=1 --quiet 2>&1 | tail -4Service account email: rag-bot@your-project-id.iam.gserviceaccount.com
Updated IAM policy for project [your-project-id].
granted roles/aiplatform.user to rag-bot
Done.
Service [rag-bot] revision [rag-bot-00001-hrk] has been deployed and is serving 100 percent of traffic.
Service URL: https://rag-bot-PROJECT_NUMBER.us-east1.run.app
Proxy locally with: gcloud run services proxy rag-bot --region us-east1 --project your-project-idStep 7: call the deployed chatbot
Why: prove that strangers are refused and signed-in callers get answers. gcloud auth print-identity-token makes a short-lived token that says who you are.
# call the deployed chatbot: first without a token (refused), then with your identity token
URL=$(gcloud run services describe rag-bot --region=us-east1 --format="value(status.url)")
curl -s -o /dev/null -w "no token: HTTP %{http_code}\n" "$URL/ask?q=hello"
TOKEN=$(gcloud auth print-identity-token --audiences=$URL)
curl -s -H "Authorization: Bearer $TOKEN" "$URL/"
curl -s -G -H "Authorization: Bearer $TOKEN" "$URL/ask" --data-urlencode "q=Who must approve 12 working days of leave?" | jq .no token: HTTP 403
RAG chatbot is running with 12 chunks. Try /ask?q=your+question
{
"answer": "Leave longer than 10 working days in a row must be approved by both your manager and the department head [leave-policy.pdf p2].",
"question": "Who must approve 12 working days of leave?",
"sources": [
"leave-policy.pdf p2 (0.68)",
"leave-policy.pdf p2 (0.61)",
"leave-policy.pdf p1 (0.59)"
],
"tokens": {
"in": 256,
"out": 31
}
}
12 working days is "longer than 10 working days in a row", so both the manager and the department head approve. The bot combined two rules from the same page correctly.
POC: prove it works
- Three PDFs became 12 chunks and a 768-number vector each, in one embedding call.
- Questions were answered from the right page, with sources, in about 250 input tokens each.
- A question not in the PDFs got "I could not find that in the documents", not a made-up answer.
- A vector size mismatch broke the search; using the same model and size fixed it.
- The chatbot runs on Cloud Run as a least-privilege service account, sign-in required, with no API key anywhere.
Clean up (delete everything)
Why: Cloud Run costs nothing while idle, but the image repository and the upload bucket keep small storage charges, and an unused service account is a security risk.
# delete the service, its image repository and source bucket, the service account and its role
gcloud run services delete rag-bot --region=us-east1 --quiet 2>&1 | tail -1
gcloud artifacts repositories delete cloud-run-source-deploy --location=us-east1 --quiet 2>&1 | tail -1
for b in $(gcloud storage buckets list --format="value(name)" | grep -E "run-sources|cloudbuild"); do gcloud storage rm -r gs://$b --quiet > /dev/null 2>&1 && echo "deleted bucket (source upload)"; done
gcloud projects remove-iam-policy-binding $PROJECT_ID --member="serviceAccount:rag-bot@$PROJECT_ID.iam.gserviceaccount.com" \
--role=roles/aiplatform.user --condition=None --format="value(etag)" > /dev/null && echo "removed role"
gcloud iam service-accounts delete rag-bot@$PROJECT_ID.iam.gserviceaccount.com --quiet 2>&1 | tail -1
rm -rf app .venv index.json *.pdf
echo "Cloud Run services left: $(gcloud run services list --format="value(name)" | wc -l)"
echo "repositories left: $(gcloud artifacts repositories list --format="value(name)" 2>/dev/null | wc -l)"
echo "buckets left: $(gcloud storage buckets list --format="value(name)" | wc -l)"Deleted service [rag-bot].
Deleted repository [cloud-run-source-deploy].
deleted bucket (source upload)
Updated IAM policy for project [your-project-id].
removed role
deleted service account [rag-bot@your-project-id.iam.gserviceaccount.com]
Cloud Run services left: 0
repositories left: 0
buckets left: 0
If you made a project only for this lab, you can also shut the whole project down: gcloud projects delete $PROJECT_ID.
FAQ
Why not just paste all my PDFs into the prompt? For 3 short files you could. For hundreds of pages it gets slow and costly (you pay for every input token on every question), and models answer worse when the right sentence is buried in a long prompt. RAG sends only the 3 best chunks.
Which chunk size is best? There is no single answer. Start with 300 to 1,000 characters with sentence ends, test real questions, and look at the sources: if the right rule is cut in half, make chunks bigger or overlap them.
Do I need a vector database? Not for a few thousand chunks; a file or a numpy array is fine. For large or changing document sets, use Vertex AI Vector Search, AlloyDB or Cloud SQL with pgvector, or Firestore vector search.
Is my data used to train Google's models? Google Cloud's terms for Vertex AI say customer data is not used to train its models without permission. Check the current terms for your use.
**Why gemini-3.5-flash-lite?** It is the smallest, cheapest current Gemini model and is good at "answer from this text" tasks. Use a bigger model when questions need more reasoning.
Can I use Vertex AI RAG Engine or Vertex AI Search instead? Yes, they do the chunking, embedding and search for you. Building it by hand once, as here, shows what those services do inside.
Troubleshooting
| You see | Why | Fix |
|---|---|---|
DefaultCredentialsError | The SDK cannot find your login | In Cloud Shell it is automatic; elsewhere run gcloud auth application-default login |
403 PERMISSION_DENIED on Vertex AI | API off or the account lacks the role | gcloud services enable aiplatform.googleapis.com; give the account roles/aiplatform.user |
404 ... model not found | Model name or location wrong | text-embedding-005 on us-central1, Gemini on global, as in rag.py |
size 256 is different from 768 | Question and chunks embedded with different sizes or models | Use the same model and size; rebuild the index after any change |
| Answer says "could not find" but the PDF has it | The right chunk was not in the top 3 | Check sources; try bigger chunks or k=5 |
Cloud Run 403 Forbidden | No identity token, or no Invoker role | Send Authorization: Bearer $(gcloud auth print-identity-token); grant roles/run.invoker to the caller |
What next
- New to Gemini on Google Cloud? Start with Gemini API for Beginners: Your First AI App on Google Cloud.
- Give your chatbot tools for AI agents: Model Context Protocol (MCP) Server on Cloud Run.
- Keep keys and passwords out of code: Secret Manager: Store Passwords and API Keys Safely.
- Understand the roles we gave
rag-bot: Google Cloud IAM for Beginners. - More hands-on posts: ByteBlog. Practice in ByteLabs and see the courses.
Prepared by the CareerByteCode Team from our tested lab, with AI help for the write-up.
⭐ Keep learning with CareerByteCode
Hand-picked to go deeper on this topicGet CareerByteCode Team's next blog by email
One email a week when they publish. Add the weekly Best of ByteBlog too. No spam, unsubscribe with one click.
We email you once to confirm. By subscribing you agree to the privacy policy.
Related blogs
CareerByteCode TeamVector Search with AlloyDB pgvector and Gemini
What you build: semantic search with AlloyDB and pgvector, Vertex AI embeddings, an HNSW index and Gemini answers that cite their source. Every step tested.
#gemini
CareerByteCode TeamGenerative AI Project for Your Resume on Google Cloud
What you build: an AI support-ticket triage API with Gemini on Cloud Run, structured output, a measured test set and a prompt-injection test. Every step tested.
#gemini
CareerByteCode TeamText-to-SQL: Ask BigQuery Questions in Plain English
What you build: a chat box where Gemini turns plain-English questions into BigQuery SQL, with three safety nets (rules, a SELECT-only check, a read-only account), on Cloud Run. Every step tested.
#gemini
CareerByteCode TeamModel Context Protocol (MCP) Server on Cloud Run
What you build: your own MCP server with one tool, on Cloud Run with sign-in required, tested with real initialize, tools/list and tools/call requests, in your own Google Cloud account.
💬 Comments (0)
Top · NewestSign in to comment