Cloud Logging Hands-On: Find Any Error in Seconds
What you build: log queries, log-based metrics and sinks, in your own Google Cloud account. Every step, command and screenshot explained for beginners.

✍️ Translate this blog into Hindi, Tamil or Telugu
In this lab you use Cloud Logging the way a support engineer does at 2 a.m.: an app writes log lines, you find the one error among them in seconds, you look up one order by its id, you count errors with a log-based metric, and you route every new error to a Pub/Sub topic so other tools hear about it at once. Along the way the routing breaks (a missing permission), and you fix it. You finish by deleting everything.
Who it is for: beginners who run anything on Google Cloud and want to stop scrolling through logs. Time: about 1 hour. Cost: free tier (the first 50 GiB of logs per project each month are free; Pub/Sub and custom metrics stay far inside their free amounts here).
Every command below was run for real by the CareerByteCode Team on a fresh Google Cloud project, and every output and screenshot is the real result we got. We ran the commands from a terminal with the Google Cloud CLI; Cloud Shell in your browser has the same tools, so you can copy and paste each command there.
🧪⭐ Recommended by CareerByteCode · ByteLabs project bundleCloud and DevOps Realtime Starterโน499Build the projects →Part A: logs in plain English
A log line (Google calls it a log entry) is one short note an app writes when something happens: "user logged in", "payment failed". Each entry has a timestamp, a severity (DEBUG, INFO, WARNING, ERROR, CRITICAL ...), a log name (which log it belongs to, like cbc-shop) and a payload: plain text, or JSON with fields you can search.

The tools you will use:
- **Logs Explorer /
gcloud logging read: search with the Logging query language**, for exampleseverity>=ERROR AND jsonPayload.order_id="A-1002". - Log-based metric: a counter that adds 1 for every matching line. Cloud Monitoring stores it as a graph you can alert on.
- Sink: a rule that copies matching lines to a destination (Pub/Sub, BigQuery, Cloud Storage or another log bucket).
- Writer identity: the robot account a sink uses to write to its destination. It needs permission there, like anyone else.
Real-life example: a post office. Every letter (log line) arrives at the sorting room (Cloud Logging). A clerk counts red envelopes (the metric) and a rule sends every red envelope straight to the manager's desk (the sink).
Why JSON? "order_id":"A-1002" can be searched exactly. The same text buried in a sentence can only be searched as words.
Before you start
- A Google Cloud account (the free trial works) and a project with billing linked.
- Open Cloud Shell (the
>_icon at the top of the Google Cloud console). - Save your project id (put your own id between the quotes):
export PROJECT_ID="your-project-id"Part B: the lab
Step 1: turn on the APIs
Why: Logging is on in every project; Pub/Sub (for the sink) and Monitoring (for the metric) may not be.
gcloud config set project $PROJECT_ID
gcloud services enable logging.googleapis.com pubsub.googleapis.com monitoring.googleapis.com
gcloud services list --enabled --filter="name:(logging.googleapis.com OR pubsub.googleapis.com)" --format="value(name)"Updated property [core/project].
Operation "operations/acat.p2-PROJECT_NUMBER-f96600cc-bb49-4dd8-bef9-82494780391f" finished successfully.
projects/PROJECT_NUMBER/services/logging.googleapis.com
projects/PROJECT_NUMBER/services/pubsub.googleapis.comStep 2: write logs like a small shop app would
Why: in a real app your code (or the platform) writes these lines. We write six with gcloud logging write: logins, a cart, two checkouts, a slow page warning and one failed payment.
gcloud logging write cbc-shop "{\"event\":\"login\",\"user\":\"asha\"}" --payload-type=json --severity=INFO
gcloud logging write cbc-shop "{\"event\":\"add_to_cart\",\"user\":\"asha\",\"item\":\"notebook\"}" --payload-type=json --severity=INFO
gcloud logging write cbc-shop "{\"event\":\"checkout\",\"order_id\":\"A-1001\",\"amount\":240}" --payload-type=json --severity=INFO
gcloud logging write cbc-shop "{\"event\":\"slow_page\",\"page\":\"/cart\",\"ms\":2300}" --payload-type=json --severity=WARNING
gcloud logging write cbc-shop "{\"event\":\"payment_failed\",\"order_id\":\"A-1002\",\"reason\":\"gateway timeout\"}" --payload-type=json --severity=ERROR
gcloud logging write cbc-shop "{\"event\":\"checkout\",\"order_id\":\"A-1003\",\"amount\":1450}" --payload-type=json --severity=INFOCreated log entry.
Created log entry.
Created log entry.
Created log entry.
Created log entry.
Created log entry.
Step 3: read every line of the shop log
Why: the query logName=projects/<your project>/logs/cbc-shop keeps only our log. --freshness=15m looks back 15 minutes. The --format turns each entry into one table row with the fields we care about. The first sleep 10 gives the new entries a few seconds to become searchable.
sleep 10
gcloud logging read "logName=projects/$PROJECT_ID/logs/cbc-shop" --freshness=15m --order=asc \
--format="table(timestamp.date(%H:%M:%S),severity,jsonPayload.event,jsonPayload.order_id)"TIMESTAMP SEVERITY EVENT ORDER_ID
22:40:54 INFO login
22:40:57 INFO add_to_cart
22:40:59 INFO checkout A-1001
22:41:02 WARNING slow_page
22:41:05 ERROR payment_failed A-1002
22:41:07 INFO checkout A-1003Step 4: find only the errors
Why: this is the query you will use most. severity>=ERROR also catches CRITICAL, ALERT and EMERGENCY.
gcloud logging read "logName=projects/$PROJECT_ID/logs/cbc-shop AND severity>=ERROR" --freshness=15m \
--format="table(timestamp.date(%H:%M:%S),severity,jsonPayload.event,jsonPayload.order_id,jsonPayload.reason)"TIMESTAMP SEVERITY EVENT ORDER_ID REASON
22:41:05 ERROR payment_failed A-1002 gateway timeout
Six lines became one, with the reason right there: gateway timeout.
Step 5: find one order by its id
Why: a customer calls about order A-1002. Search the JSON field, not the text.
gcloud logging read "logName=projects/$PROJECT_ID/logs/cbc-shop AND jsonPayload.order_id=\"A-1002\"" --freshness=15m --format=json | head -25[
{
"insertId": "14agzukf984w0s",
"jsonPayload": {
"event": "payment_failed",
"order_id": "A-1002",
"reason": "gateway timeout"
},
"logName": "projects/your-project-id/logs/cbc-shop",
"receiveTimestamp": "2026-11-17T22:41:05.119050927Z",
"resource": {
"labels": {
"project_id": "your-project-id"
},
"type": "global"
},
"severity": "ERROR",
"timestamp": "2026-11-17T22:41:05.119050927Z"
}
]
Step 6: count errors with a log-based metric
Why: you do not want to search by hand every hour. A counter turns "how many payment errors right now?" into a graph, and later into an alert.
gcloud logging metrics create shop_errors \
--description="Errors in the shop log" \
--log-filter="logName=projects/$PROJECT_ID/logs/cbc-shop AND severity>=ERROR"
gcloud logging metrics list --format="table(name,filter)"Created [shop_errors].
NAME FILTER
shop_errors logName=projects/your-project-id/logs/cbc-shop AND severity>=ERRORA log-based metric counts only lines written after it exists. The earlier A-1002 error is not counted.
Step 7: a Pub/Sub topic to receive errors
Why: Pub/Sub is a message queue. Other systems (a chat bot, a ticket tool, a function) can listen to the topic. The subscription shop-errors-sub lets us read what arrives.
gcloud pubsub topics create shop-errors
gcloud pubsub subscriptions create shop-errors-sub --topic=shop-errors
gcloud pubsub topics list --format="value(name.basename())"Created topic [projects/your-project-id/topics/shop-errors].
Created subscription [projects/your-project-id/subscriptions/shop-errors-sub].
shop-errorsStep 8: create a sink
Why: the sink says "every shop line with severity ERROR or higher goes to the topic". Read its output carefully: gcloud reminds you that the writer identity needs the Pub/Sub Publisher role on the topic. We skip that on purpose.
gcloud logging sinks create shop-errors-sink pubsub.googleapis.com/projects/$PROJECT_ID/topics/shop-errors \
--log-filter="logName=projects/$PROJECT_ID/logs/cbc-shop AND severity>=ERROR" 2>&1
gcloud logging sinks describe shop-errors-sink --format="yaml(destination,filter,writerIdentity)"Created [https://logging.googleapis.com/v2/projects/your-project-id/sinks/shop-errors-sink].
Please remember to grant `serviceAccount:service-PROJECT_NUMBER@gcp-sa-logging.iam.gserviceaccount.com` the Pub/Sub Publisher role on the topic.
More information about sinks can be found at https://cloud.google.com/logging/docs/export/configure_export
destination: pubsub.googleapis.com/projects/your-project-id/topics/shop-errors
filter: logName=projects/your-project-id/logs/cbc-shop AND severity>=ERROR
writerIdentity: serviceAccount:service-PROJECT_NUMBER@gcp-sa-logging.iam.gserviceaccount.com
Break it: an error happens, but nothing arrives
A new payment fails. We wait a minute and pull the subscription:
📝⭐ Recommended by CareerByteCode · Certification examGoogle Analytics Certification (GA4) PracticeTake the examTake the exam →gcloud logging write cbc-shop "{\"event\":\"payment_failed\",\"order_id\":\"A-1004\",\"reason\":\"card declined\"}" --payload-type=json --severity=ERROR
sleep 60
gcloud pubsub subscriptions pull shop-errors-sub --auto-ack --limit=5 --format="value(message.data)"
echo "messages received: $(gcloud pubsub subscriptions pull shop-errors-sub --auto-ack --limit=5 --format="value(message.messageId)" | wc -l)"Created log entry.
messages received: 0
0 messages. The sink exists and the filter is right, but its writer identity has no permission to publish to the topic, so Google cannot deliver. Nothing tells the app; the error just does not travel.
Fix it: let the sink publish
Give the writer identity the Publisher role on this one topic:
WRITER=$(gcloud logging sinks describe shop-errors-sink --format="value(writerIdentity)")
gcloud pubsub topics add-iam-policy-binding shop-errors --member="$WRITER" --role="roles/pubsub.publisher" --format="table(bindings.role)"Updated IAM policy for topic [shop-errors].
ROLE
['roles/pubsub.publisher']POC: prove a new error arrives on the topic
We give the permission 30 seconds to spread, write a new error and pull the subscription (up to 6 tries, 10 seconds apart):
sleep 30
gcloud logging write cbc-shop "{\"event\":\"payment_failed\",\"order_id\":\"A-1005\",\"reason\":\"gateway timeout\"}" --payload-type=json --severity=ERROR
for i in 1 2 3 4 5 6; do
MSG=$(gcloud pubsub subscriptions pull shop-errors-sub --auto-ack --limit=5 --format="value(message.data)")
[ -n "$MSG" ] && break; sleep 10
done
echo "$MSG" | python3 -c "import sys,json; [print(json.loads(l)[\"severity\"], json.loads(l)[\"jsonPayload\"]) for l in sys.stdin if l.strip()]"Created log entry.
ERROR {'event': 'payment_failed', 'order_id': 'A-1005', 'reason': 'gateway timeout'}
Proof: the error for order A-1005 arrived on the topic, with its severity and every JSON field. Any tool listening to shop-errors now hears about payment failures within seconds.
Now read the counter from Cloud Monitoring (the metric's data is kept in Monitoring, so we call its API with your login token):
curl -s -G -H "Authorization: Bearer $(gcloud auth print-access-token)" \
"https://monitoring.googleapis.com/v3/projects/$PROJECT_ID/timeSeries" \
--data-urlencode "filter=metric.type=\"logging.googleapis.com/user/shop_errors\"" \
--data-urlencode "interval.startTime=$(date -u -d "-30 min" +%Y-%m-%dT%H:%M:%SZ)" \
--data-urlencode "interval.endTime=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
| python3 -c "import sys,json; d=json.load(sys.stdin); pts=[p for s in d.get(\"timeSeries\",[]) for p in s[\"points\"]]; [print(p[\"interval\"][\"endTime\"][11:19], \"errors in that minute:\", p[\"value\"][\"int64Value\"]) for p in sorted(pts, key=lambda p: p[\"interval\"][\"endTime\"])]; print(\"total errors counted:\", sum(int(p[\"value\"][\"int64Value\"]) for p in pts))"22:41:08 errors in that minute: 0
22:42:08 errors in that minute: 0
22:43:08 errors in that minute: 1
22:44:08 errors in that minute: 0
22:45:08 errors in that minute: 0
22:46:08 errors in that minute: 1
total errors counted: 2
The metric counted 2: A-1004 (the one that never reached the topic) and A-1005. Counting and routing are independent: the metric worked even while the sink was broken.
Clean up (delete everything)
gcloud logging sinks delete shop-errors-sink --quiet 2>&1 | tail -1
gcloud logging metrics delete shop_errors --quiet 2>&1 | tail -1
gcloud pubsub subscriptions delete shop-errors-sub --quiet 2>&1 | tail -1
gcloud pubsub topics delete shop-errors --quiet 2>&1 | tail -1
gcloud logging logs delete cbc-shop --quiet 2>&1 | tail -1
echo "sinks: $(gcloud logging sinks list --format="value(name)" | grep -c shop), metrics: $(gcloud logging metrics list --format="value(name)" | wc -l), topics: $(gcloud pubsub topics list --format="value(name)" | wc -l)"Deleted [https://logging.googleapis.com/v2/projects/your-project-id/sinks/shop-errors-sink].
Deleted [shop_errors].
Deleted subscription [projects/your-project-id/subscriptions/shop-errors-sub].
Deleted topic [projects/your-project-id/topics/shop-errors].
Deleted [cbc-shop].
sinks: 0, metrics: 0, topics: 0
FAQ
Can I do all this in the console? Yes. Logging > Logs Explorer takes the same query language; Log-based Metrics and Log Router (for sinks) are in the same menu.
How long are logs kept? By default 30 days in the _Default bucket (400 days for audit logs in _Required). You can keep them longer in your own log bucket or a BigQuery sink.
Why did the metric not count A-1002? A log-based metric starts counting when it is created. It never goes back in time.
What is the difference between a sink and a metric? A metric counts lines. A sink copies them somewhere else. You often use both.
How do I get an email when errors happen? In Cloud Monitoring create an alerting policy on the metric logging/user/shop_errors (for example "more than 0 in 5 minutes") and add an email channel.
Should my app write JSON logs? Yes. On Cloud Run, Cloud Functions and GKE a JSON line printed to standard output becomes a structured entry, with severity read from the JSON.
Troubleshooting
| You see | Why | Fix |
|---|---|---|
Listed 0 items or an empty table | Too short --freshness, a typo in the log name, or the entries are not searchable yet | Wait 10 seconds, use --freshness=1h, check the log name |
| A sink exists but nothing reaches the destination | The writer identity has no permission there | Give it the Publisher (Pub/Sub), Data Editor (BigQuery) or Object Creator (Storage) role |
| Metric shows no data | It counts only lines written after it was created, and data arrives with a small delay | Write a new matching line, wait 2 to 3 minutes |
Invalid filter or a parse error | Quotes in the query | Put field values in double quotes inside the filter: jsonPayload.order_id="A-1002" |
| The JSON field is not searchable | The line was written as text, not JSON | Use --payload-type=json (or print JSON from your app) |
| Too many logs, high volume | Debug lines left on | Lower the log level, or add an exclusion filter in the Log Router |
What next
- Go deeper on traces and metrics: "OpenTelemetry for Beginners: Traces, Metrics and Logs", in this series on ByteBlog.
- Our tested Kubernetes lab: Zero to Kubernetes in 2.5 hours.
- More hands-on posts on ByteBlog, guided projects in ByteLabs and the full courses.
Prepared by the CareerByteCode Team from our tested lab, with AI help for the write-up.
⭐ Keep learning with CareerByteCode
Hand-picked to go deeper on this topicGet CareerByteCode Team's next blog by email
One email a week when they publish. Add the weekly Best of ByteBlog too. No spam, unsubscribe with one click.
We email you once to confirm. By subscribing you agree to the privacy policy.
Related blogs
CareerByteCode TeamMonitor Linux Servers with Node Exporter and Ops Agent
What you build: CPU, memory and disk alerts for VMs, in your own Google Cloud account. Every step, command and screenshot explained for beginners.
#beginners
CareerByteCode TeamCloud Build Hands-On: Pipelines Without Managing Servers
What you build: a build, test and push pipeline in Cloud Build, in your own Google Cloud account. Every step, command and screenshot explained for beginners.
#beginners
CareerByteCode TeamGoogle Cloud VPC for Beginners: Subnets, Routes and Firewalls
What you build: a custom VPC with two subnets and firewall rules, in your own Google Cloud account. Every step, command and screenshot explained for beginners.
#beginners
CareerByteCode TeamCloud Storage Deep Dive: Classes, Lifecycle and Signed URLs
What you build: a private bucket with storage classes, a lifecycle policy, object versions and expiring signed URLs, in your own Google Cloud account. Every step tested.
💬 Comments (0)
Top · NewestSign in to comment