CareerByteCode
ByteBlogby CareerByteCode

Cloud Logging Hands-On: Find Any Error in Seconds

What you build: log queries, log-based metrics and sinks, in your own Google Cloud account. Every step, command and screenshot explained for beginners.

Cloud Logging Hands-On: Find Any Error in Seconds

✍️ Translate this blog into Hindi, Tamil or Telugu

In this lab you use Cloud Logging the way a support engineer does at 2 a.m.: an app writes log lines, you find the one error among them in seconds, you look up one order by its id, you count errors with a log-based metric, and you route every new error to a Pub/Sub topic so other tools hear about it at once. Along the way the routing breaks (a missing permission), and you fix it. You finish by deleting everything.

Who it is for: beginners who run anything on Google Cloud and want to stop scrolling through logs. Time: about 1 hour. Cost: free tier (the first 50 GiB of logs per project each month are free; Pub/Sub and custom metrics stay far inside their free amounts here).

Every command below was run for real by the CareerByteCode Team on a fresh Google Cloud project, and every output and screenshot is the real result we got. We ran the commands from a terminal with the Google Cloud CLI; Cloud Shell in your browser has the same tools, so you can copy and paste each command there.

🧪⭐ Recommended by CareerByteCode · ByteLabs project bundleCloud and DevOps Realtime Starterโ‚น499Build the projects →

Part A: logs in plain English

A log line (Google calls it a log entry) is one short note an app writes when something happens: "user logged in", "payment failed". Each entry has a timestamp, a severity (DEBUG, INFO, WARNING, ERROR, CRITICAL ...), a log name (which log it belongs to, like cbc-shop) and a payload: plain text, or JSON with fields you can search.

Where a log line goes
An app writes a line; Cloud Logging stores it; a metric counts it; a sink routes it to Pub/Sub

The tools you will use:

  • **Logs Explorer / gcloud logging read: search with the Logging query language**, for example severity>=ERROR AND jsonPayload.order_id="A-1002".
  • Log-based metric: a counter that adds 1 for every matching line. Cloud Monitoring stores it as a graph you can alert on.
  • Sink: a rule that copies matching lines to a destination (Pub/Sub, BigQuery, Cloud Storage or another log bucket).
  • Writer identity: the robot account a sink uses to write to its destination. It needs permission there, like anyone else.

Real-life example: a post office. Every letter (log line) arrives at the sorting room (Cloud Logging). A clerk counts red envelopes (the metric) and a rule sends every red envelope straight to the manager's desk (the sink).

Why JSON? "order_id":"A-1002" can be searched exactly. The same text buried in a sentence can only be searched as words.

Before you start

  1. A Google Cloud account (the free trial works) and a project with billing linked.
  2. Open Cloud Shell (the >_ icon at the top of the Google Cloud console).
  3. Save your project id (put your own id between the quotes):
export PROJECT_ID="your-project-id"

Part B: the lab

Step 1: turn on the APIs

Why: Logging is on in every project; Pub/Sub (for the sink) and Monitoring (for the metric) may not be.

gcloud config set project $PROJECT_ID
gcloud services enable logging.googleapis.com pubsub.googleapis.com monitoring.googleapis.com
gcloud services list --enabled --filter="name:(logging.googleapis.com OR pubsub.googleapis.com)" --format="value(name)"
Updated property [core/project].
Operation "operations/acat.p2-PROJECT_NUMBER-f96600cc-bb49-4dd8-bef9-82494780391f" finished successfully.
projects/PROJECT_NUMBER/services/logging.googleapis.com
projects/PROJECT_NUMBER/services/pubsub.googleapis.com

Step 2: write logs like a small shop app would

Why: in a real app your code (or the platform) writes these lines. We write six with gcloud logging write: logins, a cart, two checkouts, a slow page warning and one failed payment.

gcloud logging write cbc-shop "{\"event\":\"login\",\"user\":\"asha\"}" --payload-type=json --severity=INFO
gcloud logging write cbc-shop "{\"event\":\"add_to_cart\",\"user\":\"asha\",\"item\":\"notebook\"}" --payload-type=json --severity=INFO
gcloud logging write cbc-shop "{\"event\":\"checkout\",\"order_id\":\"A-1001\",\"amount\":240}" --payload-type=json --severity=INFO
gcloud logging write cbc-shop "{\"event\":\"slow_page\",\"page\":\"/cart\",\"ms\":2300}" --payload-type=json --severity=WARNING
gcloud logging write cbc-shop "{\"event\":\"payment_failed\",\"order_id\":\"A-1002\",\"reason\":\"gateway timeout\"}" --payload-type=json --severity=ERROR
gcloud logging write cbc-shop "{\"event\":\"checkout\",\"order_id\":\"A-1003\",\"amount\":1450}" --payload-type=json --severity=INFO
Created log entry.
Created log entry.
Created log entry.
Created log entry.
Created log entry.
Created log entry.
Write six log lines
Steps 1 and 2: the APIs are on and six entries are created

Step 3: read every line of the shop log

Why: the query logName=projects/<your project>/logs/cbc-shop keeps only our log. --freshness=15m looks back 15 minutes. The --format turns each entry into one table row with the fields we care about. The first sleep 10 gives the new entries a few seconds to become searchable.

sleep 10
gcloud logging read "logName=projects/$PROJECT_ID/logs/cbc-shop" --freshness=15m --order=asc \
  --format="table(timestamp.date(%H:%M:%S),severity,jsonPayload.event,jsonPayload.order_id)"
TIMESTAMP  SEVERITY  EVENT           ORDER_ID
22:40:54   INFO      login
22:40:57   INFO      add_to_cart
22:40:59   INFO      checkout        A-1001
22:41:02   WARNING   slow_page
22:41:05   ERROR     payment_failed  A-1002
22:41:07   INFO      checkout        A-1003

Step 4: find only the errors

Why: this is the query you will use most. severity>=ERROR also catches CRITICAL, ALERT and EMERGENCY.

gcloud logging read "logName=projects/$PROJECT_ID/logs/cbc-shop AND severity>=ERROR" --freshness=15m \
  --format="table(timestamp.date(%H:%M:%S),severity,jsonPayload.event,jsonPayload.order_id,jsonPayload.reason)"
TIMESTAMP  SEVERITY  EVENT           ORDER_ID  REASON
22:41:05   ERROR     payment_failed  A-1002    gateway timeout
Read and filter
Steps 3 and 4: all six lines, then only the payment_failed error

Six lines became one, with the reason right there: gateway timeout.

Step 5: find one order by its id

Why: a customer calls about order A-1002. Search the JSON field, not the text.

gcloud logging read "logName=projects/$PROJECT_ID/logs/cbc-shop AND jsonPayload.order_id=\"A-1002\"" --freshness=15m --format=json | head -25
[
  {
    "insertId": "14agzukf984w0s",
    "jsonPayload": {
      "event": "payment_failed",
      "order_id": "A-1002",
      "reason": "gateway timeout"
    },
    "logName": "projects/your-project-id/logs/cbc-shop",
    "receiveTimestamp": "2026-11-17T22:41:05.119050927Z",
    "resource": {
      "labels": {
        "project_id": "your-project-id"
      },
      "type": "global"
    },
    "severity": "ERROR",
    "timestamp": "2026-11-17T22:41:05.119050927Z"
  }
]
One order
Step 5: the full entry for order A-1002 (project id blurred)

Step 6: count errors with a log-based metric

Why: you do not want to search by hand every hour. A counter turns "how many payment errors right now?" into a graph, and later into an alert.

gcloud logging metrics create shop_errors \
  --description="Errors in the shop log" \
  --log-filter="logName=projects/$PROJECT_ID/logs/cbc-shop AND severity>=ERROR"
gcloud logging metrics list --format="table(name,filter)"
Created [shop_errors].
NAME         FILTER
shop_errors  logName=projects/your-project-id/logs/cbc-shop AND severity>=ERROR

A log-based metric counts only lines written after it exists. The earlier A-1002 error is not counted.

Step 7: a Pub/Sub topic to receive errors

Why: Pub/Sub is a message queue. Other systems (a chat bot, a ticket tool, a function) can listen to the topic. The subscription shop-errors-sub lets us read what arrives.

gcloud pubsub topics create shop-errors
gcloud pubsub subscriptions create shop-errors-sub --topic=shop-errors
gcloud pubsub topics list --format="value(name.basename())"
Created topic [projects/your-project-id/topics/shop-errors].
Created subscription [projects/your-project-id/subscriptions/shop-errors-sub].
shop-errors

Step 8: create a sink

Why: the sink says "every shop line with severity ERROR or higher goes to the topic". Read its output carefully: gcloud reminds you that the writer identity needs the Pub/Sub Publisher role on the topic. We skip that on purpose.

gcloud logging sinks create shop-errors-sink pubsub.googleapis.com/projects/$PROJECT_ID/topics/shop-errors \
  --log-filter="logName=projects/$PROJECT_ID/logs/cbc-shop AND severity>=ERROR" 2>&1
gcloud logging sinks describe shop-errors-sink --format="yaml(destination,filter,writerIdentity)"
Created [https://logging.googleapis.com/v2/projects/your-project-id/sinks/shop-errors-sink].
Please remember to grant `serviceAccount:service-PROJECT_NUMBER@gcp-sa-logging.iam.gserviceaccount.com` the Pub/Sub Publisher role on the topic.
More information about sinks can be found at https://cloud.google.com/logging/docs/export/configure_export
destination: pubsub.googleapis.com/projects/your-project-id/topics/shop-errors
filter: logName=projects/your-project-id/logs/cbc-shop AND severity>=ERROR
writerIdentity: serviceAccount:service-PROJECT_NUMBER@gcp-sa-logging.iam.gserviceaccount.com
Metric, topic and sink
Steps 6 to 8: the shop_errors metric, the topic, and the sink with its writer identity (ids blurred)

Break it: an error happens, but nothing arrives

A new payment fails. We wait a minute and pull the subscription:

📝⭐ Recommended by CareerByteCode · Certification examGoogle Analytics Certification (GA4) PracticeTake the examTake the exam →
gcloud logging write cbc-shop "{\"event\":\"payment_failed\",\"order_id\":\"A-1004\",\"reason\":\"card declined\"}" --payload-type=json --severity=ERROR
sleep 60
gcloud pubsub subscriptions pull shop-errors-sub --auto-ack --limit=5 --format="value(message.data)"
echo "messages received: $(gcloud pubsub subscriptions pull shop-errors-sub --auto-ack --limit=5 --format="value(message.messageId)" | wc -l)"
Created log entry.
messages received: 0
Nothing arrives
The error is logged, but 0 messages reach the topic

0 messages. The sink exists and the filter is right, but its writer identity has no permission to publish to the topic, so Google cannot deliver. Nothing tells the app; the error just does not travel.

Fix it: let the sink publish

Give the writer identity the Publisher role on this one topic:

WRITER=$(gcloud logging sinks describe shop-errors-sink --format="value(writerIdentity)")
gcloud pubsub topics add-iam-policy-binding shop-errors --member="$WRITER" --role="roles/pubsub.publisher" --format="table(bindings.role)"
Updated IAM policy for topic [shop-errors].
ROLE
['roles/pubsub.publisher']

POC: prove a new error arrives on the topic

We give the permission 30 seconds to spread, write a new error and pull the subscription (up to 6 tries, 10 seconds apart):

sleep 30
gcloud logging write cbc-shop "{\"event\":\"payment_failed\",\"order_id\":\"A-1005\",\"reason\":\"gateway timeout\"}" --payload-type=json --severity=ERROR
for i in 1 2 3 4 5 6; do
  MSG=$(gcloud pubsub subscriptions pull shop-errors-sub --auto-ack --limit=5 --format="value(message.data)")
  [ -n "$MSG" ] && break; sleep 10
done
echo "$MSG" | python3 -c "import sys,json; [print(json.loads(l)[\"severity\"], json.loads(l)[\"jsonPayload\"]) for l in sys.stdin if l.strip()]"
Created log entry.
ERROR {'event': 'payment_failed', 'order_id': 'A-1005', 'reason': 'gateway timeout'}
The error arrives
After the publisher role, the new error A-1005 arrives on the topic with all its fields

Proof: the error for order A-1005 arrived on the topic, with its severity and every JSON field. Any tool listening to shop-errors now hears about payment failures within seconds.

Now read the counter from Cloud Monitoring (the metric's data is kept in Monitoring, so we call its API with your login token):

curl -s -G -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  "https://monitoring.googleapis.com/v3/projects/$PROJECT_ID/timeSeries" \
  --data-urlencode "filter=metric.type=\"logging.googleapis.com/user/shop_errors\"" \
  --data-urlencode "interval.startTime=$(date -u -d "-30 min" +%Y-%m-%dT%H:%M:%SZ)" \
  --data-urlencode "interval.endTime=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  | python3 -c "import sys,json; d=json.load(sys.stdin); pts=[p for s in d.get(\"timeSeries\",[]) for p in s[\"points\"]]; [print(p[\"interval\"][\"endTime\"][11:19], \"errors in that minute:\", p[\"value\"][\"int64Value\"]) for p in sorted(pts, key=lambda p: p[\"interval\"][\"endTime\"])]; print(\"total errors counted:\", sum(int(p[\"value\"][\"int64Value\"]) for p in pts))"
22:41:08 errors in that minute: 0
22:42:08 errors in that minute: 0
22:43:08 errors in that minute: 1
22:44:08 errors in that minute: 0
22:45:08 errors in that minute: 0
22:46:08 errors in that minute: 1
total errors counted: 2
Error count
The log-based metric counted 2 errors: A-1004 and A-1005, one per minute

The metric counted 2: A-1004 (the one that never reached the topic) and A-1005. Counting and routing are independent: the metric worked even while the sink was broken.

Clean up (delete everything)

gcloud logging sinks delete shop-errors-sink --quiet 2>&1 | tail -1
gcloud logging metrics delete shop_errors --quiet 2>&1 | tail -1
gcloud pubsub subscriptions delete shop-errors-sub --quiet 2>&1 | tail -1
gcloud pubsub topics delete shop-errors --quiet 2>&1 | tail -1
gcloud logging logs delete cbc-shop --quiet 2>&1 | tail -1
echo "sinks: $(gcloud logging sinks list --format="value(name)" | grep -c shop), metrics: $(gcloud logging metrics list --format="value(name)" | wc -l), topics: $(gcloud pubsub topics list --format="value(name)" | wc -l)"
Deleted [https://logging.googleapis.com/v2/projects/your-project-id/sinks/shop-errors-sink].
Deleted [shop_errors].
Deleted subscription [projects/your-project-id/subscriptions/shop-errors-sub].
Deleted topic [projects/your-project-id/topics/shop-errors].
Deleted [cbc-shop].
sinks: 0, metrics: 0, topics: 0
Clean up
0 shop sinks, 0 metrics, 0 topics; the cbc-shop log entries are deleted

FAQ

Can I do all this in the console? Yes. Logging > Logs Explorer takes the same query language; Log-based Metrics and Log Router (for sinks) are in the same menu.

How long are logs kept? By default 30 days in the _Default bucket (400 days for audit logs in _Required). You can keep them longer in your own log bucket or a BigQuery sink.

Why did the metric not count A-1002? A log-based metric starts counting when it is created. It never goes back in time.

What is the difference between a sink and a metric? A metric counts lines. A sink copies them somewhere else. You often use both.

How do I get an email when errors happen? In Cloud Monitoring create an alerting policy on the metric logging/user/shop_errors (for example "more than 0 in 5 minutes") and add an email channel.

Should my app write JSON logs? Yes. On Cloud Run, Cloud Functions and GKE a JSON line printed to standard output becomes a structured entry, with severity read from the JSON.

Troubleshooting

You seeWhyFix
Listed 0 items or an empty tableToo short --freshness, a typo in the log name, or the entries are not searchable yetWait 10 seconds, use --freshness=1h, check the log name
A sink exists but nothing reaches the destinationThe writer identity has no permission thereGive it the Publisher (Pub/Sub), Data Editor (BigQuery) or Object Creator (Storage) role
Metric shows no dataIt counts only lines written after it was created, and data arrives with a small delayWrite a new matching line, wait 2 to 3 minutes
Invalid filter or a parse errorQuotes in the queryPut field values in double quotes inside the filter: jsonPayload.order_id="A-1002"
The JSON field is not searchableThe line was written as text, not JSONUse --payload-type=json (or print JSON from your app)
Too many logs, high volumeDebug lines left onLower the log level, or add an exclusion filter in the Log Router

What next

Prepared by the CareerByteCode Team from our tested lab, with AI help for the write-up.

Read next →Monitor Linux Servers with Node Exporter and Ops Agent⏱ 13 min

⭐ Keep learning with CareerByteCode

Hand-picked to go deeper on this topic
News, guides and updates from the CareerByteCode team
🏅 Contributor writer

The CareerByteCode team writes about what is new on careerbytecode.in: courses, labs, exams, events and the community, which is free to join for influencers and content creators in tech.

Get CareerByteCode Team's next blog by email

One email a week when they publish. Add the weekly Best of ByteBlog too. No spam, unsubscribe with one click.

We email you once to confirm. By subscribing you agree to the privacy policy.

💬 Comments (0)

Top · Newest

Sign in to comment

Related blogs

🧪⭐ Recommended by CareerByteCode · ByteLabs project bundleCloud and DevOps Realtime Starterโ‚น499Build the projects →