CareerByteCode
ByteBlogby CareerByteCode

Bash Scripting for DevOps: Automate Boring Tasks

What you build: five Bash scripts (arguments, checks, backups with rotation, a log report) made safe with set -euo pipefail and scheduled with cron, on a free Linux VM. Every step tested.

Bash Scripting for DevOps: Automate Boring Tasks

✍️ Translate this blog into Hindi, Tamil or Telugu

In this lab you turn boring, repeated server tasks into Bash scripts: a greeting that takes arguments, a disk check that returns a proper exit code, a backup that keeps only the newest 3 copies, and a log report that reads a file line by line. You make them safe, you meet a real quoting bug and fix it, and you let cron run the backup by itself. Everything runs on a free Linux VM in your own Google Cloud project, and you delete it at the end.

Who it is for: beginners who know basic Linux commands (ls, cd, cat, grep) and want to automate. If you are new to Linux, do our tested lab "Linux for Cloud Beginners: 50 Commands on a Free VM" (on ByteBlog) first. Time: about 1 to 1.5 hours. Cost: free tier (one e2-micro VM in us-east1, us-west1 or us-central1 is free every month; our VM has no public IP).

Every command below was run for real by the CareerByteCode Team on a fresh Debian 12 VM, and every output and screenshot is the real result we got. To record the outputs exactly, we copied the scripts and each group of commands to the VM and ran each group with bash, which printed each command (after $ ) before its output. You simply type the commands one by one after you log in.

🧪⭐ Recommended by CareerByteCode · ByteLabs project bundleCloud and DevOps Realtime Starterโ‚น499Build the projects →

Part A: Bash scripting in plain English

A Bash script is a text file with the commands you would type, one after another, so you can run them all with one word. Add variables, checks and loops, and the script makes decisions for you.

From a boring task to an automatic script
Manual task, script, safe script, functions and loops, then cron

Eight ideas you will use in this lab:

  • Shebang #!/bin/bash: the first line says which program runs the file.
  • Variable: name="Priya", then use it as "$name". ${1:-default} means "the first argument, or this default if none was given".
  • Arguments: words after the script name. $1 is the first, $# is how many, $0 is the script itself.
  • Exit code: every command ends with a number. 0 means success, anything else means a problem. $? holds the last one. Real-life example: a green or red light at the end of a car wash.
  • **set -euo pipefail**: stop on the first error (-e), treat an unset variable as an error (-u), and fail a pipe if any part fails (pipefail). Put it at the top of almost every script.
  • Quotes: "$file" keeps a name with spaces as one word. Without quotes, sales q1.txt becomes two words. This causes more script bugs than anything else.
  • Function: a named group of commands you can call, like make_backup.
  • cron: the Linux scheduler. * * * * * means every minute; 0 2 * * * means every day at 02:00.

Before you start

  1. A Google Cloud account (the free trial works) and a project with billing linked. Create one at https://console.cloud.google.com/projectcreate if you do not have one.
  2. Open Cloud Shell (the >_ icon at the top right of the Google Cloud console).
  3. Save your project id in a variable. Replace the text in quotes with your own project id:
export PROJECT_ID="your-project-id"
  1. Make a folder called scripts in Cloud Shell and save the five scripts from Part C in it (Cloud Shell has an editor: click Open Editor).

Part B: the lab machine

Step 1: create a free VM and wait for SSH

Why: an e2-micro VM with Debian 12 and no public IP. The firewall rule allows SSH only from Google's IAP range, and the until line waits until the new VM accepts SSH.

gcloud config set project $PROJECT_ID
gcloud compute instances create bash-lab --zone=us-east1-b --machine-type=e2-micro \
  --image-family=debian-12 --image-project=debian-cloud --boot-disk-size=10GB --boot-disk-type=pd-standard --no-address
gcloud compute firewall-rules create bash-lab-ssh-from-iap --network=default --direction=INGRESS \
  --allow=tcp:22 --source-ranges=35.235.240.0/20 --format="value(name)"
until gcloud compute ssh bash-lab --zone=us-east1-b --tunnel-through-iap --command="echo ready" 2>/dev/null | grep -q ready; do sleep 10; done
echo "SSH is ready"
Updated property [core/project].
WARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.
Created [https://www.googleapis.com/compute/v1/projects/your-project-id/zones/us-east1-b/instances/bash-lab].
NAME      ZONE        MACHINE_TYPE  PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP  STATUS
bash-lab  us-east1-b  e2-micro                   10.142.0.6                RUNNING
Creating firewall...
..Created [https://www.googleapis.com/compute/v1/projects/your-project-id/global/firewalls/bash-lab-ssh-from-iap].
done.
bash-lab-ssh-from-iap
SSH is ready

Step 2: copy the scripts to the VM

Why: gcloud compute scp copies files over the same secure IAP connection. The second command lists what arrived. (We also copied the small groups folder we used to record the outputs.)

gcloud compute scp --recurse scripts groups bash-lab: --zone=us-east1-b --tunnel-through-iap 2>&1 | grep -v "^ *$" | tail -3
gcloud compute ssh bash-lab --zone=us-east1-b --tunnel-through-iap --command="ls -1 ~/scripts"
g6.txt                    | 0 kB |   0.1 kB/s | ETA: 00:00:00 | 100%
g7.run                    | 0 kB |   0.5 kB/s | ETA: 00:00:00 | 100%
g7.txt                    | 0 kB |   0.2 kB/s | ETA: 00:00:00 | 100%
backup.sh
disk-check.sh
hello.sh
log-report.sh
report-files-fixed.sh
report-files.sh
VM and copy
Steps 1 and 2: the VM is ready and the five scripts (plus the fixed one) are on it

Now log in with gcloud compute ssh bash-lab --zone=us-east1-b --tunnel-through-iap and type the groups below.

Part C: five scripts

Script 1: hello.sh (variables and arguments)

#!/bin/bash
# hello.sh: variables, arguments and a default value
name="${1:-DevOps learner}"
today=$(date +%A)
echo "Hello, $name! Today is $today."
echo "This script is $0 and got $# argument(s)."
cd scripts
chmod +x *.sh
ls -l
./hello.sh
./hello.sh Priya
bash -n backup.sh && echo "syntax OK"
$ cd scripts
$ chmod +x *.sh
$ ls -l
total 24
-rwxr-xr-x 1 you you 663 Nov 18 07:37 backup.sh
-rwxr-xr-x 1 you you 351 Nov 18 07:37 disk-check.sh
-rwxr-xr-x 1 you you 198 Nov 18 07:37 hello.sh
-rwxr-xr-x 1 you you 455 Nov 18 07:37 log-report.sh
-rwxr-xr-x 1 you you 225 Nov 18 07:37 report-files-fixed.sh
-rwxr-xr-x 1 you you 197 Nov 18 07:37 report-files.sh
$ ./hello.sh
Hello, DevOps learner! Today is Wednesday.
This script is ./hello.sh and got 0 argument(s).
$ ./hello.sh Priya
Hello, Priya! Today is Wednesday.
This script is ./hello.sh and got 1 argument(s).
$ bash -n backup.sh && echo "syntax OK"
syntax OK
First scripts
chmod +x makes the scripts runnable; hello.sh with and without an argument; bash -n checks syntax

chmod +x adds the run permission. Without an argument, ${1:-DevOps learner} used the default. bash -n reads a script and reports syntax errors without running it: a free safety check before you run anything new.

Script 2: disk-check.sh (if, exit codes)

Why: a check that monitoring tools, cron or a CI pipeline can understand: it prints a line and returns 0 (fine) or 1 (problem).

#!/bin/bash
# disk-check.sh: warn when the root disk is fuller than a limit (default 80 percent)
set -euo pipefail
limit="${1:-80}"
used=$(df --output=pcent / | tail -1 | tr -dc '0-9')
if [ "$used" -ge "$limit" ]; then
  echo "WARNING: disk / is ${used}% full (limit ${limit}%)"
  exit 1
else
  echo "OK: disk / is ${used}% full (limit ${limit}%)"
fi
  • df --output=pcent / prints the percent used of the root disk; tr -dc '0-9' keeps only the digits.
  • [ "$used" -ge "$limit" ] means "used is greater than or equal to limit".
cd scripts
./disk-check.sh
echo "exit code: $?"
./disk-check.sh 10 || echo "exit code: $?"
$ cd scripts
$ ./disk-check.sh
OK: disk / is 23% full (limit 80%)
$ echo "exit code: $?"
exit code: 0
$ ./disk-check.sh 10 || echo "exit code: $?"
WARNING: disk / is 23% full (limit 10%)
exit code: 1
Disk check
23 percent used: OK with exit code 0; with a 10 percent limit: WARNING and exit code 1

|| means "run the next command only if the first one failed", so we see exit code: 1 only for the warning.

Script 3: backup.sh (functions, dates, rotation)

Why: backups that pile up fill the disk. This script makes a dated backup and then deletes all but the newest 3.

#!/bin/bash
# backup.sh: zip a folder with the date and time in the name, keep only the newest 3 backups
set -euo pipefail
src="${1:?usage: backup.sh <folder>}"
dest="$HOME/backups"
keep=3
mkdir -p "$dest"
make_backup() {
  local file="$dest/$(basename "$src")-$(date +%Y%m%d-%H%M%S).tar.gz"
  tar -czf "$file" -C "$(dirname "$src")" "$(basename "$src")"
  echo "created $(basename "$file") ($(du -h "$file" | cut -f1))"
}
remove_old() {
  ls -1t "$dest"/*.tar.gz | tail -n +$((keep + 1)) | while read -r old; do
    rm -f "$old"; echo "removed old backup $(basename "$old")"
  done
}
make_backup
remove_old
echo "backups kept: $(ls -1 "$dest"/*.tar.gz | wc -l)"
  • ${1:?usage: ...} stops the script with a message if no folder was given.
  • $(date +%Y%m%d-%H%M%S) puts the date and time in the name, so names never clash and sort by time.
  • ls -1t ... | tail -n +4 lists newest first and keeps everything from the 4th line on: those are the old ones to remove.
cd scripts
mkdir -p ~/project && echo "app config v1" > ~/project/app.conf
for i in 1 2 3 4; do ./backup.sh ~/project; sleep 1; done
ls -1 ~/backups
$ cd scripts
$ mkdir -p ~/project && echo "app config v1" > ~/project/app.conf
$ for i in 1 2 3 4; do ./backup.sh ~/project; sleep 1; done
created project-20261118-073802.tar.gz (4.0K)
backups kept: 1
created project-20261118-073803.tar.gz (4.0K)
backups kept: 2
created project-20261118-073804.tar.gz (4.0K)
backups kept: 3
created project-20261118-073805.tar.gz (4.0K)
removed old backup project-20261118-073802.tar.gz
backups kept: 3
$ ls -1 ~/backups
project-20261118-073803.tar.gz
project-20261118-073804.tar.gz
project-20261118-073805.tar.gz
Backup
Four backups made; on the 4th run the oldest one is removed, 3 are kept

Script 4: log-report.sh (while read, case)

Why: read a log line by line, count each kind of status and name every server error. The exit code tells cron or a pipeline whether to raise an alarm.

#!/bin/bash
# log-report.sh: read a web log line by line and count requests by status class
set -euo pipefail
log="${1:?usage: log-report.sh <logfile>}"
ok=0; client=0; server=0
while read -r host status path; do
  case "$status" in
    2??) ok=$((ok + 1)) ;;
    4??) client=$((client + 1)) ;;
    5??) server=$((server + 1)); echo "server error on $host for $path" ;;
  esac
done < "$log"
echo "2xx: $ok  4xx: $client  5xx: $server"
[ "$server" -eq 0 ]
  • while read -r host status path splits each line into three variables.
  • case picks a branch by pattern: 2?? means any three-character code starting with 2.
  • The last line [ "$server" -eq 0 ] is a test: the script's exit code is 0 only when there were no 5xx errors.
cd scripts
printf "web-01 200 /home\nweb-02 500 /checkout\nweb-01 200 /courses\nweb-03 404 /old\nweb-02 500 /checkout\n" > access.log
./log-report.sh access.log || echo "the report found server errors (exit code $?)"
$ cd scripts
$ printf "web-01 200 /home\nweb-02 500 /checkout\nweb-01 200 /courses\nweb-03 404 /old\nweb-02 500 /checkout\n" > access.log
$ ./log-report.sh access.log || echo "the report found server errors (exit code $?)"
server error on web-02 for /checkout
server error on web-02 for /checkout
2xx: 2  4xx: 1  5xx: 2
the report found server errors (exit code 1)
Log report
Two server errors named, the counts by class, and exit code 1

Break it and fix it: a file name with a space

Script 5 prints the size of every file in a folder. It looks fine and works for most files:

🎓⭐ Recommended by CareerByteCode · Live trainingCloud & DevOps Live Cohort Internshipโ‚น2,360See the training →
#!/bin/bash
# report-files.sh: print the size of every file in a folder (has a bug on purpose: no quotes)
set -euo pipefail
for f in $(ls "$1"); do
  echo "$(du -b $1/$f | cut -f1) bytes  $f"
done

Now give it a folder where one file name has a space:

cd scripts
mkdir -p ~/reports && echo "q1" > ~/reports/"sales q1.txt" && echo "notes" > ~/reports/notes.txt
./report-files.sh ~/reports
$ cd scripts
$ mkdir -p ~/reports && echo "q1" > ~/reports/"sales q1.txt" && echo "notes" > ~/reports/notes.txt
$ ./report-files.sh ~/reports
6 bytes  notes.txt
du: cannot access '/home/you/reports/sales': No such file or directory
 bytes  sales
du: cannot access '/home/you/reports/q1.txt': No such file or directory
 bytes  q1.txt
Break it
sales q1.txt is split into sales and q1.txt; du cannot find either

Two mistakes worked together: $(ls "$1") splits names on spaces, and $1/$f without quotes splits them again. Notice also that set -e did not stop the script: an error inside $( ) that is used as an argument to echo does not end a script. set -e is a safety net with holes, not a guarantee. Fix: loop over a glob ("$1"/*) and quote every variable:

cd scripts
diff report-files.sh report-files-fixed.sh
./report-files-fixed.sh ~/reports
$ cd scripts
$ diff report-files.sh report-files-fixed.sh
2c2
< # report-files.sh: print the size of every file in a folder (has a bug on purpose: no quotes)
---
> # report-files-fixed.sh: the same report, safe for file names with spaces (a glob, and every variable in quotes)
4,5c4,5
< for f in $(ls "$1"); do
<   echo "$(du -b $1/$f | cut -f1) bytes  $f"
---
> for f in "$1"/*; do
>   echo "$(du -b "$f" | cut -f1) bytes  $(basename "$f")"
$ ./report-files-fixed.sh ~/reports
6 bytes  notes.txt
3 bytes  sales q1.txt
Fix it
The diff shows the two changes; the fixed script reports sales q1.txt correctly

Rule to remember: never loop over $(ls ...), and put every $variable in double quotes unless you have a reason not to. The free tool ShellCheck finds both mistakes automatically.

POC: cron runs the backup by itself

The last step of automation: nobody has to remember. We add a cron job that runs backup.sh every minute, wait 75 seconds, read its log and remove the job again.

cd scripts
(crontab -l 2>/dev/null; echo "* * * * * $HOME/scripts/backup.sh $HOME/project >> $HOME/backup-cron.log 2>&1") | crontab -
crontab -l
sleep 75
cat ~/backup-cron.log
crontab -r && echo "cron job removed"
$ cd scripts
$ (crontab -l 2>/dev/null; echo "* * * * * $HOME/scripts/backup.sh $HOME/project >> $HOME/backup-cron.log 2>&1") | crontab -
$ crontab -l
* * * * * /home/you/scripts/backup.sh /home/you/project >> /home/you/backup-cron.log 2>&1
$ sleep 75
$ cat ~/backup-cron.log
created project-20261118-073901.tar.gz (4.0K)
removed old backup project-20261118-073803.tar.gz
backups kept: 3
created project-20261118-074001.tar.gz (4.0K)
removed old backup project-20261118-073804.tar.gz
backups kept: 3
$ crontab -r && echo "cron job removed"
cron job removed
cron
cron ran the backup at 07:39:01 and 07:40:01 by itself, each time keeping only 3 backups

Proof: the log shows two runs at the start of two minutes (07:39:01 and 07:40:01) that nobody started by hand, and the rotation kept the folder at 3 backups. In real life you would use 0 2 * * * (every night at 02:00), not every minute.

  • crontab -l lists your jobs; (crontab -l; echo "...") | crontab - adds one line and keeps the old ones.
  • Always use full paths in cron ($HOME/scripts/...), because cron does not start in your home folder, and send the output to a log file with >> file 2>&1.
  • crontab -r removes all your jobs. Use crontab -e to edit one by one.

Clean up (delete everything)

Why: delete the VM (its disk and every backup go with it) and the SSH rule. Run this in Cloud Shell.

gcloud compute instances delete bash-lab --zone=us-east1-b --quiet 2>&1 | tail -1
gcloud compute firewall-rules delete bash-lab-ssh-from-iap --quiet 2>&1 | tail -1
echo "VMs left: $(gcloud compute instances list --format="value(name)" | wc -l)"
echo "lab firewall rules left: $(gcloud compute firewall-rules list --filter="name=bash-lab-ssh-from-iap" --format="value(name)" | wc -l)"
Deleted [https://www.googleapis.com/compute/v1/projects/your-project-id/zones/us-east1-b/instances/bash-lab].
Deleted [https://www.googleapis.com/compute/v1/projects/your-project-id/global/firewalls/bash-lab-ssh-from-iap].
VMs left: 0
lab firewall rules left: 0
Clean up
VMs left 0, lab firewall rules left 0

VMs left: 0 and lab firewall rules left: 0 are your proof.

FAQ

Bash or Python for automation? Bash is perfect for gluing commands together (files, services, backups) in under about 100 lines. When you need data structures, APIs or real error handling, switch to Python.

Why does my script work in the terminal but not in cron? cron starts with a tiny environment: a short PATH, no aliases, a different folder. Use full paths, set variables in the script, and log the output to a file.

**What does #!/bin/bash do if I run bash script.sh?** Nothing then: you already chose bash. It matters when you run ./script.sh.

**Should every script use set -euo pipefail?** Almost every one. As you saw, it does not catch errors inside $( ) used as arguments, so still check important results yourself.

How do I debug a script? Run it with bash -x script.sh: Bash prints each command before it runs it.

Is cron still used, or is it all systemd timers now? Both. systemd timers have better logs and can catch up after downtime; cron is everywhere and simpler. Learn cron first.

Troubleshooting

You seeWhyFix
Permission denied when running ./script.shNo run permissionchmod +x script.sh
bad interpreter: /bin/bash^MThe file has Windows line endingssed -i 's/\r$//' script.sh, and save as LF in your editor
unbound variableset -u found a variable that was never setSet it, or give a default: ${NAME:-default}
A name with spaces breaks into twoA variable without quotes, or a loop over $(ls)Quote "$var", loop over a glob "$dir"/*
The cron job never runs or does nothingRelative paths or a missing PATH in cronFull paths, and log with >> file 2>&1
crontab: command not foundcron is not installed (rare on Debian VMs)sudo apt-get install -y cron

What next

Prepared by the CareerByteCode Team from our tested lab, with AI help for the write-up.

Read next →Nginx for Beginners: Web Server, Reverse Proxy and HTTPS⏱ 16 min

⭐ Keep learning with CareerByteCode

Hand-picked to go deeper on this topic
News, guides and updates from the CareerByteCode team
🏅 Contributor writer

The CareerByteCode team writes about what is new on careerbytecode.in: courses, labs, exams, events and the community, which is free to join for influencers and content creators in tech.

Get CareerByteCode Team's next blog by email

One email a week when they publish. Add the weekly Best of ByteBlog too. No spam, unsubscribe with one click.

We email you once to confirm. By subscribing you agree to the privacy policy.

💬 Comments (0)

Top · Newest

Sign in to comment

Related blogs

🧪⭐ Recommended by CareerByteCode · ByteLabs project bundleCloud and DevOps Realtime Starterโ‚น499Build the projects →