CareerByteCode
ByteBlogby CareerByteCode

Cron Jobs and Log Rotation on Linux

What you build: a disk report job run by cron every minute with its log rotated by logrotate, on a free VM, plus 3 real mistakes: relative paths, the % sign and an insecure log folder.

Cron Jobs and Log Rotation on Linux

✍️ Translate this blog into Hindi, Tamil or Telugu

In this lab you schedule a real job with cron and keep its log under control with logrotate, on a free Google Cloud VM. You also hit the three mistakes that catch almost everyone: a job that works by hand but fails under cron, a % sign that silently breaks a crontab line, and logrotate refusing to touch a log folder. You see each failure in the real logs and fix it.

Who it is for: beginners who can use a Linux terminal and want jobs that run on time, every time, without filling the disk. Time: about 1 hour (cron runs once a minute, so some steps wait a minute or two). Cost: free. An e2-micro VM in us-east1 is in Google Cloud's Always Free tier (one per account), and we delete it at the end.

Every command below was run for real by the CareerByteCode Team on a fresh Google Cloud project, and every output and screenshot is the real result we got. Only dates and times will differ on your machine.

🧪⭐ Recommended by CareerByteCode · ByteLabs project bundleCloud and DevOps Realtime Starterโ‚น499Build the projects →

Part A: cron and logrotate in plain English

cron is the Linux alarm clock. A small program (the cron daemon) wakes up every minute, reads everyone's crontab (cron table) and runs whatever is due. logrotate is the cleaner: once a day (or when you run it) it renames a big log to app.log.1, starts a fresh one, compresses old copies and deletes the oldest, so logs never fill the disk.

cron and logrotate in one picture
cron starts the job on time, the job writes a log, logrotate keeps 3 old copies and compresses them

A crontab line has five time fields, then the command:

*  *  *  *  *  command
|  |  |  |  |
|  |  |  |  +-- day of week (0-7, 0 and 7 = Sunday)
|  |  |  +----- month (1-12)
|  |  +-------- day of month (1-31)
|  +----------- hour (0-23)
+-------------- minute (0-59)

Examples: */5 * * * * = every 5 minutes, 30 2 * * * = every day at 02:30, 0 9 * * 1-5 = 09:00 on weekdays. Real-life example: setting a phone alarm with "repeat on Mon to Fri".

Two facts explain most cron problems:

  1. cron runs your command from your home folder with a very small environment (a short PATH, no aliases, none of your terminal settings).
  2. cron runs without a terminal, so you never see errors unless you send the output to a file (>> file.log 2>&1).

Times are in the server's time zone. Google Cloud VMs use UTC by default.

Before you start

  1. A Google Cloud account (the free trial works) and a project.
  2. Open Cloud Shell (the >_ icon at the top right of the Google Cloud console) and set your project:
export PROJECT_ID="your-project-id"
gcloud config set project $PROJECT_ID
gcloud services enable compute.googleapis.com

Part B: the lab

Step 1: a free e2-micro VM

Why: a small Linux server (Debian 12, cron already installed) to practise on. --no-service-account --no-scopes gives it no Google Cloud permissions; this lab needs none.

gcloud compute instances create cron-lab --zone=us-east1-b --machine-type=e2-micro \
  --image-family=debian-12 --image-project=debian-cloud --boot-disk-size=10GB --boot-disk-type=pd-standard \
  --no-service-account --no-scopes 2>&1 | tail -2
until gcloud compute ssh cron-lab --zone=us-east1-b --tunnel-through-iap --command="echo ssh ok" 2>/dev/null | grep -q ok; do sleep 10; done; echo "SSH works"
NAME      ZONE        MACHINE_TYPE  PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS
cron-lab  us-east1-b  e2-micro                   10.142.0.28  EXTERNAL_IP  RUNNING
SSH works

Connect with Compute Engine > VM instances > SSH next to cron-lab, or from Cloud Shell gcloud compute ssh cron-lab --zone=us-east1-b. Everything from Step 2 runs inside the VM in ~/lab269. Create two files there: limits.conf with the single line 80 (the alert limit), and the script below (nano disk-report.sh).

#!/bin/bash
# disk-report.sh: write one line with the root disk usage and the alert limit to a log
LIMIT=$(cat limits.conf)
USED=$(df --output=pcent / | tail -1 | tr -d ' %')
echo "$(date '+%F %T') root disk used ${USED}% (alert at ${LIMIT}%)"

Step 2: a job that works by hand

Why: always run a job by hand first. crontab -l lists your crontab; a new user has none yet.

# A small job to schedule: it works when you run it by hand
chmod +x disk-report.sh && mkdir -p logs
cat disk-report.sh
./disk-report.sh
systemctl is-active cron
crontab -l
#!/bin/bash
# disk-report.sh: write one line with the root disk usage and the alert limit to a log
LIMIT=$(cat limits.conf)
USED=$(df --output=pcent / | tail -1 | tr -d ' %')
echo "$(date '+%F %T') root disk used ${USED}% (alert at ${LIMIT}%)"
2026-11-18 20:35:24 root disk used 23% (alert at 80%)
active
no crontab for you

The script works: 23 percent used, alert at 80 percent. no crontab for you is normal for a new user (and makes crontab -l exit with code 1).

Break it: the job fails under cron

We add one crontab line: every minute, run the script and append everything it prints, errors included (2>&1), to logs/disk.log. crontab - reads the new crontab from the pipe. (On your own you would use crontab -e, which opens an editor.) Then we wait two minutes and read the log.

# Break it: schedule it every minute, wait, and read what cron did
( echo "* * * * * $HOME/lab269/disk-report.sh >> $HOME/lab269/logs/disk.log 2>&1" ) | crontab -
crontab -l
sleep 125
cat logs/disk.log
* * * * * /home/you/lab269/disk-report.sh >> /home/you/lab269/logs/disk.log 2>&1
cat: limits.conf: No such file or directory
2026-11-18 20:36:01 root disk used 23% (alert at %)
cat: limits.conf: No such file or directory
2026-11-18 20:37:01 root disk used 23% (alert at %)
The job fails under cron
By hand it works; under cron cat cannot find limits.conf, so the alert limit is empty

Under cron, cat cannot find limits.conf, and the line says (alert at %) with no number. The script is fine, so why? cron started it from the home folder, not from ~/lab269, so the relative name limits.conf pointed to a file that does not exist. Without 2>&1 you would never have seen the error. The job would just quietly write wrong lines.

Fix it: make the script find its own folder

Why: cd "$(dirname "$0")" moves to the folder the script lives in ($0 is the script's own path), whoever starts it and from wherever. || exit 1 stops the script if that fails. Using full paths everywhere works too.

# Fix it: cron runs your job from your HOME folder, so the script must cd to its own folder
sed -i 's|^LIMIT=|cd "$(dirname "$0")" \|\| exit 1\nLIMIT=|' disk-report.sh
sed -n 3,4p disk-report.sh
: > logs/disk.log; sleep 70
cat logs/disk.log
cd "$(dirname "$0")" || exit 1
LIMIT=$(cat limits.conf)
2026-11-18 20:38:01 root disk used 23% (alert at 80%)

The next cron run writes the right limit: (alert at 80%).

Break it: a % sign in the crontab line

Now a one-line job that writes today's date with date +%F. It works in the terminal. In a crontab line, though, **% means "new line"**: cron cuts the command at the first % and sends the rest as input.

# Break it: a % sign in a crontab line means "new line" to cron, so date +%F breaks the line
( crontab -l; echo "* * * * * echo \"tick \$(date +%F)\" >> $HOME/lab269/logs/tick.log 2>&1" ) | crontab -
sleep 70; cat logs/tick.log 2>/dev/null; echo "tick.log lines: $(cat logs/tick.log 2>/dev/null | wc -l)"
sudo journalctl -u cron --no-pager -n 30 -o cat | grep -m1 "echo \"tick" | sed 's/^(\([^)]*\))/(you)/'
tick.log lines: 0
(you) CMD (echo "tick $(date +)

tick.log has 0 lines. The cron log (journalctl -u cron) shows what cron really ran: echo "tick $(date +), cut off right at the %. That command is broken, so nothing was written.

Fix it: escape the % as \%

# Fix it: write \% in crontab (or move the command into a script)
crontab -l | grep -v tick > /tmp/ct; echo "* * * * * echo \"tick \$(date +\%F)\" >> $HOME/lab269/logs/tick.log 2>&1" >> /tmp/ct; crontab /tmp/ct
crontab -l | grep tick
sleep 70; cat logs/tick.log
* * * * * echo "tick $(date +\%F)" >> /home/you/lab269/logs/tick.log 2>&1
tick 2026-11-18
Percent sign
cron cut the command at the % sign; with \% the date is written

With \% cron passes a real % to the shell. Even simpler: put the command in a script file, where % needs no escaping.

Step 3: rotate the log with logrotate

Why: a job that writes a line every minute adds about 80,000 lines a year. We write 60 lines at once to make the log bigger than 1 KB, then give logrotate a small config file. The options:

OptionMeaning
size 1krotate when the log is bigger than 1 KB (on real servers: daily or size 100M)
rotate 3keep 3 old copies, delete older ones
compress / delaycompressgzip old copies, but leave the newest old copy (.1) uncompressed for one more round
missingok / notifemptyno error if the log is missing; do not rotate an empty log
copytruncatecopy the log, then empty it in place, so a program that keeps the file open keeps writing to it

-s state.txt keeps logrotate's memory of the last rotation in our folder, so we can run it as a normal user, and -v explains what it does.

# Rotate the job's log: keep 3 old copies, compress them, rotate when bigger than 1 KB
for i in $(seq 1 60); do ./disk-report.sh >> logs/disk.log; done; ls -l logs/disk.log | awk '{print $5, $9}'
cat > logrotate-lab.conf <<CONF
$HOME/lab269/logs/disk.log {
    size 1k
    rotate 3
    compress
    delaycompress
    missingok
    notifempty
    copytruncate
}
CONF
cat logrotate-lab.conf
/usr/sbin/logrotate -s state.txt -v logrotate-lab.conf 2>&1 | grep -E "rotating|log needs|copying|truncating" | sed "s|$HOME|~|g"
ls logs/
3540 logs/disk.log
/home/you/lab269/logs/disk.log {
    size 1k
    rotate 3
    compress
    delaycompress
    missingok
    notifempty
    copytruncate
}
rotating pattern: ~/lab269/logs/disk.log  1024 bytes (3 rotations)
  log needs rotating
rotating log ~/lab269/logs/disk.log, log->rotateCount is 3
copying ~/lab269/logs/disk.log to ~/lab269/logs/disk.log.1
truncating ~/lab269/logs/disk.log
disk.log
disk.log.1
tick.log

Step 4: three more rotations

Why: to see the whole life of a log, we fill and rotate it three more times, as if three days passed.

# Run it 3 more times (as if 3 days passed): old logs move down, get compressed, the 4th is deleted
for n in 1 2 3; do for i in $(seq 1 60); do ./disk-report.sh >> logs/disk.log; done; /usr/sbin/logrotate -s state.txt logrotate-lab.conf; done
ls -l logs/ | awk 'NR>1 {print $5, $9}'
zcat logs/disk.log.2.gz | head -2
0 disk.log
3240 disk.log.1
98 disk.log.2.gz
138 disk.log.3.gz
32 tick.log
2026-11-18 20:42:03 root disk used 23% (alert at 80%)
2026-11-18 20:42:03 root disk used 23% (alert at 80%)
🎓⭐ Recommended by CareerByteCode · Live trainingCloud & DevOps Live Cohort Internshipโ‚น2,360See the training →
logrotate
After 4 rotations: the current log, .1 uncompressed, .2.gz and .3.gz; the oldest copy was deleted

The current disk.log (0 bytes: copytruncate just emptied it), then .1 (plain, because of delaycompress), then .2.gz and .3.gz. A fourth old copy was deleted, as rotate 3 says. zcat reads a compressed log without unpacking it.

Break it: logrotate refuses a log folder

On a real server logrotate runs as root from /etc/logrotate.d/, once a day. Here our app's log folder /var/log/myapp is owned by our group and the group may write to it (775), which is common when an app user writes its own logs. -d is a dry run: logrotate only says what it would do.

# Break it: the system logrotate (run as root) refuses a log folder that a group can write to
sudo mkdir -p /var/log/myapp && sudo chown root:$USER /var/log/myapp && sudo chmod 775 /var/log/myapp
for i in $(seq 1 60); do ./disk-report.sh; done | sudo tee /var/log/myapp/app.log > /dev/null
printf '/var/log/myapp/app.log {\n    size 1k\n    rotate 3\n    compress\n    missingok\n}\n' | sudo tee /etc/logrotate.d/myapp > /dev/null
sudo logrotate -d /etc/logrotate.d/myapp 2>&1 | grep -iE "insecure|error" | head -2
error: skipping "/var/log/myapp/app.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation.

logrotate refuses (exit code 1), on purpose: if a group can write to the folder, someone in that group could swap a log for a link to an important file, and root would then rotate (and possibly delete) that file.

Fix it: the su line

Why: su root <group> tells logrotate to rotate this folder as that user and group, so it is safe. Then -f forces a rotation to prove it works, and systemctl list-timers shows the systemd timer that runs logrotate every day.

# Fix it: tell logrotate which user and group own that folder (the su line)
printf '/var/log/myapp/app.log {\n    su root %s\n    size 1k\n    rotate 3\n    compress\n    missingok\n}\n' "$USER" | sudo tee /etc/logrotate.d/myapp | sed "s/$USER/you/"
sudo logrotate -f /etc/logrotate.d/myapp && ls /var/log/myapp/
systemctl list-timers logrotate.timer --no-pager | head -2 | cut -c1-110
/var/log/myapp/app.log {
    su root you
    size 1k
    rotate 3
    compress
    missingok
}
app.log.1.gz
NEXT                        LEFT          LAST PASSED UNIT            ACTIVATES
Thu 2026-11-19 00:00:00 UTC 3h 17min left -    -      logrotate.timer logrotate.service
su line
insecure permissions refused; with the su line logrotate rotates the log, and logrotate.timer runs it daily

Step 5: clean up the crontab

Why: crontab -r removes your whole crontab (careful: there is no undo, so on real servers use crontab -e and delete one line).

# Clean up: remove the crontab and the test files
crontab -r && (crontab -l 2>&1 || true)
sudo rm -rf /etc/logrotate.d/myapp /var/log/myapp
no crontab for you

POC: prove it works

  • A job that worked by hand failed under cron (No such file or directory), and worked after the script moved to its own folder.
  • A % in a crontab line cut the command (the cron log shows date +) and \% fixed it.
  • logrotate kept 3 old copies, compressed all but the newest one and deleted the oldest.
  • logrotate refused a group-writable folder until a su line said who owns it.

Quiz task: check what you learned

  1. Write a crontab line that runs /opt/backup.sh every day at 01:30 and keeps its output. *(Answer: 30 1 * * * /opt/backup.sh >> /var/log/backup.log 2>&1.)*
  2. Your job works by hand but not in cron. Name two likely causes. *(Answer: a relative path, because cron starts in the home folder, and a short PATH. Also an unescaped %.)*
  3. How do you see whether cron started your job? *(Answer: sudo journalctl -u cron, and your own 2>&1 log.)*
  4. What does rotate 5 with daily keep? (Answer: the current log plus 5 days of old logs.)
  5. Your app keeps its log file open. Which logrotate option helps? *(Answer: copytruncate, or a postrotate command that tells the app to reopen its log.)*

Clean up (delete everything)

Exit the VM (exit), then in Cloud Shell:

gcloud compute instances delete cron-lab --zone=us-east1-b --quiet 2>&1 | tail -1
echo "cron-lab VMs left: $(gcloud compute instances list --filter="name=cron-lab" --format="value(name)" | wc -l)"
echo "cron-lab disks left: $(gcloud compute disks list --filter="name~^cron-lab" --format="value(name)" | wc -l)"
Deleted [https://www.googleapis.com/compute/v1/projects/your-project-id/zones/us-east1-b/instances/cron-lab].
WARNING: The following filter keys were not present in any resource : name
cron-lab VMs left: 0
WARNING: The following filter keys were not present in any resource : name
cron-lab disks left: 0
Clean up
The cron-lab VM and its disk are deleted: 0 left

FAQ

cron or systemd timers? Both schedule jobs. cron is simple and everywhere. systemd timers add logs in the journal, catch up after a reboot (Persistent=true) and can depend on other services. Debian runs logrotate itself with a timer, as Step 5 showed.

Which time zone does cron use? The server's. Google Cloud VMs use UTC. Check with timedatectl. A job for 09:00 in India (UTC+5:30) is 30 3 * * * on a UTC server.

How do I stop two runs of a long job from overlapping? Start it with flock: * * * * * flock -n /tmp/job.lock /opt/job.sh. A second run exits at once if the first still holds the lock.

Where are the system's own cron jobs? In /etc/crontab, /etc/cron.d/ and the /etc/cron.daily/, cron.weekly/ and cron.monthly/ folders. Your own are in crontab -l.

How do I schedule a job in Google Cloud without a VM? Cloud Scheduler calls a URL (for example a Cloud Run service) on a cron schedule, using the same five fields.

Should logs go to files at all? On one server, yes, with logrotate. In containers and Cloud Run, write to the screen (stdout) and let the platform collect the logs.

Troubleshooting

You seeWhyFix
Job runs by hand, nothing happens in cronRelative paths, short PATH, no output keptFull paths or cd "$(dirname "$0")", add >> log 2>&1
Command cut short in the cron logUnescaped %Write \%, or move the command into a script
bash: logrotate: command not found/usr/sbin is not in a normal user's PATHRun /usr/sbin/logrotate, or use sudo
skipping ... because parent directory has insecure permissionsThe log folder is group- or world-writableAdd su <user> <group> to the config, or fix the folder
Log keeps growing after rotationThe app writes to the old (renamed) filecopytruncate, or reopen the log in postrotate
Job runs at the wrong hourThe server is on UTCConvert the time, or set CRON_TZ / the server time zone

What next

Prepared by the CareerByteCode Team from our tested lab, with AI help for the write-up.

Read next →Linux Users, Groups and sudo Explained⏱ 13 min

⭐ Keep learning with CareerByteCode

Hand-picked to go deeper on this topic
News, guides and updates from the CareerByteCode team
🏅 Expert writer

The CareerByteCode team writes about what is new on careerbytecode.in: courses, labs, exams, events and the community, which is free to join for influencers and content creators in tech.

Get CareerByteCode Team's next blog by email

One email a week when they publish. Add the weekly Best of ByteBlog too. No spam, unsubscribe with one click.

We email you once to confirm. By subscribing you agree to the privacy policy.

💬 Comments (0)

Top · Newest

Sign in to comment

Related blogs

🧪⭐ Recommended by CareerByteCode · ByteLabs project bundleCloud and DevOps Realtime Starterโ‚น499Build the projects →