Web Server Log Analyzer

Write a Bash script that turns a raw web server access log into a readable report: visitors, top pages, status codes, missing pages and the busiest hour.

Pipesgrep, cut, sort, uniqawkBash Scripts

Overview

Every web server writes an access log, one line per request. Reading thousands of lines by eye is impossible, but a few Linux tools joined with pipes can summarise them in seconds. In this project you will build analyze-log.sh, a reusable script that prints a clear report for any log in the standard “combined” format used by Nginx and Apache.

What You'll Build

A generator script that creates the same 200-line sample log for everyone.
One-line pipelines that answer individual questions about the log.
analyze-log.sh: a script combining them into one report, with input checking.

Prerequisites

Pipes, cut, sort, uniq and awk — lesson 8.
Shell scripts, arguments and if — lesson 12.
A Linux terminal (lesson 2).

Step 1: Create Sample Data

Make a project folder with mkdir log-analyzer && cd log-analyzer. Instead of downloading a log, generate one: this script produces the same 200 requests on every computer, so your results will match this tutorial exactly.

make-sample-log.sh

#!/bin/bash
# make-sample-log.sh: writes a realistic access.log so everyone gets the same data
pages=(/ /about.html /projects.html /contact.html /blog.html /css/style.css)
ips=(203.0.113.5 198.51.100.7 192.0.2.44 203.0.113.9 198.51.100.23)
for i in $(seq 0 199); do
  ip=${ips[$(( (i * 7 + i / 3) % 5 ))]}
  page=${pages[$(( (i * 3 + i / 5) % 6 ))]}
  status=200
  if (( i % 17 == 0 )); then page=/old-page.html; status=404; fi
  if (( i % 41 == 0 )); then page=/contact.html; status=500; fi
  hour=$(( 8 + (i * 5) % 11 ))
  printf '%s - - [24/Oct/2026:%02d:%02d:%02d +0000] "GET %s HTTP/1.1" %s %s\n' \
    "$ip" "$hour" $(( i % 60 )) $(( (i * 13) % 60 )) "$page" "$status" $(( 500 + (i * 37) % 4000 ))
done > access.log
echo "Wrote $(wc -l < access.log) lines to access.log"

chmod +x make-sample-log.sh
./make-sample-log.sh
head -n 3 access.log

Output

Click Run to see what this code prints.

Step 2: Explore with One-Liners

Before writing a script, answer each question on the command line. How many requests, and which distinct visitors (IP addresses)?

wc -l < access.log
cut -d " " -f 1 access.log | sort -u

Output

Click Run to see what this code prints.

Which successful pages are most popular? Field 9 is the status code and field 7 the page:

awk '$9 == 200 {print $7}' access.log | sort | uniq -c | sort -rn | head -n 3

Output

Click Run to see what this code prints.

Which hours are busiest? Splitting the line on colons puts the hour in field 2:

cut -d ':' -f 2 access.log | sort | uniq -c | sort -rn | head -n 3

Output

Click Run to see what this code prints.

Step 3: Write the Script

Now combine the pipelines. The script takes the log file as an optional argument (defaulting to access.log), refuses missing files with a clear error, and prints each section under a heading:

analyze-log.sh

#!/bin/bash
# analyze-log.sh: summarise a web server access log
set -euo pipefail

log="${1:-access.log}"
if [ ! -f "$log" ]; then
  echo "Error: log file '$log' not found" >&2
  exit 1
fi

echo "=== Report for $log ==="
echo "Total requests:  $(wc -l < "$log")"
echo "Unique visitors: $(cut -d ' ' -f 1 "$log" | sort -u | wc -l)"
echo
echo "--- Top 3 pages ---"
awk '$9 == 200 {print $7}' "$log" | sort | uniq -c | sort -rn | head -n 3
echo
echo "--- Status codes ---"
awk '{print $9}' "$log" | sort | uniq -c | sort -rn
echo
echo "--- Missing pages (404) ---"
awk '$9 == 404 {print $7}' "$log" | sort | uniq -c
echo
echo "--- Busiest hour ---"
cut -d ':' -f 2 "$log" | sort | uniq -c | sort -rn | head -n 1 |
  awk '{printf "%s:00 with %s requests\n", $2, $1}'

set -euo pipefail stops the script if any command fails (lesson 12).
log="${1:-access.log}" uses the first argument, or access.log if none is given.
wc -l < "$log" reads the file through input redirection so wc prints only the number.
The last pipeline uses awk’s printf to turn “19 13” into a readable sentence.

Step 4: Run the Report

chmod +x analyze-log.sh
./analyze-log.sh

Output

Click Run to see what this code prints.

Eleven requests hit a page that no longer exists — a broken link worth fixing — and five caused server errors. Bad input is handled too:

./analyze-log.sh nope.log
echo "exit code: $?"

Output

Click Run to see what this code prints.

Extend This Project

Run it on a real log: on a server with Nginx, try sudo ./analyze-log.sh /var/log/nginx/access.log.
Add a section listing IP addresses that caused more than five 404s — a sign of a bot scanning the site.
Add an option to analyse only one day, using grep on the date before the rest of the pipeline.
Save each report to a dated file and schedule it nightly with cron (see the backup project).

Summary

You turned a raw log into an actionable report with nothing but standard Linux tools: cut, sort, uniq and awk joined by pipes, wrapped in a robust script with argument handling and error checking.

Overview

Prerequisites

Step 1: Create Sample Data

Step 2: Explore with One-Liners

Step 3: Write the Script

Step 4: Run the Report

Extend This Project

Summary

More Linux & Command Line Practice Builds

Automated Backups with Rotation and cron

Downloads Folder Organizer

Host a Website with Nginx