Log File Analyzer

Overview

Perl was built for exactly this kind of job: chewing through a text file line by line and pulling structured facts out of it with a regular expression. A web server access log is a stream of lines that all follow the same shape — client address, timestamp, request, status code, size — and a regex with capture groups is the natural tool for pulling the pieces you care about out of that shape without writing a hand-rolled parser.

By the end of this tutorial you will have a script that reads a log file line by line with an open file handle, matches each line against a regular expression built to mirror the Common Log Format, and tallies every 4xx and 5xx response into a hash keyed by the hour it happened in. Every step leans on the three habits every Perl script should have from the very first line: use strict; use warnings; to catch mistakes early, and checked open calls so a missing file fails loudly instead of silently.

Prerequisites

Project Structure

The whole project is one script, log_analyzer.pl, plus a sample log file, access.log, sitting next to it. The script is organized top to bottom exactly the way it runs: open the log file, loop over its lines matching each one against a regex, accumulate counts into a hash, then loop over that hash a second time to print the report. There is no need for subroutines or modules here — the entire job is a single pass over a file followed by a single pass over a hash, which is about as close to Perl's original one-liner-turned-script roots as a project gets.

use strict; and use warnings; appear on the first two lines and stay there for the rest of the tutorial. strict forces every variable to be declared with my before it is used, which catches typos in variable names at compile time instead of letting them silently create a new global. warnings flags things like using an undefined value in a numeric context, which is exactly the kind of mistake a log line that does not match the expected format can trigger.

Step 1: Open the Log File Safely

Perl's three-argument open — open(my $fh, '<', $path) — is the modern idiom: the mode ('<' for reading) is kept separate from the filename, so a filename that happens to contain characters like > or | can never be misread as part of the mode. open returns a true value on success and a false value on failure, so or die "..." is the standard Perl pattern for turning a failed open into an immediate, informative error instead of a confusing failure several lines later.

use strict;   # Require every variable to be declared with 'my' before use
use warnings; # Warn about risky things like using undef in a numeric context

my $log_file = 'access.log';   # Path to the log file this script analyzes

# Three-argument open: mode ('<' for read) is separate from the filename,
# so a filename containing '>' or '|' can never be mistaken for a mode.
# "or die" aborts immediately with a clear message if the file can't be opened,
# instead of silently continuing with an unusable file handle.
open(my $fh, '<', $log_file)
    or die "Could not open '$log_file': $!\n";   # $! holds the OS error string

# ... regex matching and hour tallying happen here in the next steps ...

close($fh);   # Release the file handle once every line has been processed

Step 2: Match Each Line With a Regex

Reading <$fh> inside a while condition pulls one line at a time out of the file handle, which keeps memory usage flat even for a multi-gigabyte log — the whole file is never loaded into a single string or array at once. The regex below is built to match a Common Log Format line such as 192.168.1.10 - - [10/Aug/2026:14:23:01 +0000] "GET /index.html HTTP/1.1" 404 512, with parentheses marking every piece the rest of the script needs to pull out.

# Matches a Common Log Format line and captures the two-digit hour and the
# three-digit HTTP status code. Everything outside a capture group here is
# just structural text the regex must see but doesn't need to remember.
my $log_pattern = qr/
    ^\S+ \s+ \S+ \s+ \S+ \s+          # client IP, ident, userid (ignored)
    \[ \d{2}\/\w{3}\/\d{4}:(\d{2}):\d{2}:\d{2} \s [+-]\d{4} \] \s+   # (1) hour
    "[^"]*" \s+                        # the quoted request line (ignored)
    (\d{3}) \s+                        # (2) HTTP status code
    \S+                                 # response size (ignored)
/x;   # /x lets the pattern span multiple lines with whitespace and comments

while (my $line = <$fh>) {         # Reads and processes one line per iteration
    chomp $line;                    # Strip the trailing newline before matching
    if ($line =~ $log_pattern) {
        my ($hour, $status) = ($1, $2);   # $1/$2 hold the two capture groups on a successful match
        # ... tally this hour/status pair in the next step ...
    }
    # Lines that don't match the expected format are silently skipped —
    # a real log can contain the occasional malformed or truncated line.
}

Step 3: Extract the Hour and Status Code

Step 2 already captures the hour and status code into $1 and $2; this step is about deciding what counts as an "error" worth tallying. HTTP status codes in the 400s mean a client error and the 500s mean a server error, so checking the first digit of the three-digit code with a small regex — $status =~ /^[45]/ — is enough to classify a response as an error without needing a lookup table of every individual code.

# A status code is an "error" if it starts with 4 (client error) or 5 (server
# error) — 2xx success and 3xx redirect codes are not counted here.
sub is_error_status {
    my ($status) = @_;          # @_ holds this subroutine's arguments
    return $status =~ /^[45]/;  # Matches if the first digit is 4 or 5
}

# Used inside the while loop from Step 2:
if ($line =~ $log_pattern) {
    my ($hour, $status) = ($1, $2);
    if (is_error_status($status)) {
        print "Error at hour $hour: status $status\n";   # Replaced by hash tallying in Step 4
    }
}

Step 4: Tally Errors in a Hash Keyed by Hour

A Perl hash is the natural structure for "count how many times each hour appears" — $errors_by_hour{$hour}++ both creates the key with a starting value of 0 the first time a given hour is seen and increments it on every subsequent match, because an undefined hash value autovivifies to 0 the moment warnings lets it be used in a numeric context like ++. No pre-initialization loop over all 24 hours is needed; the hash only ever grows entries for hours that actually appear in the log.

my %errors_by_hour;   # Hash: hour string ("00".."23") => count of 4xx/5xx responses that hour
my $total_lines = 0;    # Total lines read, matching or not — used in the summary
my $total_errors = 0;   # Total error responses across every hour

while (my $line = <$fh>) {
    chomp $line;
    $total_lines++;
    if ($line =~ $log_pattern) {
        my ($hour, $status) = ($1, $2);
        if (is_error_status($status)) {
            $errors_by_hour{$hour}++;   # Autovivifies to 1 on the first error seen for this hour
            $total_errors++;
        }
    }
}

Step 5: Print a Sorted Summary Report

keys %errors_by_hour returns the hash's keys in no particular order, so sort is needed before printing anything readable. Because the hour keys are two-digit strings like "09" and "14", Perl's default sort (which compares as strings) already puts them in the right chronological order — the leading zero keeps "09" sorting before "10" the same way a numeric sort would. printf then lines up the hour and count into fixed-width columns.

print "\n===== ERROR SUMMARY BY HOUR =====\n";
printf "%-8s %s\n", "Hour", "Errors";
foreach my $hour (sort keys %errors_by_hour) {   # String sort works here because hours are zero-padded "00".."23"
    printf "%-8s %d\n", "$hour:00", $errors_by_hour{$hour};
}

print "\nTotal lines processed: $total_lines\n";
print "Total error responses: $total_errors\n";

Complete Code

Here is the full script assembled in the order it runs, ready to save as log_analyzer.pl and run with perl log_analyzer.pl alongside an access.log file in the same directory.

use strict;
use warnings;

my $log_file = 'access.log';

open(my $fh, '<', $log_file)
    or die "Could not open '$log_file': $!\n";

# Matches a Common Log Format line and captures the two-digit hour and the
# three-digit HTTP status code.
my $log_pattern = qr/
    ^\S+ \s+ \S+ \s+ \S+ \s+
    \[ \d{2}\/\w{3}\/\d{4}:(\d{2}):\d{2}:\d{2} \s [+-]\d{4} \] \s+
    "[^"]*" \s+
    (\d{3}) \s+
    \S+
/x;

# A status code is an "error" if it starts with 4 (client error) or 5 (server error).
sub is_error_status {
    my ($status) = @_;
    return $status =~ /^[45]/;
}

my %errors_by_hour;   # hour string ("00".."23") => count of 4xx/5xx responses
my $total_lines = 0;
my $total_errors = 0;

while (my $line = <$fh>) {
    chomp $line;
    $total_lines++;
    if ($line =~ $log_pattern) {
        my ($hour, $status) = ($1, $2);
        if (is_error_status($status)) {
            $errors_by_hour{$hour}++;
            $total_errors++;
        }
    }
}

close($fh);

print "\n===== ERROR SUMMARY BY HOUR =====\n";
printf "%-8s %s\n", "Hour", "Errors";
foreach my $hour (sort keys %errors_by_hour) {
    printf "%-8s %d\n", "$hour:00", $errors_by_hour{$hour};
}

print "\nTotal lines processed: $total_lines\n";
print "Total error responses: $total_errors\n";

Sample Run

Extend This Project

Summary

You built a log analyzer that reads a file one line at a time, pulls structured data out of unstructured text with a regex and capture groups, and summarizes it with a hash keyed by hour. This read-match-tally-report shape — an opened file handle feeding a while loop, a regex doing the extraction, and a hash doing the counting — is the backbone of a huge share of real-world Perl scripts, from log analysis to report generation.