Word Frequency Analyzer

Count words in a text with string patterns, skip common stop words, and print a ranked top-10 list with a bar chart.

String PatternsTables as MapsSortingFile I/O

Overview

Counting words is a small task that shows off two of Lua's strengths at once: string patterns and tables. string.gmatch walks through a text one match at a time, and a table used as a dictionary — counts[word] = (counts[word] or 0) + 1 — keeps a running total for every word it has seen.

The analyzer reads a text file given on the command line (or uses a built-in sample paragraph), ignores common "stop words" like "the" and "and", and prints the total word count, the number of unique words, and a ranked top-10 list with a simple bar chart.

What You'll Build

A read_text function that loads a file with io.open or falls back to sample text.
A stop-word set built from a string with gmatch.
A count_words function that lower-cases text and counts words with %a+ patterns.
A ranking step that converts the counts into an array and sorts it with a custom comparator.
A formatted report using string.format and string.rep for bars.

Prerequisites

Tables as arrays and dictionaries — lesson 5.
Functions with multiple return values — lesson 7.
String patterns, gmatch and string.format — lesson 10.
Basic error handling with assert — lesson 11.

Project Structure

Everything is in wordfreq.lua, written as a pipeline of four functions: read the text, count the words, rank the counts, print the report. Each function takes the previous one's result as input and none of them prints anything except the last — which makes each step easy to test or reuse on its own.

Step 1: Read the Text

The global arg table holds command-line arguments, so arg[1] is the first one. io.open returns nil plus an error message when a file cannot be opened; wrapping it in assert turns that into a clear error. f:read("a") reads the whole file at once.

local SAMPLE = [[
Lua is a powerful, efficient, lightweight, embeddable scripting language.
Lua combines simple procedural syntax with powerful data description
constructs based on associative arrays and extensible semantics. Lua is
dynamically typed, runs by interpreting bytecode with a register-based
virtual machine, and has automatic memory management with incremental
garbage collection, making Lua ideal for configuration, scripting, and
rapid prototyping. Tables are the only data structuring mechanism in Lua,
and tables can represent arrays, sets, records, and objects.
]]

local function read_text(path)
  if path == nil then
    return SAMPLE, "built-in sample"
  end
  local file = assert(io.open(path, "r"))
  local text = file:read("a")
  file:close()
  return text, path
end

Step 2: Count Words with gmatch

The pattern %a+ matches a run of letters, so punctuation and numbers are skipped automatically. Stop words are stored as a set — a table whose keys are the words and whose values are true — so checking membership is a single lookup instead of a loop.

local STOP_WORDS = {}
for word in ("a an and are as at be by for from has in is it of on or the to with"):gmatch("%a+") do
  STOP_WORDS[word] = true
end

local function count_words(text)
  local counts, total = {}, 0
  for word in text:lower():gmatch("%a+") do
    if not STOP_WORDS[word] and #word > 1 then
      counts[word] = (counts[word] or 0) + 1
      total = total + 1
    end
  end
  return counts, total
end

Step 3: Rank the Results

A dictionary has no order, so the counts are copied into an array of { word, count } records first. The comparator passed to table.sort must return true when a should come before b: higher counts first, and alphabetical order to break ties, so the output is the same on every run.

local function rank(counts)
  local list = {}
  for word, count in pairs(counts) do
    list[#list + 1] = { word = word, count = count }
  end
  table.sort(list, function(a, b)
    if a.count ~= b.count then
      return a.count > b.count
    end
    return a.word < b.word
  end)
  return list
end

Step 4: Print a Report

%-12s left-aligns a word in 12 characters and %3d right-aligns a number in 3, producing tidy columns. string.rep("#", n) repeats a character to draw each bar, scaled so the most frequent word gets the longest bar.

local function report(ranked, total, source, top)
  print(("Source: %s"):format(source))
  print(("Words counted: %d   Unique words: %d"):format(total, #ranked))
  print()
  print(("Top %d words:"):format(math.min(top, #ranked)))
  local max_count = ranked[1] and ranked[1].count or 0
  for i = 1, math.min(top, #ranked) do
    local entry = ranked[i]
    local bar = string.rep("#", math.ceil(entry.count / max_count * 20))
    print(("%2d. %-12s %3d  %s"):format(i, entry.word, entry.count, bar))
  end
end

Complete Code

wordfreq.lua

-- wordfreq.lua — run with: lua wordfreq.lua [file.txt]
local SAMPLE = [[
Lua is a powerful, efficient, lightweight, embeddable scripting language.
Lua combines simple procedural syntax with powerful data description
constructs based on associative arrays and extensible semantics. Lua is
dynamically typed, runs by interpreting bytecode with a register-based
virtual machine, and has automatic memory management with incremental
garbage collection, making Lua ideal for configuration, scripting, and
rapid prototyping. Tables are the only data structuring mechanism in Lua,
and tables can represent arrays, sets, records, and objects.
]]

local STOP_WORDS = {}
for word in ("a an and are as at be by for from has in is it of on or the to with"):gmatch("%a+") do
  STOP_WORDS[word] = true
end

local function read_text(path)
  if path == nil then
    return SAMPLE, "built-in sample"
  end
  local file = assert(io.open(path, "r"))
  local text = file:read("a")
  file:close()
  return text, path
end

local function count_words(text)
  local counts, total = {}, 0
  for word in text:lower():gmatch("%a+") do
    if not STOP_WORDS[word] and #word > 1 then
      counts[word] = (counts[word] or 0) + 1
      total = total + 1
    end
  end
  return counts, total
end

local function rank(counts)
  local list = {}
  for word, count in pairs(counts) do
    list[#list + 1] = { word = word, count = count }
  end
  table.sort(list, function(a, b)
    if a.count ~= b.count then
      return a.count > b.count
    end
    return a.word < b.word
  end)
  return list
end

local function report(ranked, total, source, top)
  print(("Source: %s"):format(source))
  print(("Words counted: %d   Unique words: %d"):format(total, #ranked))
  print()
  print(("Top %d words:"):format(math.min(top, #ranked)))
  local max_count = ranked[1] and ranked[1].count or 0
  for i = 1, math.min(top, #ranked) do
    local entry = ranked[i]
    local bar = string.rep("#", math.ceil(entry.count / max_count * 20))
    print(("%2d. %-12s %3d  %s"):format(i, entry.word, entry.count, bar))
  end
end

local path = arg and arg[1] -- arg is nil when Lua is embedded in another program
local text, source = read_text(path)
local counts, total = count_words(text)
report(rank(counts), total, source, 10)

Sample Run

lua wordfreq.lua

Click Run to see what this code prints.

Pass any text file to analyse it instead, for example lua wordfreq.lua notes.txt. If the file does not exist, assert stops the program with a message such as "notes.txt: No such file or directory".

Extend This Project

Accept a second argument for how many top words to show: tonumber(arg[2]) or 10.
Count two-word phrases ("bigrams") by remembering the previous word in the loop.
Load stop words from a separate stopwords.txt file using io.lines.
Report the longest word and the average word length.
Turn the four functions into a reusable module that returns a table (lesson 11).

Summary

You built a small text-analysis pipeline using Lua's core tools: gmatch with %a+ to find words, a table as a counter, a table as a set for stop words, a custom comparator for table.sort, and string.format/string.rep for tidy output. Splitting the work into pure functions with one job each is a habit worth keeping in every Lua script.