Text Search and Transformation: grep, sed, and awk Foundations
Finding lines that match a pattern, and the two workhorse tools for transforming text at the line and field level — enough to be genuinely useful, not a complete reference.
What you'll learn
- Use grep with common flags to find matching lines by pattern
- Use sed for a basic, line-oriented find-and-replace
- Use awk to extract and print specific fields from structured text
Prerequisites
Explanation
Every real command below runs only in your own terminal — this lesson's exercises model text processing as JavaScript string logic and never execute grep, sed, or awk.
grep searches for lines matching a pattern: grep "ERROR" app.log prints every line containing "ERROR"; grep -i matches case-insensitively; grep -v inverts the match, printing lines that don't match; grep -r searches recursively through a directory tree; grep -n prefixes each match with its line number, genuinely useful for then jumping directly to that line in an editor. grep -c prints just the count of matching lines rather than the lines themselves — combined with a pipe from an earlier lesson, grep -c "ERROR" app.log is a more direct way to get the same count grep "ERROR" app.log | wc -l computes via a pipeline.
sed (stream editor) applies an edit to text as it streams through, line by line — its most common use, by far, is find-and-replace: sed 's/old/new/' file.txt replaces the first occurrence of "old" with "new" on each line; sed 's/old/new/g' (the trailing g for "global") replaces every occurrence on each line, not just the first. This lesson deliberately covers this one foundational pattern precisely rather than sed's full, genuinely large feature set — knowing this one substitution pattern solidly covers the large majority of real, everyday sed usage.
awk treats each line of input as a record automatically split into whitespace-separated fields — $1, $2, ..., referring to the first, second, ... field on the current line, and $0 referring to the whole line. awk '{ print $2 }' data.txt prints just the second whitespace-separated field of every line — genuinely useful for structured, column-like text (log lines, ps output, CSV-ish data) where you need one specific column without writing a more complex parser. awk -F, changes the field separator from whitespace to a comma, for genuinely comma-separated data. Like sed, this lesson covers awk's most common, foundational pattern (field extraction) rather than its full programming-language-level feature set.
Example
Modeling grep's line-matching, sed's substitution, and awk's field extraction as pure string operations -- no real tool is invoked.
function grepLines(text, pattern, invert = false) {
const lines = text.split("\n");
return lines.filter((line) => invert ? !line.includes(pattern) : line.includes(pattern));
}
const log = "INFO: started\nERROR: disk full\nINFO: running\nERROR: timeout";
console.log(grepLines(log, "ERROR")); // both ERROR lines
console.log(grepLines(log, "ERROR", true)); // both INFO lines -- grep -v
function sedReplaceFirst(line, oldStr, newStr) {
return line.replace(oldStr, newStr); // JS replace() without /g -- first occurrence only, matching sed's default
}
function sedReplaceAll(line, oldStr, newStr) {
return line.split(oldStr).join(newStr); // matches sed's trailing 'g' flag -- every occurrence
}
console.log(sedReplaceFirst("foo foo foo", "foo", "bar")); // "bar foo foo"
console.log(sedReplaceAll("foo foo foo", "foo", "bar")); // "bar bar bar"
function awkField(line, fieldNumber, separator = /\s+/) {
const fields = line.trim().split(separator);
return fields[fieldNumber - 1]; // awk fields are 1-indexed, unlike JS arrays
}
console.log(awkField("alice 30 engineer", 2)); // "30"Try it yourself
Call awkField with fieldNumber 3 to extract 'engineer' from the same sample line.
Code editor. Press Escape then Tab to leave the editor if keyboard focus becomes trapped. Press Control+Shift+M inside the editor to toggle Tab-key focus trapping.
Guided exercise
Guided exercise
This models grep's line matching only -- no real search tool is invoked. Write grepLines(text, pattern, invert) that splits text on newlines and returns matching lines (or non-matching lines if invert is true).
Checks: finds matching lines correctly · inverted mode finds non-matching lines correctly, modeling grep -v
Code editor. Press Escape then Tab to leave the editor if keyboard focus becomes trapped. Press Control+Shift+M inside the editor to toggle Tab-key focus trapping.
Stuck? Get a hint.
Independent exercise
Independent exercise
This models sed and awk's core patterns only -- no real tool is invoked. Write sedReplaceAll(line, oldStr, newStr) (replace EVERY occurrence, modeling sed's 's/old/new/g'). Write awkField(line, fieldNumber) (1-indexed, whitespace-separated field extraction, modeling awk '{ print $N }').
Checks: replaces every occurrence, matching sed's global flag · extracts the correct field by 1-indexed position · correctly handles the first field (awk fields start at 1, not 0)
Code editor. Press Escape then Tab to leave the editor if keyboard focus becomes trapped. Press Control+Shift+M inside the editor to toggle Tab-key focus trapping.
Stuck? Get a hint.
Common mistakes
- Using plain sed 's/old/new/' when every occurrence on a line needs replacing -- without the trailing 'g' flag, sed replaces only the FIRST occurrence per line, a common source of 'why didn't this replace everything' confusion.
- Forgetting that awk fields are whitespace-separated by default -- genuinely comma-separated or otherwise-delimited data needs an explicit field separator (awk -F,), or field extraction silently produces wrong results.
- Reaching for a complex awk or sed one-liner when a simple grep would answer the actual question -- matching this lesson's guidance of covering the common, foundational pattern of each tool, not always reaching for the most powerful option available.
Knowledge check
Takeaway
grep finds matching (or, with -v, non-matching) lines; sed's most common use is line-oriented substitution, with 'g' controlling whether every occurrence per line is replaced or just the first; awk automatically splits each line into whitespace-separated fields ($1, $2, ...) for structured, column-like text extraction.
Summary
grep -i/-v/-r/-n/-c cover the most common line-matching needs. sed 's/old/new/' replaces the first match per line; the trailing 'g' flag replaces every match. awk '{ print $N }' extracts the Nth whitespace-separated field (1-indexed); -F changes the field separator for non-whitespace-delimited data like CSV.
References
Your notes
Notes save automatically.
Finished this lesson?
Mark it complete to track your progress and schedule a future review.
AI tutor
The optional AI tutor isn't enabled in this deployment. All lessons, exercises, quizzes, and search work fully without it.