grep, sed and awk: Text Processing Examples in the Linux Terminal

grep, sed and awk are three classic Linux utilities for text processing. grep finds lines that match a pattern, sed replaces and deletes text in a stream without opening an editor, and awk splits a line into columns and can do arithmetic. Below are grep sed awk examples that you actually need when analyzing logs, config files and command output, plus common mistakes.

When to use which

All three tools read text line by line and write the result to standard output, so they are easy to chain with |. The difference lies in the task each one solves best.

A simple rule: if you only need to filter, use grep; if you need to change text, use sed; if the text has columns or you need arithmetic, use awk.

grep: finding lines by pattern

Basic grep examples

# Lines containing the word error in a file
grep error /var/log/syslog

# Case-insensitive: Error, ERROR, error
grep -i error app.log

# With line numbers
grep -n "Connection refused" app.log

# Recursively through a directory, with file names
grep -rn "DB_HOST" /etc/myapp/

# Only the names of files that contain a match
grep -rl "TODO" src/

# Invert: everything EXCEPT comments and empty lines
grep -v -e '^#' -e '^$' /etc/ssh/sshd_config

# Count the matching lines
grep -c " 500 " access.log

The -w flag searches for a whole word: grep -w log will not find login. The -F flag turns off regular expressions and searches for the string literally, which is handy when the pattern contains dots, brackets or asterisks.

Context around a found line

An error in a log is rarely clear from one line alone: you need the stack trace or the request that led to it. Context flags are made for this:

# 3 lines after the match (After)
grep -A 3 "Traceback" app.log

# 2 lines before (Before)
grep -B 2 "OOM" /var/log/kern.log

# 5 lines on both sides (Context)
grep -C 5 "segfault" /var/log/syslog

grep separates groups of matches with a -- line, so the output is easy to read.

grep and regular expressions

By default grep understands basic regular expressions (BRE). With the -E flag extended regular expressions (ERE) are enabled, where +, ?, | and brackets work without backslashes.

# Lines with errors or warnings
grep -E "ERROR|WARN" app.log

# Only IP addresses from the log, one per line (-o prints only the match)
grep -oE '([0-9]{1,3}\.){3}[0-9]{1,3}' access.log

grep -P: extract a value between known parts

GNU grep has the -P flag, which enables Perl-style regular expressions. With it you get \K ("forget everything matched so far") and lookahead, which lets you extract only the value:

$ echo 'user_id=4821 status=ok' | grep -oP 'user_id=\K[0-9]+'
4821

$ grep -oP '"version":\s*"\K[^"]+' package.json
2.4.1

The -P flag is not available in every grep build (for example, BSD grep on macOS does not have it), so in portable scripts it is better to use sed or awk.

grep -q in script conditions

grep -q prints nothing and only returns an exit code: 0 means found, 1 means not found. This lets you put the check directly into an if:

if grep -q "^PermitRootLogin no" /etc/ssh/sshd_config; then
  echo "root login is disabled"
fi

This is shorter than saving the output into a variable and checking it for emptiness, and grep stops at the first match.

sed: replacing and deleting text

Replacement: the s command

The main sed command is s/pattern/replacement/flags. Without the g flag only the first match in each line is replaced.

# Print the text with replacements (the file is not changed)
sed 's/foo/bar/' file.txt

# Replace all occurrences in the line
sed 's/foo/bar/g' file.txt

# Case-insensitive (GNU sed)
sed 's/error/ERROR/gI' app.log

# Another delimiter if the text contains slashes
sed 's|/var/www|/srv/www|g' nginx.conf

sed -i: replacing directly in a file

The -i flag writes the result back to the file. To keep the original, set a backup suffix:

# Creates config.ini.bak and changes config.ini
sed -i.bak 's/^debug=true/debug=false/' config.ini

# Replace in all .conf files in a directory
find /etc/myapp -name "*.conf" -exec sed -i 's/old-host/new-host/g' {} +

Before a mass edit, run the command without -i and look at the output first. On macOS (BSD sed) -i requires an argument: sed -i '' 's/a/b/' file.

Working with lines by number and by pattern

# Show lines 10–20 (-n turns off auto-print, p means print)
sed -n '10,20p' file.txt

# Delete line 5
sed '5d' file.txt

# Delete the last line
sed '$d' file.txt

# Delete empty lines and comments
sed -e '/^$/d' -e '/^#/d' nginx.conf

# Replace only in lines that contain "listen"
sed '/listen/s/80/8080/' nginx.conf

# Insert a line after a match (GNU sed)
sed '/^\[mysqld\]/a max_connections = 500' my.cnf

Back references and &

A part of the pattern in parentheses can be reused in the replacement with \1, \2, and the & character stands for the whole matched text.

# Swap first and last name (ERE via -E)
$ echo "Smith John" | sed -E 's/(\S+) (\S+)/\2 \1/'
John Smith

# Wrap all numbers in square brackets
$ echo "port 8080, timeout 30" | sed -E 's/[0-9]+/[&]/g'
port [8080], timeout [30]

# Turn the date 2026-10-14 into 10/14/2026
$ echo 2026-10-14 | sed -E 's/([0-9]{4})-([0-9]{2})-([0-9]{2})/\2\/\3\/\1/'
10/14/2026

awk: columns, conditions and counting

Printing the column you need

awk splits each line into fields by spaces and tabs: $1 is the first field, $2 the second, $NF the last one, and $0 the whole line.

# IP addresses from the nginx log (first column)
awk '{print $1}' /var/log/nginx/access.log

# User name and shell from /etc/passwd (separator is a colon)
awk -F: '{print $1, $7}' /etc/passwd

# Last column
df -h | awk '{print $NF}'

# Custom separator in the output
awk -F: -v OFS=';' '{print $1, $3}' /etc/passwd

Conditions: printing only the lines you need

A condition can be placed before a block in curly braces — awk runs the action only for matching lines.

# Users with UID 1000 and above (regular users, not system ones)
awk -F: '$3 >= 1000 {print $1}' /etc/passwd

# Requests with status code 500 and above (field 9 in the combined format)
awk '$9 >= 500' access.log

# Lines where the 6th column matches a regex
awk '$6 ~ /POST/' access.log

# Files larger than 100 MB from the ls -l output
ls -l | awk '$5 > 100*1024*1024 {print $9}'

BEGIN, END and NR: awk as a small program

The BEGIN block runs before the first line is read, and END runs after the last one. NR holds the number of the current line, and inside the END block it holds the total count.

# Sum of response sizes (field 10) in megabytes
awk '{sum += $10} END {printf "%.1f MB\n", sum/1024/1024}' access.log

# Average response time from the last column
awk '{s += $NF} END {if (NR) print s/NR}' timings.log

# Skip the CSV header
awk -F, 'NR > 1 {print $2}' users.csv

# Counting requests per IP: an associative array
awk '{count[$1]++} END {for (ip in count) print count[ip], ip}' access.log | sort -rn | head

The last example is a classic for log analysis: the count array keyed by IP replaces the sort | uniq -c combination and is easy to extend, for example to the total traffic per address.

Pipelines: grep, sed and awk together

The real power is in combining them. Each tool does its part, and | passes the result along.

Top 10 IPs with 5xx errors

grep ' 50[0-9] ' /var/log/nginx/access.log \
  | awk '{print $1}' \
  | sort | uniq -c | sort -rn | head -10

    412 203.0.113.45
    118 198.51.100.7
     36 192.0.2.19

Here grep roughly filters the lines, awk cuts out the field, and sort | uniq -c counts the repeats. The same can be written in one awk program: awk '$9 ~ /^5/ {c[$1]++} END {for (i in c) print c[i], i}' access.log | sort -rn | head — this is more precise, because it checks exactly the status code column.

Which URLs are opened most often

awk '{print $7}' access.log | sed 's/?.*//' | sort | uniq -c | sort -rn | head

sed here cuts off the query string, so that /search?q=a and /search?q=b are counted as one page.

Values from a config without comments

grep -v '^\s*#' /etc/myapp/app.env | grep '=' | awk -F= '{print $1}'

Errors for a specific hour

grep '14/Oct/2026:03:' access.log | awk '$9 >= 500 {print $4, $7, $9}'

Common mistakes

• Double quotes instead of single quotes in awk. In awk "{print $1}" the shell substitutes its own $1 before awk runs. Write the awk program and sed expressions in single quotes.
• Forgetting g in sed. Without it only the first match in a line is replaced, and the rest stay as they are.
• sed -i without a check. A wrong regex will silently damage the file. Run it without -i first, or use -i.bak.
• + and | do not work in grep. This is extended syntax: you need grep -E (or sed -E).
• The dot in a pattern. grep 192.168.1.1 will also match 192x168y1z1: the dot means "any character". Use grep -F or escape it as \..
• Spaces inside columns. If a field can contain spaces (CSV, paths), set the separator explicitly: awk -F, or awk -F'\t'.
• Useless cat. cat file | grep x works, but grep x file is shorter and does not start an extra process.

What to use instead of grep, sed and awk

For JSON these utilities are inconvenient: it is better to parse the structure with jq (jq '.items[] | select(.status=="active") | .name'), and YAML with yq. For fast recursive search in large projects people often use ripgrep (the rg command), which respects .gitignore by default. If you prefer Perl syntax, perl -pe 's/a/b/g' and perl -lane 'print $F[0]' replace sed and awk in one line. But grep, sed and awk are available on almost any server without installing anything, so they are the first things to learn.

Summary

grep finds lines, sed changes them, awk works with columns and counts. Remember the basic set: grep -rniE and -A/-B/-C, sed 's/.../.../g' with -i.bak and -n 'Np', awk -F with $1, $NF, conditions and the END block. Almost any log analysis is built from these blocks. These grep sed awk examples, together with find, jq and curl, are worked through in practice in the Linux course lecture on text processing in the terminal.