Bash Stream Editor (sed)

sed Basics

sed 's/old/new/' file

sed '2d' file

sed '2i\text' file

sed -i 's/old/new/' file

🔹 Basic Substitution

The s (substitute) command is the workhorse of sed, following the syntax s/pattern/replacement/flags. By default, it replaces only the first occurrence of the pattern on each line. This is used for countless text transformations: updating version strings in files, changing configuration values, standardizing date formats, or cleaning extraneous characters. Because sed works on streams, it can process input of any size, making it ideal for in-place edits or filtering pipelines where files should not be loaded entirely into memory.

# Replace first occurrence on each line
sed 's/error/warning/' app.log

# Replace all occurrences (global flag)
sed 's/error/warning/g' app.log

# Replace only on specific line
sed '3s/old/new/' file.txt

# Case-insensitive replacement
sed 's/error/warning/gi' app.log

error in line 1
error in line 2

warning in line 1
warning in line 2

🔹 In-Place Editing

The -i option is a powerful but dangerous feature that modifies input files directly. Always test a sed command without -i first to verify its effect. On BSD systems like macOS, -i requires a backup extension argument (e.g., -i.bak). This option is invaluable for batch editing across multiple files, such as renaming a variable in all scripts in a directory or updating a server URL in numerous configuration files. It automates what would otherwise be a tedious, error-prone manual process.

# Edit file in place (Linux)
sed -i 's/old/new/g' file.txt

# Edit with backup (creates file.txt.bak)
sed -i.bak 's/old/new/g' file.txt

# Edit multiple files
sed -i 's/error/warning/g' *.log

# macOS requires backup extension
sed -i '' 's/old/new/g' file.txt

🔹 Deleting Lines

The d command instructs sed to delete entire lines from the pattern space, preventing them from being output. Lines can be targeted by number (5d), range (10,20d), or pattern (/^#/d deletes comment lines). This is a fundamental cleanup operation. It's used to strip blank lines from documents, remove debug print statements from code before production, filter out specific log levels, or extract a specific section of a file by deleting everything outside it.

# Delete line 3
sed '3d' file.txt

# Delete lines 2 to 5
sed '2,5d' file.txt

# Delete last line
sed '$d' file.txt

# Delete lines matching pattern
sed '/error/d' app.log

# Delete blank lines
sed '/^$/d' file.txt

# Delete comment lines
sed '/^#/d' config.txt

Line 1
Line 2
Line 3
Line 4

Line 1
Line 4

🔹 Inserting and Appending Text

The i and a commands allow adding new lines of text before or after a specified address, respectively. The address can be a line number or a pattern. For example, sed '3i\# This is a header' file.txt inserts a header comment before line 3. These commands are perfect for modifying files in an automated way: adding license headers to source files, inserting new configuration directives, or appending timestamps or markers to log entries as they are processed in a pipeline.

# Insert text before line 2
sed '2i\New line here' file.txt

# Append text after line 3
sed '3a\New line here' file.txt

# Insert at beginning of file
sed '1i\Header text' file.txt

# Append at end of file
sed '$a\Footer text' file.txt

# Insert before matching line
sed '/pattern/i\New line' file.txt

Line 1
Line 2
Line 3

Line 1
Inserted
Line 2
Line 3

🔹 Printing Specific Lines

Using the -n option to suppress automatic printing, combined with the p command, lets sed act as a line extractor. For instance, sed -n '5p' prints only the 5th line, and sed -n '/start/,/end/p' prints a range between two patterns. This gives sed functionality similar to head, tail, and grep, but with a unified syntax for addressing by line number and pattern, making it a versatile tool for extracting specific sections from text data.

# Print line 5
sed -n '5p' file.txt

# Print lines 2 to 5
sed -n '2,5p' file.txt

# Print last line
sed -n '$p' file.txt

# Print lines matching pattern
sed -n '/error/p' app.log

# Print every 2nd line
sed -n '1~2p' file.txt

Line 2
Line 3
Line 4

🔹 Multiple Commands

Multiple sed commands can be executed in sequence on the same input using the -e flag or by separating them with semicolons within a single quote block. For complex transformations, commands can also be placed in a script file and invoked with -f. This allows you to perform a series of edits—like substituting text, deleting lines, and then appending a footer—in one efficient pass over the data, minimizing I/O operations and keeping related transformations together in a readable, maintainable script.

# Multiple commands with -e
sed -e 's/error/warning/g' -e 's/fail/issue/g' app.log

# Multiple commands with semicolon
sed 's/old/new/g; s/foo/bar/g' file.txt

# Delete and substitute
sed '/^#/d; s/error/warning/g' file.txt

# Use script file
sed -f script.sed file.txt

s/error/warning/g
s/fail/issue/g
/^#/d

🔹 Using Regular Expressions

sed integrates regular expressions deeply, using them not just in the s command but also for selecting lines to process. Anchors (^ for line start, $ for line end), character classes, and quantifiers allow for precise pattern specification. This enables sophisticated tasks like normalizing whitespace, validating input formats, or extracting substrings. Mastery of regex in sed unlocks its full potential, transforming it from a simple line editor into a powerful text processing engine.

# Replace at start of line
sed 's/^error/warning/' app.log

# Replace at end of line
sed 's/error$/warning/' app.log

# Replace whole words only
sed 's/\btest\b/exam/g' file.txt

# Remove leading whitespace
sed 's/^[ \t]*//' file.txt

# Remove trailing whitespace
sed 's/[ \t]*$//' file.txt

# Replace multiple spaces with single space
sed 's/  */ /g' file.txt

Line without leading spaces
Another line
Third line

🔹 Backreferences and Capture Groups

Parentheses in a sed pattern create capture groups, which can be referenced in the replacement part with \1, \2, etc. This is a game-changer for reformatting text. A classic example is rearranging date formats: sed 's/\([0-9]*\)-\([0-9]*\)-\([0-9]*\)/\3/\2/\1/' changes YYYY-MM-DD to DD/MM/YYYY. This ability to remember and reuse parts of the match makes sed exceptionally powerful for data reshaping, templating, and complex search-and-replace operations where the replacement is derived from the matched content.

# Swap two words
sed 's/$$.*$$ $$.*$$/\2 \1/' file.txt

# Extract and reformat dates
sed 's/$$[0-9]\{4\}$$-$$[0-9]\{2\}$$-$$[0-9]\{2\}$$/\3\/\2\/\1/' dates.txt

# Add quotes around words
sed 's/$$[a-zA-Z]*$$/"\1"/g' file.txt

# Duplicate words
sed 's/$$.*$$/\1 \1/' file.txt

John Doe
2024-01-15

Doe John
15/01/2024

🔹 Practical sed Examples

sed shines in automation scripts for system administration, data cleaning, and log file management. Practical examples include: removing trailing whitespace from all lines in a project, converting between DOS and Unix line endings (sed 's/\r$//'), extracting specific columns from a non-CSV formatted report, or incrementing version numbers in build files. These tasks, often repetitive and critical, are perfectly suited for sed, which performs them reliably and can be integrated into CI/CD pipelines or cron jobs for hands-off operation.

# Remove HTML tags
sed 's/<[^>]*>//g' page.html

# Convert Windows line endings to Unix
sed 's/\r$//' file.txt

# Add line numbers
sed = file.txt | sed 'N;s/\n/\t/'

# Comment out lines
sed 's/^/#/' config.txt

# Uncomment lines
sed 's/^#//' config.txt

# Extract email addresses
sed -n 's/.*$$[a-zA-Z0-9._%+-]*@[a-zA-Z0-9.-]*\.[a-zA-Z]\{2,\}$$.*/\1/p' file.txt

# Replace environment variables
sed 's/${USER}/john/g; s/${HOME}/\/home\/john/g' script.sh

🔹 Common sed Options

The sed command provides a variety of command-line options that significantly enhance its functionality for text processing. Key options include -i for in-place file editing, which allows direct modification of input files, often with a backup extension like .bak. The -e flag enables the specification of multiple editing scripts in a single command, while -n suppresses automatic printing of pattern space, useful with the p command to control output precisely. Extended regular expressions can be activated with -E or -r (depending on your sed version), providing access to more powerful pattern-matching syntax for complex transformations.