Handy Sed One-Liners for Text Manipulation and Processing
Tags: sed, unix, scripting
Published: 2024-04-01 02:34:35

FILE SPACING:

Double space a file:

```bash
sed G
```

Double space a file which already has blank lines, output file should contain no more than one blank line between lines of text:

```bash
sed '/^$/d;G'
```

Triple space a file:

```bash
sed 'G;G'
```

Undo double-spacing (assumes even-numbered lines are always blank):

```bash
sed 'n;d'
```

Insert a blank line above every line which matches "regex":
```bash
sed '/regex/{x;p;x;}'
```

Insert a blank line below every line which matches "regex":

```bash
sed '/regex/G'
```

Insert a blank line above and below every line which matches "regex":

```bash
sed '/regex/{x;p;x;G;}'
```

NUMBERING:

Number each line of a file (simple left alignment):

```bash
sed = filename | sed 'N;s/\n/\t/'
```

Number each line of a file (number on left, right-aligned):

```bash
sed = filename | sed 'N; s/^/     /; s/ *\(.\{6,\}\)\n/\1  /'
```

Number each line of file, but only print numbers if line is not blank:

```bash
sed '/./=' filename | sed '/./N; s/\n/ /'
```

Count lines (emulates "wc -l"):

```bash
sed -n '$='
```

TEXT CONVERSION AND SUBSTITUTION:

Convert DOS newlines (CR/LF) to Unix format:

```bash
sed 's/.$//'
```

Convert Unix newlines (LF) to DOS format:

```bash
sed "s/$/`echo -e \\\r`/"
```

Delete leading whitespace (spaces, tabs) from front of each line:

```bash
sed 's/^[ \t]*//'
```

Delete trailing whitespace (spaces, tabs) from end of each line:

```bash
sed 's/[ \t]*$//'
```

Delete both leading and trailing whitespace from each line:

```bash
sed 's/^[ \t]*//;s/[ \t]*$//'
```

Insert 5 blank spaces at beginning of each line (make page offset):

```bash
sed 's/^/     /'
```

Align all text flush right on a 79-column width:

```bash
sed -e :a -e 's/^.\{1,78\}$/ &/;ta'
```

Center all text in the middle of 79-column width:

```bash
sed  -e :a -e 's/^.\{1,77\}$/ & /;ta'
```

Substitute (find and replace) "foo" with "bar" on each line:

```bash
sed 's/foo/bar/'
```

Substitute "foo" with "bar" EXCEPT for lines which contain "baz":
```bash
sed '/baz/!s/foo/bar/g'
```

Reverse order of lines (emulates "tac"):

```bash
sed '$!G;h;$!d'
```

Join pairs of lines side-by-side (like "paste"):

```bash
sed '$!N;s/\n/ /'
```

If a line ends with a backslash, append the next line to it:

```bash
sed -e :a -e '/\\$/N; s/\\\n//; ta'
```

Add a blank line every 5 lines:

```bash
gsed '0~5G'
```

SELECTIVE PRINTING OF CERTAIN LINES:

Print first 10 lines of file (emulates behavior of "head"):

```bash
sed 10q
```

Print first line of file (emulates "head -1"):

```bash
sed q
```

Print the last 10 lines of a file (emulates "tail"):

```bash
sed -e :a -e '$q;N;11,$D;ba'
```

Print only lines which match regular expression (emulates "grep"):

```bash
sed -n '/regexp/p'
```

Print only lines which do NOT match regexp (emulates "grep -v"):

```bash
sed '/regexp/!d'
```

Print paragraph if it contains AAA:

```bash
sed -e '/./{H;$!d;}' -e 'x;/AAA/!d;'
```

Print section of file from regular expression to end of file:

```bash
sed -n '/regexp/,$p'
```

SELECTIVE DELETION OF CERTAIN LINES:

Print all of file EXCEPT section between 2 regular expressions:

```bash
sed '/Iowa/,/Montana/d'
```

Delete duplicate, consecutive lines from a file (emulates "uniq"):

```bash
sed '$!N; /^\(.*\)\n\1$/!P; D'
```

Delete all blank lines from a file:

```bash
sed '/^$/d'
```

Delete all consecutive blank lines from file except the first:

```bash
sed '/./,/^$/!d'
```

Delete all trailing blank lines at end of file:

```bash
sed -e :a -e '/^\n*$/{$d;N;ba' -e '}' 
```

SPECIAL APPLICATIONS:

Remove nroff overstrikes (char, backspace) from man pages:

```bash
sed "s/.`echo \\\b`//g"
```

Get Usenet/e-mail message header:

```bash
sed '/^$/q'
```

Get Usenet/e-mail message body:

```bash
sed '1,/^$/d'
```

Add a leading angle bracket and space to each line (quote a message):

```bash
sed 's/^/> /'
```

Delete leading angle bracket & space from each line (unquote a message):

```bash
sed 's/^> //'
```

Remove most HTML tags:

```bash
sed -e :a -e 's/<[^>]*>//g;/</N;//ba'
```

Extract multi-part uuencoded binaries:

```bash
sed '/^end/,/^begin/d' file1 file2 ... fileX | uudecode
```

SORTING PARAGRAPHS OF FILE ALPHABETICALLY:

```bash
sed '/./{H;d;};x;s/\n/={NL}=/g' file | sort | sed '1s/={NL}=//;s/={NL}=/\n/g'
```

OPTIMIZING FOR SPEED:
Substitution executes more quickly if the "find" expression is specified before the "s/.../.../" instruction:

```bash
sed '/foo/ s/foo/bar/g' filename
```