I’ve always been a bit hit-or-miss with sed and haven’t understood it to the level to which I wanted. Today I aimed to change that and these are my notes from sitting down to learn it.

Replace a string in a file (first occurrence per line only):

cat file.txt | sed 's/oldstring/newstring/'

Replace a string in a file (all occurrences per line, i.e., globally):

cat file.txt | sed 's/oldstring/newstring/g'

Use your own delimiters by just dropping them in place of /:

cat file.txt | sed 's(oldstring(newstring('

Another example of using your own delimiter (useful when file contains / chars):

[11:25] simas@macbook ~/D/sed Tutorial> cat file3.txt
/usr/bin/python
/usr/bin/python3
[11:26] simas@macbook ~/D/sed Tutorial> cat file3.txt | sed 's_/usr_/var/bin_'
/var/bin/bin/python
/var/bin/bin/python3

& denotes the matched pattern:

[11:27] simas@macbook ~/D/sed Tutorial> cat file3.txt
/usr/bin/python
/usr/bin/python3
[11:27] simas@macbook ~/D/sed Tutorial> cat file3.txt | sed 's/usr/&&/'
/usrusr/bin/python
/usrusr/bin/python3
[11:27] simas@macbook ~/D/sed Tutorial>

To enable extended regular expressions use -i (Linux) or -E (macOS).

Reference a regex capture group using \1, \2, etc. Capture groups denoted using \( and \) (or ( and ) if you use -E flag).

[11:32] simas@macbook ~/D/sed Tutorial> echo abcd123 | sed 's/\([a-z]*\).*/\1/'
abcd

Add additional sed flags after last delimiter. e.g., g in 's/thing/thing2/g'

Useful flags:

  • g: global (find and replace all instances of pattern in line, not just first one)
  • I: case insensitive (does not work on macOS, as its BSD-based sed doesn’t support it)

Specify which occurrence to operate on using 1, 2, 3, etc:

[11:39] simas@macbook ~/D/sed Tutorial> cat file4.txt
blah blah blah
blah2 blah2 blah2
hello
[11:39] simas@macbook ~/D/sed Tutorial> cat file4.txt | sed 's/blah/oopsies/2'
blah oopsies blah
blah2 oopsies2 blah2
hello

Change a file in place and make a backup using -i:

[11:42] simas@macbook ~/D/sed Tutorial> cat file4.txt
blah blah blah
blah2 blah2 blah2
hello
[11:42] simas@macbook ~/D/sed Tutorial> sed -i -backup 's/blah/doyouevensedbruh/g' file4.txt
[11:42] simas@macbook ~/D/sed Tutorial [1]> diff -uw file4.txt file4.txt-backup
--- file4.txt   2019-06-15 11:42:47.000000000 -0700
+++ file4.txt-backup    2019-06-15 11:39:41.000000000 -0700
@@ -1,3 +1,3 @@
-doyouevensedbruh doyouevensedbruh doyouevensedbruh
-doyouevensedbruh2 doyouevensedbruh2 doyouevensedbruh2
+blah blah blah
+blah2 blah2 blah2
 hello

Perform multiple substitutions in a single command using -e:

[11:48] simas@macbook ~/D/sed Tutorial [1]> cat file5.txt
abc aBc ABC ABc
[11:48] simas@macbook ~/D/sed Tutorial> sed -e 's/abc/123/' < file5.txt
123 aBc ABC ABc
[11:48] simas@macbook ~/D/sed Tutorial> sed -e 's/abc/123/' -e 's/ABC/123/' < file5.txt
123 aBc 123 ABc

You can operate on multiple files at once:

[11:50] simas@macbook ~/D/sed Tutorial> cat file.txt
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
[11:50] simas@macbook ~/D/sed Tutorial> cat file2.txt
oops ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
[11:50] simas@macbook ~/D/sed Tutorial>
[11:50] simas@macbook ~/D/sed Tutorial> sed 's/ipsum/123/' file.txt file2.txt
Lorem 123 dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
oops 123 dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Matching a line using regex before performing substitution:

[18:16] james_simas@widget ~/Downloads $ cat example_file.txt
gtm pool /Common/p_example {
    members {
        /Common/2001:668:fc78:0:f811:3ef2:fed7:98cc:vs6_2001_668_fc78_0_f811-3ef2-fed7_98cc-80 {
            order 0
        }
    }
    monitor /Common/gateway_icmp
}
[18:16] james_simas@widget ~/Downloads $ cat example_file.txt | sed '/Common.*vs6/s/:/\\\\:/g'
gtm pool /Common/p_example {
    members {
        /Common/2001\\:668\\:fc78\\:0\\:f811\\:3ef2\\:fed7\\:98cc\\:vs6_2001_668_fc78_0_f811-3ef2-fed7_98cc-80 {
            order 0
        }
    }
    monitor /Common/gateway_icmp
}
[18:16] james_simas@widget ~/Downloads $