sed

stream editor. Noninteractively edits text as it streams thru. Performs editing on lines, outputs each to standard output. Input file is unchanged. Uses ed commands, and pattern matching as in grep. sed goes thru file once. File lines stream thru, sed will or won't edit the line. Useful for editing streaming text in a pipeline, very large files, automatic editing.
sed can do everything grep and tr can do.
sed and awk, by Dougherty

sed 'editing_command(s)' file(s)

editing_commands / script:
1.) line specifier (default is every line) to indicate which lines and/or
2.) editing instruction to operate on the matching lines
1a.) line number or range of numbers: 

 3      # third line in file
 5,8    # lines 5 thru 8 [lines 5 and 8 by separate duplicated editing cmds]
 10,$   # lines 10 thru end of file ($ is ed's last line specifier)
Unlike ed, no relative line numbering (e.g. $-9 ninth from last, +2 two lines forward)
1b.) regular expression pattern enclosed in / to specify lines: 
 /big/          # all lines containing string big
 /[Dd]o/        # all lines containing Do or do
 /^f[a-z]/      # all lines starting f followed by letter
 /[0-9][0-9][0-9][0-9][0-9]/     #all lines with "zip code"
 /^[0-9][0-9][0-9][0-9][0-9]$/   #all lines consisting of "zip code"
 /regex/I   case-insensitive
a&b combined): 
 1,/enough/     # from line 1 to first line with enough    
 /word/,20      # from first line with word to line 20
! opposite of specified lines: 
  /word/,20!  # from line 1 to line before first line with word and lines 21 thru end of file
2.) editing instructions:
d --delete line
s --substitution
q --quit
sed automatically outputs ("prints") each line, so p duplicates lines
$ sed '' myfile             # cat
$ sed 'p' myfile            # each line duplicated.  [use?]
$ sed 'd' myfile            # delete every line, i.e. no output
$ sed '10 q' myfile         # head
$ sed '/aardvark/d' myfile  # delete lines with aardvark (output all others):  grep -v aardvark
$ sed '/aardvark/!d' myfile # delete every line that doesn't contain aardvark: grep aardvark  

$ sed '/^$/d' myfile        # delete empty lines: grep '.'
$ sed '/^[  ]*$/d' myfile   # delete blank and empty lines  (space and Tab)

$ sed '1,/Chapter 1/d' myfile # delete lines 1 thru first w/Chapter 1
$ sed '1,5d' myfile           #delete first 5 lines. (output lines 6 to end: tail +6)

-n option: only print lines explicitly told to with p. Extract a line or range of lines. 
$ sed -n 24p  myfile             # extract line 24
$ sed -n 24,28p  myfile          # extract lines 24 thru 28
$ sed -n '1,10p' myfile          # head
    # only output lines 1 thru 10. Without -n would dup 1-10, then rest of file output
$ sed -n '/LINUX/p' myfile        # only lines with LINUX output
$ sed -n '/LINUX/Ip' myfile       # only lines with linux in any case output
$ sed -n '/pattern/ p' myfile    # grep 'pattern'
$ sed -n '$p' myfile             # only last line printed. tail -1
$ sed -n '/^[   ]*$/!p' myfile   # delete blank lines.  Non-blank lines are output.
$ sed -n '/./p' myfile           # non-empty lines output
$ sed -n '/[^ ]/p' myfile        # lines without a space

s/oldstring/newstring/ Oldstring can be grep-like pattern: ^ $ . * ? [^] But not egrep | () + unless -r option
Can use other delimiters instead of /
g -- global (each occurence per line, else only first) 
$ sed 's/Bob/Robert/g' myfile      #replace every Bob with Robert  
$ sed '1,6s/Bob/Robert/' myfile    #first Bob on a line to Robert in lines 1 thru 6  
$ sed 's/^/   /' myfile       #add 3 spaces at beginning of each line.  g would be useless
$ sed -r 's/ +/ /g' myfile    # compress multiple blanks (run of spaces) to single blank
                              #  need 1 or 2 spaces before the +     tr -s ' ' ' '
$ sed -r 's/ +$//' myfile     # delete trailing blanks  Right-trim
$ sed -r 's/^ +//' myfile     # delete leading blanks   Left-trim
$ sed -r 's/ +/,/g' myfile    # change runs of spaces to comma (field separator)
$ sed 's/<[^>]*>//' myfile # delete HTML tags (assumes no more than one per line...)
newstring can include & which means entire matched oldstring: 
$ sed 's/UNIX/"UNIX"/g' myfile    # replace all UNIX with "UNIX".  Shell "" escaped because in ''
$ sed 's/UNIX/"&"/g' myfile   # same
& useful if oldstring is a pattern: 
$ sed 's/.*/(&)/' myfile                  # parenthesize line
$ sed 's/U[nN][iI][xX]/"&"/g' myfile      # replace UNIX, UnIx ... with quoted UNIX, etc
$ sed 's/[^ ][^ ]*/<&>/g' myfile    # angle bracket each word
$ sed -r 's/[^ ]+/<&>/g' myfile     # angle bracket each word
Escape sed metacharacter with \ (sed's \)
ex. escape the & 
$ sed 's/ and/ \&/g' myfile               # replace  and with  &
tagging: parts of oldstring enclosed in \( \) and referred to by number in new string 
$ sed 's/\(Jack\) and \(Jill\)/\2 and \1/g' myfile        # replace Jack and Jill with Jill and Jack
$ sed 's/\([jJ]ack\) and \([jJ]ill\)/\2 and \1/g' myfile  # same, but jack/Jack and jill/Jill
Ex. try to change all seperate with separate:
s/[sS]eperate/[sS]eparate/ Pattern not allowed in newstring (what to make the new string?) so: 
$ sed 's/\([sS]ep\)erate/\1arate/g' myfile
Ex. file of Lastname, Firstname to Firstname Lastname 
$ sed 's/\([^,]*\), \(.*\)/\2 \1/' myfile
w --write to a file
Ex. delete lines with 'billybob', copy them to a file
$ sed '/billybob/dw debobbed' myfile #?????
Ex. change billybob to garp, copy those lines to file (and go to output too)
$ sed 's/billybob/garp/gw garpfile' myfile #????

Ex. trim output of who to only user name: (first word of line with cut -f1) 
$ who | sed 's/ .*$//'      
# all chars from first space to end of line matched, replaced by null string. $ not necessary.

$ sed 's/^...//' myfile        # cut -c4-
   # delete first 3 chars of each line.  ^ not needed.  <3 chars in line ignored
$ sed 's/\(...\).*/\1/' myfile   # delete all but first three chars: cut -c1-3
$ sed 's/...//g' myfile  # chars of line deleted until 0,1,or 2 left  (last ones in line)
$ sed 's/.$//' myfile    # delete last char of each line
$ sed 's/[^A-Za-z0-9]//g' myfile  # delete all non-alphanumerics: tr -dc 'a-zA-Z0-9 \n'

Several editing commands simultaneously. Must be on separate lines. Perform first, then second,...on input line.
$ sed 's/Dodger/Giant/g
>s/Giant/Yankee/g'  myfile 
# some Giant were originally Dodger
Ex. change billybob to garp, copy to file, delete lines with garp
$ sed 's/billybob/garp/gw garpfile
>        /garp/d'   myfile
Ex. change billybob to garp, copy to file, delete original lines with garp 
$ sed -n '/garp/!p                  # lines without garp printed
>          s/billybob/garp/gw garpfile'  myfile
-f option: editing cmds/script from file (shell escape quotes not needed in file): remember complex, multiple use... 
$ sed -f edit1 -f edit2 myfile      # as if cmds were on cmd line
-e option: intermingling cmds from file and cmd line: 
$ sed -e 's/New York/Boston/g' -f edit1 myfile
Ex. Double space a file (add newline at end of each line): 
$ sed 's/$/\  #sed escape (\) the newline that would terminate editing instruction
>/' myfile
N.B. Newline is never matched, thus can't delete them.(Use tr)

Ex. Make one word per line (word = non-whitespace chars) 
$ sed 's/ /\
>/g' myfile                   
# each space replaced by newline.  Might be some runs of blank lines if were run of spaces.
Ex. Replace each sequence of one or more spaces and tabs with newline: 
$ sed -r 's/[  ]+/\
>/g' myfile
Ex. One char per line: 
$ sed 's/./&\
>/g' myfile   # each and every char change to itself and newline
Ex. Delete all non-letters, keep words separate: 
$ sed 's/[^A-Za-z]/ /g' myfile     # replace non-letters by space
Ex. Delete blank lines, strip punctuation, one word per line: 
$ sed 's/[^A-Za-z]/ /g
>s/ /\
>/g
>/^ *$/d' myfile

s/[^a-zA-Z ]//g    # delete non-letters except spaces
s/  */\            # runs of spaces to newline
/g
/^ *$/d            # delete blank lines