SED

From Network Security Wiki

Basics

  • Using SED to make the small change of replacing x with y:
generate_data | sed 's/x/y/g'
  • SED on file:
sed 's/regexp/replacement/g' inputFileName > outputFileName
  • Symbols:
Symbol Description
^ matches the beginning of the line.
$ matches the end of the line.
* matches zero or more occurrences of the previous character.
?
q Quit
.* zero or more of any character
# comments
: label
{....} Block
= print line number
a \ Append
b label Branch
c \ change
d and D Delete
g and G Get/Grep
h and H Hold
i \ Insert
l Look
n and N Next
p and P Print
r filename Read File
s/..../..../ Substitute
t label Test
w filename Write Filename
x eXchange
y/..../..../ Transform
  • Sed Pattern Flags:
Symbol Description
/g Global
/I Ignore Case
/p Print
/w filename Write Filename

Examples

  • This will remove 2 numbers e.g: (123),(376) from input:
sed -e 's/([0-9][0-9][0-9])//g'
  • SED Joke to represent MSDOS Prompt:
export PS1="C:\$( pwd | sed 's:/:\\\\\\:g' )\\> "

Numbering

  • Number each line of a file (simple left alignment). Using a tab (note on '\t' at end of file) instead of space will preserve margins.
sed = filename | sed 'N;s/\n/\t/'
  • Number each line of a file (number on left, right-aligned)
sed = filename | sed 'N; s/^/     /; s/ *\(.\{6,\}\)\n/\1  /'
  • Number each line of file, but only print numbers if line is not blank
sed '/./=' filename | sed '/./N; s/\n/ /'

File Spacing

  • double space a file
sed G
  • double space a file which already has blank lines in it. Output file should contain no more than one blank line between lines of text.
sed '/^$/d;G'
  • triple space a file
sed 'G;G'
  • undo double-spacing (assumes even-numbered lines are always blank)
sed 'n;d'
  • insert a blank line above every line which matches "regex"
sed '/regex/{x;p;x;}'
  • insert a blank line below every line which matches "regex"
sed '/regex/G'
  • insert a blank line above and below every line which matches "regex"
sed '/regex/{x;p;x;G;}'

Text Conversion

  • IN UNIX ENVIRONMENT: convert DOS newlines (CR/LF) to Unix format.
sed 's/.$//'                         # assumes that all lines end with CR/LF
  • IN UNIX ENVIRONMENT: Convert Unix newlines (LF) to DOS format.
sed 's/$'"/`echo \\\r`/"             # command line under bash
  • IN DOS ENVIRONMENT: convert Unix newlines (LF) to DOS format.
sed "s/$//" 
sed -n p
  • delete leading whitespace (spaces, tabs) from front of each line, aligns all text flush left
sed 's/^[ \t]*//'                    # '\t' is a tab character (0x09),you should press the TAB key instead.
  • delete trailing whitespace (spaces, tabs) from end of each line
sed 's/[ \t]*$//'                    # '\t' is a tab character (0x09),you should press the TAB key instead.
  • delete BOTH leading and trailing whitespace from each line
sed 's/^[ \t]*//;s/[ \t]*$//'
  • insert 5 blank spaces at beginning of each line (make page offset)
sed 's/^/     /'
  • align all text flush right on a 79-column width
sed -e :a -e 's/^.\{1,78\}$/ &/;ta'  # set at 78 plus 1 space
  • center all text in the middle of 79-column width.
sed  -e :a -e 's/^.\{1,77\}$/ & /;ta'  
sed  -e :a -e 's/^.\{1,77\}$/ &/;ta' -e 's/\( *\)\1/\1/'
  • add a blank line every 5 lines (after lines 5, 10, 15, 20, etc.)
sed 'n;n;n;n;G;'
  • substitute (find and replace) "foo" with "bar" on each line
sed 's/foo/bar/'             	      # replaces only 1st instance in a line
sed 's/foo/bar/4'          	      # replaces only 4th instance in a line
sed 's/foo/bar/g'          	      # replaces ALL instances in a line
sed 's/\(.*\)foo\(.*foo\)/\1bar\2/'   # replace the next-to-last case
sed 's/\(.*\)foo/\1bar/'              # replace only the last case
  • substitute "foo" with "bar" ONLY for lines which contain "baz"
sed '/baz/s/foo/bar/g'
  • substitute "foo" with "bar" EXCEPT for lines which contain "baz"
sed '/baz/!s/foo/bar/g'
  • change "scarlet" or "ruby" or "puce" to "red"
sed 's/scarlet/red/g;s/ruby/red/g;s/puce/red/g'

Printing

  • print first 10 lines of file (emulates behavior of "head")
sed 10q
  • print first line of file (emulates "head -1")
sed q
  • grep for AAA and BBB and CCC (in any order)
sed '/AAA/!d; /BBB/!d; /CCC/!d'
  • grep for AAA and BBB and CCC (in that order)
sed '/AAA.*BBB.*CCC/!d'
  • grep for AAA or BBB or CCC (emulates "egrep")
sed -e '/AAA/b' -e '/BBB/b' -e '/CCC/b' -e d

Deletion

  • delete all lines except duplicate lines (emulates "uniq -d").
sed '$!N; s/^\(.*\)\n\1$/\1/; t; D'

Special

  • get Usenet/e-mail message header
sed '/^$/q'                # deletes everything after first blank line
  • get Usenet/e-mail message body
sed '1,/^$/d'              # deletes everything up to first blank line
  • get return address header
sed '/^Reply-To:/q; /^From:/h; /./d;g;q'


One-Liners

Changing 1st parameter values 

sed 's/2M/200M/' php.ini > php.ini.new

TEE

Using tee to append to config file 

echo "net.ipv4.ip_forward = 1" |  tee -a /etc/sysctl.conf


{{#widget:DISQUS |id=networkm |uniqid=SED |url=https://aman.awiki.org/wiki/SED }}

Retrieved from "https://aman.awiki.org/wiki/SED?oldid=1600"