Sed

• Stands for 'stream editor' which allows us to filter and transform text.
• Often used in conjunction with regex

Substitution

Replaces all instances of day with night inside myfile.txt. Note the g at the end.

sed 's/day/night/g' myfile.txt

Sed Removing stuff

Do not print the first line

sed '1d' file.txt

Remove the first character of every line

sed 's/^.//' file

Sed Removing stuff

Remove the last character of every line

sed 's/.$//' file

Remove lines 7 thru 9 of a file

sed '7,9d' filename.txt

Remove the line containing the string Fred

sed '/Fred/d' filename.txt

Sed Print every nth line beginning with in the file

Print only the first line

sed -n '1p' file.txt

Print every third line starting from line 3 (which will print lines 3, 6, 9, etc)

sed -n '0~3p' file.txt

Print every fifth line starting with the second line (which will print lines 2, 7, 12, 17, etc)

sed -n '2~5p' file.txt

Sed Print a range of lines

Print lines 1 through 4

sed -n '1,4p' file.txt

Print lines 2 through 4

sed -n '2,4p' file.txt

awk

Awk is hugely powerful, but we will just look at how it can be used for text pattern scanning.

It is also often used with Regular Expressions.

Here are some examples:

print the first and third columns in the output of a command

• ls -l | awk '{ print $1 $3 }'

awk

print the first column in the output of a command

• ps aux | awk '{ print $1 }'

print the first column in the output of a command and add text.

• ps aux | awk '{ print "this process is owned by " $1 }'

print first column of values in file separated by :

• awk -F: '{ print $1 }' /etc/passwd

awk

print second and eighth column separated by ;

• awk -F';' '{ print $2, $8 }' master_file_room_104.dhcp

print first column of values in a file

• awk '{ print $1 }' /etc/fstab

Using regular expressions search for lines that start with UUID and print 3rd column of results

• awk '/^UUID/ {print $3}' /etc/fstab

awk

Figure this one out - if you don't know what the lspci -v command does - read the man page.

• lspci -v | awk '/VGA/ { print $6 }'

Awk does Arithmetic operations too (assuming grades is a file with 3 scores on each row)

• awk '{ print "the average is ", ($1+$2+$3)/3 }' grades

Other resources

• Unix School SED
• Awk Full Tutorial
• Awk and Regular Expressions