Much of a script's work is text: logs, lists and command output. A handful of tools, joined with pipes, handle nearly all of it.
cut takes columns. -d names the separator and -f the fields: cut -d: -f1 /etc/passwd prints every user name, since that file uses colons and the name is first.
sort sorts (-n by number, -r reversed), and uniq -c counts runs of the same line. Together they answer "how many of each?": sort | uniq -c | sort -rn puts the most common first. head then keeps the top few.
$ cut -d' ' -f1 access.log | sort | uniq -c | sort -rn | head -n 3
912 203.0.113.9
455 198.51.100.4
87 192.0.2.77That one line lists the three addresses that visited a website the most: take the first field of each log line, sort them so repeats are together, count each, sort by the count, keep three.
sed edits text as it flows past. Its most used command is substitute: sed 's/teal/mallard/g' notes.txt prints the file with every teal replaced (g means every match on a line, not just the first). sed -i changes the file itself, so try it without -i first.
awk splits each line into fields on spaces, $1, $2 and so on, and is handy when columns are lined up with several spaces, where cut struggles: ps aux | awk '{print $2, $11}' prints each process's PID and command. $NF is the last field.
/etc/passwd has fields separated by colons, the user name first. Print just the names with cut.
| cut -d, -f1,3 | Fields 1 and 3, split on commas. |
|---|---|
| sort -n -r, sort -rn | By number; reversed. |
| uniq -c | Count runs of the same line. |
| sed 's/old/new/g' file | Replace text (-i edits the file). |
| awk '{print $1, $NF}' | The first and last field of each line. |