Awk Cheat sheet And Examples

Last Updated: 2020-05-25

Awk is a great utility for text parsing and manipulation. All Unix operating systems have Awk installed by default. If you are on Windows. Please check out at the bottom of this tutorial on how to install and enable awk on Windows.

awk syntax is following...

awk pattern action

I will be using cars.csv file and other dummy examples to illustrate Awk usage. So let us get started.

Let us check our cars.csv file first.

cat cars.csv | head -2
> mpg, cylinders, cubicinches, hp, weightlbs, time-to-60, year, brand
> 14,8,350,165,4209,12,1972, US.

Print Columns In Awk

Let us print column 2 which cyclinders column in cars.csv file. By default field separator is space. Therefore we need to specify FS variable here because field spearator is comma in our csv file. I will restrict it first 2 rows using head command.

awk 'BEGIN {FS=","} {print $2}' < cars.csv | head -2
 > cylinders
 > 8

Note BEGIN code of block BEGIN {FS=","}. This part of code pre-processes or sets variables before we process anything on lines read from the file. Similarly there is another block of code which we can specify to post process that is called the END block. I will show you example of END block later in this tutorial.

We can print multiple columns too as shown below.

awk 'BEGIN {FS=","} {print $2,$3,$4}' < cars.csv | head -2
 > cylinders  cubicinches  hp
 > 8 350 165

Note by default output field separator OFS is space. We can change in BEGIN block to something else example comma as shown below.

awk 'BEGIN {FS=",";OFS=","} {print $2,$3,$4}' < cars.csv | head -2
 > cylinders, cubicinches, hp
 > 8,350,165

Note also how we have defined multiple statements FS=",";OFS="," in the BEGIN block.

Regular Expressions In Awk

Regular expressions can be expressed inside two forward slashes. Let us just print the line which has the word "mpg" that is the first line only in our csv file.

awk '/mpg/ {print $0}' < cars.csv
> mpg, cylinders, cubicinches, hp, weightlbs, time-to-60, year, brand

Note /mpg/ is a regular expression. print $0 means print the whole line.

NF and NR variables in Awk

NF indexes the last column in awk. Let us do an example and print the last column.

awk 'BEGIN {FS=","} {print $(NF)}' < cars.csv | head -2
 > brand
 > US.

For printing the second last column just do $(NF-1)

awk 'BEGIN {FS=","} {print $(NF-1);exit}' < cars.csv
 > year

Note: exit will make sure that awk just prints the first output line only not all the lines.

NR will print the number of records.

awk 'END {printf "No of records %d",NR}' < cars.csv
> No of records 262

Note, we have used here the END code of block because awk needs to go through the whole file before it can count the number of records.

Arrays in AWK

Arrays in awk are very flexible. Arrays index can be both an integer and string. Awk arrays can be initialized dynamically. Let us take a look at below example.

awk 'BEGIN {a[1]=2;print a[1]}'
> 2

How to loop through arrays in AWK

awk 'BEGIN {
        for (i=0 ;i <5; i++) {
            b[i] = 1
        }
        #Loop through awk array b
        for (elem in b) {
            print elem
        }
    }'
> 4
> 0
> 1
> 2
> 3

In the above code, we are first initializing the array b. Then in second loop, we are loop through awk array b. Note: the order of values in awk array is not maintained.

How to delete an AWK array and its array elements

To delete an array in AWk, just do delete . To delete an element, just do delete array[index]. Please check out the example below.

awk 'BEGIN {
        delete b
        for (i=0 ;i <5; i++) {
            b[i] = 1
        }
        if (4 in b){
            print "yes 4 is present"
        }
        delete b[3]
        print b[3]
        print b[4]
    }'
> yes 4 is present

> 1

How to add values of a column in a file in AWK

Let us create a file with only column of some numbers as shown below.

echo "1" > /tmp/numbers.txt
echo "2" >> /tmp/numbers.txt
echo "3" >> /tmp/numbers.txt

Now let us add the values.

awk 'BEGIN {
    x=0
} {x=x+$1} END {
    print x
}' /tmp/numbers.txt
> 6

How to install Awk on Windows

Please checkout following tutorial by me if you are on Windows and want to run Awk on Windows 10 and previous versions.

Table of Contents