Join Hackr.io to get personalized course recommendations for you.

AWK Command with Examples

Posted in AWK
AWK Command with Examples

About the AWK Command

  • It is a scripting language.
  • It is the most powerful command in the Linux environment. 
  • Manipulates data.
  • Does not require compiling
  • Generates reports by processing some files.
  • Suitable for pattern searching and processing
  • Analyzes files 

What Operations Can AWK Perform?

  • Scan files line by line. 
  • Splits input record into fields
  • Compare the fields to patterns
  • Perform on action matched pattern 

How is The AWK Command Useful?

  • It helps in manipulating data files as per the user requirements. 
  • It generates formatted outputs.

What are the various Program Constructs that AWK Offers?

Various programming concepts offered by AWK are:

  • Output formatting 
  • Inbuilt variables 
  • Pattern matching 
  • String operations
  • Arithmetic operations

Basic Syntax of AWK Command 

$ awk options 'selection _criteria {action }' input-file > output-file

Various AWK Commands

Consider the following data table in a sample text file called sample.txt

ID

Name

Age

Company

Skill 

101

John

28

Apple

iOS

102

Chris

33

NextDoor

Android

103

Smith

45

Facebook

Haskell

104

Jack 

33

Apple

Java

105

May

27

Google

Python

1. Printing the table line by line 

$ awk ‘{print}’       sample.txt

The command above prints each line of the file. 

ID

Name

Age

Company

Skill 

101

John

28

Apple

iOS

102

Chris

33

NextDoor

Android

103

Smith

45

Facebook

Haskell

104

Jack 

33

Apple

Java

105QQ

May

27

Google

Python

2. Printing table row-wise 

$ awk ‘{print}’ sample.txt
$ awk ‘{print $0}’        sample.txt

The above code prints all the rows of the table. 

ID

Name

Age

Company

Skill 

101

John

28

Apple

iOS

102

Chris

33

NextDoor

Android

103

Smith

45

Facebook

Haskell

104

Jack 

33

Apple

Java

105

May

27

Google

Python

3. Printing Columns 

$ awk ‘{print $1}’       sample.txt

The above code prints the first column of the table. 

ID

101

102

103

104

105

Similarly, we can print other columns of the table. 

$ awk ‘{print $2}’       sample.txt

Prints second column

Name

John

Chris

Smith

Jack 

May

$ awk ‘{print $3}’       sample.txt

Prints third column

Age

28

33

45

33

27

4. Printing Matched Pattern 

Wrap the specific pattern to be matched within the forward-slash to print the lines that match the particular pattern. 

Syntax:

$ awk ‘{print $3}’       sample.txt
$ awk  '/variable_to_be_matched/ {print $0}'     file.txt

Example: 

Print the data which contains Apple in its fields. 

$ awk  '/Apple/ {print $0}'     sample.txt

The above command prints the following rows:

101

John

28

Apple

iOS

104

Jack 

33

Apple

Java

5. Inbuilt Variables 

AWK offers certain inbuilt variables that help to manipulate and analyze data. 

Enlisted are a few inbuilt variables. 

5.1. NR

The variable returns the rows with the current count of the number of input records. 

$ awk ‘{print NR,  $0}’ sample.txt

The above command counts the number of input records in the data. In this example, the table consists of a total of six records.

1

ID

Name

Age

Company

Skill 

2

101

John

28

Apple

iOS

3

102

Chris

33

NextDoor

Android

4

103

Smith

45

Facebook

Haskell

5

104

Jack 

33

Apple

Java

6

105

May

27

Google

Python

5.2. NF

The variable prints the count of fields for each record. 

$ awk ‘{print NF,  $0}’ sample.txt

The above command prints the last column of the table. 

5.3. $NF

The variable returns the last column of the table 

$ awk ‘{print  $NF}’ sample.txt

Skill 

iOS

Android

Haskell

Java

Python

Similarly, we can print the second last and third last column of the table. 

$ awk ‘{print $2,  $(NF -1) }’ sample.txt

Name

Company

John

Apple

Chris

NextDoor

Smith

Facebook

Jack 

Apple

May

Google

$ awk ‘{print $2,  $(NF -2) }’ sample.txt

Name

Age

John

28

Chris

33

Smith

45

Jack 

33

May

27

5.4. ORS

Output Record Separator 

The variable separates every record in the desired output format. 

$ awk ‘ BEGIN {ORS = “:” } {print $0}’       sample.txt

Following output is printed: 

ID  Name  Age  Company  Skill: 101  John  28  Apple  iOS:  102  Chris  33  NextDoor  Android: 103  Smith  45  Facebook Haskell:  104  Jack 33  Apple  Java: 105  May 27  Google  Python:

5.5. OFS

Output Field Separator 

The variable separates every field in the desired output format. 

$ awk ‘ BEGIN {ORS = “:” } {print $0}’       sample.txt
$ awk ‘ BEGIN {OFS = “==” } {print $1, $2}’       sample.txt
ID==Name
101==John
102==Chris
103==Smith
104==Jack
105==May
$ awk ‘ BEGIN {ORS = “:” } {print $0}’       sample.txt

5.6. RS

It is the current record separator character. The default record separator character is the newline, since, by default, an input line is the input record.

RS$ awk ‘ BEGIN {ORS = “:” } {print $0}’       sample.tx

6. IF Command 

This command applies conditions on the data set and prints the data that satisfies the condition. 

Syntax:

$ awk ‘{ if(condition) } END   {print $0}’ file.txt

Example: 

Print the length of the longest line present in the file. 

$ awk '{ if (length($0) > max) max = length($0) } END { print max }' sample.txt

7. String Operations 

We can also do string operations on the file with AWK commands. 

Syntax:

$ awk ‘{ if(condition)  print $0}’ sample.txt

Examples: 

$ awk ‘{ if($1 == “John” ) print $0}’     sample.txt

The above code prints the rows with any field containing “John”

101

John

28

Apple

iOS

$ awk ‘{ if($1 == “Apple” ) print $0}’     sample.txt

The above code prints the rows with any field containing “Apple”

101

John

28

Apple

iOS

104

Jack 

33

Apple

Java

8. Length

The command counts the length of characters in each line. 

Example: 

Print the lines with more than 10 characters

$ awk ‘length($0) > 24 ’     sample.txt

The command above prints the lines with a character length greater than 24.

102

Chris

33

NextDoor

Android

103

Smith

45

Facebook

Haskell

9. Count 

Counts the lines in the file. 

$ awk ‘END {print NR}’     sample.txt

Six is printed as there are 6 lines in the file.

6

10. For

Print the squares of first numbers from 1 to n. 

Let n = 5

$ awk 'BEGIN { for(i=1;i<=6;i++) print "square of", i, "is",i*i; }'

Output: 

square of 1 is 1
square of 2 is 4
square of 3 is 9
square of 4 is 16
square of 5 is 25

11. Non-empty Line

$ awk ‘ NF > 0 ’     sample.txt

Output:

0

12. Arithmetic Operations

AWK commands allow performing arithmetic operations if the table elements are numbers. 

$ awk ‘ {print $1 + $3 } ’     sample.txt

Output:

129

135

148

137

132

13. Saving Output

Use the redirection operator “>” to save the output results. 

Results can be verified using the cat command. 

$ cat output.txt

Conclusion

Awk commands; they enable a programmer to write useful programs in the form of statements defining specific patterns to be searched for in each line of the file and the action which is to be taken when a match is found within a line. Awk is used for pattern processing and scanning. It searches one or more files to see if they contain lines that match the specified patterns and then perform the associated actions.

People are also reading:

Simran Kaur Arora

Simran Kaur Arora

Simran, born in Delhi, did her schooling and graduation from India in Computer Science. Curious and passionate about technology urged her to study for an MS in the same from the renowned Silicon Valley, California, USA. Graduated in 2017, she flew back to India and now works for hackr.io as a freelance technical writer. View all posts by the Author

Leave a comment

Your email will not be published
Cancel