Disclosure: This post contains affiliate links. I may earn commission from any sales made or actions taken as a result from users clicking the links on this page.



AWK Command in Unix/Linux with Examples

Posted in AWK

What is AWK?

It is a scripting language and considered the most powerful command in the Linux environment. It does not require any compiling and generates reports by processing some files and then analyze those files. It is used to manipulate data and is thus suitable for pattern searching and processing.

What Operations Can AWK Perform?

  • Scan files line by line.
  • Splits input record into fields
  • Compare the fields to patterns
  • Perform on action matched pattern

How is AWK Command Useful in Linux and Unix?

  • It helps in manipulating data files as per the user requirements.
  • It generates formatted outputs.

What are the various Program Constructs that AWK Offers?

Various programming concepts offered by AWK are:

  • Output formatting
  • Inbuilt variables
  • Pattern matching
  • String operations
  • Arithmetic operations

Basic Syntax of AWK Command

$ awk options 'selection _criteria ' input-file > output-file

Various AWK Commands

Consider the following data table in a sample text file called sample.txt

ID Name Age Company Skill
101 John 28 Apple iOS
102 Chris 33 NextDoor Android
103 Smith 45 Facebook Haskell
104 Jack 33 Apple Java
105 May 27 Google Python

1. Printing the table line by line

$ awk ‘’    sample.txt

The command above prints each line of the file.

ID Name Age Company Skill
101 John 28 Apple iOS
102 Chris 33 NextDoor Android
103 Smith 45 Facebook Haskell
104 Jack 33 Apple Java
105QQ May 27 Google Python

2. Printing table row-wise

$ awk ‘’ sample.txt
$ awk ‘’ sample.txt

The above code prints all the rows of the table.

ID Name Age Company Skill
101 John 28 Apple iOS
102 Chris 33 NextDoor Android
103 Smith 45 Facebook Haskell
104 Jack 33 Apple Java
105 May 27 Google Python

3. Printing Columns

$ awk ‘’    sample.txt

The above code prints the first column of the table.

ID
101
102
103
104
105

Similarly, we can print other columns of the table.

$ awk ‘’    sample.txt

Prints second column

Name
John
Chris
Smith
Jack
May
$ awk ‘’    sample.txt

Prints third column

Age
28
33
45
33
27

4. Printing Matched Pattern

Wrap the specific pattern to be matched within the forward-slash to print the lines that match the particular pattern. Syntax:

$ awk ‘’    sample.txt
$ awk '/variable_to_be_matched/ ' file.txt

Example: Print the data which contains Apple in its fields.

$ awk '/Apple/ '   sample.txt

The above command prints the following rows:

101 John 28 Apple iOS
104 Jack 33 Apple Java

5. Inbuilt Variables

AWK offers certain inbuilt variables that help to manipulate and analyze data. Enlisted are a few inbuilt variables.

5.1. NR

The variable returns the rows with the current count of the number of input records.

$ awk ‘’ sample.txt

The above command counts the number of input records in the data. In this example, the table consists of a total of six records.

1 ID Name Age Company Skill
2 101 John 28 Apple iOS
3 102 Chris 33 NextDoor Android
4 103 Smith 45 Facebook Haskell
5 104 Jack 33 Apple Java
6 105 May 27 Google Python

5.2. NF

The variable prints the count of fields for each record.

$ awk ‘’ sample.txt

The above command prints the last column of the table.

5.3. $NF

The variable returns the last column of the table

$ awk ‘’ sample.txt
Skill
iOS
Android
Haskell
Java
Python

Similarly, we can print the second last and third last column of the table.

$ awk ‘’ sample.txt
Name Company
John Apple
Chris NextDoor
Smith Facebook
Jack Apple
May Google
$ awk ‘’ sample.txt
Name Age
John 28
Chris 33
Smith 45
Jack 33
May 27

5.4. ORS (Output Record Separator)

The variable separates every record in the desired output format.

$ awk ‘ BEGIN ’    sample.txt

Following output is printed:

ID  Name  Age  Company  Skill: 101  John  28  Apple  iOS:  102  Chris  33  NextDoor  Android: 103  Smith  45  Facebook Haskell:  104  Jack 33  Apple  Java: 105  May 27  Google  Python:

5.5. OFS

Output Field Separator The variable separates every field in the desired output format.

$ awk ‘ BEGIN ’    sample.txt
$ awk ‘ BEGIN ’ sample.txt
ID==Name
101==John
102==Chris
103==Smith
104==Jack
105==May
$ awk ‘ BEGIN ’    sample.txt

5.6. RS

It is the current record separator character. The default record separator character is the newline, since, by default, an input line is the input record.

RS$ awk ‘ BEGIN ’    sample.tx

6. IF Command

This command applies conditions on the data set and prints the data that satisfies the condition. Syntax:

$ awk ‘{ if(condition) } END  ’ file.txt

Example: Print the length of the longest line present in the file.

$ awk '{ if (length($0) > max) max = length($0) } END { print max }' sample.txt

7. String Operations

We can also do string operations on the file with AWK commands. Syntax:

$ awk ‘{ if(condition) print $0}’ sample.txt

Examples:

$ awk ‘{ if($1 == “John” ) print $0}’   sample.txt

The above code prints the rows with any field containing “John.”

101 John 28 Apple iOS
$ awk ‘{ if($1 == “Apple” ) print $0}’   sample.txt

The above code prints the rows with any field containing “Apple.”

101 John 28 Apple iOS
104 Jack 33 Apple Java

8. Length

The command counts the length of characters in each line. Example: Print the lines with more than ten characters

$ awk ‘length($0) > 24 ’   sample.txt

The command above prints the lines with a character length greater than 24.

102 Chris 33 NextDoor Android
103 Smith 45 Facebook Haskell

9. Count

Counts the lines in the file

$ awk ‘END ’   sample.txt

Six is printed as there are six lines in the file. 6

10. For

Print the squares of first numbers from 1 to n. Let n = 5

$ awk 'BEGIN { for(i=1;i<=6;i++) print "square of", i, "is",i*i; }'

Output:

square of 1 is 1
square of 2 is 4
square of 3 is 9
square of 4 is 16
square of 5 is 25

11. Non-empty Line

$ awk ‘ NF > 0 ’ sample.txt Output: 0

12. Arithmetic Operations

AWK commands allow performing arithmetic operations if the table elements are numbers.

$ awk ‘ ’   sample.txt

Output:

129
135
148
137
132

13. Saving Output

Use the redirection operator “>” to save the output results. Results can be verified using the cat command.

$ cat output.txt

Conclusion

AWK command in Linux enable a programmer to write useful programs in the form of statements defining specific patterns to be searched for in each line of the file and the action which is to be taken when a match is found within a line. AWK command in Unix is used for pattern processing and scanning. It searches one or more files to see if they contain lines that match the specified patterns and then perform the associated actions.

People are also reading:

Simran Kaur Arora

Simran Kaur Arora

Simran works at Hackr as a technical writer. The graduate in MS Computer Science from the well known CS hub, aka Silicon Valley, is also an editor of the website. She enjoys writing about any tech topic, including programming, algorithms, cloud, data science, and AI. Traveling, sketching, and gardening are the hobbies that interest her. View all posts by the Author

Leave a comment

Your email will not be published
Cancel
John P
John P

Seems like the command is not properly edited. We only see "awk '' sample.txt" nothing else. Please correct those errors