Learn AWK command — Part 3

Variables of AWK :

At the starting I pointed that awk is like c programing language

Like c ,awk also have some inbuilt varibales and user defined variables.User defined variable is what we defined and system (position) defined varibales are not special variables but a function triggered by the dollar sign.

1.FS  – Input Field Separator variable:

Previously we discussed by using command line option -f we will separate fields other than space. We can also do this thing without using command line option by using FS variable.

Lets see ,how we can do

awk ‘ BEGIN { FS = “:” } ; { print $3 “ “ $6}’ passwd

awk1.png

Here we used the field separator FS and we assigned delimiter to FS .

2.OFS – Output Field Separator Variable :

Awk OFS is an output equivalent of awk FS variable. By default awk OFS is a single space character .

See the example

awk ‘BEGIN {FS=”:” ; OFS=”=”}; {print $3,$6}’ passwd 

Here we assigned input field separator as “:” because in passwd file ,fields are separated by “:” and output field separator as “=” which differenciates the fields of output .

If you didn’t specify the OFS by default it will take single character space.

awk1.png

3.NR – Number of Records Variable :

The NR Variable will give you total number of Records(lines) being processed .

See the below very simple example ,which gives the number of records in the file

awk ‘{print NR ,$0}’ worker.txt

Which gives you number before each record in the file .

awk1.png

4.NF – Number of Fields Variable :

The NF variable is used to know the number of fields in a line . Awk NF will be very useful for validating whether all the fields are exist in a record.

Suppose take an example

awk1.png

In the above example the first column is the number of fields in each record . NF won’t count space as a value so it gives value 3 for both 2 and 4th rows in worker.txt file.

and also NF used to remove blank lines between the lines

See below example

awk1.png

Now suppose if you want check the total number of fileds in each line to find the missing fields

awk ‘{print NR “st  line contains ” ” –> ” NF “fields” }; END {print NR ” records processed “}’

worker.txt

awk1.png

5.RS – Record Separator variable :

We discussed that awk reads input line by line by default and each line is called a record. Row Separator is helpful in defining separator between rows in a file. By default AWK takes row separator as new line. We can change this by using RS built-in variable.

if you want to convert a sentence to a word per line. We can use RS variable for doing it.

echo “This is how it works” | awk ‘BEGIN{RS=” ”}{print $0}’

awk1.png

6.ORS – Output Record Separator

This variable is useful for defining the record separator for the AWK command output. By default ORS is set to new line.

awk ‘BEGIN{ORS=”=”;} {print;}’ worker.txt

awk1.png

7.FNR  (Number of Records relative to the current input file )

This variable keeps count of number of lines present in a given file/data. This will come handy when you want to print no of line present in a given file. This command is equivalent to wc -l command

awk1.png

8.FILENAME

FILENAME variable gives the name of the file being read. Awk can accept number of input files to process.

awk1.png


 

Advanced Features of AWK

Running awk programs :

There are four ways in which we can run awk programs

  • One-shot: Running a short throw-away awk program.
    • $ awk ‘programinput-file1 input-file2

… where program consists of a series of patterns and actions.

  • Read Terminal: Using no input files (input from terminal instead).
  • $ awk ‘program‘ <ENTER>
  • <input lines>
  • <input lines>
  • ctrl-d
  • Long: Putting permanent awk programs in files.
  • $ awk -f source-file input-file1 input-file2
  • Executable Scripts: Making self-contained awk programs.
    • (eg) : Write a script named hello with the following contents
    • #! /bin/awk -f
    • # a sample awk program
    • /foo/ { print $1}
  • Execute the following command
  • $ chmod +x hello
  • To run this script simply type
  • $ ./hello file.txt

Loops And Arithmetic Expressions :

  • Awk borrows a lot from the C language.
  • The if loop, for loop and while loop have the same constructs as in C.
  • Awk’s variables are stored internally as strings.
    • x = “1.01”
      • x = x + 1
      • print x
    • The above will print the value 2.01
  • Comparison operators in awk are : “==”, “<“, “>”, “<=”, “>=”, “!=“, “~” and “!~“.
  • “~” and “!~” operators mean “matches” and “does not match”.
  • Common Arithmetic operators in awk are : “+”, “-“, “/”, “*“;
  • “^” is the exponentiation operator.
  • “%” is the modulo operator
  • All the C operators like “++”, “–”, “+=“, “-=”, “/=“ are also valid.
  • The awk language has one-dimensional arrays for storing groups of related strings or numbers.
  • Arrays in awk are associative. This means that each array is a collection of pairs: an index, and its corresponding array element value.
    • (eg) : Element 1 value 2
      • Element 2 value “foo”
      • Element “cat” value “chicken”

 Advantages of AWK :

  • awk is an interpreted language so you can avoid the usually lengthy edit-compile-test-debug cycle of software development .
  • Can be used for rapid prototyping.
  • The awk language is very useful for producing reports from large amounts of raw data, such as summarizing information from the output of other utility programs like ls.

This is the brief introduction about AWK command  . There is a lot to learn about AWK command.I tried to cover some basics of it. There are more like functions ,strings,patterns and environmental variables etc.

Please give your valuable feedback . Happy Learning

Advertisements

Express your Opinion here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s