How to use the awk command

How to use the awk command

Basic syntax of the awk command

awk '
  BEGIN{
    # BEGIN block for variable initialization, etc.
  }
  {
    # main block
  }
  END{
    # END block to be processed last
  }
'

After “#”, it is a one-line comment.

Each block can be omitted if unnecessary.

Echoes space-delimited numbers and outputs a summary of each number.

The sum variable is initialized in the BEGIN clause, calculated in the main block, and output in the END block.

$ echo 111,222,333 | awk 'BEGIN{FS=",";sum=0} \
> {sum=$1+$2+$3} \
> END{print sum}'
666

$0,$1,$2…variable

,,…variable

$0,$1,$2… and variables are provided. The image is as follows.

Echo a space-delimited numeric value and receive the result in awk.

$ echo 1 2 3 | awk '{print $1}'
1
$ echo 1 2 3 | awk '{print $2}'
2
$ echo 1 2 3 | awk '{print $3}'
3
$ echo 1 2 3 | awk '{print $0}'
1 2 3

FS Variables

BEGIN句でFS変数を初期化し、$1,$2などで標準出力します。区切り文字をカンマにして$1を出力します。

$ echo 111,222,333 | awk 'BEGIN{FS=","} {print $1}'
111

OFS Variables

The OFS variable is the separator for output. In the following, it is set to “/”.

$ echo 111,222,333 | awk 'BEGIN{FS=",";OFS="/"} {print $0,$1,$2,$3}'
111,222,333/111/222/333

RS Variables

Specify delimiters with RS variables.

Similar to the FS variable, but when delimiters are specified with the RS variable, output is separated by newlines.

$ echo 111,222,333 | awk 'BEGIN{FS=","} {print $1}'
111
$ echo 111,222,333 | awk 'BEGIN{RS=","} {print $1}'
111
222
333

ORS Variables

The ORS variable specifies the delimiter character for output.

Let ORS=””.

$ echo 111,222,333 | awk 'BEGIN{RS=",";ORS=""} {print $1}'
111222333$

ORS=” “.

$ echo 111,222,333 | awk 'BEGIN{RS=",";ORS=" "} {print $1}'
111 222 333 $

NF Variables

The NF variable represents the final field.

$ echo 111,222,333 | awk 'BEGIN{FS=","} {print $1}'
333

NR Variables

The NR variable represents the current line when using the awk command for multi-line files, etc.

The following 3-line CSV file is available.

$ cat sample.csv
111,222,333
100,200,300
10,10,50

Let’s add a leading line number to this file.

$ cat sample.csv | awk '{print NR"行目",$0}'
1行目 111,222,333
2行目 100,200,300
3行目 10,10,50

FPAT Variables

When dealing with CSV, if there is a value that contains commas, FS=”,” is not handled properly.

"100","200","300" 
"100","2,200","300" ←この行がうまく処理できない
"100","200","300"

Use the FPAT variable when processing CSVs that may contain commas in the field values.

$ echo '111,"222,333",444' \
> | awk 'BEGIN{FPAT="([^,]+)|(\"[^\"]+\")"} {print $2}'
"222,333"

ENVIRON Variables

ENVIRON variables are associative arrays.

$ echo | awk '{print ENVIRON["SHELL"]}'
/bin/bash
$ echo | awk '{print ENVIRON["PWD"]}'
/home/ec2-user

Outputs all the keys of the associative array in the ENVIRON variable.

Sort by putting the index of the first argument array into the second argument array with the asorti function.

$ echo | awk '{asorti(ENVIRON,env); for(i in env) {print env[i]; }}'
SSH_TTY
HOME
TERM
HOSTNAME
USER
LANG
LESSOPEN
LOGNAME
LS_COLORS
MAIL
XDG_RUNTIME_DIR
PATH
XDG_SESSION_ID
PWD
_
SHELL
SHLVL
AWKPATH
SSH_CLIENT
HISTCONTROL
SSH_CONNECTION
HISTSIZE

Now we do the reverse, outputting all the values of the associative array of ENVIRON variables.

The asort function puts the elements of the first argument array into the second argument array and sorts it.

$ echo | awk '{asort(ENVIRON,env); for(i in env) {print env[i]; }}'
en_US.UTF-8
.:/usr/share/awk
ignoredups
/bin/bash
ip-172-31-47-49.ap-northeast-1.compute.internal
/dev/pts/0
/home/ec2-user
/home/ec2-user
/run/user/1000
/usr/bin/awk
rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:～～～～省略
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/ec2-user/.local/bin:/home/ec2-user/bin
xterm-256color
/var/spool/mail/ec2-user
||/usr/bin/lesspipe.sh %s
218.219.193.161 62309 172.31.47.49 23
218.219.193.161 62309 22
1
ec2-user
166
ec2-user
1000

高橋