# Shell Scripting

Notes on how to perform tasks within shell scripting, mostly bash, on Linux

# 1 - Bash Scripts

Notes on how to perform tasks with Bash

# 1.1 - Read data to script

Notes on how to read data into a Bash script

# 1.1.1 - Read file line by line

How to read a file line by line in Bash

Reading a file into a bash script on a per-line basis is pretty simple:

1while read -r line; do command "${line}"; done < src.file here we read a line from stdin and as long as a line was read the line is passed to command as a single command argument. the -r option prevents backslash escapes in the line being interpreted. If you want to prevent leading or trailling whitespace from being removed from the line then add IFS= to the line: 1while IFS= read -r line; do command "${line}"; done < src.file

## Within scripts

The above is useful when writing something quick on the command line, but in a script it's more customary to write this across multiple lines.

What I tend to do is wrap the while loop within ( ) with the input redirect after the parentheses. To me this makes it clearer on what the bounds of the script relates to the input.

For example, here we have a script which writes the content of a file to stdout but adds line numbers to the output:

#!/bin/bash input="$1" ( lc=1 while IFS= read -r line do printf "%03d %s\n"${lc} "${line}" lc=$((lc+1)) done ) < "${input}" When run against itself:$ ./morelines.sh morelines.sh 001 #!/bin/bash 002 input="$1" 003 ( 004 lc=1 005 while IFS= read -r line 006 do 007 printf "%03d %s\n"${lc} "${line}" 008 lc=$((lc+1)) 009 done 010 ) < "${input}" # 2 - Cut Remove sections from each line of files ### SYNOPSIS cut OPTION... [FILE]... ### DESCRIPTION Print selected parts of lines from each FILE to standard output. With no FILE, or when FILE is -, read standard input. Mandatory arguments to long options are mandatory for short options too. -b, --bytes=LIST select only these bytes -c, --characters=LIST select only these characters -d, --delimiter=DELIM use DELIM instead of TAB for field delimiter -f, --fields=LIST select only these fields; also print any line that contains no delimiter character, unless the -s option is specified -n ignored --complement complement the set of selected bytes, characters or fields -s, --only-delimited do not print lines not containing delimiters --output-delimiter=STRING use STRING as the output delimiter the default is to use the input delimiter -z, --zero-terminated line delimiter is NUL, not newline --help display this help and exit --version output version information and exit Use one, and only one of -b, -c or -f. Each LIST is made up of one range, or many ranges separated by commas. Selected input is written in the same order that it is read, and is written exactly once. Each range is one of: N N'th byte, character or field, counted from 1 N- from N'th byte, character or field, to end of line N-M from N'th to M'th (included) byte, character or field -M from first to M'th (included) byte, character or field # 2.1 - Cut last column How to get the last column using cut To get the last field in a line using cut is simple, just wrap cut with a pair of rev commands and then extract the first field: 1rev | cut -f1 -d' ' | rev This works as rev simply reverses the characters of each line, pipe it through cut which extracts the required fields (counting in reverse), and then reverses it again so the content is in the original order. The only thing you have to remember is that, once reversed, field 1 is actually the last, 2 the one before the last and so on. # 3 - Pipes Pipes and Pipelines In most script languages, a pipe is where the output of one command is passed as the input of a second command. It allows the chaining of multiple commands without the need of intermediary files. ### File handles Every command has 3 files open by default: File # Name Script Description Read Create Append 0stdin < Input into the command 1stdout >&1 >>&1 Output from the command 2stderr >&2 >>&2 Errors from the command # 3.1 - Here document Sending multiple lines into a pipe A Here document is a set of lines in a script which is to be sent to a command as it's input. It consists of <<deliminator followed by multiple lines and then a line with just demiminator to show the end of the document. For example, we have a 4 line document that will be sent to the command as it's standard input: 1command <<TERMINATOR 2line 1 3line 2 4line$someVar
5line 4
6TERMINATOR

# 3.2 - Here strings

Sending a string into a pipe

Here strings are similar to Here documents but allows you to do this on one line:

1command <<<"some string"

For example, the following commands do the same thing, count the number of words in a string:

1echo "This is a test" | wc -w
2
3wc -w <<<"This is a test"

You can also pipe the output of commands as a here string:

1ps -fe | wc -l
2
3wc -l <<<\$(ps -fe)