Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting
Prev |  | Next

# Chapter 11. Loops and Branches

| What needs this iteration, woman? --Shakespeare, `Othello`

Operations on code blocks are the key to structured and organized shell scripts. Looping and branching constructs provide the tools for accomplishing this.

# 11.1. Loops

A loop is a block of code that iterates [1] a list of commands as long as the loop control condition is true.

for loops

for `arg` in `[list]`

  This is the basic looping construct. It differs significantly from its C counterpart.

  for `arg` in [`list`]
  do
  `command(s)`...
  done

  | During each pass through the loop, `arg` takes on the value of each successive variable in the `list`.

         1 for arg in "$var1" "$var2" "$var3" ... "$varN"
         2 # In pass 1 of the loop, arg = $var1
         3 # In pass 2 of the loop, arg = $var2
         4 # In pass 3 of the loop, arg = $var3
         5 # ...
         6 # In pass N of the loop, arg = $varN
         7
         8 # Arguments in [list] quoted to prevent possible word splitting.

  The argument `list` may contain wild cards.

  If do is on same line as for, there needs to be a semicolon after list.

  for `arg` in [`list`] ; do

  Example 11-1. Simple for loops

         1 #!/bin/bash
         2 # Listing the planets.
         3
         4 for planet in Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune Pluto
         5 do
         6   echo $planet  # Each planet on a separate line.
         7 done
         8
         9 echo; echo
        10
        11 for planet in "Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune Pluto"
        12     # All planets on same line.
        13     # Entire 'list' enclosed in quotes creates a single variable.
        14     # Why? Whitespace incorporated into the variable.
        15 do
        16   echo $planet
        17 done
        18
        19 echo; echo "Whoops! Pluto is no longer a planet!"
        20
        21 exit 0

  Each `[list]` element may contain multiple parameters. This is useful when processing parameters in groups. In such cases, use the set command (see Example 15-16) to force parsing of each `[list]` element and assignment of each component to the positional parameters.

  Example 11-2. for loop with two parameters in each [list] element

         1 #!/bin/bash
         2 # Planets revisited.
         3
         4 # Associate the name of each planet with its distance from the sun.
         5
         6 for planet in "Mercury 36" "Venus 67" "Earth 93"  "Mars 142" "Jupiter 483"
         7 do
         8   set -- $planet  #  Parses variable "planet"
         9                   #+ and sets positional parameters.
        10   #  The "--" prevents nasty surprises if $planet is null or
        11   #+ begins with a dash.
        12
        13   #  May need to save original positional parameters,
        14   #+ since they get overwritten.
        15   #  One way of doing this is to use an array,
        16   #         original_params=("$@")
        17
        18   echo "$1		$2,000,000 miles from the sun"
        19   #-------two  tabs---concatenate zeroes onto parameter $2
        20 done
        21
        22 # (Thanks, S.C., for additional clarification.)
        23
        24 exit 0

  A variable may supply the `[list]` in a for loop.

  Example 11-3. Fileinfo: operating on a file list contained in a variable

         1 #!/bin/bash
         2 # fileinfo.sh
         3
         4 FILES="/usr/sbin/accept
         5 /usr/sbin/pwck
         6 /usr/sbin/chroot
         7 /usr/bin/fakefile
         8 /sbin/badblocks
         9 /sbin/ypbind"     # List of files you are curious about.
        10                   # Threw in a dummy file, /usr/bin/fakefile.
        11
        12 echo
        13
        14 for file in $FILES
        15 do
        16
        17   if [ ! -e "$file" ]       # Check if file exists.
        18   then
        19     echo "$file does not exist."; echo
        20     continue                # On to next.
        21    fi
        22
        23   ls -l $file | awk '{ print $8 "         file size: " $5 }'  # Print 2 fields.
        24   whatis `basename $file`   # File info.
        25   # Note that the whatis database needs to have been set up for this to work.
        26   # To do this, as root run /usr/bin/makewhatis.
        27   echo
        28 done
        29
        30 exit 0

  The `[list]` in a for loop may be parameterized.

  Example 11-4. Operating on a parameterized file list

         1 #!/bin/bash
         2
         3 filename="*txt"
         4
         5 for file in $filename
         6 do
         7  echo "Contents of $file"
         8  echo "---"
         9  cat "$file"
        10  echo
        11 done

  If the `[list]` in a for loop contains wild cards (* and ?) used in filename expansion, then globbing takes place.

  Example 11-5. Operating on files with a for loop

         1 #!/bin/bash
         2 # list-glob.sh: Generating [list] in a for-loop, using "globbing" ...
         3 # Globbing = filename expansion.
         4
         5 echo
         6
         7 for file in *
         8 #           ^  Bash performs filename expansion
         9 #+             on expressions that globbing recognizes.
        10 do
        11   ls -l "$file"  # Lists all files in $PWD (current directory).
        12   #  Recall that the wild card character "*" matches every filename,
        13   #+ however, in "globbing," it doesn't match dot-files.
        14
        15   #  If the pattern matches no file, it is expanded to itself.
        16   #  To prevent this, set the nullglob option
        17   #+   (shopt -s nullglob).
        18   #  Thanks, S.C.
        19 done
        20
        21 echo; echo
        22
        23 for file in [jx]*
        24 do
        25   rm -f $file    # Removes only files beginning with "j" or "x" in $PWD.
        26   echo "Removed file \"$file\"".
        27 done
        28
        29 echo
        30
        31 exit 0

  Omitting the `in [list]` part of a for loop causes the loop to operate on $@ -- the positional parameters. A particularly clever illustration of this is Example A-15. See also Example 15-17.

  Example 11-6. Missing `in [list]` in a for loop

         1 #!/bin/bash
         2
         3 #  Invoke this script both with and without arguments,
         4 #+ and see what happens.
         5
         6 for a
         7 do
         8  echo -n "$a "
         9 done
        10
        11 #  The 'in list' missing, therefore the loop operates on '$@'
        12 #+ (command-line argument list, including whitespace).
        13
        14 echo
        15
        16 exit 0

  It is possible to use command substitution to generate the `[list]` in a for loop. See also Example 16-54, Example 11-11 and Example 16-48.

  Example 11-7. Generating the `[list]` in a for loop with command substitution

         1 #!/bin/bash
         2 #  for-loopcmd.sh: for-loop with [list]
         3 #+ generated by command substitution.
         4
         5 NUMBERS="9 7 3 8 37.53"
         6
         7 for number in `echo $NUMBERS`  # for number in 9 7 3 8 37.53
         8 do
         9   echo -n "$number "
        10 done
        11
        12 echo
        13 exit 0

  Here is a somewhat more complex example of using command substitution to create the `[list]`.

  Example 11-8. A grep replacement for binary files

         1 #!/bin/bash
         2 # bin-grep.sh: Locates matching strings in a binary file.
         3
         4 # A "grep" replacement for binary files.
         5 # Similar effect to "grep -a"
         6
         7 E_BADARGS=65
         8 E_NOFILE=66
         9
        10 if [ $# -ne 2 ]
        11 then
        12   echo "Usage: `basename $0` search_string filename"
        13   exit $E_BADARGS
        14 fi
        15
        16 if [ ! -f "$2" ]
        17 then
        18   echo "File \"$2\" does not exist."
        19   exit $E_NOFILE
        20 fi
        21
        22
        23 IFS=$'\012'       # Per suggestion of Anton Filippov.
        24                   # was:  IFS="\n"
        25 for word in $( strings "$2" | grep "$1" )
        26 # The "strings" command lists strings in binary files.
        27 # Output then piped to "grep", which tests for desired string.
        28 do
        29   echo $word
        30 done
        31
        32 # As S.C. points out, lines 23 - 30 could be replaced with the simpler
        33 #    strings "$2" | grep "$1" | tr -s "$IFS" '[\n*]'
        34
        35
        36 #  Try something like  "./bin-grep.sh mem /bin/ls"
        37 #+ to exercise this script.
        38
        39 exit 0

  More of the same.

  Example 11-9. Listing all users on the system

         1 #!/bin/bash
         2 # userlist.sh
         3
         4 PASSWORD_FILE=/etc/passwd
         5 n=1           # User number
         6
         7 for name in $(awk 'BEGIN{FS=":"}{print $1}' < "$PASSWORD_FILE" )
         8 # Field separator = :    ^^^^^^
         9 # Print first field              ^^^^^^^^
        10 # Get input from password file  /etc/passwd  ^^^^^^^^^^^^^^^^^
        11 do
        12   echo "USER #$n = $name"
        13   let "n += 1"
        14 done
        15
        16
        17 # USER #1 = root
        18 # USER #2 = bin
        19 # USER #3 = daemon
        20 # ...
        21 # USER #33 = bozo
        22
        23 exit $?
        24
        25 #  Discussion:
        26 #  ----------
        27 #  How is it that an ordinary user, or a script run by same,
        28 #+ can read /etc/passwd? (Hint: Check the /etc/passwd file permissions.)
        29 #  Is this a security hole? Why or why not?

  Yet another example of the `[list]` resulting from command substitution.

  Example 11-10. Checking all the binaries in a directory for authorship

         1 #!/bin/bash
         2 # findstring.sh:
         3 # Find a particular string in the binaries in a specified directory.
         4
         5 directory=/usr/bin/
         6 fstring="Free Software Foundation"  # See which files come from the FSF.
         7
         8 for file in $( find $directory -type f -name '*' | sort )
         9 do
        10   strings -f $file | grep "$fstring" | sed -e "s%$directory%%"
        11   #  In the "sed" expression,
        12   #+ it is necessary to substitute for the normal "/" delimiter
        13   #+ because "/" happens to be one of the characters filtered out.
        14   #  Failure to do so gives an error message. (Try it.)
        15 done
        16
        17 exit $?
        18
        19 #  Exercise (easy):
        20 #  ---------------
        21 #  Convert this script to take command-line parameters
        22 #+ for $directory and $fstring.

  A final example of `[list]` / command substitution, but this time the "command" is a function.

         1 generate_list ()
         2 {
         3   echo "one two three"
         4 }
         5
         6 for word in $(generate_list)  # Let "word" grab output of function.
         7 do
         8   echo "$word"
         9 done
        10
        11 # one
        12 # two
        13 # three

  The output of a for loop may be piped to a command or commands.

  Example 11-11. Listing the symbolic links in a directory

         1 #!/bin/bash
         2 # symlinks.sh: Lists symbolic links in a directory.
         3
         4
         5 directory=${1-`pwd`}
         6 #  Defaults to current working directory,
         7 #+ if not otherwise specified.
         8 #  Equivalent to code block below.
         9 # ----------------------------------------------------------
        10 # ARGS=1                 # Expect one command-line argument.
        11 #
        12 # if [ $# -ne "$ARGS" ]  # If not 1 arg...
        13 # then
        14 #   directory=`pwd`      # current working directory
        15 # else
        16 #   directory=$1
        17 # fi
        18 # ----------------------------------------------------------
        19
        20 echo "symbolic links in directory \"$directory\""
        21
        22 for file in "$( find $directory -type l )"   # -type l = symbolic links
        23 do
        24   echo "$file"
        25 done | sort                                  # Otherwise file list is unsorted.
        26 #  Strictly speaking, a loop isn't really necessary here,
        27 #+ since the output of the "find" command is expanded into a single word.
        28 #  However, it's easy to understand and illustrative this way.
        29
        30 #  As Dominik 'Aeneas' Schnitzer points out,
        31 #+ failing to quote  $( find $directory -type l )
        32 #+ will choke on filenames with embedded whitespace.
        33 #  containing whitespace.
        34
        35 exit 0
        36
        37
        38 # --------------------------------------------------------
        39 # Jean Helou proposes the following alternative:
        40
        41 echo "symbolic links in directory \"$directory\""
        42 # Backup of the current IFS. One can never be too cautious.
        43 OLDIFS=$IFS
        44 IFS=:
        45
        46 for file in $(find $directory -type l -printf "%p$IFS")
        47 do     #                              ^^^^^^^^^^^^^^^^
        48        echo "$file"
        49        done|sort
        50
        51 # And, James "Mike" Conley suggests modifying Helou's code thusly:
        52
        53 OLDIFS=$IFS
        54 IFS='' # Null IFS means no word breaks
        55 for file in $( find $directory -type l )
        56 do
        57   echo $file
        58   done | sort
        59
        60 #  This works in the "pathological" case of a directory name having
        61 #+ an embedded colon.
        62 #  "This also fixes the pathological case of the directory name having
        63 #+  a colon (or space in earlier example) as well."
        64

  The `stdout` of a loop may be redirected to a file, as this slight modification to the previous example shows.

  Example 11-12. Symbolic links in a directory, saved to a file

         1 #!/bin/bash
         2 # symlinks.sh: Lists symbolic links in a directory.
         3
         4 OUTFILE=symlinks.list                         # save-file
         5
         6 directory=${1-`pwd`}
         7 #  Defaults to current working directory,
         8 #+ if not otherwise specified.
         9
        10
        11 echo "symbolic links in directory \"$directory\"" > "$OUTFILE"
        12 echo "---------------------------" >> "$OUTFILE"
        13
        14 for file in "$( find $directory -type l )"    # -type l = symbolic links
        15 do
        16   echo "$file"
        17 done | sort >> "$OUTFILE"                     # stdout of loop
        18 #           ^^^^^^^^^^^^^                       redirected to save file.
        19
        20 # echo "Output file = $OUTFILE"
        21
        22 exit $?

  There is an alternative syntax to a for loop that will look very familiar to C programmers. This requires double parentheses.

  Example 11-13. A C-style for loop

         1 #!/bin/bash
         2 # Multiple ways to count up to 10.
         3
         4 echo
         5
         6 # Standard syntax.
         7 for a in 1 2 3 4 5 6 7 8 9 10
         8 do
         9   echo -n "$a "
        10 done
        11
        12 echo; echo
        13
        14 # +==========================================+
        15
        16 # Using "seq" ...
        17 for a in `seq 10`
        18 do
        19   echo -n "$a "
        20 done
        21
        22 echo; echo
        23
        24 # +==========================================+
        25
        26 # Using brace expansion ...
        27 # Bash, version 3+.
        28 for a in {1..10}
        29 do
        30   echo -n "$a "
        31 done
        32
        33 echo; echo
        34
        35 # +==========================================+
        36
        37 # Now, let's do the same, using C-like syntax.
        38
        39 LIMIT=10
        40
        41 for ((a=1; a <= LIMIT ; a++))  # Double parentheses, and naked "LIMIT"
        42 do
        43   echo -n "$a "
        44 done                           # A construct borrowed from ksh93.
        45
        46 echo; echo
        47
        48 # +=========================================================================+
        49
        50 # Let's use the C "comma operator" to increment two variables simultaneously.
        51
        52 for ((a=1, b=1; a <= LIMIT ; a++, b++))
        53 do  # The comma concatenates operations.
        54   echo -n "$a-$b "
        55 done
        56
        57 echo; echo
        58
        59 exit 0

  See also Example 27-16, Example 27-17, and Example A-6.

  ---

  Now, a for loop used in a "real-life" context.

  Example 11-14. Using efax in batch mode

         1 #!/bin/bash
         2 # Faxing (must have 'efax' package installed).
         3
         4 EXPECTED_ARGS=2
         5 E_BADARGS=85
         6 MODEM_PORT="/dev/ttyS2"   # May be different on your machine.
         7 #                ^^^^^      PCMCIA modem card default port.
         8
         9 if [ $# -ne $EXPECTED_ARGS ]
        10 # Check for proper number of command-line args.
        11 then
        12    echo "Usage: `basename $0` phone# text-file"
        13    exit $E_BADARGS
        14 fi
        15
        16
        17 if [ ! -f "$2" ]
        18 then
        19   echo "File $2 is not a text file."
        20   #     File is not a regular file, or does not exist.
        21   exit $E_BADARGS
        22 fi
        23
        24
        25 fax make $2              #  Create fax-formatted files from text files.
        26
        27 for file in $(ls $2.0*)  #  Concatenate the converted files.
        28                          #  Uses wild card (filename "globbing")
        29 			 #+ in variable list.
        30 do
        31   fil="$fil $file"
        32 done
        33
        34 efax -d "$MODEM_PORT"  -t "T$1" $fil   # Finally, do the work.
        35 # Trying adding  -o1  if above line fails.
        36
        37
        38 #  As S.C. points out, the for-loop can be eliminated with
        39 #     efax -d /dev/ttyS2 -o1 -t "T$1" $2.0*
        40 #+ but it's not quite as instructive [grin].
        41
        42 exit $?   # Also, efax sends diagnostic messages to stdout.

  The keywords do and done delineate the for-loop command block. However, these may, in certain contexts, be omitted by framing the command block within curly brackets

         1 for((n=1; n<=10; n++))
         2 # No do!
         3 {
         4   echo -n "* $n *"
         5 }
         6 # No done!
         7
         8
         9 # Outputs:
        10 # * 1 ** 2 ** 3 ** 4 ** 5 ** 6 ** 7 ** 8 ** 9 ** 10 *
        11 # And, echo $? returns 0, so Bash does not register an error.
        12
        13
        14 echo
        15
        16
        17 #  But, note that in a classic for-loop:    for n in [list] ...
        18 #+ a terminal semicolon is required.
        19
        20 for n in 1 2 3
        21 {  echo -n "$n "; }
        22 #               ^
        23
        24
        25 # Thank you, YongYe, for pointing this out.

while

  This construct tests for a condition at the top of a loop, and keeps looping as long as that condition is true (returns a 0 exit status). In contrast to a for loop, a while loop finds use in situations where the number of loop repetitions is not known beforehand.

  while [`condition` ]
  do
  `command(s)`...
  done

  The bracket construct in a while loop is nothing more than our old friend, the test brackets used in an if/then test. In fact, a while loop can legally use the more versatile double-brackets construct (while [[ condition ]]).

  As is the case with for loops, placing the do on the same line as the condition test requires a semicolon.

  while [`condition` ] ; do

  Note that the test brackets are not mandatory in a while loop. See, for example, the getopts construct.

  Example 11-15. Simple while loop

         1 #!/bin/bash
         2
         3 var0=0
         4 LIMIT=10
         5
         6 while [ "$var0" -lt "$LIMIT" ]
         7 #      ^                    ^
         8 # Spaces, because these are "test-brackets" . . .
         9 do
        10   echo -n "$var0 "        # -n suppresses newline.
        11   #             ^           Space, to separate printed out numbers.
        12
        13   var0=`expr $var0 + 1`   # var0=$(($var0+1))  also works.
        14                           # var0=$((var0 + 1)) also works.
        15                           # let "var0 += 1"    also works.
        16 done                      # Various other methods also work.
        17
        18 echo
        19
        20 exit 0

  Example 11-16. Another while loop

         1 #!/bin/bash
         2
         3 echo
         4                                # Equivalent to:
         5 while [ "$var1" != "end" ]     # while test "$var1" != "end"
         6 do
         7   echo "Input variable #1 (end to exit) "
         8   read var1                    # Not 'read $var1' (why?).
         9   echo "variable #1 = $var1"   # Need quotes because of "#" . . .
        10   # If input is 'end', echoes it here.
        11   # Does not test for termination condition until top of loop.
        12   echo
        13 done
        14
        15 exit 0

  A while loop may have multiple conditions. Only the final condition determines when the loop terminates. This necessitates a slightly different loop syntax, however.

  Example 11-17. while loop with multiple conditions

         1 #!/bin/bash
         2
         3 var1=unset
         4 previous=$var1
         5
         6 while echo "previous-variable = $previous"
         7       echo
         8       previous=$var1
         9       [ "$var1" != end ] # Keeps track of what $var1 was previously.
        10       # Four conditions on *while*, but only the final one controls loop.
        11       # The *last* exit status is the one that counts.
        12 do
        13 echo "Input variable #1 (end to exit) "
        14   read var1
        15   echo "variable #1 = $var1"
        16 done
        17
        18 # Try to figure out how this all works.
        19 # It's a wee bit tricky.
        20
        21 exit 0

  As with a for loop, a while loop may employ C-style syntax by using the double-parentheses construct (see also Example 8-5).

  Example 11-18. C-style syntax in a while loop

         1 #!/bin/bash
         2 # wh-loopc.sh: Count to 10 in a "while" loop.
         3
         4 LIMIT=10                 # 10 iterations.
         5 a=1
         6
         7 while [ "$a" -le $LIMIT ]
         8 do
         9   echo -n "$a "
        10   let "a+=1"
        11 done                     # No surprises, so far.
        12
        13 echo; echo
        14
        15 # +=================================================================+
        16
        17 # Now, we'll repeat with C-like syntax.
        18
        19 ((a = 1))      # a=1
        20 # Double parentheses permit space when setting a variable, as in C.
        21
        22 while (( a <= LIMIT ))   #  Double parentheses,
        23 do                       #+ and no "$" preceding variables.
        24   echo -n "$a "
        25   ((a += 1))             # let "a+=1"
        26   # Yes, indeed.
        27   # Double parentheses permit incrementing a variable with C-like syntax.
        28 done
        29
        30 echo
        31
        32 # C and Java programmers can feel right at home in Bash.
        33
        34 exit 0

  Inside its test brackets, a while loop can call a function.

         1 t=0
         2
         3 condition ()
         4 {
         5   ((t++))
         6
         7   if [ $t -lt 5 ]
         8   then
         9     return 0  # true
        10   else
        11     return 1  # false
        12   fi
        13 }
        14
        15 while condition
        16 #     ^^^^^^^^^
        17 #     Function call -- four loop iterations.
        18 do
        19   echo "Still going: t = $t"
        20 done
        21
        22 # Still going: t = 1
        23 # Still going: t = 2
        24 # Still going: t = 3
        25 # Still going: t = 4

  By coupling the power of the read command with a while loop, we get the handy while read construct, useful for reading and parsing files.

         1 cat $filename |   # Supply input from a file.
         2 while read line   # As long as there is another line to read ...
         3 do
         4   ...
         5 done
         6
         7 # =========== Snippet from "sd.sh" example script ========== #
         8
         9   while read value   # Read one data point at a time.
        10   do
        11     rt=$(echo "scale=$SC; $rt + $value" | bc)
        12     (( ct++ ))
        13   done
        14
        15   am=$(echo "scale=$SC; $rt / $ct" | bc)
        16
        17   echo $am; return $ct   # This function "returns" TWO values!
        18   #  Caution: This little trick will not work if $ct > 255!
        19   #  To handle a larger number of data points,
        20   #+ simply comment out the "return $ct" above.
        21 } <"$datafile"   # Feed in data file.

  | A while loop may have its `stdin` redirected to a file by a < at its end. A while loop may have its `stdin` supplied by a pipe.

until

  This construct tests for a condition at the top of a loop, and keeps looping as long as that condition is false (opposite of while loop).

  until [`condition-is-true` ]
  do
  `command(s)`...
  done

  Note that an until loop tests for the terminating condition at the top of the loop, differing from a similar construct in some programming languages.

  As is the case with for loops, placing the do on the same line as the condition test requires a semicolon.

  until [`condition-is-true` ] ; do

  Example 11-19. until loop

         1 #!/bin/bash
         2
         3 END_CONDITION=end
         4
         5 until [ "$var1" = "$END_CONDITION" ]
         6 # Tests condition here, at top of loop.
         7 do
         8   echo "Input variable #1 "
         9   echo "($END_CONDITION to exit)"
        10   read var1
        11   echo "variable #1 = $var1"
        12   echo
        13 done
        14
        15 #                     ---                        #
        16
        17 #  As with "for" and "while" loops,
        18 #+ an "until" loop permits C-like test constructs.
        19
        20 LIMIT=10
        21 var=0
        22
        23 until (( var > LIMIT ))
        24 do  # ^^ ^     ^     ^^   No brackets, no $ prefixing variables.
        25   echo -n "$var "
        26   (( var++ ))
        27 done    # 0 1 2 3 4 5 6 7 8 9 10
        28
        29
        30 exit 0

How to choose between a for loop or a while loop or until loop? In C, you would typically use a for loop when the number of loop iterations is known beforehand. With Bash, however, the situation is fuzzier. The Bash for loop is more loosely structured and more flexible than its equivalent in other languages. Therefore, feel free to use whatever type of loop gets the job done in the simplest way.

### Notes

[1] | Iteration: Repeated execution of a command or group of commands, usually -- but not always, while a given condition holds, or until a given condition is met.

Prev | Home | Next
Parameter Substitution | Up | Nested Loops