Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting
Prev |  | Next

# Chapter 3. Special Characters

What makes a character special? If it has a meaning beyond its literal meaning, a meta-meaning, then we refer to it as a special character. Along with commands and keywords, special characters are building blocks of Bash scripts.

Special Characters Found In Scripts and Elsewhere

#

  Comments. Lines beginning with a # (with the exception of #!) are comments and will not be executed.

         1 # This line is a comment.

  Comments may also occur following the end of a command.

         1 echo "A comment will follow." # Comment here.
         2 #                            ^ Note whitespace before #

  Comments may also follow whitespace at the beginning of a line.

         1      # A tab precedes this comment.

  Comments may even be embedded within a pipe.

         1 initial=( `cat "$startfile" | sed -e '/#/d' | tr -d '\n' |\
         2 # Delete lines containing '#' comment character.
         3            sed -e 's/\./\. /g' -e 's/_/_ /g'` )
         4 # Excerpted from life.sh script

  | A command may not follow a comment on the same line. There is no method of terminating the comment, in order for "live code" to begin on the same line. Use a new line for the next command.

  Of course, a quoted or an escaped # in an echo statement does not begin a comment. Likewise, a # appears in certain parameter-substitution constructs and in numerical constant expressions.

         1 echo "The # here does not begin a comment."
         2 echo 'The # here does not begin a comment.'
         3 echo The \# here does not begin a comment.
         4 echo The # here begins a comment.
         5
         6 echo ${PATH#*:}       # Parameter substitution, not a comment.
         7 echo $(( 2#101011 ))  # Base conversion, not a comment.
         8
         9 # Thanks, S.C.

  The standard quoting and escape characters (" ' \) escape the #.

  Certain pattern matching operations also use the #.

;

  Command separator [semicolon]. Permits putting two or more commands on the same line.

         1 echo hello; echo there
         2
         3
         4 if [ -x "$filename" ]; then    #  Note the space after the semicolon.
         5 #+                   ^^
         6   echo "File $filename exists."; cp $filename $filename.bak
         7 else   #                       ^^
         8   echo "File $filename not found."; touch $filename
         9 fi; echo "File test complete."

  Note that the ";" sometimes needs to be escaped.

;;

  Terminator in a case option [double semicolon].

         1 case "$variable" in
         2   abc)  echo "\$variable = abc" ;;
         3   xyz)  echo "\$variable = xyz" ;;
         4 esac

;;&, ;&

  Terminators in a case option (version 4+ of Bash).

.

  "dot" command [period]. Equivalent to source (see Example 15-22). This is a bash builtin.

.

  "dot", as a component of a filename. When working with filenames, a leading dot is the prefix of a "hidden" file, a file that an ls will not normally show.

       bash$ touch .hidden-file
       bash$ ls -l
       total 10
       -rw-r--r--    1 bozo      4034 Jul 18 22:04 data1.addressbook
       -rw-r--r--    1 bozo      4602 May 25 13:58 data1.addressbook.bak
       -rw-r--r--    1 bozo       877 Dec 17  2000 employment.addressbook

       bash$ ls -al
       total 14
       drwxrwxr-x    2 bozo  bozo      1024 Aug 29 20:54 ./
       drwx------   52 bozo  bozo      3072 Aug 29 20:51 ../
       -rw-r--r--    1 bozo  bozo      4034 Jul 18 22:04 data1.addressbook
       -rw-r--r--    1 bozo  bozo      4602 May 25 13:58 data1.addressbook.bak
       -rw-r--r--    1 bozo  bozo       877 Dec 17  2000 employment.addressbook
       -rw-rw-r--    1 bozo  bozo         0 Aug 29 20:54 .hidden-file

  When considering directory names, a single dot represents the current working directory, and two dots denote the parent directory.

       bash$ pwd
       /home/bozo/projects

       bash$ cd .
       bash$ pwd
       /home/bozo/projects

       bash$ cd ..
       bash$ pwd
       /home/bozo/

  The dot often appears as the destination (directory) of a file movement command, in this context meaning current directory.

       bash$ cp /home/bozo/current_work/junk/* .

  Copy all the "junk" files to $PWD.
.

  "dot" character match. When matching characters, as part of a regular expression, a "dot" matches a single character.

"

  partial quoting [double quote]. "STRING" preserves (from interpretation) most of the special characters within STRING. See Chapter 5.

'

  full quoting [single quote]. 'STRING' preserves all special characters within STRING. This is a stronger form of quoting than "STRING". See Chapter 5.

,

  comma operator. The comma operator [1] links together a series of arithmetic operations. All are evaluated, but only the last one is returned.

         1 let "t2 = ((a = 9, 15 / 3))"
         2 # Set "a = 9" and "t2 = 15 / 3"

  The comma operator can also concatenate strings.

         1 for file in /{,usr/}bin/*calc
         2 #             ^    Find all executable files ending in "calc"
         3 #+                 in /bin and /usr/bin directories.
         4 do
         5         if [ -x "$file" ]
         6         then
         7           echo $file
         8         fi
         9 done
        10
        11 # /bin/ipcalc
        12 # /usr/bin/kcalc
        13 # /usr/bin/oidcalc
        14 # /usr/bin/oocalc
        15
        16
        17 # Thank you, Rory Winston, for pointing this out.

,, ,

  Lowercase conversion in parameter substitution (added in version 4 of Bash).

\

  escape [backslash]. A quoting mechanism for single characters.

  `\X` escapes the character X. This has the effect of "quoting" X, equivalent to 'X'. The \ may be used to quote " and ', so they are expressed literally.

  See Chapter 5 for an in-depth explanation of escaped characters.

/

  Filename path separator [forward slash]. Separates the components of a filename (as in `/home/bozo/projects/Makefile`).

  This is also the division arithmetic operator.

`

  command substitution. The `command` construct makes available the output of command for assignment to a variable. This is also known as backquotes or backticks.

:

  null command [colon]. This is the shell equivalent of a "NOP" (`no op`, a do-nothing operation). It may be considered a synonym for the shell builtin true. The ":" command is itself a Bash builtin, and its exit status is true (0).

         1 :
         2 echo $?   # 0

  Endless loop:

         1 while :
         2 do
         3    operation-1
         4    operation-2
         5    ...
         6    operation-n
         7 done
         8
         9 # Same as:
        10 #    while true
        11 #    do
        12 #      ...
        13 #    done

  Placeholder in if/then test:

         1 if condition
         2 then :   # Do nothing and branch ahead
         3 else     # Or else ...
         4    take-some-action
         5 fi

  Provide a placeholder where a binary operation is expected, see Example 8-2 and default parameters.

         1 : ${username=`whoami`}
         2 # ${username=`whoami`}   Gives an error without the leading :
         3 #                        unless "username" is a command or builtin...
         4
         5 : ${1?"Usage: $0 ARGUMENT"}     # From "usage-message.sh example script.

  Provide a placeholder where a command is expected in a here document. See Example 19-10.

  Evaluate string of variables using parameter substitution (as in Example 10-7).

         1 : ${HOSTNAME?} ${USER?} ${MAIL?}
         2 #  Prints error message
         3 #+ if one or more of essential environmental variables not set.

  Variable expansion / substring replacement.

  In combination with the > redirection operator, truncates a file to zero length, without changing its permissions. If the file did not previously exist, creates it.

         1 : > data.xxx   # File "data.xxx" now empty.
         2
         3 # Same effect as   cat /dev/null >data.xxx
         4 # However, this does not fork a new process, since ":" is a builtin.

  See also Example 16-15.

  In combination with the >> redirection operator, has no effect on a pre-existing target file (`: >> target_file`). If the file did not previously exist, creates it.

  | This applies to regular files, not pipes, symlinks, and certain special files.

  May be used to begin a comment line, although this is not recommended. Using # for a comment turns off error checking for the remainder of that line, so almost anything may appear in a comment. However, this is not the case with :.

         1 : This is a comment that generates an error, ( if [ $x -eq 3] ).

  The ":" serves as a field separator, in `/etc/passwd`, and in the $PATH variable.

       bash$ echo $PATH
       /usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/sbin:/usr/sbin:/usr/games

  A colon is acceptable as a function name.

         1 :()
         2 {
         3   echo "The name of this function is "$FUNCNAME" "
         4   # Why use a colon as a function name?
         5   # It's a way of obfuscating your code.
         6 }
         7
         8 :
         9
        10 # The name of this function is :

  This is not portable behavior, and therefore not a recommended practice. In fact, more recent releases of Bash do not permit this usage. An underscore _ works, though.

  A colon can serve as a placeholder in an otherwise empty function.

         1 not_empty ()
         2 {
         3   :
         4 } # Contains a : (null command), and so is not empty.

!

  reverse (or negate) the sense of a test or exit status [bang]. The ! operator inverts the exit status of the command to which it is applied (see Example 6-2). It also inverts the meaning of a test operator. This can, for example, change the sense of equal ( = ) to not-equal ( != ). The ! operator is a Bash keyword.

  In a different context, the ! also appears in indirect variable references.

  In yet another context, from the command line, the ! invokes the Bash history mechanism (see Appendix L). Note that within a script, the history mechanism is disabled.

*

  wild card [asterisk]. The * character serves as a "wild card" for filename expansion in globbing. By itself, it matches every filename in a given directory.

       bash$ echo *
       abs-book.sgml add-drive.sh agram.sh alias.sh

  The * also represents any number (or zero) characters in a regular expression.

*

  arithmetic operator. In the context of arithmetic operations, the * denotes multiplication.

  ** A double asterisk can represent the exponentiation operator or extended file-match globbing.

?

  test operator. Within certain expressions, the ? indicates a test for a condition.

  In a double-parentheses construct, the ? can serve as an element of a C-style trinary operator. [2]

  `condition`?`result-if-true`:`result-if-false`

         1 (( var0 = var1<98?9:21 ))
         2 #                ^ ^
         3
         4 # if [ "$var1" -lt 98 ]
         5 # then
         6 #   var0=9
         7 # else
         8 #   var0=21
         9 # fi

  In a parameter substitution expression, the ? tests whether a variable has been set.

?

  wild card. The ? character serves as a single-character "wild card" for filename expansion in globbing, as well as representing one character in an extended regular expression.

$

  Variable substitution (contents of a variable).

         1 var1=5
         2 var2=23skidoo
         3
         4 echo $var1     # 5
         5 echo $var2     # 23skidoo

  A $ prefixing a variable name indicates the value the variable holds.

$

  end-of-line. In a regular expression, a "$" addresses the end of a line of text.

${}

  Parameter substitution.

$' ... '

  Quoted string expansion. This construct expands single or multiple escaped octal or hex values into ASCII [3] or Unicode characters.

$*, $@

  positional parameters.

$?

  exit status variable. The $? variable holds the exit status of a command, a function, or of the script itself.

$$

  process ID variable. The $$ variable holds the process ID [4] of the script in which it appears.

()

  command group.

         1 (a=hello; echo $a)

  A listing of commands within `parentheses` starts a subshell.

  Variables inside parentheses, within the subshell, are not visible to the rest of the script. The parent process, the script, cannot read variables created in the child process, the subshell.

         1 a=123
         2 ( a=321; )
         3
         4 echo "a = $a"   # a = 123
         5 # "a" within parentheses acts like a local variable.

  array initialization.

         1 Array=(element1 element2 element3)

{xxx,yyy,zzz,...}

  Brace expansion.

         1 echo \"{These,words,are,quoted}\"   # " prefix and suffix
         2 # "These" "words" "are" "quoted"
         3
         4
         5 cat {file1,file2,file3} > combined_file
         6 # Concatenates the files file1, file2, and file3 into combined_file.
         7
         8 cp file22.{txt,backup}
         9 # Copies "file22.txt" to "file22.backup"

  A command may act upon a comma-separated list of file specs within `braces`. [5] Filename expansion (globbing) applies to the file specs between the braces.

  | No spaces allowed within the braces unless the spaces are quoted or escaped. `echo {file1,file2}\ :{\ A," B",' C'}` `file1 : A file1 : B file1 : C file2 : A file2 : B file2 : C`

{a..z}

  Extended Brace expansion.

         1 echo {a..z} # a b c d e f g h i j k l m n o p q r s t u v w x y z
         2 # Echoes characters between a and z.
         3
         4 echo {0..3} # 0 1 2 3
         5 # Echoes characters between 0 and 3.
         6
         7
         8 base64_charset=( {A..Z} {a..z} {0..9} + / = )
         9 # Initializing an array, using extended brace expansion.
        10 # From vladz's "base64.sh" example script.

  The {a..z} extended brace expansion construction is a feature introduced in version 3 of Bash.

{}

  Block of code [curly brackets]. Also referred to as an inline group, this construct, in effect, creates an anonymous function (a function without a name). However, unlike in a "standard" function, the variables inside a code block remain visible to the remainder of the script.

       bash$ { local a;
      	      a=123; }
       bash: local: can only be used in a
      function

         1 a=123
         2 { a=321; }
         3 echo "a = $a"   # a = 321   (value inside code block)
         4
         5 # Thanks, S.C.

  The code block enclosed in braces may have I/O redirected to and from it.

  Example 3-1. Code blocks and I/O redirection

         1 #!/bin/bash
         2 # Reading lines in /etc/fstab.
         3
         4 File=/etc/fstab
         5
         6 {
         7 read line1
         8 read line2
         9 } < $File
        10
        11 echo "First line in $File is:"
        12 echo "$line1"
        13 echo
        14 echo "Second line in $File is:"
        15 echo "$line2"
        16
        17 exit 0
        18
        19 # Now, how do you parse the separate fields of each line?
        20 # Hint: use awk, or . . .
        21 # . . . Hans-Joerg Diers suggests using the "set" Bash builtin.

  Example 3-2. Saving the output of a code block to a file

         1 #!/bin/bash
         2 # rpm-check.sh
         3
         4 #  Queries an rpm file for description, listing,
         5 #+ and whether it can be installed.
         6 #  Saves output to a file.
         7 #
         8 #  This script illustrates using a code block.
         9
        10 SUCCESS=0
        11 E_NOARGS=65
        12
        13 if [ -z "$1" ]
        14 then
        15   echo "Usage: `basename $0` rpm-file"
        16   exit $E_NOARGS
        17 fi
        18
        19 { # Begin code block.
        20   echo
        21   echo "Archive Description:"
        22   rpm -qpi $1       # Query description.
        23   echo
        24   echo "Archive Listing:"
        25   rpm -qpl $1       # Query listing.
        26   echo
        27   rpm -i --test $1  # Query whether rpm file can be installed.
        28   if [ "$?" -eq $SUCCESS ]
        29   then
        30     echo "$1 can be installed."
        31   else
        32     echo "$1 cannot be installed."
        33   fi
        34   echo              # End code block.
        35 } > "$1.test"       # Redirects output of everything in block to file.
        36
        37 echo "Results of rpm test in file $1.test"
        38
        39 # See rpm man page for explanation of options.
        40
        41 exit 0

  | Unlike a command group within (parentheses), as above, a code block enclosed by {braces} will not normally launch a subshell. [6] It is possible to iterate a code block using a non-standard for-loop.

{}

  placeholder for text. Used after xargs `-i` (replace strings option). The {} double curly brackets are a placeholder for output text.

         1 ls . | xargs -i -t cp ./{} $1
         2 #            ^^         ^^
         3
         4 # From "ex42.sh" (copydir.sh) example.

{} \;

  pathname. Mostly used in find constructs. This is not a shell builtin.

  | The ";" ends the `-exec` option of a find command sequence. It needs to be escaped to protect it from interpretation by the shell.

[ ]

  test.

  Test expression between [ ]. Note that [ is part of the shell builtin test (and a synonym for it), not a link to the external command `/usr/bin/test`.

[[ ]]

  test.

  Test expression between [[ ]]. More flexible than the single-bracket [ ] test, this is a shell keyword.

  See the discussion on the [[ ... ]] construct.

[ ]

  array element.

  In the context of an array, brackets set off the numbering of each element of that array.

         1 Array[1]=slot_1
         2 echo ${Array[1]}

[ ]

  range of characters.

  As part of a regular expression, brackets delineate a range of characters to match.

$[ ... ]

  integer expansion.

  Evaluate integer expression between $[ ].

         1 a=3
         2 b=7
         3
         4 echo $[$a+$b]   # 10
         5 echo $[$a*$b]   # 21

  Note that this usage is deprecated, and has been replaced by the (( ... )) construct.

(( ))

  integer expansion.

  Expand and evaluate integer expression between (( )).

  See the discussion on the (( ... )) construct.

> &> >& >> < <>

  redirection.

  `scriptname >filename` redirects the output of `scriptname` to file `filename`. Overwrite `filename` if it already exists.

  `command &>filename` redirects both the `stdout` and the `stderr` of `command` to `filename`.

  This is useful for suppressing output when testing for a condition. For example, let us test whether a certain command exists.

       bash$ type bogus_command &>/dev/null

       bash$ echo $?
       1

  Or in a script:

         1 command_test () { type "$1" &>/dev/null; }
         2 #                                      ^
         3
         4 cmd=rmdir            # Legitimate command.
         5 command_test $cmd; echo $?   # 0
         6
         7
         8 cmd=bogus_command    # Illegitimate command
         9 command_test $cmd; echo $?   # 1

  `command >&2` redirects `stdout` of `command` to `stderr`.

  `scriptname >>filename` appends the output of `scriptname` to file `filename`. If `filename` does not already exist, it is created.

  `[i]<>filename` opens file `filename` for reading and writing, and assigns file descriptor i to it. If `filename` does not exist, it is created.

  process substitution.

  `(command)>`

  `<(command)`

  In a different context, the "<" and ">" characters act as string comparison operators.

  In yet another context, the "<" and ">" characters act as integer comparison operators. See also Example 16-9.

<<

  redirection used in a here document.

<<<

  redirection used in a here string.

<, >

  ASCII comparison.

         1 veg1=carrots
         2 veg2=tomatoes
         3
         4 if [[ "$veg1" < "$veg2" ]]
         5 then
         6   echo "Although $veg1 precede $veg2 in the dictionary,"
         7   echo -n "this does not necessarily imply anything "
         8   echo "about my culinary preferences."
         9 else
        10   echo "What kind of dictionary are you using, anyhow?"
        11 fi

\<, \>

  word boundary in a regular expression.

  `bash$` `grep '\<the\>' textfile`

|

  pipe. Passes the output (`stdout`) of a previous command to the input (`stdin`) of the next one, or to the shell. This is a method of chaining commands together.

         1 echo ls -l | sh
         2 #  Passes the output of "echo ls -l" to the shell,
         3 #+ with the same result as a simple "ls -l".
         4
         5
         6 cat *.lst | sort | uniq
         7 # Merges and sorts all ".lst" files, then deletes duplicate lines.

  The output of a command or commands may be piped to a script.

         1 #!/bin/bash
         2 # uppercase.sh : Changes input to uppercase.
         3
         4 tr 'a-z' 'A-Z'
         5 #  Letter ranges must be quoted
         6 #+ to prevent filename generation from single-letter filenames.
         7
         8 exit 0

  Now, let us pipe the output of ls -l to this script.

       bash$ ls -l | ./uppercase.sh
       -RW-RW-R--    1 BOZO  BOZO       109 APR  7 19:49 1.TXT
       -RW-RW-R--    1 BOZO  BOZO       109 APR 14 16:48 2.TXT
       -RW-R--R--    1 BOZO  BOZO       725 APR 20 20:56 DATA-FILE

  The `stdout` of each process in a pipe must be read as the `stdin` of the next. If this is not the case, the data stream will block, and the pipe will not behave as expected.

         1 cat file1 file2 | ls -l | sort
         2 # The output from "cat file1 file2" disappears.

  A pipe runs as a child process, and therefore cannot alter script variables.

         1 variable="initial_value"
         2 echo "new_value" | read variable
         3 echo "variable = $variable"     # variable = initial_value

  If one of the commands in the pipe aborts, this prematurely terminates execution of the pipe. Called a broken pipe, this condition sends a `SIGPIPE` signal.

>|

  force redirection (even if the noclobber option is set). This will forcibly overwrite an existing file.

||

  OR logical operator. In a test construct, the || operator causes a return of 0 (success) if either of the linked test conditions is true.

&

  Run job in background. A command followed by an & will run in the background.

       bash$ sleep 10 &
       [1] 850
       [1]+  Done                    sleep 10

  Within a script, commands and even loops may run in the background.

  Example 3-3. Running a loop in the background

         1 #!/bin/bash
         2 # background-loop.sh
         3
         4 for i in 1 2 3 4 5 6 7 8 9 10            # First loop.
         5 do
         6   echo -n "$i "
         7 done & # Run this loop in background.
         8        # Will sometimes execute after second loop.
         9
        10 echo   # This 'echo' sometimes will not display.
        11
        12 for i in 11 12 13 14 15 16 17 18 19 20   # Second loop.
        13 do
        14   echo -n "$i "
        15 done
        16
        17 echo   # This 'echo' sometimes will not display.
        18
        19 # ======================================================
        20
        21 # The expected output from the script:
        22 # 1 2 3 4 5 6 7 8 9 10
        23 # 11 12 13 14 15 16 17 18 19 20
        24
        25 # Sometimes, though, you get:
        26 # 11 12 13 14 15 16 17 18 19 20
        27 # 1 2 3 4 5 6 7 8 9 10 bozo $
        28 # (The second 'echo' doesn't execute. Why?)
        29
        30 # Occasionally also:
        31 # 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
        32 # (The first 'echo' doesn't execute. Why?)
        33
        34 # Very rarely something like:
        35 # 11 12 13 1 2 3 4 5 6 7 8 9 10 14 15 16 17 18 19 20
        36 # The foreground loop preempts the background one.
        37
        38 exit 0
        39
        40 #  Nasimuddin Ansari suggests adding    sleep 1
        41 #+ after the   echo -n "$i"   in lines 6 and 14,
        42 #+ for some real fun.

  | A command run in the background within a script may cause the script to hang, waiting for a keystroke. Fortunately, there is a remedy for this.

&&

  AND logical operator. In a test construct, the && operator causes a return of 0 (success) only if both the linked test conditions are true.

-

  option, prefix. Option flag for a command or filter. Prefix for an operator. Prefix for a default parameter in parameter substitution.

  `COMMAND -[Option1][Option2][...]`

  `ls -al`

  `sort -dfu $filename`

         1 if [ $file1 -ot $file2 ]
         2 then #      ^
         3   echo "File $file1 is older than $file2."
         4 fi
         5
         6 if [ "$a" -eq "$b" ]
         7 then #    ^
         8   echo "$a is equal to $b."
         9 fi
        10
        11 if [ "$c" -eq 24 -a "$d" -eq 47 ]
        12 then #    ^              ^
        13   echo "$c equals 24 and $d equals 47."
        14 fi
        15
        16
        17 param2=${param1:-$DEFAULTVAL}
        18 #               ^

  --

  The double-dash `--` prefixes long (verbatim) options to commands.

  `sort --ignore-leading-blanks`

  Used with a Bash builtin, it means the end of options to that particular command.

  This provides a handy means of removing files whose names begin with a dash.

       bash$ ls -l
       -rw-r--r-- 1 bozo bozo 0 Nov 25 12:29 -badname

       bash$ rm -- -badname

       bash$ ls -l
       total 0

  The double-dash is also used in conjunction with set.

  `set -- $variable` (as in Example 15-18)

-

  redirection from/to `stdin` or `stdout` [dash].

       bash$ cat -
       abc
       abc

       ...

       Ctl-D

  As expected, `cat -` echoes `stdin`, in this case keyboarded user input, to `stdout`. But, does I/O redirection using - have real-world applications?

         1 (cd /source/directory && tar cf - . ) | (cd /dest/directory && tar xpvf -)
         2 # Move entire file tree from one directory to another
         3 # [courtesy Alan Cox <a.cox@swansea.ac.uk>, with a minor change]
         4
         5 # 1) cd /source/directory
         6 #    Source directory, where the files to be moved are.
         7 # 2) &&
         8 #   "And-list": if the 'cd' operation successful,
         9 #    then execute the next command.
        10 # 3) tar cf - .
        11 #    The 'c' option 'tar' archiving command creates a new archive,
        12 #    the 'f' (file) option, followed by '-' designates the target file
        13 #    as stdout, and do it in current directory tree ('.').
        14 # 4) |
        15 #    Piped to ...
        16 # 5) ( ... )
        17 #    a subshell
        18 # 6) cd /dest/directory
        19 #    Change to the destination directory.
        20 # 7) &&
        21 #   "And-list", as above
        22 # 8) tar xpvf -
        23 #    Unarchive ('x'), preserve ownership and file permissions ('p'),
        24 #    and send verbose messages to stdout ('v'),
        25 #    reading data from stdin ('f' followed by '-').
        26 #
        27 #    Note that 'x' is a command, and 'p', 'v', 'f' are options.
        28 #
        29 # Whew!
        30
        31
        32
        33 # More elegant than, but equivalent to:
        34 #   cd source/directory
        35 #   tar cf - . | (cd ../dest/directory; tar xpvf -)
        36 #
        37 #     Also having same effect:
        38 # cp -a /source/directory/* /dest/directory
        39 #     Or:
        40 # cp -a /source/directory/* /source/directory/.[^.]* /dest/directory
        41 #     If there are hidden files in /source/directory.

         1 bunzip2 -c linux-2.6.16.tar.bz2 | tar xvf -
         2 #  --uncompress tar file--      | --then pass it to "tar"--
         3 #  If "tar" has not been patched to handle "bunzip2",
         4 #+ this needs to be done in two discrete steps, using a pipe.
         5 #  The purpose of the exercise is to unarchive "bzipped" kernel source.

  Note that in this context the "-" is not itself a Bash operator, but rather an option recognized by certain UNIX utilities that write to `stdout`, such as tar, cat, etc.

       bash$ echo "whatever" | cat -
       whatever

  Where a filename is expected, `-` redirects output to `stdout` (sometimes seen with `tar cf`), or accepts input from `stdin`, rather than from a file. This is a method of using a file-oriented utility as a filter in a pipe.

       bash$ file
       Usage: file [-bciknvzL] [-f namefile] [-m magicfiles] file...

  By itself on the command-line, file fails with an error message.

  Add a "-" for a more useful result. This causes the shell to await user input.

       bash$ file -
       abc
       standard input:              ASCII text

       bash$ file -
       #!/bin/bash
       standard input:              Bourne-Again shell script text executable

  Now the command accepts input from `stdin` and analyzes it.

  The "-" can be used to pipe `stdout` to other commands. This permits such stunts as prepending lines to a file.

  Using diff to compare a file with a section of another:

  `grep Linux file1 | diff file2 -`

  Finally, a real-world example using `-` with tar.

  Example 3-4. Backup of all files changed in last day

         1 #!/bin/bash
         2
         3 #  Backs up all files in current directory modified within last 24 hours
         4 #+ in a "tarball" (tarred and gzipped file).
         5
         6 BACKUPFILE=backup-$(date +%m-%d-%Y)
         7 #                 Embeds date in backup filename.
         8 #                 Thanks, Joshua Tschida, for the idea.
         9 archive=${1:-$BACKUPFILE}
        10 #  If no backup-archive filename specified on command-line,
        11 #+ it will default to "backup-MM-DD-YYYY.tar.gz."
        12
        13 tar cvf - `find . -mtime -1 -type f -print` > $archive.tar
        14 gzip $archive.tar
        15 echo "Directory $PWD backed up in archive file \"$archive.tar.gz\"."
        16
        17
        18 #  Stephane Chazelas points out that the above code will fail
        19 #+ if there are too many files found
        20 #+ or if any filenames contain blank characters.
        21
        22 # He suggests the following alternatives:
        23 # -------------------------------------------------------------------
        24 #   find . -mtime -1 -type f -print0 | xargs -0 tar rvf "$archive.tar"
        25 #      using the GNU version of "find".
        26
        27
        28 #   find . -mtime -1 -type f -exec tar rvf "$archive.tar" '{}' \;
        29 #         portable to other UNIX flavors, but much slower.
        30 # -------------------------------------------------------------------
        31
        32
        33 exit 0

  Filenames beginning with "-" may cause problems when coupled with the "-" redirection operator. A script should check for this and add an appropriate prefix to such filenames, for example `./-FILENAME`, `$PWD/-FILENAME`, or `$PATHNAME/-FILENAME`.

  If the value of a variable begins with a `-`, this may likewise create problems.

         1 var="-n"
         2 echo $var
         3 # Has the effect of "echo -n", and outputs nothing.

-

  previous working directory. A cd - command changes to the previous working directory. This uses the $OLDPWD environmental variable.

  | Do not confuse the "-" used in this sense with the "-" redirection operator just discussed. The interpretation of the "-" depends on the context in which it appears.

-

  Minus. Minus sign in an arithmetic operation.

=

  Equals. Assignment operator

         1 a=28
         2 echo $a   # 28

  In a different context, the "=" is a string comparison operator.

+

  Plus. Addition arithmetic operator.

  In a different context, the + is a Regular Expression operator.

+

  Option. Option flag for a command or filter.

  Certain commands and builtins use the `+` to enable certain options and the `-` to disable them. In parameter substitution, the `+` prefixes an alternate value that a variable expands to.

%

  modulo. Modulo (remainder of a division) arithmetic operation.

         1 let "z = 5 % 3"
         2 echo $z  # 2

  In a different context, the % is a pattern matching operator.

~

  home directory [tilde]. This corresponds to the $HOME internal variable. `~bozo` is bozo's home directory, and ls ~bozo lists the contents of it. ~/ is the current user's home directory, and ls ~/ lists the contents of it.

       bash$ echo ~bozo
       /home/bozo

       bash$ echo ~
       /home/bozo

       bash$ echo ~/
       /home/bozo/

       bash$ echo ~:
       /home/bozo:

       bash$ echo ~nonexistent-user
       ~nonexistent-user

~+

  current working directory. This corresponds to the $PWD internal variable.

~-

  previous working directory. This corresponds to the $OLDPWD internal variable.

=~

  regular expression match. This operator was introduced with version 3 of Bash.

^

  beginning-of-line. In a regular expression, a "^" addresses the beginning of a line of text.

^, ^^

  Uppercase conversion in parameter substitution (added in version 4 of Bash).

Control Characters

  change the behavior of the terminal or text display. A control character is a CONTROL + key combination (pressed simultaneously). A control character may also be written in octal or hexadecimal notation, following an escape.

  Control characters are not normally useful inside a script.

  - `Ctl-A`

    Moves cursor to beginning of line of text (on the command-line).

  - `Ctl-B`

    `Backspace` (nondestructive).

  - `Ctl-C`

    `Break`. Terminate a foreground job.

  - `Ctl-D`

    Log out from a shell (similar to exit).

    `EOF` (end-of-file). This also terminates input from `stdin`.

    When typing text on the console or in an xterm window, `Ctl-D` erases the character under the cursor. When there are no characters present, `Ctl-D` logs out of the session, as expected. In an xterm window, this has the effect of closing the window.

  - `Ctl-E`

    Moves cursor to end of line of text (on the command-line).

  - `Ctl-F`

    Moves cursor forward one character position (on the command-line).

  - `Ctl-G`

    `BEL`. On some old-time teletype terminals, this would actually ring a bell. In an xterm it might beep.

  - `Ctl-H`

    `Rubout` (destructive backspace). Erases characters the cursor backs over while backspacing.

           1 #!/bin/bash
           2 # Embedding Ctl-H in a string.
           3
           4 a="^H^H"                  # Two Ctl-H's -- backspaces
           5                           # ctl-V ctl-H, using vi/vim
           6 echo "abcdef"             # abcdef
           7 echo
           8 echo -n "abcdef$a "       # abcd f
           9 #  Space at end  ^              ^  Backspaces twice.
          10 echo
          11 echo -n "abcdef$a"        # abcdef
          12 #  No space at end               ^ Doesn't backspace (why?).
          13                           # Results may not be quite as expected.
          14 echo; echo
          15
          16 # Constantin Hagemeier suggests trying:
          17 # a=$'\010\010'
          18 # a=$'\b\b'
          19 # a=$'\x08\x08'
          20 # But, this does not change the results.
          21
          22 ########################################
          23
          24 # Now, try this.
          25
          26 rubout="^H^H^H^H^H"       # 5 x Ctl-H.
          27
          28 echo -n "12345678"
          29 sleep 2
          30 echo -n "$rubout"
          31 sleep 2

  - `Ctl-I`

    `Horizontal tab`.

  - `Ctl-J`

    `Newline` (line feed). In a script, may also be expressed in octal notation -- '\012' or in hexadecimal -- '\x0a'.

  - `Ctl-K`

    `Vertical tab`.

    When typing text on the console or in an xterm window, `Ctl-K` erases from the character under the cursor to end of line. Within a script, `Ctl-K` may behave differently, as in Lee Lee Maschmeyer's example, below.

  - `Ctl-L`

    `Formfeed` (clear the terminal screen). In a terminal, this has the same effect as the clear command. When sent to a printer, a `Ctl-L` causes an advance to end of the paper sheet.

  - `Ctl-M`

    `Carriage return`.

           1 #!/bin/bash
           2 # Thank you, Lee Maschmeyer, for this example.
           3
           4 read -n 1 -s -p \
           5 $'Control-M leaves cursor at beginning of this line. Press Enter. \x0d'
           6            # Of course, '0d' is the hex equivalent of Control-M.
           7 echo >&2   #  The '-s' makes anything typed silent,
           8            #+ so it is necessary to go to new line explicitly.
           9
          10 read -n 1 -s -p $'Control-J leaves cursor on next line. \x0a'
          11            #  '0a' is the hex equivalent of Control-J, linefeed.
          12 echo >&2
          13
          14 ###
          15
          16 read -n 1 -s -p $'And Control-K\x0bgoes straight down.'
          17 echo >&2   #  Control-K is vertical tab.
          18
          19 # A better example of the effect of a vertical tab is:
          20
          21 var=$'\x0aThis is the bottom line\x0bThis is the top line\x0a'
          22 echo "$var"
          23 #  This works the same way as the above example. However:
          24 echo "$var" | col
          25 #  This causes the right end of the line to be higher than the left end.
          26 #  It also explains why we started and ended with a line feed --
          27 #+ to avoid a garbled screen.
          28
          29 # As Lee Maschmeyer explains:
          30 # --------------------------
          31 #  In the [first vertical tab example] . . . the vertical tab
          32 #+ makes the printing go straight down without a carriage return.
          33 #  This is true only on devices, such as the Linux console,
          34 #+ that can't go "backward."
          35 #  The real purpose of VT is to go straight UP, not down.
          36 #  It can be used to print superscripts on a printer.
          37 #  The col utility can be used to emulate the proper behavior of VT.
          38
          39 exit 0

  - `Ctl-N`

    Erases a line of text recalled from history buffer [8] (on the command-line).

  - `Ctl-O`

    Issues a newline (on the command-line).

  - `Ctl-P`

    Recalls last command from history buffer (on the command-line).

  - `Ctl-Q`

    Resume (`XON`).

    This resumes `stdin` in a terminal.

  - `Ctl-R`

    Backwards search for text in history buffer (on the command-line).

  - `Ctl-S`

    Suspend (`XOFF`).

    This freezes `stdin` in a terminal. (Use Ctl-Q to restore input.)

  - `Ctl-T`

    Reverses the position of the character the cursor is on with the previous character (on the command-line).

  - `Ctl-U`

    Erase a line of input, from the cursor backward to beginning of line. In some settings, `Ctl-U` erases the entire line of input, regardless of cursor position.

  - `Ctl-V`

    When inputting text, `Ctl-V` permits inserting control characters. For example, the following two are equivalent:

           1 echo -e '\x0a'
           2 echo <Ctl-V><Ctl-J>

    `Ctl-V` is primarily useful from within a text editor.

  - `Ctl-W`

    When typing text on the console or in an xterm window, `Ctl-W` erases from the character under the cursor backwards to the first instance of whitespace. In some settings, `Ctl-W` erases backwards to first non-alphanumeric character.

  - `Ctl-X`

    In certain word processing programs, Cuts highlighted text and copies to clipboard.

  - `Ctl-Y`

    Pastes back text previously erased (with `Ctl-U` or `Ctl-W`).

  - `Ctl-Z`

    Pauses a foreground job.

    Substitute operation in certain word processing applications.

    `EOF` (end-of-file) character in the MSDOS filesystem.

Whitespace

  functions as a separator between commands and/or variables. Whitespace consists of either spaces, tabs, blank lines, or any combination thereof. [9] In some contexts, such as variable assignment, whitespace is not permitted, and results in a syntax error.

  Blank lines have no effect on the action of a script, and are therefore useful for visually separating functional sections.

  $IFS, the special variable separating fields of input to certain commands. It defaults to whitespace.

  To preserve whitespace within a string or in a variable, use quoting.

  UNIX filters can target and operate on whitespace using the POSIX character class [:space:].

### Notes

[1] | An operator is an agent that carries out an operation. Some examples are the common arithmetic operators, + - * /. In Bash, there is some overlap between the concepts of operator and keyword.
[2] | This is more commonly known as the ternary operator. Unfortunately, ternary is an ugly word. It doesn't roll off the tongue, and it doesn't elucidate. It obfuscates. Trinary is by far the more elegant usage.
[3] | American Standard Code for Information Interchange. This is a system for encoding text characters (alphabetic, numeric, and a limited set of symbols) as 7-bit numbers that can be stored and manipulated by computers. Many of the ASCII characters are represented on a standard keyboard.
[4] | A PID, or process ID, is a number assigned to a running process. The PIDs of running processes may be viewed with a ps command. `Definition:` A process is a currently executing command (or program), sometimes referred to as a job.
[5] | The shell does the brace expansion. The command itself acts upon the result of the expansion.

[6]

Exception: a code block in braces as part of a pipe may run as a subshell.

       1 ls | { read firstline; read secondline; }
       2 #  Error. The code block in braces runs as a subshell,
       3 #+ so the output of "ls" cannot be passed to variables within the block.
       4 echo "First line is $firstline; second line is $secondline"  # Won't work.
       5
       6 # Thanks, S.C.

[7] | Even as in olden times a philtre denoted a potion alleged to have magical transformative powers, so does a UNIX filter transform its target in (roughly) analogous fashion. (The coder who comes up with a "love philtre" that runs on a Linux machine will likely win accolades and honors.)
[8] | Bash stores a list of commands previously issued from the command-line in a buffer, or memory space, for recall with the builtin history commands.
[9] | A linefeed (newline) is also a whitespace character. This explains why a blank line, consisting only of a linefeed, is considered whitespace.

Prev | Home | Next
Basics | Up | Introduction to Variables and Parameters