Notes on shell programming

I occasionally have to write Bourne shell scripts. Often enough that I've a small bag on knowledge about it, rarely enough that I tend to rediscover it far too often for my liking. Here is a repository of some of them, mainly for my use, but they could be useful for others.

Template

#! /bin/ksh

usage() {
    printf "Usage: prog [options] args...\n"
}

help() {
    usage
    printf "Description...\n"
    printf "\nArguments\n"
    printf "\nOptions\n"
}

while getopts hab: opt ; do
    case $opt in
        h) 
            help
            exit 0
            ;;
        a) 
            printf "Received -a\n"
            ;;
        b)
            printf "Received -b %s\n" "$OPTARG" 
            ;;
        ?) 
            usage
            exit 1
            ;;
    esac
done
shift $(expr $OPTIND - 1)

Index

Some recommendations
Rationale for things one often see in scripts

Some recommendations

Parameter quoting

Parameter should be expanded between double quotes "${var}" if one doesn't want to get their value split. There are a few contexts where the quoting isn't needed.

parameter assignment:
```
var=$other
```
case control:
```
case $var in
```
one knows there is nothing to split ($# and other parameters restricted to numeral values by the logic of the script)

Parameter expansion

	`var` is not set	`var` is null	`var` is not null
`${var:-txt}`	txt	txt	$var
`${var-txt}`	txt		$var
`${var:=txt}`	txt	txt	$var
`${var=txt}`	txt		$var
`${var:+txt}`			txt
`${var+txt}`		txt	txt

The form with = also set $var. This is often used in a context where the parameter expansion isn't needed, just to set a default value. For instance as argument of : (which is a variant of true which never use its argument, GNU true outputs something with --version).

: ${var:=default}

A common use is to set the default value of optional arguments when the default is set from other (potentially optional) arguments. For instance:

: ${prefix:=/usr/local}
: ${bindir:=${prefix}/bin}
: ${libdir:=${prefix}/lib}

will set prefix, bindir and libdir if they are not already defined by the preceding arguments parsing.

To test if a parameter is set, the idiom is [ -n "${var+set}" ] and to test if it is not set, use [ -z "${var+set}" ].

There is also some way to remove a pattern at the start or end of a parameter:

${V%pattern}: remove the shortest pattern from the end of V.
${V%%pattern}: remove the longest pattern from the end of V.
${V#pattern}: remove the shortest pattern from the start of V.
${V##pattern}: remove the longest pattern from the start of V.

Test

[ (or test) can be used to compare strings and numbers:

	strings	integers
equal	=	-eq
not equal	!=	-ne
less	<	-lt
less or equal	<=	-le
greater or equal	>=	-ge
greater	>	-gt

-z and -n test for (non) null strings.

Starting with ! negates the test.

Testing files

-t fd	fd is a terminal
-e path	entry exists
-f path	normal file
-d path	directory
-L path	symbolic link
-r path	readable
-w path	writable
-x path	executable
-s path	file not empty

See also the note on test X"$var" = X"value".

Arithmetic expansion

Expressions in $(( ... )) are replaced by their value. Parameters in these expressions do not need to be preceded by $. The operators and their precedence are

Operator
unary +, unary -, ~, !
*, /, %
+, -
<<, >>
<, <=, >=, >
==, !=
&
^
\|
&&
\|\|
=, *=, /=, %=, +=, -=, <<=, >>=, &=, ^=, \|=

expr

Forwarding arguments

Forwarding arguments to another program should be done with "$@" so that they are not split before the passing. If one want to process some arguments and pass the others, something like this is in order to handle the argument parsing.

#! /bin/sh -
set -e

checkarg() {
    if [ $(( $2 + 1 )) = $3 ] ; then
       printf "%s: argument expected\n" "$1"
       exit 1
    fi
}

cnt=0
argc=$#
while [ $cnt -ne $argc ] ; do
    case $1 in
        -mine)
            printf "Got -mine\n"
            ;;
        -with)
            checkarg "$1" $cnt $argc
            printf "Got -with %s\n" "$2"
            shift
            cnt=$(( cnt + 1 ))
            ;;
        --)
            shift
            cnt=$(( cnt + 1 ))
            break
            ;;
        *)
            set -- "$@" "$1"
            ;;
    esac
    shift
    cnt=$(( cnt + 1 ))
done

while [ $cnt -ne $argc ] ; do
    set -- "$@" "$1"
    shift
    cnt=$(( cnt + 1 ))
done

exec prog "$@"

Choice of shell

Although I'm somewhat concerned about portability, I don't write scripts for unconditional one. For instance all the machines I care about have a /bin/ksh and thus I write my scripts for it (trying to keep to POSIX features) instead of writing them to the lowest common denominator of /bin/sh (that would be the one from Solaris) or doing epic effort to respawn a better shell.

Miscellaneous

! applies to the pipe (! false | true is false) but not the whole expression (! false || true is true and ! false && false is false).

Rationale for things one often see in scripts

Here are some rationales for usage often seen in scripts but that I don't follow.

`test X"$var" = X"value"`

The idiom test X"$var" = X"value" is used traditionally to handle correctly the case where $var begins in a way which could confuse test about the fact it is something to be compared. Personally, I avoid -a, -o and the parenthesis in test arguments (thus I use the shell && and || and grouping) as they are marked as obsolete by POSIX and this avoidance suppresses the need of the trick.

Test or [

Some scripts are using

if test cond ; then

while others are using

if [ cond ] ; then

The only reason I know to choose the former is that the use of [ doesn't work with some preprocessing tools (m4 for instance for autotools scripts).

Use of negation

Some scripts are using

if cond ; then
   :
else
   ...
fi

Instead of

if ! cond ; then
   ...
fi

The only shell I know which doesn't support ! is /bin/sh on Solaris. Better use some other shell on Solaris, that one is stable but has been frozen for so long that supporting it is painful.

! in test is AFAIK supported everywhere.

Conditional expression with [[

Some shells, at least bash and zsh, allow conditional expressions with [[. Those behave mostly like test but with some additional capabilities such as using && and || to combine subexpressions.

Arithmetic expression with ((

Some shells, at least bash and zsh, allow arithmetic expressions with ((.

Navigation

Topics