Notes on shell programming
I occasionally have to write Bourne shell scripts. Often enough that I've a small bag on knowledge about it, rarely enough that I tend to rediscover it far too often for my liking. Here is a repository of some of them, mainly for my use, but they could be useful for others.
Template
#! /bin/ksh usage() { printf "Usage: prog [options] args...\n" } help() { usage printf "Description...\n" printf "\nArguments\n" printf "\nOptions\n" } while getopts hab: opt ; do case $opt in h) help exit 0 ;; a) printf "Received -a\n" ;; b) printf "Received -b %s\n" "$OPTARG" ;; ?) usage exit 1 ;; esac done shift $(expr $OPTIND - 1)
Index
Some recommendations
Parameter quoting
Parameter should be expanded between double quotes "${var}"
if one doesn't want to get their value split. There are a few contexts
where the quoting isn't needed.
- parameter assignment:
var=$other
- case control:
case $var in
- one knows there is nothing to split (
$#
and other parameters restricted to numeral values by the logic of the script)
Parameter expansion
var is not set |
var is null |
var is not null |
|
${var:-txt} |
txt | txt | $var |
${var-txt} |
txt | $var | |
${var:=txt} |
txt | txt | $var |
${var=txt} |
txt | $var | |
${var:+txt} |
txt | ||
${var+txt} |
txt | txt |
The form with =
also set $var
. This is often
used in a context where the parameter expansion isn't needed, just to set
a default value. For instance as argument of :
(which is a
variant of true which never use its argument, GNU true
outputs something with --version
).
: ${var:=default}
A common use is to set the default value of optional arguments when the default is set from other (potentially optional) arguments. For instance:
: ${prefix:=/usr/local} : ${bindir:=${prefix}/bin} : ${libdir:=${prefix}/lib}
will set prefix, bindir and libdir if they are not already defined by the preceding arguments parsing.
To test if a parameter is set, the idiom is [ -n "${var+set}"
]
and to test if it is not set, use [ -z "${var+set}"
]
.
There is also some way to remove a pattern at the start or end of a parameter:
${V%pattern}
- remove the shortest
pattern
from the end ofV
. ${V%%pattern}
- remove the longest
pattern
from the end ofV
. ${V#pattern}
- remove the shortest
pattern
from the start ofV
. ${V##pattern}
- remove the longest
pattern
from the start ofV
.
Test
[
(or test
) can be used to compare strings and
numbers:
strings | integers | |
equal | = | -eq |
not equal | != | -ne |
less | < | -lt |
less or equal | <= | -le |
greater or equal | >= | -ge |
greater | > | -gt |
-z
and -n
test for (non) null strings.
Starting with !
negates the test.
Testing files
-t fd | fd is a terminal |
-e path | entry exists |
-f path | normal file |
-d path | directory |
-L path | symbolic link |
-r path | readable |
-w path | writable |
-x path | executable |
-s path | file not empty |
See also the note on test X"$var" =
X"value"
.
Arithmetic expansion
Expressions in $(( ... ))
are replaced by their
value. Parameters in these expressions do not need to be preceded
by $
. The operators and their precedence are
Operator |
unary +, unary -, ~, ! |
*, /, % |
+, - |
<<, >> |
<, <=, >=, > |
==, != |
& |
^ |
| |
&& |
|| |
=, *=, /=, %=, +=, -=, <<=, >>=, &=, ^=, |= |
expr
Forwarding arguments
Forwarding arguments to another program should be done
with "$@"
so that they are not split before the passing. If
one want to process some arguments and pass the others, something like
this is in order to handle the argument parsing.
#! /bin/sh - set -e checkarg() { if [ $(( $2 + 1 )) = $3 ] ; then printf "%s: argument expected\n" "$1" exit 1 fi } cnt=0 argc=$# while [ $cnt -ne $argc ] ; do case $1 in -mine) printf "Got -mine\n" ;; -with) checkarg "$1" $cnt $argc printf "Got -with %s\n" "$2" shift cnt=$(( cnt + 1 )) ;; --) shift cnt=$(( cnt + 1 )) break ;; *) set -- "$@" "$1" ;; esac shift cnt=$(( cnt + 1 )) done while [ $cnt -ne $argc ] ; do set -- "$@" "$1" shift cnt=$(( cnt + 1 )) done exec prog "$@"
Choice of shell
Although I'm somewhat concerned about portability, I don't write scripts
for unconditional one. For instance all the machines I care about have
a /bin/ksh
and thus I write my scripts for it (trying to
keep to POSIX features) instead of writing them to the lowest common
denominator of /bin/sh
(that would be the one from Solaris)
or doing epic effort to respawn a better shell.
Miscellaneous
!
applies to the pipe (! false | true
is false) but not the whole expression (! false || true
is true and! false && false
is false).
Rationale for things one often see in scripts
Here are some rationales for usage often seen in scripts but that I don't follow.
test X"$var" = X"value"
The idiom test X"$var" = X"value"
is used traditionally to
handle correctly the case where $var
begins in a way which
could confuse test
about the fact it is something to be
compared. Personally, I avoid -a
, -o
and the
parenthesis in test arguments (thus I use the shell &&
and ||
and grouping) as they are marked as obsolete by POSIX
and this avoidance suppresses the need of the trick.
Test or [
Some scripts are using
if test cond ; thenwhile others are using
if [ cond ] ; then
The only reason I know to choose the former is that the use
of [
doesn't work with some preprocessing tools
(m4
for instance for autotools scripts).
Use of negation
Some scripts are using
if cond ; then : else ... fi
Instead of
if ! cond ; then ... fi
The only shell I know which doesn't support !
is /bin/sh
on Solaris. Better use some other shell on
Solaris, that one is stable but has been frozen for so long that
supporting it is painful.
!
in test
is AFAIK supported everywhere.
Conditional expression with [[
Some shells, at least bash and zsh, allow conditional expressions with[[
. Those behave mostly like test
but with
some additional capabilities such as using &&
and ||
to combine subexpressions.
Arithmetic expression with ((
Some shells, at least bash and zsh, allow arithmetic expressions with((
.