A bashism a week: maths

You've probably already done some basic maths in shell scripts, but do you know what else you can actually do?

Pick at least 4 operations that you can do in bashisms-free shell scripts:

$((n+1))
$((n>8))
$((n^4))
$((--n))
$((n*=5))
$((n++))
$((n==1?2:3))

The POSIX:2001 standard defines the arithmetic expansion requirements, which leads us to selecting all of the above operations except two:

$((--n))
$((n++))

"--" and "++" are not required to be implemented, and in some cases they may lead to unexpected results, such as the following:


$ bash -c 'n=1; echo $((++n))'
2
$ dash -c 'n=1; echo $((++n))'
1


Remember, if you rely on any non-standard behaviour or feature make sure you document it and, if feasible, check for it at run-time.

Faster, more stable and new opportunities

A new version of the code behind http.debian.net was deployed a few days ago. It has proved to be far more stable, faster, and scalable compared to the previous mod_perl-based deployment.

There were a couple of glitches during the earlier roll-out for IPv6 users, fixed thanks to the reports by Cyril Brulebois, Michael Stapelberg and Robert Drake.

What's behind?
The redirector is now a plack application (with no middlewere) running under the Starman server with an apache fronted. Requests are processed faster than before and there mod_perl-induced system overload is finally gone.

The redirector is now easier to test and develop. Deploying the live instance is not yet fully streamlined, but it has seen a lot of improvement. Some important changes to the way the redirector works are already on their way to see the light and I am going to be announcing them when they do. Fork the repository and hack a few changes, contributions are welcome :)

It's probably time to move it under debian.org, and finish making it the default everywhere. It's even made its way into the installation manual.

A bashism a week: dangerous exports

As a user of a shell you have most likely had the need to export a variable to another process; i.e. set/modify an environment variable.

Now, how do you stop exporting an environment variable? can you export anything else?

The bash shell offers the -n option of the export built-in, claiming it "remove[s] the export property". However, this feature is not part of the POSIX:2001 standard and is, therefore, a bashism.

A portable way to stop exporting an environment variable is to unset it. E.g. the effect of "export MY_VAR=foo" can be reverted by calling "unset MY_VAR" - surely enough, this will also destroy the content of the variable.

An equivalent could then be:

# to stop exporting/"unexport" the MY_VAR environment variable:
my_var="$MY_VAR" ; unset MY_VAR ;
MY_VAR="$my_var" ; unset my_var ;

The above code will make a copy of the variable before destroying it and then restoring its content.

How about exporting other things? did you know that you can export shell functions?

With the bash shell, you can export a function with the -f parameter of the export built-in. Needless to say, this is a bashism. Its secret? it's just an environment variable with the name of the function and the rest of the function definition as its value.

Try this:

$ echo="() { /bin/echo 'have some bash' ; }" bash -c 'echo "Hello World!"'
have some bash

Yes, this means that if you can control the content of an environment variable passed to bash you can probably execute whatever code you want. It comes handy when you want to alter a script's behaviour without modifying the script itself.

Possibilities are endless thanks to bash's support for non-standard characters in function names. Functions with slashes can also be exported, for example:

/sbin/ifconfig() {
echo "some people say you should be using ip(1) instead" ;
}

Are you into bug hunting? export exec='() { echo mount this ; }'

A bashism a week: aliases

In a response to my blogpost about bashisms in function names, reader Detlef L pointed out in a comment that aliases allow non-standard characters in their names, contrary to functions. They could then be used to, for example, set an alias of the run-parts(1) command (cf. the blog post).

Aliases indeed allow characters such as commas ( , ) to be used in the alias name. However, aliases are an extension to the POSIX:2001 specification and are therefore bashisms. Moreover, the characters set defined by POSIX does not include dashes.

Last but not least, aliases belong to the list of shell features that are usually "disabled" when the shell is run in non-interactive mode. I.e.


$ bash <<EOF
alias true=false;
if true; then echo alias disabled; else echo alias enabled; fi
EOF

alias disabled
$ bash -i <<EOF # force interactive mode
alias true=false;
if true; then echo alias disabled; else echo alias enabled; fi
EOF

$ alias true=false;
$ if true; then echo alias disabled; else echo alias enabled; fi
alias enabled
$ exit


To add to the fun of different behaviours, other shells always expand aliases.

If you decide to play with aliases you should note one thing: they are only enabled from the line after the definition. E.g. note the difference below

$ dash -c 'alias foo="echo foo"; foo'
dash: foo: not found
$ dash -c 'alias foo="echo foo";
foo'
foo


Worse than no support

You are in need of support and you go ask in project x's forums (mailing list, online board, etc.) and get something worse than no support:

Responses from an incompetent with plenty of time and lots of FUD to spread.

Now that we have "umarell" thanks to Enrico Zini and other Italian folks, we need a name for the above.

Apologies to some readers who might be disappointed by this post. I don't like calling people names but I've seen too many of the above in less than two hours not to be tired.

A bashism a week: testing strings

The bashism of this week deals again with tests. How do you compare two strings to tell which one goes before or after the other in the alphabet in a shell script?
(also know as lexicographic comparison)

If you are familiar with perl and the relationship between its comparison operators and that of shell scripts, you would probably write the test as follows:


$ test bar '<' foo
$ test foo '>' bar


And yes, that would work, but not with all shells. I'm afraid to tell you that the < and > comparison operators are not required by the POSIX:2001 specification and are, therefore, bashisms.

What does this have to do with perl? well, the shell way is the inverse of the perl way.
What? In Perl you test the equality of two numbers with '==', in shell you use '-eq'; in perl the equality of strings is 'eq', in shell it is '='. In shell scripts you can compare two numbers with -gt (test 1 -gt 0) while in perl the equivalent comparison is with > (1 > 0).

So in this case the perl way is "bar lt foo" and based on the above the shell way should be "bar < foo". However, care must be taken when using such operators in shell scripts. Since < and > are used for redirections in shell scripts, one must quote them even when using bash. An alternative is to use the [[ special test function of bash which alters the shell syntax.

This time I didn't write a function to portably (or "somewhat portably") replace such comparisons. Feel free to share your solution in the comments, with bonus points if you come up with a solution without using external commands.

A bashism a week: tilde expansion

Did you know that you can get a user's home directory with the "~username" tilde expansion? You probably do, but how about other tilde expansions?

Thanks to a few bashisms you can make your script even more difficult to read by using "~+" to get $PWD, and "~-" to get $OLDPWD.

Are you using bash and want to access the directories stack created when using the pushd and popd bashisms? The ~i (where i is an integer) expansion can give you that. It gives you forward (~+i) and backward (~-i) access to the directories stack.

But beware, if the directories stack is smaller than the number you used, there won't be any expansion.

When using tildes in shell scripts, make sure you quote to avoid unwanted expansions. Note that the posh shell in wheezy and older do support those non-POSIX expansions.

in the first examples the tilde is quoted for easier reading; the expansion doesn't occur if the tilde is quoted.