biotext.org.uk

Michael Barton at Bioinformatics Zen recently posted an interesting article entitled “why write good software“. He makes an important point, but I think he underplays the importance of testing somewhat.

I used to code (as a lot of bioinformatics people do) like this: write a huge pile of code, run it, see if it works. If it crashes, spend hours debugging. If not, see if the results look ‘kinda right’. Now obviously unit testing reduces (almost eliminates) those tedious hours of debugging, but more importantly, it also gives you some assurance that the results you’ve submitted for publication are more likely to be correct.

I’ve suffered the embarrassment of publishing a paper with incorrect scores in (thankfully only half a percent) due to a bug not even in my algorithm itself but in the scoring routines I’d written to benchmark it. If I’d grokked unit testing at that point, this probably wouldn’t have happened. Who knows how many published papers do have incorrect scores in them because of undiscovered bugs in the code — little things like off-by-one errors or rounding errors that you wouldn’t even notice, if inspecting the output of an algorithm by eye.

Michael says that “statistical tests for significance outweigh software testing for reliability” in the eyes of bioinformaticians, but there are important parallels, and one of the parallels is that coding bugs can subtlely shaft your results just as surely as arithmetic errors or picking the wrong statistical test. Fortunately in software development, there are automated ways to catch such bugs before you publish, and to make sure that any changes you make later don’t introduce new ones. Test, test, test some more.

Andrew.