import java.util.concurrent.Callable;
import java.util.concurrent.CancellationException;
import java.util.logging.Logger;

/**
 * This class is a wrapper for a Callable that adds retry functionality.
 * The user supplies an existing Callable, a maximum number of tries,
 * and optionally a Logger to which exceptions will be logged. Calling
 * the call() method of RetriableTask causes the wrapped object's call()
 * method to be called, and any exceptions thrown from the inner call()
 * will cause the entire inner call() to be repeated from scratch, as
 * long as the maximum number of tries hasn't been exceeded.
 * InterruptedException and CancellationException are allowed to
 * propogate instead of causing retries, in order to allow cancellation
 * by an executor service etc.
 *
 * @param <T> the return type of the call() method
 */
public class RetriableTask<T> implements Callable<T>
{

    private final Callable<T> _wrappedTask;

    private final int _tries;

    private final Logger _log;

    /**
     * Creates a new RetriableTask around an existing Callable. Supplying
     * zero or a negative number for the tries parameter will allow the
     * task to retry an infinite number of times -- use with caution!
     *
     * @param taskToWrap the Callable to wrap
     * @param tries the max number of tries
     * @param log a Logger to log exceptions to (null == no logging)
     */
    public RetriableTask ( final Callable<T> taskToWrap, final int tries, final Logger log )
    {
        _wrappedTask = taskToWrap;
        _tries = tries;
        _log = log;
    }

    /**
     * Invokes the wrapped Callable's call method, optionally retrying
     * if an exception occurs. See class documentation for more detail.
     *
     * @return the return value of the wrapped call() method
     */
    public T call() throws Exception
    {
        int triesLeft = _tries;
        while( true )
        {
            try
            {
                return _wrappedTask.call();
            }
            catch( final InterruptedException e )
            {
                // We don't attempt to retry these
                throw e;
            }
            catch( final CancellationException e )
            {
                // We don't attempt to retry these either
                throw e;
            }
            catch( final Exception e )
            {
                triesLeft--;

                // Are we allowed to try again?
                if( triesLeft == 0 ) // No -- rethrow
                    throw e;

                // Yes -- log and allow to loop
                if( _log != null )
                    _log.warning( "Caught exception, retrying... Error was: " + e.getMessage() );
            }
        }

    }

}

EDIT: Just to make it clear how I would use this, it works best in the context of an ExecutorService, which manages multi-threading and timeouts for you. So for example, I could create an executor service like so:

ExecutorService pool = Executors.newCachedThreadPool();

Then create all the tasks I need, and wrap them in RetriableTasks:

// Create a task that returns a String
Callable<String> task1 = new Callable<String>()
{
    public String call()
    {
        // Do stuff here
    }
};
// Make it try up to three times
RetriableTask<String> retriable1 =
    new RetriableTask<String>( task1, 3, null );

Then add all the RetriableTasks to a Collection, and execute them all on the thread pool:

Collection<Callable<String>> tasks =
    new ArrayList<Callable<String>>();
tasks.add( retriable1 );
// TIMEOUT == timeout in seconds
List<Future<String>> results =
    pool.invokeAll( tasks, TIMEOUT, TimeUnit.SECONDS );

Finally, iterate through the results list to get all the return values:

for( Future<String> result : results )
{
    // Un-retried exceptions will pop out here
    String returnValue = result.get();
}

Or use a CompletionService.

Obviously there’s lots of other things you could do within the same framework — introducing a delay before retrying would be useful in a lot of circumstances, and also you could supply a list of exceptions that would be allowed to propagate. Or only retry on checked exceptions, and automatically bubble unchecked ones. Really it depends on the use case.

Feel free to write an extended version and post it here or link to it :-)

Andrew.

It worried me that there was no easy way to remotely check the machine’s health, so I hacked together a quick PHP page to report various vital statistics on demand — load average, memory usage, disk usage etc. — and a Perl monitor that can raise the alarm if anything exceeds safe bounds.

The status script itself really is dead simple, it looks like this:

<?php
    # status.php -- very simple server status monitor
    header( 'Content-type: text/plain' );
    # Get and display load average times 3
    $load = sys_getloadavg();
    echo "LoadAverage1: $load[0]\n";
    echo "LoadAverage5: $load[1]\n";
    echo "LoadAverage15: $load[2]\n";
    # Get and display all sorts of memory usage info
    echo join( '', file( '/proc/meminfo' ) );
    # Get and display disk usage percentages
    $df = `/bin/df`;
    foreach( split( "\n", $df ) as $line )
    {
        if( preg_match( "/(\d+%)\s+(\S+)$/", $line, $matches ) )
        {
            $fs = $matches[ 2 ];
            $usage = $matches[ 1 ];
            echo "Usage_$fs: $usage\n";
        }
    }
    # Count running processes
    $procs = `/bin/ps -e|wc -l`;
    echo "RunningProcesses: $procs\n";
?>

It uses PHP’s built-in sys_getloadavg function to return the load average, df to get the disk usage, and ps and wc to count the number of processes running. These should work on any Unix-ish system (let me know if they don’t!). Also, it uses the /proc filesystem to read lots of metrics about memory use, and this is Linux-specific. It produces output that looks like this:

LoadAverage1: 0.59
LoadAverage5: 0.4
LoadAverage15: 0.29
MemTotal:      2021816 kB
MemFree:        337592 kB
Buffers:         35408 kB
Cached:         286676 kB
SwapCached:       3152 kB
Active:        1115608 kB
Inactive:       102288 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2021816 kB
LowFree:        337592 kB
SwapTotal:     4192956 kB
SwapFree:      4185596 kB
Dirty:             304 kB
Writeback:           0 kB
AnonPages:      893740 kB
Mapped:         119900 kB
Slab:           256036 kB
PageTables:      21240 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   5203864 kB
Committed_AS:  1433344 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    280284 kB
VmallocChunk: 34359458043 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB
Usage_/: 48%
Usage_/var: 9%
Usage_/tmp: 1%
Usage_/dev/shm: 0%
Usage_/export/people: 16%
Usage_/home/bsm: 79%
Usage_/LINUX/local64: 94%
Usage_/cath/opt: 13%
Usage_/cath/svnbin: 23%
Usage_/nfs/mail: 82%
Usage_/LINUX/local: 89%
RunningProcesses: 149

The Perl monitoring script is a bit more complex, so I’ve made it available for download here. It lets you set up a config file with rules specifying named fields from the PHP script’s output, along with maximum and/or minimum allowable values for them. From the script’s comments:

#!/usr/bin/perl

# server_status_check.pl
# Andrew Clegg
#
# This script parses the output of status.php and compares it to a
# list of minimum and maximum allowable values for server resources
# specified in a config file. The config file contains one rule per
# line, like so:
#
# min MemFree 7500000
# min Usage_/LINUX/local64 95
# max LoadAverage5 0.1
#
# Any line not in this format causes an error. Do not include any
# percent signs, units (e.g. kB) etc. in the config file; these
# are automatically stripped out from the results of status.php
# before applying the rules.
#
# For each resource that is lower than a min value or larger than
# a max value, a warning is printed. Also, if the config file
# contains any rules which name resources that are not found in
# the output of status.php at all, a warning is printed for each.
#
# It returns 0 if everything is fine, 255 if an error occurred, or
# the number of warnings issued if one or more of the resource
# rules are violated.

You invoke the monitor script like this:

./server_status_check.pl http://my.server/status.php my.config.file

And it returns output that looks like this if anything rules from the config file are violated:

MemFree has value 229472 which is less than minimum 250000
Usage_/ has value 86 which is greater than maximum 50
SomeIncorrectVariableName not found in server status report

Since the return code is non-zero in case of a problem, you can easily use it in a cron job or shell script to take action when a server’s vital statistics move into dangerous ranges.

Of course, being PHP, you can use it from the command line for a quick summary of the local machine’s resources by just typing

php status.php

There are plenty more complex server monitoring tools out there, but you probably have to be a skilled sysadmin to use them, whereas these tools took a few hours to write, and five minutes to install. As usual, suggestions are welcome, and you are free to use them wherever and however you like, but please credit me and include a link back here.

Andrew.

One easy option is to iterate over the whole Iterator and buffer the results in a String, and then use a StringReader. But that’s not terribly efficient, when you might well be dealing with XML documents in the 10-20MB range. So I wrote an IteratorReader class, which is a Reader that can be wrapped around any Iterator. Each time it’s read from, it pulls enough elements from the Iterator to enable the request to be fulfilled, and buffers any remainder. This keeps its memory usage down, although this of course depends on (a) the number of characters requested at once from its read method, and (b) the size of the elements coming off the Iterator. (Each element is simply converted into a String via its toString method before being stored in a character buffer.)

Surprisingly, given the vast amount of Java source out there, I couldn’t find an existing solution for this — not even in the usually comprehensive Apache Commons. The code is below, and you are free to do what you like with it, but a credit would be nice if you use it, and if you come up with any improvements I’d be interested to hear about them. In particular, I’m sure it could be optimized more, as it spends a lot of time garbage collecting in its current form. It’s pretty thoroughly tested, with an ArrayList of two million random strings as the source of the Iterator, and seems to work fine both with single-character reads and a BufferedReader wrapped round it. Actually, testing taught me some very interesting lessons, but that’s another post.

Implementation note: As well as providing the Iterator to read from, you can optionally provide an object that implements the Closeable interface. This is because in the scenario I developed this for, the Iterator in question represented a stream of objects that was being generated on-the-fly from a live database connection, and implemented Closeable as well as Iterator so the connection could be closed when necessary. I needed a way of doing this automatically from the Reader’s point of view, so when the Iterator runs out of data (hasNext returns false) the close method of the attached Closeable, if present, is called.

Download the file: IteratorReader.java.v0_1

All comments very gratefully received.

/**
 * IteratorReader v. 0.1
 * Andrew B. Clegg
 */
 
import java.io.Closeable;
import java.io.IOException;
import java.io.Reader;
import java.util.Iterator;
 
 
public class IteratorReader extends Reader
{
 
    // The iterator from which we'll read
    private final Iterator<? extends Object> _iterator;
 
    // Optionally, an object to close when we're done
    private Closeable _closeable;
 
    // Buffer to hold character pulled from iterator before they're read
    private char[] _leftoverCharsFromLastRead = new char[ 0 ];
 
    // Flag to indicate when iterator is out of elements
    private boolean _iteratorExhausted = false;
 
    /**
     * Creates a new IteratorReader.
     * @param iterator the Iterator to read from
     */
    public IteratorReader( Iterator<? extends Object> iterator )
    {
        _iterator = iterator;
        _closeable = null;
    }
 
    /**
     * Creates a new IteratorReader whose Iterator is backed by a Closeable
     * object that must be cleanly closed when no longer needed.
     * @param iterator the Iterator to read from
     * @param closeable the Closeable object backing the Iterator
     */
    public IteratorReader( Iterator<? extends Object> iterator, Closeable closeable )
    {
        _iterator = iterator;
        _closeable = closeable;
    }
 
    /**
     * Closes the Closeable object on which this reader's Iterator depends.
     * If there is no such Closeable, or it has already been closed, this
     * method does nothing. This method is automatically called when Iterator's
     * hasNext method returns false, but can be called earlier.
     * @throws IOException if the Closeable encounters a problem when closing
     */
    @Override
    public void close() throws IOException
    {
        if( _closeable != null )
        {
            _closeable.close();
            _closeable = null;
        }
    }
 
    /**
     * Reads characters into a portion of an array. See Reader.
     * @param outBuf array to copy the characters into
     * @param outBufOffset offset at which to start storing characters
     * @param charsRequested maximum number of characters to read
     * @return the number of characters read, or -1 if the end of the iterator has been reached
     * @throws IOException if the Closeable encounters a problem when closing
     */
    @Override
    public synchronized int read( char[] outBuf, int outBufOffset, int charsRequested ) throws IOException
    {
//        System.out.format( "read called: outBufOffset=%d, charsRequested=%d\n", outBufOffset, charsRequested );
//        System.out.format( "current state: _leftoverCharsFromLastRead has %d characters, _iteratorExhausted=%b\n",
//                _leftoverCharsFromLastRead.length, _iteratorExhausted );
 
        // Have we already read enough characters from the iterator to feed this request?
        if( charsRequested <= _leftoverCharsFromLastRead.length )
        {
            // Yes, we already have enough characters, copy them into output buffer
            System.arraycopy( _leftoverCharsFromLastRead, 0, outBuf, outBufOffset, charsRequested );
            // Are there any left over?
            int remainder = _leftoverCharsFromLastRead.length - charsRequested;
            assert( remainder >= 0 );
            if( remainder > 0 )
            {
                // Copy remaining characters to new buffer (i.e. shrink buffer)
                char[] tempBuf = new char[ remainder ];
                System.arraycopy( _leftoverCharsFromLastRead, charsRequested, tempBuf, 0, remainder );
                _leftoverCharsFromLastRead = tempBuf;
            }
            else
            {
                // None left over, so reset buffer to zero-length
                _leftoverCharsFromLastRead = new char[ 0 ];
            }
            // Return the number of characters read
            // (in this case, all the characters requested)
            return charsRequested;
        }
        else
        {
            // We have been asked for more characters than we currently have, so we
            // can return what we have (if there are no more in the iterator) or
            // try to acquire more from the iterator
 
            // If iterator is exhausted and read has been called again, clean up and
            // return straight away, after copying as many characters as we have left
            if( _iteratorExhausted )
            {
                int charsAvailable = _leftoverCharsFromLastRead.length;
                if( charsAvailable == 0 )
                {
                    // Nothing in the iterator or the buffer, we're done
                    return -1;
                }
                else
                {
                    // Copy what we have into output buffer
                    System.arraycopy( _leftoverCharsFromLastRead, 0, outBuf, outBufOffset, charsAvailable );
                    // Clean up our own buffer and return number of characters copied
                    _leftoverCharsFromLastRead = new char[ 0 ];
                    return charsAvailable;
                }
            }
            else
            {
                // There's still data in the iterator, so we can attempt to satisfy the whole request
                // by doing another read -- open a stringbuilder of the desired length
                StringBuilder sb = new StringBuilder( charsRequested );
                // Insert however many characters we do have and reset our buffer to zero-length
                if( _leftoverCharsFromLastRead.length > 0 )
                {
                    sb.append( _leftoverCharsFromLastRead );
                    _leftoverCharsFromLastRead = new char[ 0 ];
                }
                int charsStillRequired = charsRequested - _leftoverCharsFromLastRead.length;
                // Iteratively add new strings until no more characters are required
                while( charsStillRequired > 0 && !_iteratorExhausted )
                {
                    // Read another string from the underlying iterator
                    String string = nextString();
                    // Add it to stringbuffer
                    sb.append( string );
                    // Adjust number still required
                    charsStillRequired = charsStillRequired - string.length();
                }
                // Did we read to the end of the iterator?
                if( _iteratorExhausted )
                {
                    // We have read all the strings from the iterator, but can only return
                    // as many characters as we managed to read, or as many as were requested,
                    // whichever is lower
                    int charsObtained = sb.length();
                    char[] tempBuf = sb.toString().toCharArray();
                    // charsToReturn is the number of chars requested, or obtained, whichever is lower
                    int charsToReturn = Math.min( charsRequested, charsObtained );
                    // Copy this many characters into output buffer
                    System.arraycopy( tempBuf, 0, outBuf, outBufOffset, charsToReturn );
                    // Do we have any left over in our buffer?
                    if( charsObtained > charsRequested )
                    {
                        // Yes -- more obtained than requested -- save them for next request
                        int charsToSave = charsObtained - charsRequested;
                        assert( charsToSave + charsToReturn == tempBuf.length );
                        _leftoverCharsFromLastRead = new char[ charsToSave ];
                        System.arraycopy( tempBuf, charsToReturn, _leftoverCharsFromLastRead, 0, charsToSave );
                    }
                    if( charsObtained == 0 )
                    {
                        // No characters left in buffer or iterator; return -1 immediately
                        _leftoverCharsFromLastRead = new char[ 0 ];
                        return -1;
                    }
                    else
                    {
                        // There are some remaining in buffer for next time, so just return
                        // the number we acquired this time
                        return charsToReturn;
                    }
                }
                else
                {
                    // sb now contains text to return, and there are more strings to iterate through.
                    // We can save a bit of effort by putting the entire contents of sb into
                    // our 'leftover' characters buffer, and calling this method again to copy it over
                    _leftoverCharsFromLastRead = sb.toString().toCharArray();
                    return read( outBuf, outBufOffset, charsRequested );
                }
            }
        }
    }
 
    private String nextString() throws IOException
    {
        // This should never get called after _iteratorExhausted has been set
        assert( !_iteratorExhausted );
 
        if( _iterator.hasNext() )
        {
            return _iterator.next().toString();
        }
        else
        {
            _iteratorExhausted = true;
            close();
            return "";
        }
    }
 
}

Note that I am far from being an expert in this field, so feel free to leave a comment if there’s anything here you disagree with. I am, as ever, happy to be corrected.

Also, please don’t get the impression that this is a methodical side-by-side comparison. I would have liked to implement the exact same non-trivial service in each stack, and report systematically on my experiences with each, but in reality I just didn’t have time to do it that rigourously. What I present here is the result of going through one or two tutorials (mostly of the Hello World/echo-a-string/double-a-number kind), reading the docs, hanging out on forums and garnering opinion from blog posts and articles. And the amount of time I spent with each toolkit is admittedly variable.

I hope it will be useful to someone, despite the caveats.

Andrew.

Background and assumptions

All of these toolkits are primarily for the development of SOAP web services, although some provide facilities for other paradigms too (e.g. REST or POX). In addition, they were evaluated from the point of view of top-down (contract-first) design, which is when a WSDL file and XML Schema are drawn up first, containing a description of the services available and their input-output formats. This is then used to generate code (more or less automatically) to handle the requests and responses at the client and/or server ends.

The opposite model, bottom-up (code-first), is very popular in the business world, as when the person writing the clients to access your services is in the same organization as you, internal standards and practices can be enforced at the code level and the actual service definitions can be a bit of an afterthought. Indeed it is relatively common for service or client programmers in this context to never even look at the generated WSDLs. However, there are good arguments for contract-first design, and all the more so when (a) you have to design services compliant with externally-imposed standards and (b) the clients accessing your services may be written using radically different toolkits, design philosophies or programming languages to your own. Much of the literature on the web relates to bottom-up design, and some of the claims made about different toolkits are only true of bottom-up projects, so caveat lector.

Also, there are two main competing service definition paradigms for web services, rpc/encoded and document/literal. Of these, the former is still only in widespread use in MOBY and in the Perl world, while the latter has become a de-facto standard elsewhere, particularly in its ‘wrapped’ dialect. (The details of the differences are beyond the scope of this introduction but there’s a good technical article about them here.) Document/literal is now becoming an ‘official’ standard too with its adoption by the Web Services Interoperability consortium. Therefore, all the frameworks below were assessed in the context of producing document/literal wrapped services.

There is a detailed (if slightly out-of-date) five-way interview with the lead developers/architects of Axis2, Metro, JBossWS, CXF and Spring-WS here. It discusses the design philosophies and architectures of these five frameworks, although it doesn’t cover subjective aspects like ease-of-use.

Axis2

This is an Apache project and largely unrelated to its predecessor Axis. It is a general-purpose implementation of SOAP web services which predates the now-standard JAX-WS model (see below). See: http://ws.apache.org/axis2/

Pros:

• Broad choice of databinding and deployment models
• Good community/developer support
• Code generation tools produce skeleton implementation classes as well as just interfaces
• REST/POX support requires little extra work

Cons:

• Broad choice of databinding and deployment models (!)
• Generated code seems to come in for criticism on the net
• JAX-WS/JAXB-style services supported but still a bit crude; tools only produce interfaces, not skeleton classes
• Error diagnosis can be awkward, requiring inspection of server logs

JAX-WS RI (Metro)

Sun’s ‘Reference Implementation’ of the JAX-WS standard, bundled alongside the GlassFish application server as part of the Java Enterprise Edition SDK. See: https://jax-ws.dev.java.net/

Pros:

• Sun are pushing it heavily = likely to be widespread adoption and good tool support
• JAXB databinding and generated code are elegant and easy to use
• Some of our collaborators use it already

Cons:

• The code generation tools require you to do a lot more work for yourself than Axis2′s, e.g. writing the implementation classes from scratch rather than from auto-generated skeletons
• The server/service configuration files are quite fiddly
• Together these mean that adding new services or changing the interface of existing ones is a bit of a hassle
• Error diagnosis can be awkward, requiring inspection of server logs

JBossWS

This is an implementation of the JAX-WS standard by RedHat and the JBoss community. Seems to work in a very similar way to Metro, similar advantages, same major disadvantage. See: http://www.jboss.org/jbossws/

I have to admit, I didn’t actually go much beyond installing this and reading the docs. If anyone has any practical experiences to share, please add some comments below! Thanks.

CXF

Formed from the merger of two older projects, Celtix and XFire, and has now been adopted by the Apache Foundation. Yes, this does mean that there are two entirely separate Apache web service platforms. Such is the bountiful joy of open-source.

Pros:

• Uses JAX-WS/JAXB natively, so compatibility and tool support shouldn’t be an issue
• Generates skeleton implementation classes as well as just interfaces
• Automatic generation of JavaScript client code, for testing or AJAX interfaces
• More meaningful error messages than Metro or Axis2
• Good community/developer support
• Adding new services/operations or modifying existing ones is pretty intuitive
• Support for REST via JAX-RS standard, and various other communication methods like JMS

Cons:

• Configuration can seem horribly complex — there’s almost too much flexibility
• Non-Spring-programmers may be intimated by Spring-based architecture (also see Spring-WS below)

Spring-WS

Spring Web Services is (obviously) the web services component of the Spring Framework, a popular application development framework that also provides useful facilities like configuration, transaction management, object-relational mapping, database abstraction, logging etc. See: http://www.springframework.org/

Pros:

• Integration with rest of Spring could be useful (although CXF also designed to facilitate this)
• Very firm (actually exclusive) focus on contract-first design
• Tool for constructing WSDL automatically from XSD (really contract-first)
• User can tie in support for various databinding technologies

Cons:

• Young compared to the other frameworks
• Additional learning curve for non-Spring-programmers
• No tools for generating Java classes/interface automatically from XSD/WSDL = more trivial hand-coding
• Document/message-centric model at odds with operation-centric (RPC-like) model used by most other frameworks — you get a blob of XML and can do with it what you want

N.B. The last of these disadvantages is actually, if you believe the Spring-WS team, an advantage after all. This is quite a persuasive argument, since you can plug in any of several unmarshalling (i.e. databinding) steps if you like, or just pull bits out of the XML tree with a parser (again there are several available). Your implementation then won’t be tied to any particular databinding object model, freeing you from the danger of ending up with incorrectly-generated classes or XSD structures that can’t be well represented in Java. Also in many cases you ought to be able to expect better performance this way, e.g. by using forward-only streaming. But it makes the endpoint classes more verbose and slower to create, which would be an issue if you’re providing a lot of services.

See the Epilogue below for more on this way of working.

Soaplab2

Largely developed at the EBI, and replacing the earlier Soaplab, Soaplab2 is a toolkit to enable wrapping of existing software in web service front-ends with the minimum of actual coding. See: http://soaplab.sourceforge.net/soaplab2/

Pros:

• Consistent access method across different back-end operation
• Server-side asynchronous functionality like status polling, job cancellation etc. is built in
• Provides web and command-line clients
• Creation of web services wrappers around existing programs is swift and usually doesn’t require any hand-coding

Cons:

• All Soaplab2 services are designed contract-first, but in ACD, not WSDL/XSD
• Doesn’t use XML Schema to describe input and output types (not in any useful sense)
• WSDLs contain no details of the actual underlying programs or their data requirements

Although it’s a powerful tool, Soaplab2′s approach to SOAP/WSDL is extremely unusual and seems, in a sense, to miss the point. The WSDLs and XSDs it produces don’t even attempt to describe the underlying processes — it just provides generically-named operations like createJob, run, destroy, getSomeResults etc. where the actual ‘service’ required (known as an ‘analysis’) is one of the parameters. Instead of listing these services in the WSDL and providing distinct SOAP endpoints to access them, it provides a list of available analyses via another web service. Elements tend to have non-descriptive names like arg0, arg1 etc. and parameters are generally passed in name-value pairs rather than proper typed data structures.

EDIT (13 Jan 2009): Apparently the Soaplab team have recently started working on ‘typed’ WSDLs (i.e. WSDLs how most other people use them) which may render some of these objections obselete. However, I don’t think they’ve actually released a new version yet which includes these.

Others?

These are all of the open-source options that I’m currently aware of. There are some commercial options around, but even some of these use open-source products under the hood (e.g. IBM WebSphere which is apparently Axis2-based).

Drop a comment if you know any different.

Conclusions

It is a shame that Soaplab2 web services have such a unique and quirky architecture, since its overall aim — enabling easy access to existing bioinformatics resources and tools — covers 90% of what we need web services for at CATH. However, taking the Soaplab2 option would enforce a certain amount of ‘vendor’ lock-in, since only Soaplab2-aware tools would be able to get the best out of the resulting services. For other clients, it would lead to an extra level of abstraction between the service consumer and the service. Rather than simply looking for suitable WSDL-based service descriptions in a registry, one would have to find every Soaplab2-based service and then query each one in order to find out what services it actually offered. The consequences for registry architecture of having some services fully described by their WSDLs and others with definitions hidden in Soaplab2 responses are not clear, but they can’t be good, and the same goes for consistent test suites and automatic service level monitoring.

It is also unclear how Soaplab2-based services would allow for semantic markup of the sort provided by methods like SAWSDL, since the same SOAP operation (e.g. createJob) could take parameters with any one of many different semantic types — sequence, accession number, structure, GO term, whatever — depending on what actual underlying process was being invoked. This would seriously reduce the potential for interoperability, automated service discovery and semantic data mining that could be provided by properly-specified services. One of the Soaplab2 developers claimed in June 2008 on the embrace-tech mailing list that ‘typed’ services were on the to-do list and would be ready in a matter of “weeks or months”, but this is apparently still a work-in-progress six months later, and of course we cannot be certain what form these will take when they do appear. EDIT (13 Jan 2009): This is apparently now being worked on, see above.

Looking instead at general-purpose web service stacks, I would consider Spring-WS if we already possessed in-house Spring expertise, since the focus on contract-first development is a good match for our requirements. Furthermore the Spring framework is very popular with Java developers and seems to provide useful solutions to a lot of common [web] application development problems. The lack of class generation tools could be problematic but is probably solvable with Ant tasks or even Perl scripts. However, the exhaustive feature list and broad compatibility of CXF, coupled with its good reputation on blogs and discussion fora, make it the best candidate available. And it offers Spring integration itself, meaning that should we wish to explore the capabilities of Spring later on, we will still have that option.

Epilogue

Having eventually gone for CXF, I quickly discovered that for performance reasons, our first Java web services project would need to use raw XML messages, rather than databinding to JAXB objects. Thankfully, it’s pretty straightforward to bypass the unmarshalling steps and just treat a SOAP message as raw XML, exactly like Spring-WS does. This is very useful if you want to produce high-throughput web services that process a large amount of data. Several of the services in the project use Oracle to output XML directly (up to tens of thousands of rows at a time) which can be injected into the body of a SOAP message by CXF, using a ‘payload provider’ service. Fast and simple.

I believe Metro also supports provider services, as well as the corresponding ‘dispatch’ clients for sending raw chunks of XML, as these are part of the JAX-WS spec — so this solution could also have been implemented with Metro. Although, having got my head around CXF’s complexity, I am still highly impressed by its flexibility as well as the quality of feedback from its developers and community. So I would have no qualms about recommending it again.