SOAP Client for Amazon ECS with XFire

SOAP based Webservices are fairly ubiquitous nowadays, but so far, I had never had a need to build one. I have built several non-SOAP Webservices in the past, but they were all for internal use, so I used various light-weight remoting technologies such as Caucho's Burlap and Hessian, Spring's HttpInvoker, RMI and so on. All of these involve making the API JARs for the service available to the client somehow. SOAP is more like CORBA, in the sense that its WSDL file is similar to the CORBA IDL, and serves the same purpose. Given a WSDL, the client should be able to generate an API locally.

One of the nice things about being a developer is that, when given a tool or technology you are unfamiliar with, you can build in some time into your project to learn it. As a manager, that luxury is denied to me. I have seen managers at past jobs deal with this by doing the unfamiliar work themselves, and handing off the rest to their engineers, but having been on the recieving end of this transaction, I did not want to perpetuate it. Besides, I was working on another equally high-priority project at the time and did not have the bandwidth to commit to a delivery date for this one. So I ended up assigning this work to one of our engineers, which neither he nor I had done before.

Nevertheless, it made me uncomfortable that I did not know enough about what I was asking someone else to do. Besides, given the ubiquity of SOAP, being able to build a SOAP client should be part of the average Java developer's skillset. So I decided to do a small proof-of-concept to learn about how to build a SOAP Webservice client. The service I chose to hit was Amazon's E-Commerce System (Amazon ECS or AWS), a very comprehensive and well-built API that exposes almost every bit of information you can find on their website. You can see their WSDL file here.

For a toolkit, I first chose Axis2, but its wsdl2java tool fails with unsatisfied dependencies when I request adf or jibx data bindings, and hangs when I request XmlBeans bindings. For those unfamiliar with the term "data bindings" (as I was when I started last weekend), its just the generated Java beans representing the types defined in the WSDL file, and the parsing code to convert between the XML and Java. Colleagues have reported success generating APIs from WSDL files using IDEs such as Netbeans and IDEA, and I am sure Eclipse can do it too, but wsdl2java from Axis2 did not work for me on the command line.

I then chose Apache CXF, formerly known as XFire. XFire seems to be the more well-known name, which is why I used it in the title - in the rest of the post I will call it Apache-CXF. I had heard about Apache-CXF when checking out Spring remoting strategies. It is supposed to be lighter weight than Axis2, but I don't know enough about either project to elaborate on what that means. Anyway, its wsdl2java worked great, generating JAXB bindings for me by default. Here is the command I used to generate the API from the AWS WSDL file:

1 2 3 4 5

sujit@sirocco:~/tmp/apache-cxf-2.0.1-incubator/bin$ ./wsdl2java \ -p net.sujit.amazon.generated \ -client \ -d /home/sujit/src/wsclient/src/main/java \ http://webservices.amazon.com/AWSECommerceService/AWSECommerceService.wsdl

This generated a bunch of files in the net/sujit/amazon/generated subdirectory of my Maven2 application. At this point, I was now ready to build my client. Amazon is my (and probably the world's) favorite bookstore, so I decided to build a client that would search for and return a list of books based on a search string. To keep my client separate from the generated code, I put it in a parallel package to the generated code. The client code follows:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59

package net.sujit.amazon.client; import java.util.ArrayList; import java.util.List; import net.sujit.amazon.generated.AWSECommerceService; import net.sujit.amazon.generated.AWSECommerceServicePortType; import net.sujit.amazon.generated.Item; import net.sujit.amazon.generated.ItemAttributes; import net.sujit.amazon.generated.ItemSearch; import net.sujit.amazon.generated.ItemSearchRequest; import net.sujit.amazon.generated.ItemSearchResponse; import net.sujit.amazon.generated.Items; import org.apache.commons.lang.StringUtils; import org.apache.log4j.Logger; public class MyWebServicesClient { private static final Logger LOGGER = Logger.getLogger(MyWebServicesClient.class); private static final String ACCESS_KEY = "MySuperSecretHexKey"; private static final String BOOK_SEARCHINDEX = "Books"; private AWSECommerceServicePortType client; public void init() throws Exception { AWSECommerceService service = new AWSECommerceService(); this.client = service.getAWSECommerceServicePort(); } public List<MyBook> getSearchResults(String keywords) { List<MyBook> myBooks = new ArrayList<MyBook>(); ItemSearch itemSearch = new ItemSearch(); itemSearch.setAWSAccessKeyId(ACCESS_KEY); ItemSearchRequest request = new ItemSearchRequest(); request.setKeywords(keywords); request.setCondition("All"); request.setSearchIndex(BOOK_SEARCHINDEX); request.getResponseGroup().add("ItemAttributes"); itemSearch.getRequest().add(request); ItemSearchResponse response = client.itemSearch(itemSearch); List<Items> itemsList = response.getItems(); for (Items items : itemsList) { List<Item> itemList = items.getItem(); for (Item item : itemList) { MyBook myBook = new MyBook(); myBook.setAsin(item.getASIN()); myBook.setUrl(item.getDetailPageURL()); ItemAttributes attributes = item.getItemAttributes(); myBook.setTitle(attributes.getTitle()); myBook.setAuthor(StringUtils.join(attributes.getAuthor().iterator(), ", ")); myBook.setPublisher(attributes.getPublisher()); myBook.setPublicationDate(attributes.getPublicationDate()); myBooks.add(myBook); } } return myBooks; } }

As you can see, my application client instantiates the service using the service stub and gets a reference to the underlying client proxy. This is done in the init() method. Once you have that, you are golden, and the rest of the code is just application code calling the remote methods via the proxy.

The getSearchResults() method represents the actual application code. The method takes a search string as its argument. It then instantiates an ItemSearch object with the AWS Access key, then builds an ItemSearchRequest with the search string, and assigns the ItemSearchRequest to the ItemSearch object. Executing the ItemSearch object's execute() method yields a ItemSearchResponse, which I then query to pull out the information from the web service into a List of MyBook view beans. The MyBook bean is a simple data holder, with the fields as shown below. The getters and setters have been omitted in the interest of space.

1 2 3 4 5 6 7 8 9 10 11

package net.sujit.amazon.client; public class MyBook { private String asin; private String title; private String url; private String author; private String publisher; private String publicationDate; ... }

More information on the various methods available in AWS and the parameters that they take are available in the Amazon ECS Web Developer's guide. As mentioned before, the service is very comprehensive, so its advisable to go through the guide if you want to do anything serious with it.

I use a JUnit test to actually call this method. This could also have been done using a main() method on the MyWebServicesClient.java file. The JUnit test looks like this:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

package net.sujit.amazon.client; import java.util.List; import org.apache.log4j.Logger; import org.junit.Before; import org.junit.Test; public class MyWebServicesClientTest { private static final Logger LOGGER = Logger.getLogger(MyWebServicesClientTest.class); private MyWebServicesClient client; @Before public void setUp() throws Exception { client = new MyWebServicesClient(); client.init(); } @Test public void testGetSearchResults() throws Exception { List<MyBook> books = client.getSearchResults("java web services"); LOGGER.debug("#-books:" + books.size()); for (MyBook book : books) { LOGGER.debug("--"); LOGGER.debug(book.toString()); } } }

The results for a search of "java web services" (see test above) return the first 10 results of the search. I did not actually build an UI for it, so I am not going to include the results here, but building an UI should be a very simple task.

As you can see, the actual code you have to write to build a SOAP client is minimal. In that sense, I can see why SOAP is so popular as an external Webservice framework, even though the XML itself is so horribly bloated compared to other XML based remoting protocols. Building a client should take very little time, if you know the things to do before you start. I outline the list of steps I had to do to get my SOAP client up and working.

1. Download apache-cxf from the project website.
2. Sign up for a free Webservices account with Amazon to get my access key.
3. Generate a Maven2 Java application
4. Add to my default pom.xml the JAR files listed in the "all CXF usage" section of the lib/WHICH_JARS file of the apache-cxf distribution. They missed the wsdl4j.jar file in the list which I added in later.
5. Locally install JARs specified in above list but not already available in my repository using mvn install:install-file.
6. Run the wsdl2java command so that the java files are generated in the right spot in my application.
7. Develop the MyWebservicesClient.java file to define the services I should call for my use case.
8. In addition, I defined a view bean (MyBook.java) to be able to collect the information out of the service into my application.
9. Write a JUnit test to run the client.

I hope this post was useful. I did find some articles on the Internet about how to develop a SOAP web client, but most of these are from companies hawking their IDE, IDE plugins or other visual products, and you have to read between the lines to see how everything fits together. My goal of building the proof of concept described here was to understand how the whole thing works, something I do not get when someone points and clicks his way through an IDE or other visual tool.