Salmon Run: Exposing Java Objects over HTTP with Jetty

Couple of weeks ago, I described how I exposed a Python object over HTTP using CherryPy. I wrote the code in there after a quick read of the CherryPy Tutorial, so it's quite basic and doesn't use any advanced CherryPy functionality. But what impressed me was the simplicity of CherryPy's approach to exposing the Python object - and since I am primarily a Java programmer, and therefore mostly need to expose Java objects, I decided to see if I could do something similar using Jetty. This post is a result of that decision.

Overview

I called the system JOH (for Java Over HTTP). It rhymes with D'oh (as in, D'oh! Why didn't I think of this before?). The diagram below illustrates the data flow. Imagine that you have one or more Java objects (the left most box) that you are currently calling directly from Java client code (the right most box). Exposing these objects over HTTP using JOH involves creating a Facade and plugging it into JOH on the server side. The Facade delegates to the Java Objects and decides how to serialize the outputs, in our example we convert our outputs to JSON. On the client side, the client now needs to go through an HTTP client which will deserialize the HTTP response into the desired Object.

As a user, the only significant thing you need to supply is the Facade class, which has a bunch of public methods which translate directly into servlet URI's. So a public method called getFoo() will be accessible over JOH using the URL http://.../getfoo. Unlike CherryPy, where each method is individually exposed, we rely on Java's visibility modifiers here - all public methods on the Facade are exposed - if you don't want to expose some method, change its visibility to protected or private.

Each method on the Facade takes a reference to the HttpServletRequest and HttpServletResponse. When delegating to methods of local Java objects, it extracts the relevant parameter from the request, validates and passes it on. On the way out, it serializes the output of the result and sticks it into the response. The serialization mechanism can probably be factored out into a Renderer abstraction, and multiple types of Renderer provided for different serialization mechanisms. So anyway, as you can see, most of the "magic" of exposing the Java object over HTTP happens in the Facade.

The Joh.expose method (called from the Facade's main() method) is responsible for exposing the Facade to Jetty's lifecycle via the JohHandler. Each incoming HTTP request is intercepted by the JohHandler, and converted into a method call on the Facade, which is then invoked reflectively. After the invocation, the Facade takes over and is responsible for sending out the response.

On the client side, we have a simple JOH Client which uses takes a URL and deserializes the response back to the user's requested class. Both the JSON serialization and deserialization use the Jackson JSON Processor. A similar facade exists on the client side to minimize disruption to client code, if it already exists - instead of calling the Java objects, the client code calls the equivalent methods on the client facade, which delegates to the JOH Client.

JOH Components

We describe the JOH components (in the pink boxes in the diagram above) individually below. If you just want to know how to use this stuff, then you can safely skip down to the next section on the Facade components (the light blue boxes in the diagram).

Joh.java

The Joh class is the main class and consists of a bunch of static methods. The main method is expose() which takes an instance of the Facade class to expose and a configuration file. If a configuration file is not supplied (i.e. null), suitable defaults are used to start the Jetty server. In addition, there are some generally useful utility methods in here.

// Source: src/main/java/com/mycompany/joh/Joh.java
package com.mycompany.joh;

import java.io.IOException;
import java.io.PrintWriter;
import java.util.Map;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.apache.commons.collections15.map.CaseInsensitiveMap;
import org.apache.commons.lang.StringUtils;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.mortbay.jetty.Connector;
import org.mortbay.jetty.Handler;
import org.mortbay.jetty.HttpStatus;
import org.mortbay.jetty.Server;
import org.mortbay.jetty.bio.SocketConnector;

/**
 * Exposes a Java object reference.
 */
public class Joh {

  private final static Log LOG = LogFactory.getLog(Joh.class);
  
  public final static String HTTP_PORT_KEY = "_http_port";
  
  private final static int DEFAULT_HTTP_PORT = 8080;
  
  public static void expose(Object obj, Map<String,Object> config)
      throws Exception {
    Server server = new Server();
    Connector connector = new SocketConnector();
    if (config != null && config.containsKey(HTTP_PORT_KEY)) {
      connector.setPort((Integer) config.get(HTTP_PORT_KEY));
    } else {
      connector.setPort(DEFAULT_HTTP_PORT);
    }
    server.setConnectors(new Connector[] {connector});
    Handler handler = new JohHandler(obj);
    server.setHandler(handler);
    server.start();
    server.join();
  }
  
  public static Map<String,String> getParameters(
      HttpServletRequest request) {
    Map<String,String> parameters = 
      new CaseInsensitiveMap<String>();
    Map<String,String[]> params = request.getParameterMap();
    for (String key : params.keySet()) {
      parameters.put(key, StringUtils.join(params.get(key), ","));
    }
    return parameters;
  }

  public static void error(Exception e, HttpServletRequest request,
      HttpServletResponse response) {
    response.setContentType("text/html");
    try {
      PrintWriter responseWriter = response.getWriter();
      responseWriter.println("<html><head><title>Error Page</title></head>");
      responseWriter.println("<body><font color=\"red\">");
      e.printStackTrace(responseWriter);
      responseWriter.println("</font></body></html>");
      responseWriter.flush();
      responseWriter.close();
      response.setStatus(HttpServletResponse.SC_BAD_REQUEST);
      LOG.error(e);
    } catch (IOException ioe) {
      LOG.error(ioe);
    }
  }
}

JohHandler.java

The JohHandler hooks into the Jetty request lifecycle, so that the JohHandler is invoked to handle a request. The URI of the request is mapped to a method of the Facade class, and any required method parameters are extracted from the request parameters. The invocation of the method on the Facade will cause the response to be populated with the JSON serialized result of the method call.

// Source: src/main/java/com/mycompany/joh/JohHandler.java
package com.mycompany.joh;

import java.io.IOException;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.Map;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.apache.commons.collections15.map.CaseInsensitiveMap;
import org.mortbay.jetty.HttpConnection;
import org.mortbay.jetty.Request;
import org.mortbay.jetty.handler.AbstractHandler;

/**
 * Hook into the Jetty infrastructure.
 */
public class JohHandler extends AbstractHandler {

  private Map<String,Method> methodMap = 
    new CaseInsensitiveMap<Method>();
  
  private Object javaObject;
  
  public JohHandler(Object obj) {
    super();
    this.javaObject = obj;
    Method[] methods = obj.getClass().getMethods();
    for (Method method : methods) {
      methodMap.put(method.getName(), method);
    }
  }
  
  public void handle(String target, HttpServletRequest request,
      HttpServletResponse response, int dispatch) 
      throws IOException, ServletException {
    Request req = (request instanceof Request ? 
      (Request) request : 
      HttpConnection.getCurrentConnection().getRequest());
    // strip off the leading "/" and lowercase the target. The target is
    // the same as the requestURI from the HttpServletRequest object.
    String methodName = request.getRequestURI().substring(1);
    if (methodMap.containsKey(methodName)) {
      Method method = methodMap.get(methodName);
      try {
        method.invoke(javaObject, new Object[] {request, response});
        response.setStatus(HttpServletResponse.SC_OK);
      } catch (InvocationTargetException e) {
        Joh.error(e, request, response);
      } catch (IllegalAccessException e) {
        Joh.error(e, request, response);
      }
    } else {
      Joh.error(new Exception("No such method: " + methodName), 
        request, response);
    }
    req.setHandled(true);
  }
}

JohClient.java

The JobClient is a standard Apache HTTP client that passes in HTTP GET calls to Jetty and gets back a JSON response. It then deserializes the JSON response into the object expected by the client. Here is the code for it.

// Source: src/main/java/com/mycompany/joh/JohClient.java
package com.mycompany.joh;

import org.apache.commons.httpclient.DefaultHttpMethodRetryHandler;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.params.HttpMethodParams;
import org.codehaus.jackson.map.ObjectMapper;

/**
 * Simple Jetty HTTP Client to test our Joh enabled BlogDict.
 * @author Sujit Pal
 * @version $Revision$
 */
public class JohClient {

  public Object request(String url, Class<?> clazz) {
    HttpClient client = new HttpClient();
    GetMethod method = new GetMethod(url);
    method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
      new DefaultHttpMethodRetryHandler(3, false));
    try {
      int status = client.executeMethod(method);
      if (status != HttpStatus.SC_OK) {
        throw new Exception(method.getStatusText() + 
          " [" + method.getStatusCode() + "]");
      }
      ObjectMapper mapper = new ObjectMapper();
      return mapper.readValue(method.getResponseBodyAsStream(), clazz);
    } catch (Exception e) {
      throw new RuntimeException(e);
    } finally {
      method.releaseConnection();
    }
  }
}

Some examples of calling this component are described in the Facade Components section below.

Facade Components

The original component to be exposed is our tired-but-tested BlogDict class which reads a text file and builds an internal data structure, then exposes methods which allows a caller to query parts of the data structure. Here is the code in Java (last week's post contains the Python version).

// Source: src/test/java/com/mycompany/joh/BlogDict.java
package com.mycompany.joh;

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;

import org.apache.commons.lang.StringUtils;

/**
 * Simple Java Object to be exposed by JOH.
 */
public class BlogDict {

  private String file;
  
  private Set<String> labels;
  private Map<String,Set<String>> synonyms;
  private Map<String,Set<String>> categories;
  
  public BlogDict(String file) {
    this.file = file;
    init();
  }
  
  public Set<String> getLabels() {
    return labels;
  }
  
  public Set<String> getSynonyms(String label) {
    if (synonyms.containsKey(label)) {
      return synonyms.get(label);
    } else {
      return Collections.emptySet();
    }
  }
  
  public Set<String> getCategories(String label) {
    if (categories.containsKey(label)) {
      return categories.get(label);
    } else {
      return Collections.emptySet();
    }
  }
  
  protected void init() {
    this.labels = new HashSet<String>();
    this.synonyms = new HashMap<String,Set<String>>();
    this.categories = new HashMap<String,Set<String>>();
    try {
      BufferedReader reader = new BufferedReader(new FileReader(file));
      String line = null;
      while ((line = reader.readLine()) != null) {
        if (line.startsWith("#")) {
          continue;
        }
        String[] cols = StringUtils.splitPreserveAllTokens(line, ":");
        this.labels.add(cols[0]);
        this.synonyms.put(cols[0], new HashSet<String>(
          Arrays.asList(StringUtils.split(cols[1], ","))));
        this.categories.put(cols[0], new HashSet<String>(
          Arrays.asList(StringUtils.split(cols[2], ","))));
      }
      reader.close();
    } catch (IOException e) {
      throw new RuntimeException(e);
    }
  }
}

The BlogDictFacade provides a Facade that delegates to the BlogDict class and serializes the output into JSON and sends it back in the HTTP response. Each of the public getXXX() methods in the BlogDict has an analog in the BlogDictFacade, although the method signature is different and there is no return type. Here is the code - showing it is probably clearer than explaining it.

// Source: src/test/java/com/mycompany/joh/BlogDictFacade.java
package com.mycompany.joh;

import java.io.IOException;
import java.io.PrintWriter;
import java.util.HashMap;
import java.util.Map;
import java.util.Set;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.codehaus.jackson.map.ObjectMapper;

/**
 * The web based facade that will be plugged into JOH.
 */
public class BlogDictFacade {

  private BlogDict blogDict;
  
  public BlogDictFacade(String file) {
    this.blogDict = new BlogDict(file);
  }
  
  protected void init() { /* nothing to do here */ }
  
  protected void destroy() { /* nothing to do here */ }
  
  public void getLabels(HttpServletRequest request, 
      HttpServletResponse response) {
    Set<String> labels = blogDict.getLabels();
    response.setContentType("application/x-javascript");
    try {
      PrintWriter responseWriter = response.getWriter();
      ObjectMapper mapper = new ObjectMapper();
      mapper.writeValue(responseWriter, labels);
      responseWriter.flush();
      responseWriter.close();
    } catch (IOException e) {
      Joh.error(e, request, response);
    }
  }
  
  public void getSynonyms(HttpServletRequest request, 
      HttpServletResponse response) {
    Map<String,String> parameters = Joh.getParameters(request);
    if (parameters.containsKey("label")) {
      Set<String> synonyms = 
        blogDict.getSynonyms(parameters.get("label"));
      response.setContentType("application/x-javascript");
      try {
        PrintWriter responseWriter = response.getWriter();
        ObjectMapper mapper = new ObjectMapper();
        mapper.writeValue(responseWriter, synonyms);
        responseWriter.flush();
        responseWriter.close();
      } catch (IOException e) {
        Joh.error(e, request, response);
      }
    } else {
      Joh.error(new Exception("Parameter 'label' not provided"), 
        request, response);
    }
  }
  
  public void getCategories(HttpServletRequest request, 
      HttpServletResponse response) {
    Map<String,String> parameters = Joh.getParameters(request);
    if (parameters.containsKey("label")) {
      Set<String> categories = 
        blogDict.getCategories(parameters.get("label"));
      response.setContentType("application/x-javascript");
      try {
        PrintWriter responseWriter = response.getWriter();
        ObjectMapper mapper = new ObjectMapper();
        mapper.writeValue(responseWriter, categories); 
        responseWriter.flush();
        responseWriter.close();
      } catch (IOException e) {
        Joh.error(e, request, response);
      }
    } else {
      Joh.error(new Exception("Parameter 'label' not provided"), 
        request, response);
    }
  }
  
  public static void main(String[] argv) throws Exception {
    Map<String,Object> config = new HashMap<String,Object>();
    config.put(Joh.HTTP_PORT_KEY, new Integer(8080));
    Joh.expose(new BlogDictFacade("/home/sujit/bin/blog_dict.txt"), 
      config);
  }
}

Since we are really just testing this stuff at this point, I don't have any client code, so I decided to do away with the client side Facade and just write a unit test that goes directly against the JohClient. This is shown below, to illustrate usage.

// Source: src/test/java/com/mycompany/joh/JohClientTest.java
package com.mycompany.joh;

import java.util.Set;

import junit.framework.Assert;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.junit.Test;

/**
 * Test for JohClient.
 */
public class JohClientTest {

  private final Log log = LogFactory.getLog(getClass());
  
  @Test
  public void testGetLabel() throws Exception {
    JohClient client = new JohClient();
    Set<String> labels = (Set<String>) client.request(
      "http://localhost:8080/getlabels", Set.class);
    log.debug("labels=" + labels);
    Assert.assertNotNull(labels);
    Assert.assertTrue(labels.contains("crawling"));
  }
  
  @Test
  public void testGetSynonyms() throws Exception {
    JohClient client = new JohClient();
    Set<String> synonyms = (Set<String>) client.request(
      "http://localhost:8080/getsynonyms?label=crawling", Set.class);
    log.debug("synonyms(crawling)=" + synonyms);
    Assert.assertNotNull(synonyms);
    Assert.assertTrue(synonyms.contains("crawler"));
  }
  
  @Test
  public void testGetCategories() throws Exception {
    JohClient client = new JohClient();
    Set<String> categories = (Set<String>) client.request(
      "http://localhost:8080/getcategories?label=crawling", Set.class);
    log.debug("categories(crawling)=" + categories);
    Assert.assertNotNull(categories);
    Assert.assertTrue(categories.contains("lucene"));
  }
}

Shell script

This is Java, so we need to build a shell script to run the server. A simple shell script with all the dependencies in the classpath is shown below.

#!/bin/bash
# Source: src/main/scripts/joh_blogdict.sh
PROJECT_HOME=$HOME/src/gclient
M2_REPO=$HOME/.m2/repository
CLASSPATH=\
$M2_REPO/commons-lang/commons-lang/2.3/commons-lang-2.3.jar:\
$M2_REPO/org/apache/commons/collections15/4.01/collections15-4.01.jar:\
$M2_REPO/commons-logging/commons-logging/1.1/commons-logging-1.1.jar:\
$M2_REPO/log4j/log4j/1.2.14/log4j-1.2.14.jar:\
$M2_REPO/org/mortbay/jetty/jetty/6.1.5/jetty-6.1.5.jar:\
$M2_REPO/org/mortbay/jetty/jetty-util/6.1.5/jetty-util-6.1.5.jar:\
$M2_REPO/org/mortbay/jetty/servlet-api-2.5/6.1.5/servlet-api-2.5-6.1.5.jar:\
$M2_REPO/org/codehaus/jackson/core/1.2.0/core-1.2.0.jar:\
$M2_REPO/org/codehaus/jackson/mapper/1.2.0/mapper-1.2.0.jar:\
$PROJECT_HOME/target/classes:\
$PROJECT_HOME/target/test-classes

java -cp $CLASSPATH com.mycompany.joh.BlogDictFacade 2>&1 | tee $0.log

After running this script on the command prompt, you can hit the BlogDict using either a browser or something like JohClientTest shown above. To shutdown the server, hit CTRL+C.

Conclusion

Although a lot of code is shown here on this blog, in reality, a user who is looking for functionality to expose a Java object needs to only create the Facade object and plug it into JOH using the Joh.expose() call. So the approach is very similar to CherryPy's. In the same spirit, there is no attempt to force the user (via interface, etc) to conform to a specific approach - this is more of a prescriptive approach.