Saturday, January 30, 2010

Drupal interface to Java using XML-RPC

I haven't been posting to the blog as often as I want to lately. I just came back from a fairly long vacation in India, and over the last month, I have been trying to catch up on stuff, both at home and at work. The good news is that I am almost caught up, so I should return to my normal posting frequency shortly - as long as I have something interesting to write about, that is.

At work, we are setting up Drupal as our internal CMS. This is a bit of a departure from our normal practice, since Drupal is written in PHP, and we are predominantly a Java shop. Personally, I would probably have gone with Alfresco or dotCMS, but I don't know much about either CMS, or much about CMSs in general, so my preference is based solely on the fact that they are written in Java and open-source, so customizing them to interface with other existing (Java) components would be easier.

However, having now spent about a week futzing with Drupal, I am quite impressed with its pragmatic design - there are many good ideas in there that a Java web developer could use. Many thanks to John K VanDyck for writing Pro Drupal Development - this helped me immensely in getting up to speed quickly. Regarding interfacing to existing Java components, its not as huge a deal as I thought it would be either. From my point of view, Drupal (or any other CMS for that matter) is mainly a container of content, and interfacing with it would be done through a narrow (code wise) interface point.

The particular case I was looking at was to allow Drupal to publish/depublish stories to an external data store, from where it would be read by one or more Java application(s). The solution I came up with was to have Drupal send an XML-RPC message to a Java middleware server component on publish or depublish, which would do what is needed to write it out to the external data store. I describe the Drupal side of the interface here.

Installing Drupal

As mentioned, I have been using Drupal for about a week, so the first step was to install it. On my CentOS desktop at work, I installed (using yum) PHP, PHP-MySQL and re-installed MySQL (since the PHP-MySQL RPM did not seem to play well with the already installed MySQL RPMs from MySQL. I also set up Lighttpd as my webserver, then copied the Drupal tarball into my document root. On my Mac OS X notebook, I had to remove MySQL and install MAMP, which bundles Apache, MySQL and PHP, then install the Drupal tarball into my Document root.

From there, all you have to do is to create your Drupal database and user and then navigate to http://localhost/install.php on your browser. The installation process will walk you through a few pages, and you are all set up.

The Interface Module

Extension to Drupal are made via Modules. Drupal modules are typically written by super-experienced Drupal/PHP coders, but this one is really short and simple. I looked around a bit on the Internet for something similar, but perhaps my use case is too trivial for someone to build and contribute a module for. All it does is define an Advanced Action which is triggered on a Node insert, update or delete, along with its form for configuration. All this stuff is adapted from Chapters 3 and 19 of the Pro Drupal Development book.

My module is named dxi (Drupal XML Interface). The code lives under sites/all/modules/custom/dxi. The dxi.info file contains the meta information for the module. It looks like this:

1
2
3
4
5
; Source: sites/all/modules/custom/dxi/dxi.info
name = dxi
description = Drupal Interface to Java via XML-RPC
core = 6.x
package = My Company

The actual code is in dxi.module. I initially started with the send_request() and dxi_nodeapi() methods. This would have the effect of being called on every node event. So in the dxi_nodeapi() method, I was checking and firing the send_request() call only when the operation was "insert", "update" and "delete". This is really all that is needed to get my interface up and running.

However, using an Advanced Action instead allows you to let the user choose whether the event should be fired in the future on different events, and to set the URL for the remote XML-RPC server from the Administrator GUI instead of having to hardcode it into the code. So I ended up using the Advanced Action approach. Actions in the dxi module are defined in dxi_action_info() - there is only one, dxi_call_action(). Because it is an Advanced (Configurable) action, it needs a configuration form, which is defined by dxi_call_action_form(), the form validation is defined in dxi_call_action_validate() and the populated form values are returned from dxi_call_action_submit().

Both approaches call the send_request() method, which does the actual XML-RPC call to the remote server. The code for the entire module is shown below, with the first (hook_nodeapi) approach commented out for posterity/just in case.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
<?php
/*
 * Source: sites/all/modules/custom/dxi/dxi.module
 */

/**
 * Function to send the actual XML-RPC request over to the remote
 * server. Will throw a drupal error message if the XML-RPC request
 * fails for any reason.
 * @param $server_url - the URL of the remote server.
 * @param $op - the name of the operation to invoke.
 * @param $node - the reference to the node.
 * @return 0 or -1.
 */
function send_request($server_url, $op, $node) {
  watchdog('dxi', 'Sending request publish_' . $op . '(' . $node->nid . ')');
  $result = xmlrpc($server_url, 'publisher.' . $op, 
    $node->nid, $node->title, $node->body);
  if ($error = xmlrpc_error()) {
    if ($error->code <= 0) {
      $error->message = t('Remote server appears to be down');
    }
    drupal_set_message(t('Operation publish_' . $op . '(' . $node->nid .
      ') failed: %message (@code).', array(
      '%message' => $error->message,
      '@code' => $error->code
      )
    ));
    return -1;
  }
  return 0;
}

/**
 * Implementation of hook_nodeapi().
 * This is automatically called by the hook_nodeapi() on all nodeapi
 * events. This is the simplest approach to sending an XML-RPC request
 * for the desired operations.
 * @deprecated - use the dxi_call_action approach instead.
 * @param &$node - the node object.
 * @param $op - the operation.
 * @param $a3 - optional argument, set to NULL.
 * @param $a4 - optional argument, set to NULL.
 */
//function dxi_nodeapi(&$node, $op, $a3 = NULL, $a4 = NULL) {
//  if ($op == 'insert' || $op == 'update' || $op == 'delete') {
//    watchdog('dxi', 'dxi_nodeapi' . $op);
//    return send_request('http://localhost:8081/publisher', $op, $node);
//  }
//}

/**
 * Implementation of hook_action_info().
 * @return the $info object.
 */
function dxi_action_info() {
  $info['dxi_call_action'] = array(
    'type' => 'node',
    'description' => t('Send XML-RPC request'),
    'configurable' => TRUE,
    'hooks' => array(
      'nodeapi' => array('insert', 'update', 'delete')
    )
  );
  return $info;
}

/**
 * This function traps the nodeapi_insert, nodeapi_update and nodeapi_delete
 * events, and sends an XML-RPC request over to the Java server. This approach
 * is slightly more flexible than the dxi_nodeapi() approach, since you can
 * set up the component and operation mappings to the action from the Drupal
 * administration GUI.
 * @param $object - the node object.
 * @param $context - the context object.
 */
function dxi_call_action($object, $context) {
  $node = $object;
  if ($context['hook'] == 'nodeapi') {
    watchdog('dxi', 'dxi_call_action');
    // we only want to trigger this action on a node publish or unpublish
    // but we let the user decide that through the GUI
    $op = $context['op'];
    $remote_url = $context['remote_url'];
    return send_request($remote_url, $op, $node);
  }
}

/**
 * Implementation of ${action_name}_form. This returns field information
 * for the configuration form for this action.
 * @param $context - the context.
 * @return the $form object.
 */
function dxi_call_action_form($context) {
  $form['remote_url'] = array (
    '#type' => 'textfield',
    '#title' => t('Remote Server'),
    '#description' => t('Enter URL of Remote Server'),
    '#default_value' => isset($context['remote_url']) ?
      $context['remote_url'] : 'http://localhost:8080/publisher',
    '#required' => TRUE
  );
  return $form;
}

/**
 * Implementation of the ${action_name}_validate. This contains validation
 * for the form inputs. This is a NO-OP here.
 * @param $form - the form.
 * @param $form_state - the $form_state
 */
function dxi_call_action_validate($form, $form_state) {
  // Nothing
}

/**
 * Implementation of the ${action_name}_submit. This returns the validated
 * values of the form.
 * @param $form - the form.
 * @param $form_state - the form state.
 * @return the map of names and values of form fields.
 */
function dxi_call_action_submit($form, $form_state) {
  return array(
    'remote_url' => $form_state['values']['remote_url']
  );
}

Interface Installation and Configuration

Install the Module: Go to Administer :: Site building :: Modules and you should see the dxi module at the bottom of the page. Click the Enabled checkbox and save the configuration.

Add the Action: Go to Administer :: Site configuration :: Actions to see a list of Actions currently available to Drupal. At the bottom is a dropdown list of Advanced Actions. Choose the one that says "Send XML-RPC request" and click the Create button. This will bring up the configuration screen for this action. Set the URL for the XML-RPC server and click Save. You will see the action now associated with node Action type.

Associate the Action with the Trigger: Go to Administer :: Site building :: Triggers. Choose the Content tab. You will see a list of trigger types for Nodes. Add the new action to each of "After saving a new post", "After saving an updated post" and "After deleting a post" triggers.

Testing

If you don't have a server available (I don't yet), then just comment out lines 17-30 of dxi.module (the block which has the XML-RPC call and the error handling), then create/update/delete a story. After each step, take a look at the log entries generated - navigate to Administer :: Reports :: Recent log entries and look for the log messages with the type "dxi" - you should see entries with "Sending request..." which should convince you that the stuff is working.

Saturday, January 09, 2010

Movie Tickets over SMS

First off, hope you had a good 2009 and wish you a very Happy New 2010.

We were recently on vacation in India, as you probably guessed from my lack of postings last month. Apart from a week in New Delhi, where we did a bit of historical tourism, we stayed in Kolkata, visiting family and friends.

As everyone knows, India has made a tremendous amount of progress in the last few decades, and I am pleasantly surprised each time I go back. The last time I went, one thing that struck me was the ubiquity of the cell phone - almost everyone I met carried one, although usage was limited to voice communication.

This time around, I saw evidence of further maturity in this area. Our driver showed us how he could get our train reservation status on his cell phone, thanks to the Rail Info on SMS system developed by Indian Railways. The application is uniquely suited to the Indian market - most people have fairly basic cell phones, but SMS and WAP are usually always available. Of course, there are people who sport smartphones such as Blackberries and IPhones, but these are also SMS and WAP compatible.

Elsewhere, such as the movie theater where I was standing in line one day, they have opted for more traditional implementations - a web based booking interface which can be used from your home computer or from a smartphone with web access. There was also a solitary self-service kiosk, which IMO made very little sense in the Indian context, with its copious availability of cheap skilled manpower (to man another counter in the ticketing booth). As I stood in line, being pushed and shoved by other anxious patrons, I began to think of how easy it would be to extend the ticketing system to have an SMS/WAP interface, thereby allowing people to get on their cell phones and out of the line.

The diagram above shows the sequence of interactions between the client's cell phone and the booking system. It is described in a bit more detail below:

  1. init ticket booking: this is the customer sending a formatted SMS to the server, something like BOOK 3IDIOTS, where the first word is the command (BOOK) and the second is the code for the movie for which I want tickets.
  2. booking url: the server responds by sending a booking URL, which is a URL with the client ID as a parameter. If I have booked a ticket before with this system, it already knows my cell phone number and uses my client ID to track me. If it doesn't know me, it generates a new client ID and adds me to the database before sending back the URL. The URL is packaged as a system instruction (SI) so the phone knows to open the URL and retrieve the WAP deck from the server.
  3. send booking info: the customer fills in the WAP form and hits submit, which sends the information over SMS back to the server. If the customer is recognized, then information such as credit card number is prefilled for convenience. Some information has to be filled by the customer, such as show date, time and number of tickets. To minimize the chance of bad bookings, the WAP deck is built so that sold out shows are not available for selection.
  4. send confirmation: the server books the tickets if they are available, and sends back a confirmation to the customer. The confirmation number can be used to pick up paper tickets - since there is less human interaction required to pick up tickets, presumably this line will move faster and there will be less anxiety.

As you can see, the sequence of interactions is similar to that of a traditional web application over HTTP. The difference is that each request and response is asynchronous. Because of this, the system can scale to handle larger volumes, since the incoming requests can be queued on the server side.

Although I know next to nothing about SMS and WAP, I don't think it should be that hard to add this sort of interface to an existing web application. Of course, such an interface would have very limited usefulness in countries such as the US - most people carry smartphones with which they can just access the standard web interface. But in places like India, having a SMS/WAP interface can extend an application's reach quite dramatically.