Salmon Run: caching

Showing posts with label caching. Show all posts

Saturday, March 27, 2010

Java REST Client Interface for Ehcache server

Ehcache is a popular caching library in the Java world. So far, I was aware only of the Ehcache library, using which you could build in-JVM caches. Lately (since late 2008 actually), they have come up with the Ehcache server, with which you can create remote caches that your application can access over HTTP.

This obviously has enormous implications for scalability. Granted, cache access times over a network are much higher than in-memory access, I think this is a fair tradeoff to make when you are dealing with potentially very large caches. Keeping your cache behind a server frees up your application JVM's memory. Also, if you want more cache, you can partition it out into more servers.

Our first cut of the ehcache setup was something like this. A bunch of applications would maintain their own in-JVM caches, but these caches would communicate with each other and replicate over RMI, as shown in the diagram below. This was mainly to test out the local replicated mode, which we used in our final setup for fault tolerance (see below).

The next step was to take the cache and put it behind the cache server. We used Ehcache Standalone Server version 0.8. It comes packaged within a Glassfish Application Server, so all we had to do was to expand the tarball, and update the ehcache.xml file in the war/WEB-INF/classes subdirectory with our local replicated cache definitions. To start the Ehcache server, use bin/startup.sh (this works fine, but needs cleaning up to log to a file, etc). Out of the box, its set to have the cache server listen on port 8080, you can change it to whatever you want instead.

Now, if we needed more cache memory, we could now add more server pairs (paired for fault tolerance, see diagram above) and partition the key space. Then on the client, we could do a simple hashmod of the key and direct it to the appropriate load balancer.

To make the transition seamless, I factored out all ehcache access code into a helper class - the application code was basically doing get(), put() and delete() calls on the cache - then switched the calls from local to remote cache. The code is shown below.

package com.mycompany.myapp.helpers;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable;

import net.sf.ehcache.Element;

import org.apache.commons.httpclient.DefaultHttpMethodRetryHandler;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.MultiThreadedHttpConnectionManager;
import org.apache.commons.httpclient.methods.DeleteMethod;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.InputStreamRequestEntity;
import org.apache.commons.httpclient.methods.PutMethod;
import org.apache.commons.httpclient.params.HttpMethodParams;
import org.apache.commons.io.IOUtils;
import org.apache.commons.lang.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Required;
import org.springframework.stereotype.Service;

@Service("cacheHelper")
public class CacheHelper {

  private final Logger logger = LoggerFactory.getLogger(getClass());
  
  @Autowired private String serviceUrl;
  @Autowired private int readTimeout;
  @Autowired private int connectTimeout;
  @Autowired private int maxRetries;
  @Autowired private HttpClient httpClient;
  
  public Element get(String cacheName, String key) throws Exception {
    String url = StringUtils.join(new String[] {
      serviceUrl, cacheName, key}, "/");
    GetMethod getMethod = new GetMethod(url);
    configureMethod(getMethod);
    ObjectInputStream oin = null;
    int status = -1;
    try {
      status = httpClient.executeMethod(getMethod);
      if (status == HttpStatus.SC_NOT_FOUND) {
        // if the content is deleted already
        return null;
      }
      InputStream in = getMethod.getResponseBodyAsStream();
      oin = new ObjectInputStream(in);
      Element element = (Element) oin.readObject();
      return element;
    } catch (IOException e) {
      logger.warn("GET Failed (" + status + ")", e);
    } finally {
      IOUtils.closeQuietly(oin);
      getMethod.releaseConnection();
    }
    return null;
  }
  
  public void put(String cacheName, String key, Serializable value) 
      throws Exception {
    Element element = new Element(key, value);
    String url = StringUtils.join(new String[] {
      serviceUrl, cacheName, key}, "/");
    PutMethod putMethod = new PutMethod(url);
    configureMethod(putMethod);
    ObjectOutputStream oos = null;
    int status = -1;
    try {
      ByteArrayOutputStream bos = new ByteArrayOutputStream();
      oos = new ObjectOutputStream(bos);
      oos.writeObject(element);
      putMethod.setRequestEntity(new InputStreamRequestEntity(
        new ByteArrayInputStream(bos.toByteArray())));
      status = httpClient.executeMethod(putMethod);
    } catch (Exception e) {
      logger.warn("PUT Failed (" + status + ")", e);
    } finally {
      IOUtils.closeQuietly(oos);
      putMethod.releaseConnection();
    }
  }
  
  public void delete(String cacheName, String key) throws Exception {
    String url = StringUtils.join(new String[] {
      serviceUrl, cacheName, key}, "/");
    DeleteMethod deleteMethod = new DeleteMethod(url);
    configureMethod(deleteMethod);
    int status = -1;
    try {
      status = httpClient.executeMethod(deleteMethod);
    } catch (Exception e) {
      logger.warn("DELETE Failed (" + status + ")", e);
    } finally {
      deleteMethod.releaseConnection();
    }
  }
  
  private void configureMethod(HttpMethod method) {
    if (readTimeout > 0) {
      method.getParams().setSoTimeout(readTimeout);
    }
    if (maxRetries > 0) {
      method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER, 
        new DefaultHttpMethodRetryHandler(maxRetries, false));
    }
  }
}

Calling code is no different from the code calling the local cache. Instead of cache.XXX() you now do cacheHelper.XXX() calls. The service URL is /ehcache/rest.

Here is the cache definition for one of our caches. You can repeat the cache block for multiple caches. The peer listener and peer provider factory defintions are shared across multiple caches.

<ehcache>
    ...
    <defaultCache .../>
    
    <cache name="my-cache"
        maxElementsInMemory="10000"
        eternal="true"
        diskPersistent="true"
        overflowToDisk="true">
      <cacheEventListenerFactory 
        class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"/>
      <bootstrapCacheLoaderFactory 
        class="net.sf.ehcache.distribution.RMIBootstrapCacheLoaderFactory"/>
    </cache>

    <cacheManagerPeerListenerFactory
      class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory"/>
    <cacheManagerPeerProviderFactory 
      class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"
      properties="hostName=localhost,
                  peerDiscovery=automatic,
                  multicastGroupAddress=224.0.0.1,
                  multicastGroupPort=4446,
                  timeToLive=1"/>

</ehcache>

Obviously, neither the code or configuration is rocket science, there are enough examples in the Ehcache site for someone to build this stuff themselves. But the code examples of using the Ehcache server are quite generic, they concentrate on caching strings, while most people who have used Ehcache in local mode tend to cache real Serializable objects. Also, most enterprise type places I know tend to use Jakarta's HTTPClient rather than Java's URLConnection. Also, it took me a fair bit of time to figure out and test the distributed cache configuration. So if you are looking for a quick ramp up to using Ehcache server, then this post may be helpful.

Sunday, February 17, 2008

A Generic BerkeleyDB store using DPL

I have written before about how much I liked the annotation driven persistence mechanism that BerkeleyDB Java Edition provides using its Direct Persistence Layer (DPL). I had an opportunity to look at it once more this weekend, this time with a view to persisting arbitary objects into Maps keyed by a unique String value.

The objects to be persisted are arbitary in the sense that the caller of the persistence code would know for sure what objects need to be persisted, and would persist the same class of objects into a given BerkeleyDB store. However, the code that did the persisting would not know what objects it was working with until it was instantiated by the caller. To do this, we define a generic StoreEntity object that persists objects of type V.

// StoreEntity.java
package com.mycompany.bdb;

import com.sleepycat.persist.model.Entity;
import com.sleepycat.persist.model.PrimaryKey;

@Entity
public class StoreEntity<V> {

  @PrimaryKey private String key;
  private V value;
  
  public StoreEntity() {
    super();
  }
  
  public String getKey() {
    return key;
  }
  
  public void setKey(String key) {
    this.key = key;
  }
  
  public V getValue() {
    return value;
  }
  
  public void setValue(V value) {
    this.value = value;
  }
}

The StoreEntity objects are persisted by a Store class which take care of initializing the database at startup in its init() method, and clean up resource handles in its destroy() method. It provides two methods getValue(String) to get an object of type V from the BerkeleyDB database and a setValue(String, V) to save the object V keyed by the String into the database.

// Store.java
package com.mycompany.bdb;

import java.io.File;

import org.apache.commons.io.FileUtils;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

import com.sleepycat.je.Environment;
import com.sleepycat.je.EnvironmentConfig;
import com.sleepycat.persist.EntityStore;
import com.sleepycat.persist.PrimaryIndex;
import com.sleepycat.persist.StoreConfig;

public class Store<V> {

  private final Log log = LogFactory.getLog(getClass());
  
  private String dataDirectory;
  
  private Environment env;
  private EntityStore store;
  
  public void setDataDirectory(String dataDirectory) {
    this.dataDirectory = dataDirectory;
  }
  
  protected void init() throws Exception {
    File dataDir = new File(dataDirectory);
    if (! dataDir.exists()) {
      FileUtils.forceMkdir(dataDir);
    }
    EnvironmentConfig environmentConfig = new EnvironmentConfig();
    environmentConfig.setAllowCreate(true);
    environmentConfig.setTransactional(true);
    env = new Environment(dataDir, environmentConfig);
    StoreConfig storeConfig = new StoreConfig();
    storeConfig.setAllowCreate(true);
    storeConfig.setTransactional(true);
    store = new EntityStore(env, dataDir.getName(), storeConfig);
  }
  
  protected void destroy() throws Exception {
    if (store != null) {
      store.close();
    }
    if (env != null) {
      env.close();
    }
  }
  
  @SuppressWarnings("unchecked")
  public V getValue(String key) throws Exception {
    Class<?> entityClass = StoreEntity.class;
    PrimaryIndex<String,StoreEntity<V>> primaryIndex = 
      (PrimaryIndex<String,StoreEntity<V>>) store.getPrimaryIndex(
      key.getClass(), entityClass);
    StoreEntity<V> entity = (StoreEntity<V>) primaryIndex.get(key);
    return entity.getValue();
  }
  
  @SuppressWarnings("unchecked")
  public void setValue(String key, V value) throws Exception {
    StoreEntity<V> entity = new StoreEntity<V>();
    entity.setKey(key);
    entity.setValue(value);
    PrimaryIndex<String,StoreEntity<V>> primaryIndex = 
      (PrimaryIndex<String,StoreEntity<V>>) store.getPrimaryIndex(
      key.getClass(), entity.getClass());
    primaryIndex.put(entity);
  }
}

To use this, the client code looks something like this. Obviously, the client code would be better structured than this, probably pulling out the init() and destroy() calls out into its own init() and destroy() lifecycle methods, rather than lumping them together as shown below, but you get the idea.

public class ClientCode() {
  ...
  public void sampleCode() throws Exception {
    // initialize the store
    store = new Store<List<String>>();
    store.setDataDirectory(MY_BDB_DATA_DIR);
    store.init();
    // save something into the store
    String id = "some_id";
    List<String> values = new ArrayList<String>();
    values.add("value_1");
    values.add("value_2");
    store.setValue(id, values);
    ...
    // retrieve the value from the store
    List<String> retrievedValues = store.getValue(id);
    ...
    // clean up
    store.destroy();
  }
  ...
}

If you have read my earlier blog referenced above, the code here is virtually identical to the code in there. The only difference is the use of generics to make the code reusable regardless of the payload to be persisted, without having to repeat all the boilerplate code that is needed to initialize the BerkeleyDB store.

My next step was to try and make it configurable using Spring, which is where I ran into issues. I wanted the client to be able to configure multiple such stores, each servicing a particular data type (Java objects, custom objects, or collections of either) by specifying the class name of V and the name of the subdirectory where the data should be persisted. Passing in the class name of V was an idea I got from this IBM Developerworks article - "Don't Repeat your DAO".

However, I could not find an easy way to build a Store<Whatever> object using the Class.forName() mechanism, where Whatever could either be a simple Java object, such as String or Integer, or a custom Java object, or a Collection of Java objects or custom objects. Gafter's Gadget looked kind of promising, but wasn't exactly what I was looking for.

From what I have read from other posts on this subject, what I am trying to do is probably impossible in Java at the moment. Basically, using Class.forName() style calls to reflectively build a class instance whose class name is known is not that simple with generic objects. So generics gives you flexibility at compile time, while Class.forName() gives you the same flexibility at run time. Apparently, you can't have your cake and eat it too.

Of course, I could just implement the factory in code, with a Map of store names and corresponding Store implementations, which I could set up at application startup. However, I would rather not do that if I can help it. If anyone knows of a good way to do this, or know of resources you think might help, would appreciate you pointing me at them.

Saturday, March 10, 2007

Caching with EhCache

We are in the process of switching out the current caching library in our web application, ShiftOne Java Object Cache (JOCache), with ehcache. Both are open source caching solutions, but ehcache offers significantly more features than JOCache. The original choice was made when there were fewer solutions to choose from, and as often happens with open source projects, it is easy to back the wrong horse.

Let me qualify that. When I say wrong horse, its not a reflection on either project. An open source project is typically born out of the author's own needs. It is a testament to the author's generosity that he shares his code with the world. Sometimes, the author may not be motivated to do more to the code once his own needs are met, or maybe his day job keeps him too busy, while others may thrive on the user input and feedback and are motivated to add more features to the code. Some lucky open source authors may even get to work for companies where they spend all or part of their time improving their project. Whatever the circumstances, from the point of view of a user of an open source solution, predicting which solution will grow with your needs is pretty much a crapshoot.

Our decision to switch arose out of the necessity to extend the scope of our current caching. JOCache only provides functionality to cache objects in memory, and expire them out based on the specified caching policy. A small part of our application used caching, but because there is no built in functionality to write out expired entries to a secondary disk cache, we (my predecessors, not me) built this functionality locally. Unfortunately, they also implemented a custom serialization mechanism for these objects, so this forces all new objects that need to use this setup to also implement similar functionality, or forgo secondary disk caching. Fortunately, the code to switch out caches is isolated to a single class, so its just a matter of replacing the JOCache implementation with an ehcache implementation.

Our decision to go with ehcache (instead of, say OSCache or Apache JCS) was somewhat arbitary, based on the previous experience of some of our developers with ehcache embedded in Hibernate. However, a quick look at the websites for most popular open source caching solutions reveal that they all offer similar functionalty.

I had never used ehcache by itself before. I needed to get familiar with its API, so I decided to see if I could apply caching with ehcache to one of the embedded databases I had tested and reported on in my blog post couple of weeks ago. The worst performer among the embedded databases was Apache Derby, so that is the one I chose for my test. Here is the implementation of IEmbeddedADb for Derby with ehcache.

public class CachedDerbyEmbeddedDb extends DerbyEmbeddedDb implements IEmbeddedDb {

  private Cache cache;

  public CachedDerbyEmbeddedDb() throws Exception {
    super();
    CacheManager cacheManager = CacheManager.create("src/main/resources/ehcache.xml");
    String[] names = cacheManager.getCacheNames();
    cache = cacheManager.getCache("derby-cache");
  }

  public String get(String key) {
    String value = null;
    try {
      Element cachedElement = cache.get(key);
      if (cachedElement != null) {
        value = (String) cachedElement.getValue();
      }
      if (value == null) {
        value = super.get(key);
        cache.put(new Element(key, value));
      }
    } catch (CacheException e) {
      // fall back to database
      LOGGER.warn("Cache failure!", e);
      value = super.get(key);
    }
    return value;
  }
}

The derby-cache is cached in the ehcache.xml file like so:

<ehcache>
  <diskStore path="java.io.tmpdir"/>

  <defaultCache ...>

  <cache
    name="derby-cache"
    maxElementsInMemory="10000"
    eternal="false"
    timeToIdleSeconds="120"
    timeToLiveSeconds="120"
    overflowToDisk="true"
    diskPersistent="false"
    diskExpiryThreadIntervalSeconds="120"/>
  ...
</ehcache>

After caching was applied, the query time to execute 1000 random queries against this database dropped from 2013 ms to 99 ms, bringing it to third place from fifth, up right behind HSQLDB cached tables with a response time of 77 ms for 1000 random queries.

Saturday, May 13, 2006

Using HttpClient to PURGE Squid entries

This article describes a hack to send HTTP PURGE requests to a Squid server using the Apache Commons HttpClient library. Squid is a web proxy cache which sits between the webserver and the client, intercepting HTTP GET requests and serving them out of the cache if available, or passing the request through to the webserver, populating the cache, and serving it from the cache if not. This configuration is known as a reverse-proxy configuration, probably to distinguish it from the caching proxies that ISPs put in front of their gateways to speed up customer's HTTP requests.

Squid allows you to purge entries from its cache by using a PURGE request. You can configure Squid to accept PURGE requests only from localhost (or from a set of specified internal hosts), and provides a squidclient command to do the purging. This is explained in detail in the FAQ entry here.

Since my objective was to send the PURGE request to Squid from within a Java program, using the squidclient command was not the most optimal option. Looking around the web, I came across a newsgroup post which showed a Perl script to do the same thing, which just sent this standard HTTP request over a socket to the Squid port.

1 2	PURGE http://my.squid.host:port/junk HTTP/1.0 Accept: /

Obviously, I could do something similar in Java as well. But I was also using HttpClient in this project to send HTTP GET requests, so I thought that it would be more maintainable and unified if I could somehow use HttpClient instead of doing direct socket calls for the PURGE. However, the PURGE request neither part of the standard HTTP 1.1 protocol, nor does it make sense in the context of a standard webserver, so it is not supported by HttpClient out of the box.

Adding support for a PURGE method was quite trivial, however. All I had to do was create a new PurgeMethod.java class, using the source code for the GetMethod.java class as a template, and HttpClient was able to serve HTTP PURGE requests to Squid. Here is the code for the PurgeMethod.java class.

// PurgeMethod.java
package my.company.com.httpclient.methods;
 
import org.apache.commons.httpclient.HttpMethodBase;
import org.apache.log4j.Logger;
 
/**
 * Specialized method to send a HTTP PURGE request to the specified URL. This
 * class implements the HttpMethod interface from the commons HttpClient
 * package.
 */
public class PurgeMethod extends HttpMethodBase {
 
    public PurgeMethod() {
        super();
        setFollowRedirects(true);
    }
 
    public PurgeMethod(String url) {
        super(url);
        setFollowRedirects(true);
    }
 
    public String getName() {
        return "PURGE";
    }
}

The calling code is very similar to the standard calling code for sending HTTP GET requests using HttpClient, which is detailed in the HttpClient Tutorial here. A stripped down version (without timeout settings and retries) is shown below:

    HttpClient client = new HttpClient();
    HttpMethod method = new PurgeMethod(url);
    try {
        int status = client.executeMethod(method);
        if (status != HttpStatus.SC_OK && status != HttpStatus.SC_NOTFOUND) {
            throw new Exception("HTTP PURGE failed for: " + url + "(" + status + ")");
        }
        return; // response body does not make any sense here
    } finally {
        method.releaseConnection();
    }

To test this, I aimed the code at a running Squid installation which was set up to reverse proxy to a Resin server. My test consisted of sending a PURGE request followed by two GET requests in succession, while watching the Squid access.log from another terminal. My expectation is that I will see a PURGE request, then a GET request which will not find the page in the cache (a TCP_MISS:DIRECT) followed by a GET request which will find the page in cache (a TCP_HIT:NONE). Here is a snippet from the Squid access.log.

1
2
3

10.16.181.34 - - [11/May/2006:15:51:08 -0700] "PURGE http://my.company.com/myapp/mypage.html HTTP/1.1" 200 122 TCP_MISS:NONE
10.16.181.34 - - [11/May/2006:15:51:09 -0700] "GET http://my.company.com/myapp/mypage.html HTTP/1.1" 200 30351 TCP_MISS:DIRECT
10.16.181.34 - - [11/May/2006:15:52:13 -0700] "GET http://my.company.com/myapp/mypage.html HTTP/1.1" 200 29043 TCP_HIT:NONE

As you can see, the actual code involved in adding the PURGE method functionality to HttpClient is trivial. However, this is one of the two ways I can think of to purge Squid entries in pure Java in a platform independent way. The other pure Java way is to use Java Sockets. I think that this approach is cleaner in the sense that it re-invents fewer wheels and piggybacks on standard open source libraries which provides the base plumbing functionality. Also I would like to commend the developers of the HttpClient library for making the framework so easy to extend, which I believe is a hallmark of great framework software.

Saturday, May 06, 2006

Performance Tuning Thoughts

I recently attended the MySQL Users Conference at Santa Clara, CA. One of the tutorials I signed up for was Mark Matthew's talk on J2EE Performance Tuning. Of course, this was a MySQL conference, and the speaker happened to be the author of the original JDBC driver for MySQL, so understandably there was a lot of emphasis on the new performance monitoring aspects of MySQL/Connector-J version 5, the upcoming JDBC driver for MySQL version 5.x databases.

But the nice part of the talk was that it got me thinking again about how to monitor and address performance related issues in a holistic manner. In previous lives, I have been a developer and part time system administrator for console-based Unix systems, and more recently, an Informix DBA. In both these times, I have had to address performance issues, and I have done so in a non-holistic (for want of a better word) manner. For example, my first reaction to a performance issue as an Informix DBA would have been to check the database read-write statistics, looking for hot spots, and trying to address problems by splitting the reads and writes on different disks. The next place I would look for is to check cardinality of data in the tables, looking for missing indexes. Since Informix, to the best of my knowledge, did not log slow queries, analyzing the queries meant that I would have to scan the entire codebase looking for them, so that was something I would do after the other approaches did not deliver the required functionality.

As a developer in a J2EE environment, I still have to address performance issues, but the focus is developer oriented. Typically, I measure wall-clock times of various methods and find methods which take the longest times and see if there is SQL or code that can be optimized. I measure front end responses with the Apache Bench tool, which allows you to set the number of clients, and the number of requests each client will make, and returns (among other things), the requests per second the page could serve, and the average, minimum and maximum processing times. Although MySQL logs slow queries in the slow query log, it gets used only when I am reacting to a performance problem, not when I am being proactive about ensuring my code is performant, because getting the slow query log needs DBA involvement (on MySQL version 4.1). The important thing to note in all these cases is that the performance measurements are developer-centric.

Occasionally, when reacting to performance problems, I would also look at application server (Resin) thread dumps, and try to find and fix code bottlenecks by tracing the dump back to the offending code. Although we don't run Resin with the stock JVM settings (these settings are determined by another group, based on the machine capacity on which Resin will be running), the only JVM settings I have ever actually changed myself are the minimum and maximum heap sizes.

What the talk did for me was to highlight that a J2EE application is really a layered cake of potentially non-performant hardware and software. At the very bottom there is the CPU and the RAM, followed by the operating system, followed by the database, followed by the application server, followed by the application code. The operating system, the database and the application server could potentially be non-performant because they have been improperly tuned for the application.

Fortunately, however, one does not have to start from scratch when trying to optimize for performance. It really boils down to choosing the right sized components as a starting point. Based on the projected demands on your application, you can usually choose the appropriate number and type of CPUs, and the amount of memory on your system, and the size of your swap space (among other things) on a Linux (Unix) based operating system. Databases generally offer configuration profiles (for example, the tiny, small, large and huge memory models of MySQL) to suit the particular application and hardware. Java based application servers allow you to tune the starting and maximum heap sizes, the type of garbage collector you want to use, and the generation sizes within your heap to optimize garbage collection. So really, the starting point of delivering optimal performance is to set up the optimal capacity.

Still, a person who needs to diagnose and fix a problem with a J2EE application will need to be familiar with and be able to tweak all these subsystems. Because performance metrics are heavily application dependent, this person will also need to be familiar with the application itself. Finding such a person in an organization of even moderate size is next to impossible. Bringing together groups of people to do a performance audit or diagnose and fix performance issues is a possibility, but since fixing a performance problem involves an iterative cycle of observation, tweak and more observation, this is often a time consuming operation, and often not acceptable to a business, which is losing money every minute with the non-performing application.

There was a time when I would sneer at the practice of "throwing more hardware" at a problem to fix performance issues, but the more I think about the expenses in continuing to operate a non-performant application, and the logistics to try to fix this in the time provided, the more I lean towards this option as the simplest and most cost effective way to deliver performance. By that, I do not imply the sloppy and non-performant code is ok. If there are indexes that need to be applied to the database, or the SQL needs to be re-written, or the components appropriately sized, then these should by done first. However, if your application is serving 1000 pages per second, and it starts keeling over when it is required to serve 2000 pages per second, it is possible that a few days of performance tuning will allow you to scale to the new level, perhaps more. But if it did, then it is more than possible that your application was not performant to begin with, and that should have been addressed before the application was deployed.

What I mean by "throwing more hardware" is the ability to scale out using clustering technologies. Putting the application behind a webserver and setting up reverse proxies to multiple underlying application servers, all running the same application, is one way. While each application server will contain the exact same copy of the application, they will be serving different slices of the application. The slicing will be set up in the reverse proxy configuration of the webserver. Alternatively, each application server serves the full application, but through multiple webservers behind a hardware load balancer. A hybrid of these two approaches is also a possibility.

On the database side, the scale out can be achieved by clustering multiple database masters (the read-any, write-all approach) or database replicants (one master, multiple slaves). I personally prefer the multiple master clustering scenario, since the application does not need to be changed at all to accomodate the change. The application thinks it is talking to a single database. On the other hand, in a replicated setup, you will have to have separate configurations to read from the slaves and to read and write from and to the master. There is also a replication latency which you will have to account for if your application has a scenario where it writes and reads back within a very short time. Most databases (including MySQL) offer the ability to do replication. C-JDBC is an open source initiative to achieve multiple master clustering, but its performance leaves a lot to be desired. I am told that m/cluster from Continuent, a commercial offering based on C-JDBC is much better in that regard.

Another important component is the application cache. Most J2EE applications that serve dynamic (ie generated from a database) and have significant traffic tend to use caching of some kind. When we attempt to serve the traffic with multiple machines, the cache has to be shared between the application server JVMs. There are a variety of distributed caches available in the market. The best known of these is Coherence from Tangosol, but there are others, such as JBoss Cache and SwarmCache. These caches either replicate or distribute - replicated caches replicate the cached contents to all the caches in the cluster, while distributed caches store it in only one place, but are able to pull it from the right place when they are requested to.

I still think that performance reviews of application code and load testing on hardware comparable to actual production hardware have lots of value, but neither of these approaches are simple to set up. The new MySQL JDBC driver provides a lot more performance metrics (including a "local" list of slow queries), so a disciplined J2EE developer using MySQL will find this very helpful in finding and fixing code and SQL performance bottlenecks before pushing the code out of development.

So I think, my personal performance tuning mantra boils down to these two simple commandments:

Find and fix bottlenecks in SQL and code in development.
Simulate load using the Apache Bench tool.
Design for clustering.

Salmon Run

Saturday, March 27, 2010

Java REST Client Interface for Ehcache server

Sunday, February 17, 2008

A Generic BerkeleyDB store using DPL

Saturday, March 10, 2007

Caching with EhCache

Saturday, May 13, 2006

Using HttpClient to PURGE Squid entries

Saturday, May 06, 2006

Performance Tuning Thoughts

Posts

Labels

Blogs I Read

About me

My Nerd Rating

Visitor Map

Contact Me