Salmon Run: My first Custom Annotation

In a previous post, I noted that although I did not know much about how annotations worked, I seem to be using them more and more nowadays. I guess it was only a matter of time before I found a use-case where I would need to write my own annotation. Anyway, the opportunity presented itself when I was trying to write out a JPA @Entity bean described in my previous post out to a Lucene index. This post describes the annotation itself, and how I used it in my index building code.

My objective was to annotate some of the fields in my @Entity beans to tell the indexing process that these fields should be picked up. Additionally, I wanted to specify the Store and Index attributes that will govern how they will be stored and indexed.

To do this, I created the following Annotation class. The Storable and Indexable enums have elements that shadow the Field.Store and Field.Index enums in Lucene.

// IndexMeta.java
package com.mycompany.myapp.indexing;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

/**
 * Property level annotation to indicate indexing meta-information.
 * Bean properties that should be indexed should be annotated with this.
 * Additional field level indexing metadata is supplied through the 
 * attributes Indexable and Storable. 
 */
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
public @interface IndexMeta {

  public enum Indexable {NO, NO_NORMS, TOKENIZED, UN_TOKENIZED};
  public enum Storable {COMPRESS, NO, YES};
  
  Indexable indexable() default Indexable.NO;
  Storable storable() default Storable.NO;
}

To test this, we created two BookArticle objects in the database. Since BookArticle objects inherit from Article objects, we need to annotate properties in both classes. This is shown below:

// Article.java
package com.mycompany.myapp.persistence;

import java.util.ArrayList;
import java.util.List;

import javax.persistence.DiscriminatorColumn;
import javax.persistence.DiscriminatorType;
import javax.persistence.Entity;
import javax.persistence.Inheritance;
import javax.persistence.InheritanceType;

import com.mycompany.myapp.indexing.IndexMeta;
import com.mycompany.myapp.indexing.IndexMeta.Indexable;
import com.mycompany.myapp.indexing.IndexMeta.Storable;

/**
 * The article superclass table contains all the common fields.
 */
@Entity
@Inheritance(strategy=InheritanceType.JOINED)
@DiscriminatorColumn(discriminatorType=DiscriminatorType.INTEGER, name="articleTypeId")
public abstract class Article extends ModelBase {

  @IndexMeta(indexable=Indexable.UN_TOKENIZED, storable=Storable.YES)
  private String articleId;
  @IndexMeta(indexable=Indexable.TOKENIZED, storable=Storable.YES)
  private String title;
  @IndexMeta(indexable=Indexable.TOKENIZED, storable=Storable.YES)
  private String summary;
  @IndexMeta(indexable=Indexable.NO, storable=Storable.YES)
  private String url;
  private String fileLocation;

  ... getters and setters removed ...  
}

// BookArticle.java
package com.mycompany.myapp.persistence;

import javax.persistence.DiscriminatorValue;
import javax.persistence.Entity;

import com.mycompany.myapp.indexing.IndexMeta;
import com.mycompany.myapp.indexing.IndexMeta.Indexable;
import com.mycompany.myapp.indexing.IndexMeta.Storable;

/**
 * Models a book article.
 */
@Entity
@DiscriminatorValue("2")
public class BookArticle extends Article {

  private static final long serialVersionUID = -2274023497279749079L;
  
  @IndexMeta(indexable=Indexable.TOKENIZED, storable=Storable.YES)
  private String authorName;
  private String publisherName;
  private String isbnNumber;
  
  ... getters and setters removed ...
}

Finally, we use Java 1.5's new reflection methods to find and apply the annotations during our indexing process. The JUnit test below shows how to get all the BookArticle beans from the database using JPA, and then passing the beans to a Lucene IndexWriter.

// IndexingTest.java
package com.mycompany.myapp.indexing;

import java.lang.reflect.Field;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import javax.persistence.EntityManager;
import javax.persistence.EntityManagerFactory;
import javax.persistence.Persistence;
import javax.persistence.Query;

import org.apache.commons.beanutils.BeanUtils;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field.Index;
import org.apache.lucene.document.Field.Store;
import org.apache.lucene.index.IndexWriter;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

import com.mycompany.myapp.indexing.IndexMeta.Indexable;
import com.mycompany.myapp.indexing.IndexMeta.Storable;
import com.mycompany.myapp.persistence.BookArticle;

/**
 * Test case to demonstrate indexing using the IndexMeta annotation.
 */
public class IndexingTest {

  private final Log log = LogFactory.getLog(getClass());
  
  private final String INDEX_LOCATION = "/tmp/testindex";
  
  private EntityManagerFactory emf;
  private EntityManager em;
  private IndexWriter indexWriter;
  
  @Before
  public void setUp() throws Exception {
    emf = Persistence.createEntityManagerFactory("myapp");
    em = emf.createEntityManager();
    indexWriter = new IndexWriter(INDEX_LOCATION, new StandardAnalyzer(), true);
  }

  @After
  public void tearDown() throws Exception {
    em.close();
    emf.close();
    indexWriter.optimize();
    indexWriter.close();
  }

  /**
   * Builds a Lucene index using the List of beans returned by a Query.
   * @throws Exception if one is thrown.
   */
  @Test
  public void testBuildIndex() throws Exception {
    // get the indexing metadata
    Map<String,IndexMeta> indexingMetadata = 
      buildIndexingMetadata(BookArticle.class);
    // get the data
    Query q = em.createQuery("select ba from BookArticle ba");
    List<BookArticle> articles = 
      (List<BookArticle>) q.getResultList();
    for (BookArticle article : articles) {
      Document doc = new Document();
      for (String fieldName : indexingMetadata.keySet()) {
        String value = getFieldValue(article, fieldName);
        if (value == null) {
          continue;
        }
        IndexMeta indexMeta = indexingMetadata.get(fieldName);
        Store storeProperty = getStoreProperty(indexMeta.storable());
        Index indexProperty = getIndexProperty(indexMeta.indexable());
        org.apache.lucene.document.Field f = 
          new org.apache.lucene.document.Field(
          fieldName, value, storeProperty, indexProperty);
        doc.add(f);
      }
      indexWriter.addDocument(doc);
    }
  }

  /**
   * Reflectively gets the String value of the named bean property. If an 
   * exception occurs, the value returned is null.
   * @param obj the Article object.
   * @param fieldName the name of the bean property.
   * @return the String value of the bean property.
   */
  private String getFieldValue(Object obj, String fieldName) {
    try {
      return String.valueOf(BeanUtils.getProperty(obj, fieldName));
    } catch (Exception e) {
      return null;
    }
  }
  
  /**
   * Reflectively build a Map of bean property name to the annotated IndexMeta
   * properties. Climbs the object tree until the Object class is reached.
   * @param type the Article type to build the Index metadata for.
   * @return a Map of bean property name to the associated IndexMeta annotation.
   */
  private Map<String,IndexMeta> buildIndexingMetadata(Class<?> type) {
    Map<String,IndexMeta> indexingMetadata = 
      new HashMap<String,IndexMeta>();
    do {
      Field[] fields = type.getDeclaredFields();
      for (Field field : fields) {
        IndexMeta indexMeta = field.getAnnotation(IndexMeta.class);
        if (indexMeta == null) {
          // no annotation, no lucene field
          continue;
        }
        indexingMetadata.put(field.getName(), indexMeta);
      }
      type = type.getSuperclass();
    } while (type != Object.class);
    return indexingMetadata;
  }

  /**
   * Helper method to map the Field.Index value from IndexMeta.Indexable
   * @param indexable an instance of Indexable.
   * @return an instance of Field.Index.
   */
  private Index getIndexProperty(Indexable indexable) {
    switch(indexable) {
      case NO_NORMS:
        return Index.NO_NORMS;
      case TOKENIZED:
        return Index.TOKENIZED;
      case UN_TOKENIZED:
        return Index.UN_TOKENIZED;
      case NO:
      default:
        return Index.NO;
    }
  }

  /**
   * Helper method to return Field.Store value for IndexMeta.Storable.
   * @param storable an instance of Storable.
   * @return an instance of Field.Store.
   */
  private Store getStoreProperty(Storable storable) {
    switch(storable) {
      case COMPRESS:
        return Store.COMPRESS;
      case YES:
        return Store.YES;
      case NO:
      default:
        return Store.NO;
    }
  }
}

And of course, the obligatory screenshot of what this index looks like in Luke.

I hope this has been useful. I had read about annotations in the official annotation documentation and in the O'Reilly Java 1.5 Tiger - A Developer's Handbook by Brett McLaughlin and David Flanagan, but I could never come up with a use-case where I would need to build my own. This example may resonate a little better with Java developers who use ORMs and Lucene.

On a somewhat related note, check out Steve Yegge's blog post "Execution in the Kingdom of Nouns". Not only is it quite hilarious, it is also the clearest analogy I have seen that explains the differences between Object Oriented programming and functional programming. Why related? Well, we started with classes and interfaces. Java 1.5 gave us Enums and Annotations. Maybe it is time Java gave us Function objects as well, so we can use them directly when requirements dictate, rather than setting up anonymous holder classes to do the same thing.

4 comments:

Hemant Patel3/03/2008 4:26 AM
Hi Sujit,
It is "out of box" thinking and I would really appreciate it. This articles means that If we have custom annotations we could easily split complex logic and have a clean, understandable and manageable(most important factor) code against a small overhead of learning it.
If you could explain process of fetching annotation's data from bean to our data structure, would be of great help.
Sujit Pal3/09/2008 10:10 AM
Hi Hemant, the annotations are extracted from the bean properties to the map in buildIndexingMetaData(), all I do is to start with the class passed in and climb the inheritance tree until I reach Object, and pull the annotations into a map keyed by field name, which I can later refer to when building the index.
Satish12/01/2010 4:54 AM
Excellent article!

I understand annotations but didn't find a need to create custom annotation in any of my projects. This article showed a very good usage of custom annotations -- I use both Lucene and ORM framework (JPA) so I was able to relate instantly.

Another place where custom annotations can be helpful is in selecting certain methods to apply aspects (around advice for example): create custom annotation and define a point cut to select only those methods that have the custom annotation
Sujit Pal12/04/2010 3:02 PM
Thanks Satish, and yes, custom annotations can be very useful for selectively applying aspects, as you pointed out. We have used a similar approach of annotating our controllers to "advise" our ad categorizer on "what kind" of page it is looking at.

Comments are moderated to prevent spam.

Saturday, March 01, 2008

My first Custom Annotation

4 comments: