Saturday, December 09, 2006

Mock objects for Javamail Unit tests

I recently got to use Javamail for the very first time. Javamail provides the Java application developer with a convenient abstraction to send and receive email. My application would use email as an asynchronous remote invocation mechanism. It would need to read some mail from a specified IMAP mailbox, parse and process it, then send a confirmation email that the requested operation succeeded (or failed). However, I noticed that applications of this nature pose certain problems during development.

  • You need to be always connected to the SMTP, IMAP or POP servers that you are developing your application to talk to. This is not a tall order most of the time, except when you are developing code at a location outside the company firewall, or not connected to the Internet at all.
  • You may be working with database dumps of real email addresses, so you may end up inadverdently sending mail to real people during development, something that usually looks bad for you and your company. You can usually get around this by post-processing the email addresses to fake ones in the dev server, or by restricting the servers to within a certain network.

Both problems can be addressed by a mock framework that would allow Javamail to "send" messages to an in-memory data structure which can be queried by other Javamail components as they "receive" messages from it. That way, you can develop and test the code in offline mode, and also write round-trip integration tests without actually connecting to any real SMTP or IMAP servers. I searched around for something like this for a while (see resources), but could not find one that would meet all my requirements, so I decided to build one, which is described here.

Extension point

Central to Javamail is the Session object. It is a factory that returns an implementation of Transport (for SMTP), or Store and Folder (for IMAP and POP). It is final, so subclassing is not a viable approach to mocking it. What we can do is override the implementations it returns by specifying property file overrides. The property file overrides should be located in a META-INF directory in your classpath. Since mocks are only used during testing, I decided to locate them under the src/test/resources/META-INF directory of my Maven app.

1
2
3
4
5
6
7
8
# src/test/resources/META-INF/javamail.default.providers
# Specifies the mock implementations that would be returned by Session
protocol=smtp; type=transport; class=com.mycompany.smail.mocks.MockTransport; vendor=Smail, Inc.;
protocol=imap; type=store; class=com.mycompany.smail.mocks.MockImapStore; vendor=Smail, Inc.;

# src/test/resources/META-INF/javamail.default.address.map
# RFC-822 docs need to use the SMTP protocol
rfc822=smtp

Since the src/test/resources directory is in the test classpath, the following calls will now return our MockTransport and MockImapStore instead of the default Javamail implementations when we invoke the following calls from JUnit tests.

1
2
3
4
5
6
  // SMTP
  Transport transport = Session.getTransport("smtp"); // returns MockTransport

  // IMAP
  Store store = Session.getStore("imap");     // returns MockImapStore
  Folder folder = store.getFolder("INBOX");   // returns the only MockFolder

The Mock Message Store

For the mock message store which the MockTransport would write to and the MockImapStore would read from, I envisioned a Map of email address to List of MimeMessages. The email address would be the owner of the mailbox when reading (or the recipient address when writing a MimeMessage object). It would be a singleton with static methods which would be called by the MockTransport and MockImapStore and would have some dump methods to allow the developer to dump the object within the JUnit test. It is shown below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
// Messages.java
public class Messages {

  public static Map<String,List<MimeMessage>> messages = new HashMap<String,List<MimeMessage>>();

  public static void addMessage(String toEmail, MimeMessage message) {
    List<MimeMessage> messagesForUser = messages.get(toEmail);
    if (messagesForUser == null) {
      messagesForUser = new ArrayList<MimeMessage>();
    }
    messagesForUser.add(message);
    messages.put(toEmail, messagesForUser);
  }

  public static List<MimeMessage> getMessages(String toEmail) {
    List<MimeMessage> messagesForUser = messages.get(toEmail);
    if (messagesForUser == null) {
      return new ArrayList<MimeMessage>();
    } else {
      return messagesForUser;
    }
  }

  public static void reset() throws Exception {
    messages.clear();
  }

  /**
   * Dumps the contents of the Messages data structure for the current run.
   * @return the string representation of the Messages structure.
   * @throws Exception if one is thrown.
   */
  public static String dumpAllMailboxes() throws Exception {
    StringBuilder builder = new StringBuilder();
    builder.append("{\n");
    for (String email : messages.keySet()) {
      builder.append(dumpMailbox(email)).append(",\n");
    }
    builder.append("}\n");
    return builder.toString();
  }

  /**
   * Dumps the contents of a single Mailbox.
   * @param ownerEmail the owner of the mailbox.
   * @return the string representation of the Mailbox.
   * @throws Exception if one is thrown.
   */
  public static String dumpMailbox(String ownerEmail) throws Exception {
    StringBuilder mailboxBuilder = new StringBuilder();
    List<MimeMessage> messagesForThisUser = messages.get(ownerEmail);
    mailboxBuilder.append(ownerEmail).append(":[\n");
    for (MimeMessage message : messagesForThisUser) {
      mailboxBuilder.append(stringifyMimeMessage(message));
    }
    mailboxBuilder.append("],\n");
    return mailboxBuilder.toString();
  }

  /**
   * Custom stringification method for a given MimeMessage object. This is
   * incomplete, more details can be added, but this is all I needed.
   * @param message the MimeMessage to stringify.
   * @return the stringified MimeMessage.
   */
  public static String stringifyMimeMessage(MimeMessage message) throws Exception {
    StringBuilder messageBuilder = new StringBuilder();
    messageBuilder.append("From:").append(message.getFrom()[0].toString()).append("\n");
    messageBuilder.append("To:").append(message.getRecipients(RecipientType.TO)[0].toString()).append("\n");
    for (Enumeration<Header> e = message.getAllHeaders(); e.hasMoreElements();) {
      Header header = e.nextElement();
      messageBuilder.append("Header:").append(header.getName()).append("=").append(header.getValue()).append("\n");
    }
    messageBuilder.append("Subject:").append(message.getSubject()).append("\n");    messageBuilder.append(message.getContent() == null ? "No content" : message.getContent().toString());
    return messageBuilder.toString();
  }
}

Mocking SMTP

Only Transport needs to be mocked for SMTP. This implementation is almost totally copied from Bill Dudney's blog entry (referenced in resources), replacing System.out.println() calls with LOGGER.debug() calls.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
// MockTransport.java
public class MockTransport extends Transport {

  private static final Logger LOGGER = Logger.getLogger(MockTransport.class);

  public MockTransport(Session session, URLName urlName) {
    super(session, urlName);
  }

  @Override
  public void connect() throws MessagingException {
    LOGGER.info("Connecting to MockTransport:connect()");
  }

  @Override
  public void connect(String host, int port, String username, String password) throws MessagingException {
    LOGGER.info("Connecting to MockTransport:connect(String " + host + ", int " + port + ", String " + username + ", String " + password + ")");
  }

  @Override
  public void connect(String host, String username, String password) throws MessagingException {
    LOGGER.info("Connecting to MockTransport:connect(String " + host + ", String " + username + ", String " + password + ")");
  }

  @Override
  public void sendMessage(Message message, Address[] addresses) throws MessagingException {
    System.err.println("Sending message '" + message.getSubject() + "'");
    for (Address address : addresses) {
      Messages.addMessage(address.toString(), (MimeMessage) message);
    }
  }

  @Override
  public void close() {
    LOGGER.info("Closing MockTransport:close()");
  }
}

Mocking IMAP

For IMAP, we need to provide mock implementations for both Store and Folder. Most of the methods in my case are unsupported, but those that are work against the Messages object. Here they are:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
// MockImapStore.java
public class MockImapStore extends Store {

  private static final Logger LOGGER = Logger.getLogger(MockImapStore.class);

  private String ownerEmail;
  private MockFolder folder;

  public MockImapStore(Session session, URLName urlName) {
    super(session, urlName);
  }

  public String getOwnerEmail() {
    return ownerEmail;
  }

  @Override
  public void connect(String host, int port, String username, String password) {    this.ownerEmail = buildOwnerEmail(host, username);
    this.folder = new MockFolder(this);
    LOGGER.debug("MockImapStore:connect(String " + host + ", int " + port + ", String " + username + ", String " + password + ")");
  }

  @Override
  public Folder getFolder(String folderName) throws MessagingException {
    return getDefaultFolder();
  }

  @Override
  public Folder getDefaultFolder() throws MessagingException {
    return folder;
  }

  @Override
  public Folder getFolder(URLName urlName) throws MessagingException {
    return getDefaultFolder();
  }

  @Override
  public void close() {
    LOGGER.info("MockImapStore.close()");
  }

  /**
   * Converts user at mail.host.com to user@host.com
   * @param host the hostname of the mail server.
   * @param username the username that is used to connect.
   * @return the email address.
   */
  private String buildOwnerEmail(String host, String username) {
    return StringUtils.join(new String[] {
      username,
      StringUtils.join(ArrayUtils.subarray(host.split("\\."), 1, 3), ".")}, "@");
  }

}

// MockFolder.java
public class MockFolder extends Folder {

  private static final Logger LOGGER = Logger.getLogger(MockFolder.class);

  private Store store;
  private List<MimeMessage> messagesInFolder;

  public MockFolder(Store store) {
    super(store);
    this.store = store;
  }

  @Override
  public void open(int mode) throws MessagingException {
    String owner = ((MockImapStore) store).getOwnerEmail();
    LOGGER.debug("MockFolder.open(int " + mode + "), owner=" + owner);
    this.messagesInFolder = Messages.getMessages(owner);
  }

  @Override
  public Message[] getMessages() throws MessagingException {
    return messagesInFolder.toArray(new Message[0]);
  }

  @Override
  public Message[] expunge() throws MessagingException {
    return new Message[0];
  }

  @Override
  public void close(boolean expunge) throws MessagingException {
    LOGGER.debug("MockFolder.close(boolean " + expunge + ")");
  }

  @Override
  public Message getMessage(int index) throws MessagingException {
    try {
      return messagesInFolder.get(index);
    } catch (ArrayIndexOutOfBoundsException e) {
      throw new MessagingException(e.getMessage());
    }
  }

  @Override
  public int getMessageCount() throws MessagingException {
    return messagesInFolder.size();
  }

  @Override
  public int getType() throws MessagingException {
    return Folder.HOLDS_MESSAGES;
  }

  @Override
  public boolean hasNewMessages() throws MessagingException {
    return (messagesInFolder.size() > 0);
  }

  @Override
  public boolean isOpen() {
    return (((MockImapStore) getStore()).getOwnerEmail() != null);
  }

  @Override
  public Folder[] list(String arg0) throws MessagingException {
    return new Folder[] {this};
  }

  @Override
  public void appendMessages(Message[] messages) throws MessagingException {
    this.messagesInFolder.addAll(Arrays.asList((MimeMessage[]) messages));
  }

  @Override
  public boolean exists() throws MessagingException {
    return true;
  }

  @Override
  public Folder getFolder(String folderName) throws MessagingException {
    return this;
  }

  @Override
  public String getFullName() {
    return "INBOX";
  }

  @Override
  public String getName() {
    return "INBOX";
  }

  @Override
  public boolean create(int type) throws MessagingException {
    throw new UnsupportedOperationException("MockFolder.create(int) not supported");
  }

  @Override
  public boolean delete(boolean recurse) throws MessagingException {
    throw new UnsupportedOperationException("MockFolder.delete(boolean) not supported");
  }

  @Override
  public Folder getParent() throws MessagingException {
    throw new UnsupportedOperationException("MockFolder.getParent() not supported");
  }

  @Override
  public Flags getPermanentFlags() {
    throw new UnsupportedOperationException("MockFolder.getPermanentFlags() not supported");
  }

  @Override
  public char getSeparator() throws MessagingException {
    throw new UnsupportedOperationException("MockFolder.getSeparator() not supported");
  }

  @Override
  public boolean renameTo(Folder newFolder) throws MessagingException {
    throw new UnsupportedOperationException("MockFolder.renameTo() not supported");
  }

}

Calling code

Once the mock objects are in place, the calling code will run unchanged against the mock objects as long as the property override properties files are visible in our classpath. In our case, our JUnit test code (under src/test/java) will automatically use the mock objects, while our production code (under src/main/java) will use the Sun implementations to connect to the real servers.

Resources

  • jGuru's Java Mail Tutorial on the Sun Developer Network - A very quick but comprehensive overview of the Javamail API. Takes about 15 minutes or so to read and contains code snippets you can use to quickstart your Javamail based app.
  • Bill Dudney's "Mocking Javamail" blog entry - this actually got me started on my own mock objects for Javamail unit testing. However, my needs went beyond just sending mail, so I had a little more work to do. However, my Mock Transport implementation (described above) started out as a direct cut-and-paste from the code shown in this entry.
  • Dumbster is a fake SMTP server, which stores messages sent to it in an in-memory data structure similar to the mock implementation described here. I considered using this for a while, but I needed something that would work with Javamail and have support for a mock IMAP store. Dumbster, as far as I know, does not work with Javamail, and it definitely does not have support for POP or IMAP.

8 comments (moderated to prevent spam):

Robert Hanson said...

Thanks, that saved me a bit of time.

Sujit Pal said...

Thanks for the feedback, Robert, glad it helped.

BigMikeW said...

Nice article.

I'm having an issue with this approach though since the only way I can override the SMTP provider is to place javamail.providers in the jre/lib. This isn't really going to work for me for obvious reasons. When I place a javamail.default.providers in the META-INF it gets loaded but the SMTP provider has already been set up with the Sun default so it is ignored. For some reason the javamail.providers in META-INF is ignored: DEBUG: not loading resource: /META-INF/javamail.providers

Any idea if I can get around this without having to place a javamail.providers in the jre/lib?

BigMikeW said...

I stumbled on this yesterday, definitely the easy option: https://mock-javamail.dev.java.net/

Sujit Pal said...

Hi BigMikeW, it looks like the mock-javamail solution is very similar in intent to the one I have here. However, the usage pattern is different, and I agree with you that its less of a hassle -- simply point to the mock server provided by mock-javamail for your unit tests, and let the code connect to a real mail server through javamail when deploying. With my approach you would have to update the default javamail properties, which is also not a hassle, unless you run into classpath issues, and then, of course, it can be quite a nightmare. BTW, src/main/resources corresponds to the root of the deployed jar file.

Nicolas said...

mock-javamail download links are dead to me, i've downloaded them from http://grepcode.com/snapshot/repo1.maven.org/maven2/org.jvnet.mock-javamail/mock-javamail/1.7

Anonymous said...

Super post

Sujit Pal said...

Thank you!