Sunday, January 31, 2010

Java and mocking frameworks

Frameworks such as Mockito can be extremely helpful when writing tests for your code. After having compared Mockito with EasyMock, it became very obvious that EasyMocks approach to mocking is just badly designed. The normal flow in Mockito is
  1. setup mocks
  2. stub the mocks to return useful values, in order to make your real code work.
  3. exercise your code using the mocks
  4. verify that the mocks have been called in appropiate ways
This is just the general approach, Mockito allows you to do more complex flows, such as setup, stub, exercise, verify, exercise some more, verify some more.
This design feels correct since the four parts of mocking are orthogonal.

EasyMock has quite a different approach:
  1. setup mocks and set global options for the mocks (verify in order or not, should the mocks be strict or nice)
  2. write expectations, which is a combination of verification and stubbing.
  3. exercise the code
  4. apply all predefined expectations for validation
This approach makes it harder to customize your tests, for situations when you only want to test ordering for parts of the methods, or only want some method calls to be strict or nice.
Strict and nice aren't even concepts in Mockito, since strictness is completely a consequence of writing your verification.

There are many more advantages to Mockito, such as easy to use API and very clean and helpful error messages and stacktraces.

Out of curiousity, I have tried writing my own mocking framework with Mockitos usage model in mind, but with other goals as well. I wanted my mocking framework to have an extremely clean design, even sacrificing some real world uses, which makes it less useful for testing legacy code, but probably equally useful for a modern test driven codebase.

I actively chose to not support mocking classes, just interfaces. There are two reasons for this:
all of your complex injected dependencies should be interfaces, not classes, so it's not a big downside for proper code. The second reason is that I wanted to avoid dependencies on bytecode manipulation libraries such as cglib or asm. I only use the Proxy class from java.lang.reflect instead.

It turns out that some aspects of mocking is more straight forward in my mocking framework than Mockito.
This is taken from the Mockito documentation: "If you are using argument matchers, all arguments have to be provided by matchers. "

My approach allows you to mix freely with one expection - you can't use both argument matchers and plain nulls or some magic numbers (like -107 for byte values, et.c.)

A second difference is this:

//Impossible: real method is called so spy.get(0) throws IndexOutOfBoundsException (the list is yet empty)

The equivalent code in my framework works just fine:

Mockachino.stubReturn(spy, "foo").get(0);

I have put up the source code on google code for all to see. It might survive to become a stable and usable codebase, or it may not. We'll see how long it can keep my interest. Things generally become less fun when all the big problems have been solved.

I almost forgot, here is the link: