• Click: Run -> Edit Configuration
• Select your test configuration (or add a new one)
• In VM parameters, add
  -Dlog4j.configuration=file:/C:/dev/config/log4j.xml
  (Or wherever your log4j properties file is)

That’s it. The test should now run with log4j logging (although obviously you need to have the necessary log4j jars available).
This example uses a windows file format, but will work equally well with *nix.

I use this approach for turning on MyBatis logging for individual tests. See here for a basic setup of a log4j.xml file to enable logging with MyBatis.

Unit Testing

The benefits of unit testing code are many and include allowing us to find problems early, enable refactoring with confidence, and even provide a form of documentation for our code. Writing code with testing in mind (or, in the TDD world, to make already written tests pass) helps us to write code that is modular, with low coupling and high cohesion.

Unit testing is a way to verify that a unit of code works as we expect and want. In Java, that unit is often centered around a method. Sometimes a small part of one method (such as a particularly if block); sometimes several methods (for example, in the case of the public method under test calling one or more private methods). However, what a unit test should not do is test outside its own class boundary since this gets in to the realm of integration testing.

Using Mock Objects

Inevitably however, your class will rely on the behavior of other classes. So, the issue then becomes how to deal with those other classes. We somehow need to isolate, or control, their behavior in order to be able to focus on the unit under test. (We will either test that other code separately or, in the case of 3rd party libraries, assume that it works well already.)

The solution to isolating your unit under test from the other collaborating classes is to use mock objects in place of those collaborators. Mock objects allow us to focus on just the code under test by mocking the collaborator’s behavior. We specify expectations for how our code will interact (methods calls and return values), control those interactions and confirm that everything in our unit test code works as expected. Mock objects are, in essence, a way to test our code in isolation from the rest of our software world.

Fortunately, in the Java world, we have several mock object frameworks available, including JMock, Mockito and EasyMock. There are some interesting discussions on the relative merits of each here and here. This article focuses on the functionality of EasyMock.

EasyMock

EasyMock uses a record/playback paradigm for unit testing with mock objects. In record mode, you record what methods you expect to be called on your mock objects (and specify how you want your mock to respond). In playback mode, your mocks wait for those expected calls, playing back the canned responses you have specified. EasyMock will alert you to any unexpected behavior, such as missing calls and incorrect argument or return values and can also alert you to other behavior such as unexpected method calls.

These are the basic steps for using an EasyMock mock object:

1. Create a Mock Object




2. Record the expected behavior




3. Switch to replay state




4. Call the methods under test




5. Verify the mock was used as expected

1. Creating Mocks

Creating a mock is simple. Simply call createMock and pass in the class of the interface to be mocked, For example:

    MyInterface mock = org.easymock.EasyMock.createMock(MyInterface.class);

Or better yet, statically import the EasyMock static methods in you class import statements. For example:

    import static org.easymock.EasyMock.*;
    …
    MyInterface mock = createMock(MyInterface.class);

(Note that this static import approach applies to pretty much all of the EasyMock methods discussed in this article).

That’s it! Mock created. I have included some details on more customized mocks below, but in most cases, you can simply skip to the next step: specifying the expected behavior.

Mock creation specifics

Using createMock creates a mock object that has 2 notable characteristics:

1. Order checking is disabled by default. That is, you can specify the method call expectations (more later) in any order you wish – they do not need to match the actual order of the method invocations at test run time.
2. Calls to unexpected methods cause the test to fail

However, EasyMock provides two other approaches for creating mocks…

Strict/Nice mocks

Strict Mocks

A strict Mock Object has order checking enabled by default (and calls to unexpected methods still cause the test to fail).

    MyInterface mock = org.easymock.EasyMock.createStrictMock(MyInterface.class);

Since method order invocation is important under normal use, I often create my mocks as ‘strict’ by default.

Nice Mocks

Like a regular mock, order checking is disabled by default but ‘nice’ mocks will also allow calls to unexpected methods to pass unnoticed. Nice mocks also have another useful behavior: They will do their best to create a return value for unexpected method invocations. Specifically, they will return 0 (for methods with a numeric return type), false (for methods with a boolean return type) or null (for all other methods).

I find nice mocks particular use for testing legacy classes (or classes that were perhaps not created with unit testing in mind) that may interact with a large number of other classes which my current tests have no concern with.

Partial Mocks

EasyMock also provides the capability to mock specific methods of a class while not mocking (i.e. keeping the real behavior) of others. I have not personally used this functionality yet, but it is available via the createMockBuilder method, which returns a IMockBuilderinterface.

Mocking concrete classes

The final point on mock creation is that although EasyMock (and indeed the use of mock objects in general) is most often associated with mocks of interfaces, you can in fact also create mocks of concrete class also. You simply use

    org.easymock.classextension.EasyMock.createMock(...)

instead of the more conventional

    org.easymock.EasyMock.createMock(...);

e.g.

    ConcreteClass mock = org.easymock.classextension.EasyMock.createMock(ConcreteClass.class);

2. Record the expected behavior for your mock

After creating our mock object, we next need to tell EasyMock how we expect that mock to be used, that is, what methods we expect to be called on it.

2.1 Simplest case

In the simplest case where the expected method call has no return values (i.e. void) and accepts no arguments, you simply just call the method directly. For example:

    mock.expectedOperation() 

EasyMock still picks up & records the expectation.

Note that if you need to apply other expectations (such as expected Exceptions, or number of invocations as discussed later) to these simple cases, you can do so by invoking expectLastCall() followed by your expectations.

In addition to these simple calls however, EasyMock also provides the capabilities to specify return types and argument values, where required…

2.2 Specifying return values

If the call to expectedOperation returns a value, you must specify what EasyMock will return to your calling code. This can be done using the andReturn() method (a method provided by the return object of EasyMock.expect(), IExpectationSetters).

For example:

    EasyMock.expect(mock.expectedOperation()).andReturn(expectedReturnValue);

Where “expectedReturnValue” can be any object or primitive that matches expectedOperation’s return type.

Gotcha:

If the expected operation does indeed return a value, and you do not specify
e.g. EasyMock.expect(mock.expectedOperation());
You will get an error like:

    java.lang.IllegalStateException: missing behavior definition for the preceding method call expectedOperation()

Note that you can also get this error if you forget to call replay() before using a mock – more below.

Advanced return values

If you need more advanced return type creation, e.g. creating a value (or an Exception) at runtime, you can use andAnswer(IAnswer answer). See IAnswer for more details.

2.3 Specifying arg values

In the simplest approach, to specify an argument value, you just pass it with the expected method call, as in:

    mock.expectedOperation(argValue)

or

    EasyMock.expect(mock.expectedOperation(argValue)).andReturn(expectedReturnValue);

EasyMock simply compares the actual method arguments and the expected arguments by using equals().

Flexible Argument Matchers

It is also possible to have EasyMock adopt a more flexible approach to matching actual and expected arguments. For example, you may be happy to just have

• any Object passed
• A String starting with a certain value
• A number in a certain range

Easyock support these, and more, using argument matchers such as

• anyObject()
• startsWith(String prefix)
• eq(X value, X delta)

For example:

    expect(mock.expectedOpreation(EasyMock.anyInt()));

See the EasyMock class API for more details (or the EasyMock readme).

2.4 Specifying how many calls are expected

EasyMock also provides the capability to either relax or tighten the rules regarding the number of calls expected for a method.

Specifying an exact number of calls

To specify that a method must be called a certain number of times, we can just repeat the expected call the required number of time, e.g.

    EasyMock.expect(mock.expectedOperation()).andReturn(expectedReturnValue);
    EasyMock.expect(mock.expectedOperation()).andReturn(expectedReturnValue);

But this can get tedious for large numbers of calls, so we can also do:

EasyMock.expect(mock.expectedOperation()).times(2).andReturn(expectedReturnValue);

Specifying an inexact number of calls

There are a number of other methods (again, all provided by the return object of EasyMock.expect()) allowing us to specify looser rules on the number of expected method calls. These are:

• anyTimes()
• atLeastOnce()

3. Switch to replay state

Switching a mock object to replay state is simply done by calling EasyMock.replay(), passing in the mock object.
e.g.

    EasyMock.replay(mock).

This should be done for all mocks you have created (or, alternatively, call EasyMock.replayAll())
If you forget to call replay() before using a mock you will get this error:

    java.lang.IllegalStateException: missing behavior definition for the preceding method call expectedOperation()

Calling replay tells EasyMock that you are no longer specifying expected method calls and moving into replay mode.

4. Call the methods under test

This part is not really EasyMock specific since this is the crux of your test and needs to be done even if you are not using mock objects.
e.g. if you are unit testing a method of a Service object, at some point you are going to have to call that method in order to test its effect.

5. Verify the mock was used as expected

The final step is where we ask EasyMock to confirm that all our expectations were met. That is, we ask EasyMock to confirm that a method call corresponding to each expectation we set is actually made. This can be done as either

EasyMock.verify(mock)

or

EasyMock.verifyAll()

Note that this step is not mandatory, but if it is omitted, there will be no verification performed that the expected behavior we specified actually took place. So for example, if you specify

    mock.expectedOperation() 

and expectedOperation is never called, the tests will still pass.

If verify() is called, exactly what checks are performed depends on the type of mock object we created in step 1.

Note that I personally find the exact behavior of verify to be one more confusing parts of EasyMock. The following is my understanding of it and you should consult the EasyMock documentation (and run some tests yourself) before relying on it!

• EasyMock.createNiceMock() – As discussed earlier, unexpected method calls are not flagged. Calling verify() on a nice mock simply checks that all your expected methods were called, in any order.
• EasyMock.createMock() – On a ‘normal’ mock, unexpected method calls will be be flagged (they would result in AssertionError: Unexpected method call) even without the call to verify. I think that the call to verify acts the same way as for a ‘nice’ mock’. That is, EasyMock simply checks that all your expected methods were called, in any order.

  [If anyone out there knows of a difference in the effect of calling verify() between a nick mock and a normal mock, I would like to know!]

• EasyMock.createStrictMock() – Again, unexpected method calls will be flagged, and the order of the expected method calls is also checked but this time, the order of the expected methods is also checked.

Again, with the same disclaimer as before of this being my understanding only, I have tried to summarize the above behavior in the following table:

References

EasyMock Documentation
Easier testing with EasyMock

Links

EasyMock Facts and Fallacies

    String sql = "select a,b,c from tableA";
    assertThat(sql, hasNumberOfSubstrings(",", 2));

See the source code below. I have been reading up on OSS licenses recently and decided to release this using the same license as Hamcrest – the new BSD license.

I have also attached a jar which includes the associated unit tests, although you will need the hamcrest-unit-test project to compile, which can be downloaded as part of the hamcrest all-in-one jar.

package com.shaunabram.hamcrest;

import java.util.regex.Pattern;
import org.hamcrest.*;

/**
 * Matcher which tests if a string contains a specified number of substrings.
 *
 * @author sabram
 */
public class HasNumberOfSubstrings extends TypeSafeMatcher<String> {

    private final String substring;
    private final int count;

    public HasNumberOfSubstrings(String substring, int count) {
        this.substring = substring;
        this.count = count;
    }

    @Override
    protected boolean matchesSafely(String item) {
        Pattern p = Pattern.compile(substring);
        java.util.regex.Matcher m = p.matcher(item);
        int actualCount = 0;
        while (m.find()) {
            actualCount++;
        }
        if (actualCount == count) return true;
        else return false;
    }

    public void describeTo(Description description) {
        description.appendText("String containing ")
            .appendText(count + " occurences of ")
            .appendText(substring);

    }

    @Override
    public void describeMismatchSafely(String item, Description mismatchDescription) {
      mismatchDescription.appendValue(count)
                         .appendText(" occurence(s) of ")
                         .appendValue(substring)
                         .appendText(" in ")
                         .appendText(item);
    }

    @Factory
    public static Matcher<String> hasNumberOfSubstrings(String substring, int count) {
        return new HasNumberOfSubstrings(substring, count);
    }

}

The talk was given by Chris Bedford, from Build Lackey Labs – “Automating the Monkey Work Out and the Quality In!”. Overall, I thought this was a great talk by Chris. He clearly has a huge amount of experience creating automated tests and integrating them with build tools and he gave a well structured, interesting, well delivered presentation. I have posted a copy of Chris’s slides and I think the video will be posted on the SF JUG site at some point, but I have also posted my notes from the presentation below…

Selenium Overview

Chris started off with an overview of Selenium – a tool for automating tests for web based applications. He then introduced us to the Selenium IDE – a Firefox add-on that allows you to set up tests by recording clicks on a web page, or by manually entering commands. The tests can then be played back as part of your automated test suite.

Creating Selenium Tests

Chris showed us the process by both executing an existing test he had already setup, and by walking us through the creation of a new test, which loaded a page, waited for text to be present, clicked on a link, and confirmed that text was present. The test was a html file, but he showed how to export it as TestNG and JUnit tests (although both exports resulted in kind of ugly java code!)

Possible hiccups with Selenium

One of the things I thought Chris did well was, in addition to pointing out all the many great things about Selenium, he also pointed out some of the slight issues with Selenium that he has encountered during his time using it, such as

• Difficulties getting the initial Open page to work
• Issues with where the test files get saved
• Making sure you have the tests run slow to start with (to avoid timing/loading issues)
• The Selenium IDE may be difficult to get working with IE (Firefox is fine)

Selenium Components

The main Selenium section then finished up with a quick over of the various Selenium Components:

• Selenium IDE
• Selenese Commands
• Selenium RC (Remote Control)

Chris also had a section on the ‘same origin‘ JavaScript policy that Selenium manages to circumvent by using the Selenium RC Server. He managed to explain the issue in simple terms.

Other functional test alternatives

Chris discussed other functional test alternatives by comparing Canoo to Selenium (Canoo is also an automated web app testing framework).

Chris also mentionted Umangite, Chris Richardson’s open source testing framework that makes Selenium tests easier to write. I have used this before, and agree it is worth checking out…

Using Selenium with Build Tools

He then discussed using Maven with Selenium, including using Cargo (e.g. using the cargo maven2 plugin). Cargo is a set of APIs that assists in

• installing web containers, such as Tomcat, JBoss
• booting and shutting them down
• deploying web apps (.wars, .ears)

Chris finished the Selenium & build tools section with a quick run through of using Selenium with Ant.

Continuous Integration

The final section of the presentation was a talk on Continuous Integration (CI). Chris described CI as:

• A dedicated box (see my follow up question below!) that runs regular full builds (including tests) of your software
• Build triggers when any developer checks into SCM
• Team is notified of any issues

And talked about how Selenium can be used with Hudson – a very cool CI tool from Sun. This leads well into next month’s SF JUG talk!

Q&A

Q: Do you have any info on WebDriver merging with Selenium?
A: No, but sounds interesting!

Q: How closely do test writers work with marketing people?
A: Chris suggested checking out the easyb framework, a behavior driven development framework that allows you to write conditions in a very abstract level, using a DSL that marketers can use and understand. He also suggested looking at the fitnesse framework.

Q: I asked Chris via email after the talk what he meant by needing ‘dedicated’ box/server for Continuous Integration.
A: He clarified that by ‘dedicated’, he just meant that the build box should not be a box that you are doing development on, and that there isn’t anything wrong with having a mix of services on one machine. e.g. having a qa box that has the CI server and maybe hosts the bug tracking system as well…