String sql = "select a,b,c from tableA";
    assertThat(sql, hasNumberOfSubstrings(",", 2));

See the source code below. I have been reading up on OSS licenses recently and decided to release this using the same license as Hamcrest – the new BSD license.

I have also attached a jar which includes the associated unit tests, although you will need the hamcrest-unit-test project to compile, which can be downloaded as part of the hamcrest all-in-one jar.

package com.shaunabram.hamcrest;

import java.util.regex.Pattern;
import org.hamcrest.*;

/**
 * Matcher which tests if a string contains a specified number of substrings.
 *
 * @author sabram
 */
public class HasNumberOfSubstrings extends TypeSafeMatcher<String> {

    private final String substring;
    private final int count;

    public HasNumberOfSubstrings(String substring, int count) {
        this.substring = substring;
        this.count = count;
    }

    @Override
    protected boolean matchesSafely(String item) {
        Pattern p = Pattern.compile(substring);
        java.util.regex.Matcher m = p.matcher(item);
        int actualCount = 0;
        while (m.find()) {
            actualCount++;
        }
        if (actualCount == count) return true;
        else return false;
    }

    public void describeTo(Description description) {
        description.appendText("String containing ")
            .appendText(count + " occurences of ")
            .appendText(substring);

    }

    @Override
    public void describeMismatchSafely(String item, Description mismatchDescription) {
      mismatchDescription.appendValue(count)
                         .appendText(" occurence(s) of ")
                         .appendValue(substring)
                         .appendText(" in ")
                         .appendText(item);
    }

    @Factory
    public static Matcher<String> hasNumberOfSubstrings(String substring, int count) {
        return new HasNumberOfSubstrings(substring, count);
    }

}

    assertTrue(a.equalTo(b));

becomes

    assertThat(a, equalTo(b));

A small change in this example, but Hamcrest’s benefits are many, enabling you to write much more flexible tests that are easier to read and have more meaningful failure messages.

    Benefits of using Hamcrest
        Improved readability of tests
        Better failure messages
        Combine Matchers
        Custom Matchers
    Limitations
        Don’t use if JUnit will do
        Finding the right Matcher
    Setup
    Trouble Shooting
    Links

Benefits of using Hamcrest

Improved readability of tests

The following code, which utilizes the Hamcrest API, is easy to understand, even if you have no understanding of Hamcrest, or Java for that matter:

    assertThat(car, is(equalTo(ferrari));
    assertThat(cars, hasItems(ferarri, porsche));

Basically, Hamcrest provides a fluent API.

Better failure messages

This is one of the biggest advantages of using Hamcrest.
For example, if you run the following test, which makes use of JUnit’s assertTrue method:

    @Test
    public void testGreatThan_JUnit() {
        int minimumPrice = 0;
        int actualPrice = -1;
        //using org.junit.Assert.assertTrue
        assertTrue(actualPrice > minimumPrice);
    }

You get the following terse and uninformative error:

java.lang.AssertionError

However if you rewrite the test to use Hamcrest’s assertThat method with the greaterThan matcher, as follows:

    @Test
    public void testGreaterThan_Hamcrest() {
        int minimumPrice = 0;
        int actualPrice = -1;
        //using org.hamcrest.MatcherAssert.assertThat
        //and  org.hamcrest.Matchers.greaterThan
        assertThat(actualPrice, greaterThan(minimumPrice));
    }

You would get the following much more informative error message:

java.lang.AssertionError:
Expected: a value greater than <0>
but: <-1> was less than <0>

Basically, assertThat will tell you what was expected and what the value actually was where as assertTrue will only tell you that you got false where you expected true.
(You can of course put an explicit error message in the JUnit assertion yourself of course, as in:
assertTrue(“Expected a value greater than ” + minimumPrice, actualPrice > minimumPrice);
But Hamcrest is more descriptive and avoids the extra effort.)

Combinations of Matchers

Hamcrest allows you to use weird and wonderful combinations of Matchers. The obvious example is using not(), but other combinations can be achieved using Matchers such as both(), either(), or(), anyOf(), and many more.

Create your own Matchers

You can also create your own custom Matchers too, by implementing the Matcher interface. I wrote a blog post with an example of a custom matcher, but you can also find good examples here and here.


Finally, as a wrap up to the benefits of using Hamcrest, see the JUnit release notes which includes a description of using the Matcher mechanism.

Limitations

Don’t use if JUnit will do

If there is already a suitable JUnit method to use, there may not be any benefit to using a Hamcrest Matcher. Specifically, if you are just comparing primitive or String values, JUnit’s overloaded assertEquals methods should be adequate. For example, the follow two test approaches are both fairly easy to read and give the same message on test failure:

	@Test
    	public void testEquals_JUnit() {
	    int expectedPrice = 0;
	    int actualPrice = -1;
	    //using org.junit.Assert.assertEquals
	    assertEquals(expectedPrice, actualPrice);
	    //java.lang.AssertionError: expected:<0> but was:<-1>
	}

	@Test
	public void testEquals_Hamcrest() {
	    int expectedPrice = 0;
	    int actualPrice = -1;
	    //using org.hamcrest.MatcherAssert.assertThat
            //and org.hamcrest.Matchers.equalTo
	    assertThat(actualPrice, equalTo(expectedPrice));
	    //java.lang.AssertionError: Expected: <0> but: was <-1>
	}

However, for anything more complicated than primitive or String comparison, Hamcrest’s Matchers are likely to be much more useful.

Finding the right Matcher

One of the issues I have with Hamcrest is that the matchers are not all available in one place. It makes finding the correct matcher difficult and also means your IDE’s code assist, or import organizer, is not likely to be much help either. Most matchers are accessible via the Matchers class (although confusingly there is also the CoreMatchers class, which seems to be a subset – I never quite figured that one out…). But other matchers are located in other classes and packages.

Example 1
The very useful hasItem matcher is available in the aforementioned Matchers class:

    import static org.hamcrest.MatcherAssert.assertThat;
    import static org.hamcrest.Matchers.hasItem;

    public class HamcrestExamples {

        List<String> list = new ArrayList<String>();

	@Before
	public void setup() {
		list.add("itemA");
		list.add("itemB");
	}

	@Test
	public void test() {
		assertThat(list, hasItem("itemA"));
	}
}

But if you want to use the very similar hasItems matcher, you need to know (or trawl the javadocs to find out) it is in the IsCollectionContaining class.

    import static org.hamcrest.core.IsCollectionContaining.hasItems;
    ...
        @Test
	public void testHasItems() {
		assertThat(list, hasItems("itemA", "itemB"));
	}

Example 2
There is a useful containsString() method in the Matchers class that returns a Matcher to check is a String contains a sub-String, but if you want to check if a String contains several sub-Strings, in order, you need to know to look at the StringContainsInOrder class.

Notes:

• It looks like the above hasItems example will be remedied in 1.3 where hasItems will be made available via the Matchers class.
• There is a very similar testing library that seems to overcome this problem called Fluent Assertions module from FEST. FEST has all the matchers available via a single static import. (Also, see this interesting comparison with Hamcrest – I see no reason not to use both)

Setup

1. Download the Hamcrest jar(s) from http://code.google.com/p/hamcrest/downloads (using the all-in-one Jar is easiest)
   

   Note that JUnit does ship with some Hamcrest classes included. However, I prefer to use the Hamcrest jars directly because

   • JUnit only includes the core Hamcrest (hamcrest-core) API classes. A much larger library of Matcher implementations is available by also using the hamcrest-library jars. Personally, I simply use the hamcrest-all.jar, which includes all the available Hamcrest classes.
   • The package names in JUnit are different than those in Hamcrest e.g. org.hamcrest.MatcherAssert.assertThat versus org.junit.Assert.assertThat
     I prefer to keep things simply by always referring to the native org.hamcrest packages directly.

   If you do choose to use the Hamcrest jars directly, it is also advisable to use the junit-dep jars. These come without the Hamcrest classes included, avoiding any uncertainty about which Hamcrest classes you are actually using.

2. Add the Hamcrest jar(s) to your classpath or maven dependencies.
3. Statically import Hamcrest’s assertThat construct and the Hamcrest matchers:
   

   import static org.hamcrest.MatcherAssert.assertThat;
   import static org.hamcrest.Matchers.*;

4. Start using Hamrests’s assertThat in your unit tests, instead of JUnit’s assertTrue and assertEquals etc
   e.g. instead of

       assertEquals(expectedValue, actualValue);

   use

       assertThat(actualValue, equalTo(expectedValue));

Perhaps in future postings I will talk about how to use Hamcrest’s Matchers with Mock object frameworks such as EasyMock and JMock.

Trouble Shooting

Error:

java.lang.SecurityException: class “org.hamcrest.Matchers”‘s signer information does not match signer information of other classes in the same package

You simply need to make sure the Hamcrest jar comes before JUnit in your classpath. Alternatively, use junit-dep.

Error:

cannot find symbol method
assertThat(java.util.List<ExampleClass>,org.hamcrest.Matcher<java.lang.Iterable<java.lang.Object>>)

or

cannot find symbol method
assertThat(java.util.List<ExampleClass>,org.hamcrest.Matcher<java.lang.Iterable<? super
java.lang.Object>>)

This error can happen with any iterable (not just Lists, as in the above example) using hasItem or hasItems (and possibly other matchers too). It is a known issue with Hamcrest at the moment and is documented here, along with some fixes. I took the not-pretty-but-works option of declaring the Matcher in a local variable first
e.g.

    assertThat(list, hasItems(item1, item2));

becomes

    Matcher<java.lang.Iterable<ExampleClass>> matcher = hasItems(item1, item2);
    assertThat(list, matcher);

Like I said, not-pretty-but-works.

Links

• Hamcrest home page
• Hamcrest Tutorial
• Hamcrest API docs – At the time of writing, the Hamcrest API docs seem surprisingly hard to come by online and there is an ticket for this on the Hamcrest site. There also seems to be some issues when building the javadocs for yourself:
  In the mean time, I have taken the liberty of posting them on my site here:
  • Hamcrest 1.2 javadocs
  • Hamcrest 1.3 RC2 javadocs

• JUnit API docs
• Fluent Assertions module from FEST

The talk was given by Chris Bedford, from Build Lackey Labs – “Automating the Monkey Work Out and the Quality In!”. Overall, I thought this was a great talk by Chris. He clearly has a huge amount of experience creating automated tests and integrating them with build tools and he gave a well structured, interesting, well delivered presentation. I have posted a copy of Chris’s slides and I think the video will be posted on the SF JUG site at some point, but I have also posted my notes from the presentation below…

Selenium Overview

Chris started off with an overview of Selenium – a tool for automating tests for web based applications. He then introduced us to the Selenium IDE – a Firefox add-on that allows you to set up tests by recording clicks on a web page, or by manually entering commands. The tests can then be played back as part of your automated test suite.

Creating Selenium Tests

Chris showed us the process by both executing an existing test he had already setup, and by walking us through the creation of a new test, which loaded a page, waited for text to be present, clicked on a link, and confirmed that text was present. The test was a html file, but he showed how to export it as TestNG and JUnit tests (although both exports resulted in kind of ugly java code!)

Possible hiccups with Selenium

One of the things I thought Chris did well was, in addition to pointing out all the many great things about Selenium, he also pointed out some of the slight issues with Selenium that he has encountered during his time using it, such as

• Difficulties getting the initial Open page to work
• Issues with where the test files get saved
• Making sure you have the tests run slow to start with (to avoid timing/loading issues)
• The Selenium IDE may be difficult to get working with IE (Firefox is fine)

Selenium Components

The main Selenium section then finished up with a quick over of the various Selenium Components:

• Selenium IDE
• Selenese Commands
• Selenium RC (Remote Control)

Chris also had a section on the ‘same origin‘ JavaScript policy that Selenium manages to circumvent by using the Selenium RC Server. He managed to explain the issue in simple terms.

Other functional test alternatives

Chris discussed other functional test alternatives by comparing Canoo to Selenium (Canoo is also an automated web app testing framework).

Chris also mentionted Umangite, Chris Richardson’s open source testing framework that makes Selenium tests easier to write. I have used this before, and agree it is worth checking out…

Using Selenium with Build Tools

He then discussed using Maven with Selenium, including using Cargo (e.g. using the cargo maven2 plugin). Cargo is a set of APIs that assists in

• installing web containers, such as Tomcat, JBoss
• booting and shutting them down
• deploying web apps (.wars, .ears)

Chris finished the Selenium & build tools section with a quick run through of using Selenium with Ant.

Continuous Integration

The final section of the presentation was a talk on Continuous Integration (CI). Chris described CI as:

• A dedicated box (see my follow up question below!) that runs regular full builds (including tests) of your software
• Build triggers when any developer checks into SCM
• Team is notified of any issues

And talked about how Selenium can be used with Hudson – a very cool CI tool from Sun. This leads well into next month’s SF JUG talk!

Q&A

Q: Do you have any info on WebDriver merging with Selenium?
A: No, but sounds interesting!

Q: How closely do test writers work with marketing people?
A: Chris suggested checking out the easyb framework, a behavior driven development framework that allows you to write conditions in a very abstract level, using a DSL that marketers can use and understand. He also suggested looking at the fitnesse framework.

Q: I asked Chris via email after the talk what he meant by needing ‘dedicated’ box/server for Continuous Integration.
A: He clarified that by ‘dedicated’, he just meant that the build box should not be a box that you are doing development on, and that there isn’t anything wrong with having a mix of services on one machine. e.g. having a qa box that has the CI server and maybe hosts the bug tracking system as well…

Initially, I had my hibernate.cfg.xml file in the following directory

my-app/src/main/java

And when I first tried to run my hibernate unit tests (mvn test), I got this error:

SessionFactory creation failed.org.hibernate.HibernateException: /hibernate.cfg.xml not found

Some checking of the maven docs and some forums revealed the following possible solutions:
1) Copy the hibernate.cfg.xml file to

my-app/target/classes

This works, however I think it is a hack as the target folder is generated rather than being somewhere you should actually manually create files.
2) The next solution I found suggested moving the hibernate.cfg.xml to

my-app/src/main/resources

This works well as the contents of this folder are copied to the base level of the my-app/target/classes folder so it is basically a ‘more correct’ solution than the first.
3) The next and I think best solution was to create a new, duplicate hibernate.cfg.xml inside

my-app/src/test/resources

This allows a different hibernate.cfg.xml file to be used for testing and, for example, facilitates connecting to a different database (such as hsqldb) for testing while continuing to use your regular database (such as MySQL) for the app itself.

A couple of points to note:

1) The hibernate mapping files (i.e. the *.hbm.xml files) should also be in the resources folder (in whatever directory structure you choose) to ensure that these too are accessible by the unit tests.

2) I am not (yet) using Spring with the hibernate components, which would likely have changed the above setup.

Refs/links

http://maven.apache.org/guides/getting-started/index.html#How_do_I_add_resources_to_my_JAR

http://www.mail-archive.com/users@maven.apache.org/msg54720.html

I will try to post some notes on the various talks I attend below…

Day 1:

• Session 1: JavaScript
• Session 2: Easing into Agile
• Session 3: Building Better Tests in Java
• Session 4: What is Python?

Day 2:

• Session 1: Basics of Threading
• Session 2: Introduction to Grails

Check it out at http://findbugs.sourceforge.net/

Update: Found recommendations for another tool called JLint, which is supposed to be particularly good for spotting potential deadlocks so perhaps worth using when testing threaded code.