Introducing Spectrum: A BDD-Style Test Runner for Java

Update: Spectrum has come a long way since its initial release. Check out the latest releases on GitHub!

Spectrum is a lightweight spec runner for JUnit with an API similar to Jasmine and other behavior-driven development frameworks. It uses plain Java 8 and describe() / it() blocks to specify and verify a system’s expected behavior.

Motivation

When I’m writing tests, I make a deliberate effort to organize and name things in a way that keeps the focus on observable behavior instead of implementation details. This is definitely possible in Java with vanilla JUnit and creative use of class/package names; the following are scenarios for a hypothetical library management system:

public class WhenCheckingOutABook_theLibrarySystem_Should {

  @Test
  public void decrementTheNumberOfAvailableCopies() { }

  @Test
  public void generateAReceiptWithADueDateTwoWeeksFromNow() { }

}

Sometimes I’ll even use the Given-When-Then scenarios from a story’s acceptance criteria directly:

package given.three.copies.of.a.book.are.available;

public class WhenSomeoneChecksOutOneCopy {

    @Test
    public void thenThereAreNowJustTwoCopiesAvailable() { }

    @Test
    public void thenTheirCopyIsDueInTwoWeeks() { }

}

This kind of naming works well enough – and is certainly better than testCheckoutBook_RemovesBookFromInventory() – but it does leave something to be desired.

For one, I’m kind of abusing the class and package system to make the tests names pretty, a side effect of which is that test files tend not to be organized for easy discovery. The standard JUnit runner can’t decouple test naming from test organization, so we have to compromise one way or the other. Also, camelCaseSentencesArentVeryEasyToRead and underscore_separation_isnt_much_better – there is a reason whitespace and punctuation exist! But again, test names must be valid identifiers, so there’s only so much we can do.

I guess my time test-driving JavaScript with Jasmine has spoiled me. Coming back to a Java project, I really miss the flexibility and expressiveness offered by its BDD-style API. Of course xUnit frameworks can be used for behavior-driven development, and spec-style frameworks can be used for traditional white-box testing, but there’s something to be said for working with the grain instead of against it.

Perhaps Another Tool?

I won’t claim to be an expert on Java BDD tools; most of my experience lies with Cucumber-JVM. A quick Google search turned up this recent comparison of a few other options. These tools have one thing in common: they all use some flavor of domain-specific language instead of plain Java. DSLs are great for collaboration between developers, testers, and business people. What I’m looking for, though, is more of a drop-in alternative to BlockJunit4ClassRunner that I can use in my Red-Green-Refactor cycle.

Spectrum

To scratch this itch, I decided to create a lightweight JUnit test runner in the spirit of Jasmine, Quick, and RSpec that runs with plain Java and JUnit.

It’s called Spectrum and it lets you write Java specs that look like this:


@RunWith(Spectrum.class)
public class LibrarySpec {{

  describe("The library system", () -> {

    it("knows how many copies of a book are available", () -> {
      // ...
    });

    it("helps you remember when your book is due", () -> {
      // ...
    });

  });
}}

Yes, those are Java 8 lambda expressions. In a time before lambdas, I can see why the authors of other tools chose to use DSLs. And yes, that is an instance initializer block; it’s a bit of a language hack to make specs more readable.

Features

Similar to other BDD-style frameworks, Spectrum supports:

The GitHub repository has an example spec showing how to use the various features I’ve implemented thus far.

Spectrum is implemented as a custom JUnit test runner, so it should “just work” alongside all your other JUnit tests.

Future Features

Some things you might expect from a spec runner, but haven’t been implemented yet (as of v0.4.0):

Java 6/7 Compatibility

The Spectrum runner itself is designed to be compatible with Java versions back to 1.6, but the specs will be a lot uglier:


@RunWith(Spectrum.class)
public class NoLambdaSpec {{

  describe("Writing specs without lambdas", new Block() {

      @Override
      public void run() throws Throwable {

        it("makes for sad pandas :(", new Block() {
            @Override
            public void run() throws Throwable {
                // ...
            }
        });
  });
}}

Adding first-class support for Retrolambda will help make Spectrum more accessible for folks who can’t run Java 8 yet.

Non-Features

Unlike some BDD-style frameworks, Spectrum is only a test runner. Assertions/expectations, mocks, and matchers are the purview of other libraries.

Contributions Welcome

If you’d like to share some ideas or code, send me a tweet, open an issue or fork the repository.

Other Posts