Acceptance Test Driven Development Part II

Last time we looked at how we do acceptance testing at Black Pepper using a domain specific language (DSL) to make the tests more approachable to the business representatives of the project. In this article I'm going to look at writing tests using a DSL.

The purpose of a DSL is to allow tests to be defined in a language that is relevant to a wider audience than just the developers. Bear in mind that the DSL should be comprehensible by non-technical members of the team, most notably the business sponsors.

We use Java and Junit as the framework for acceptance testing as it means little overhead for developers. Of course, you can use other frameworks/languages to implement your DSL – as a general rule, use a language you're comfortable with and which allows you to express your criteria efficiently and clearly.

So... An Example

In the previous article I gave an example of some acceptance criteria for a simple story:

  • If the customer has no orders, the list of outstanding orders is empty
  • If all the customer’s orders are dispatched, the list of outstanding orders is empty
  • If the customer has one or more orders that are not dispatched, they will be displayed in a list showing the order number, date ordered and status.
  • Outstanding orders will be displayed in descending order of date ordered

Let's take the first acceptance criterion and give a concrete example of the DSL for the test:

public class OutstandingOrderListAcceptanceTest extends DslAcceptanceTestBase {

    public void outstandingOrderListShouldBeEmptyWhenNoOrders() {
        data.createCustomer("customer", "password");


So, to walk through the body of the test, we:

  • Create a customer called “customer” whose password is “password”, which we're going to use in this test (never let it be said that developers lack imagination).
  • Log in as “customer” via the user interface
  • Display the outstanding orders on the logged-in customer’s account, and
  • Ensure that the list is empty (because this is a new customer who wouldn't be expected to have any orders).

I hope you'll agree that this is quite easy to read, even when you're coming at the test with little prior knowledge of what's going on. Note that the vocabulary used is not technical – it's meant to be easy for your business representatives to read, and will be couched in their vocabulary.

A guiding principle is to try and keep technical distraction out of the test. Try to avoid complicated programming idioms, and use simple parameter types because it will make the tests more approachable by non-technical readers.

Creating Test Data

Another important aim you should have when writing acceptance tests is that each test should be responsible for creating any data it needs. This has several benefits:

  • You can run the test repeatedly without having to reset application or database state (very useful when debugging)
  • There is no need to maintain database scripts to create data, which can be time consuming
  • It can aid parallelism – allowing multiple tests to be run concurrently
  • QA can use the DSL to set up test data for exploratory testing

Our DSL will have to provide ways to create entities within the application being tested. The example creates a new customer specifically for the test. That way we can guarantee that the customer is in a known state every time the test is run.

Any data created will be stored within the context of the test so that it can be looked up later for use by other aspects of the DSL. The context can be updated as the test runs and state changes.

How the entity is created is up to the DSL. If your application has an API, it might be simple to use that to create it, for example by using a web service (which the UI would normally talk to). Alternatively, it could just involve inserting a row into a database table via JDBC.

DSL Features

The functions of the DSL are primarily:

  • To manipulate application state, e.g. setting up a customer: data.createCustomer("customer", "password");
  • To drive the user interface: ui.login("customer"); ui.myAccount.displayOutstandingOrders();
  • To assert application state: ui.myAccount.ensureOutstandingOrdersListIsEmpty();

DSL Design

You're just writing code here, just like elsewhere in your application, so start by writing a test. List the steps that you want to walk the application through in exercising the functionality. The design of the DSL will fall out of the writing of the test. Don't be afraid to refactor as you go.

There are several principles that I think you should keep in mind:

  • Use active verbs to name statements (displayOutstandingOrders). Imagine you are instructing a user how to use your application, and keep the activities coarse-grained. Go for displayOutstandingOrders(); as opposed to clickMyAccountButton(); clickDisplayOrders(); selectOnlyOutstandingOrders();
  • Make each statement focused. Do one thing well. Don't be tempted to include lots of alternate paths in a statement, rather write a new statement for the alternates. For example, ui.orders.placeOrder() and ui.orders.placeOrderExpectingError(), as opposed to ui.orders.placeOrder(boolean isErrorExpected)
  • Partition the statements into related areas. This allows each statement to be more succinct, whilst providing the necessary semantic context (ui.myAccount.displayOutstandingOrders, data.createCustomer)
  • Minimise the number of parameters to each statement


There's nothing really complicated in this DSL. However, by following some simple principles we have a powerful mechanism for defining acceptance criteria, where:

  • Acceptance tests are straightforward
  • Acceptance tests can be read by non-technical team members
  • Acceptance tests are repeatable and can be run in parallel

Next time I'll look at the internals of a DSL.

This site uses cookies. Continue to use the site as normal if you are happy with this, or read more about cookies and how to manage them.