I often work on projects where we run load and performance testing for clients. Load testing produces huge amounts of data that needs to be analysed and the results of that analysis published. Much of this analysis is mechanical: calculating means, 90th percentiles, and variances; producing charts of response time over the course of a run etc.

We like to publish data in an accessible fashion that allows all the stakeholders to see progress and in particular allows everyone to participate. We've found using a Wiki for this to be excellent. However, getting data out of load testing tools and publishing it on a Wiki usually takes a lot of manual effort, so I decided to investigate how to do this as automatically as possible - so that I can actually spend my time adding value.

The tools I use are:

I've written in an earlier blog entry about writing acceptance tests in Java for a Flex front ended web app, using Selenium RC. Here I'll describe how we added those tests to our continuous integration build, using a Hudson job as part of our build-and-deploy pipeline.

Our web app comprises two components, a Java server-side component and a Flex client-side component. The Java component is built by a Hudson job as a war file. The Flex component is built by another Hudson job as a zip file. These two Hudson jobs are independent, fairly standard projects built by Ant, each with its own set of unit tests. To run acceptance tests, we need both components to be deployed into the target environment. We created a third Hudson job which aims to deploy the latest artifacts from the Java and Flex jobs, start up Tomcat and Selenium, run the acceptance test suite, and shut down Tomcat and Selenium. This Hudson job depends on the other two jobs and its build is activated when either of those two completes.

As we are already using Ant to control the two component builds, and our acceptance tests are written in Java, we decided to use the Ant build for the Java component as the home for the acceptance test build. Here's the target to run the acceptance tests in the Hudson CI environment:

I've come across an oddity in my functional tests for a GWT application I am working on.

The Selenium RC test I have should simulate a user clicking on a button, implemented as a GWT PushButton (com.google.gwt.user.client.ui.PushButton). The problem is that the click on the button never seemed to work using the following code

 
selenium.click(BUTTON_ID);
 

A GWT PushButton is not a standard HTML <input type="submit" ...> or a <button ...> rather it is a styled button constructed from DIV and other HTML tags.

The idea of a continuous integration deployment pipeline has been around for several years and is described in detail by Dave Farley and others in this ThoughtWorks paper. Following my recent experience with Hudson, I thought I'd show how to create a CI pipeline using Hudson instead of Cruise Control.

There are 2 aspects of the CI deployment pipeline that Hudson can most easily help with:

1. Managing binaries
2. Process Gates

The ThoughtWorks paper describes a binary repository as a shared file system where release candidates are stored. A key feature of the process is to promote the compiled binary artefacts through each stage of the pipeline without needing to re-build them. Each time a developer makes a checkin the system builds a new release candidate and stores it in the binary repository ready for subsequent stages to use. Each stage in the pipeline performs a series of tests (acceptance, performance etc.) against the same set of binary artefacts. A consequence of this approach is that the binary artefacts must be configurable for each target environment.

As I look deeper into Flex testing, I keep coming across little problems that are so easy to deal with in Java, but just aren't as simple as you might think in Flex.

One of these problems is testing chunks of code that call off to live services. There are two bits of technology that make this easy in Java and that's mock objects and Inversion of Control (IoC) for dependency injection. Flex doesn't natively do either of these.

Dependency injection is available via a couple of IoC frameworks that are available for Flex. Mock object frameworks don't seem to exist yet for ActionScript, which is a shame. So, what do you do?

The problem I had is how to automate acceptance or functional testing of a web application with a Flex front end. I wanted to solve this problem because functional testing increases confidence that the software meets customers' requirements. Automating the tests - coding tests and running them as part of the build - pays dividends over and over, as agile developers well know. We're able to check that user stories are implemented and catch bugs early in the lifecycle.

Kieran's blog entry unit testing Flex tells how he solved the problem of writing and automating unit tests for Flex front-ends. He found there was little available material on unit testing for Flex - and I found even less on acceptance testing for Flex. However, we had done automated acceptance testing for non-Flex web apps.

Selenium is a popular tool in the open source community for automating functional tests for web applications. It allows developers to write tests that drive your web application via the browser. You can write tests in various languages, including Java and JavaScript, and in the test you can manipulate web page components using XPath or JavaScript. So for example I can write a test to enter text into an input element of an HTML form, click the submit button, and verify data on the response page. But how can I do the same sort of thing for a Flex front-end?

This is part 2 of an article about testing Flex and integrating Flex into a continuous integration environment so it's worth reading part 1 first.

FlexUnit in Ant

Once you've written your unit tests for your Flex project, how are you going to run them? Sure, you can just fire up the test runner that I wrote about before, but you've got to remember to run it and actually look at the green/red bar to know if things are going wrong. Ideally you'd get your unit tests to run as part of your build so that you get the advantages of continuous integration.

With this in mind, I took a look at Peter Martin's FlexAntTask which allows you to run your FlexUnit tests from within an Ant build.

I wrote recently about a project where we had to put a Flex front end on top of a set of SOAP web services. Most server side programmers are hopefully now pretty comfortable with unit testing their server side code using something like JUnit. However, testing Flex applications isn't quite as mature yet and with a lot of Flex developers coming from backgrounds where unit testing isn't so ingrained, there isn't that much written about it.

There are three major different aspects of testing a Flex app that you might want to do.

1. Unit testing: Testing individual bits of ActionScript such as individual classes or methods
2. Integration testing: Testing how your app is actually wired together such as do the right events trigger the right commands
3. Acceptance testing: Testing the whole app, including how it behaves in a realistic deployment environment

These are ordered in terms of simplicity. Unit testing as hopefully I will show is pretty simple, but things get a lot trickier when it comes to Integration and Acceptance testing, so I'll leave those for another article...hopefully by someone else :)

Unit testing thankfully is also a good place to start because the more unit testing you do, the less integration and acceptance tests you are likely to have to write.