Making time to experiment

Innovation is important for every business; as a way to create new products or services, improve what you already offer in interesting ways or to rethink the way you do things. But how exactly do you achieve that? Where do ideas come from? How do you test and validate them? And how much is it going to cost to do so?

We’re used to helping our customers produce novel and innovative solutions to their problems, but it’s easy to forget to make time to innovate ourselves, particularly when it comes to new areas of technology. Recently we ran a week-long experiment to see what we could achieve whilst learning about a machine learning technique called Neuroevolution.

Why?

When working on one project for a long time, you can begin to feel you’re covering the same ground or not developing new skills. One of the driving forces behind our experiment was to look at ways to alleviate this and to try out new things. Variety is the spice of life, and a change is as good as a rest, as the folk-wisdom tells us.

Machine learning is certainly a hot topic at the moment, both from a technology and an ethics point of view, and while we have a machine learning interest group that meets regularly we wanted to try a different way to develop and extend our collective knowledge.

However, we didn’t explicitly set out to do something with machine learning, as we didn’t want to limit ourselves by focusing on outcomes. Rather, we posed the question, “Given a small team, and 1 week, what could we achieve?”

By setting a time limit we were limiting the costs of the experiment, as the main expense here is the cost of everyone’s time involved. The team size was set at 4 people, although due to prior commitments, not everyone was involved every day. This is an agile technique called timeboxing, and shows how a lot of our agile practices can help with innovation too.

How?

The basic process we followed was to:

1) Set a challenge.

The idea here is to give some focus to the next step, without being overly constrained by possible outcomes. Be as open ended as possible.

For us, this was simply the question, “Given a team of 4 people for a week, what can we achieve?”

2) Brainstorm.

Pick your preferred method for brainstorming, and generate as many ideas as you can.

We gathered together the team (and other interested parties) around a whiteboard for half an hour and just started throwing out ideas for things we could do. There are loads of useful ideation techniques out there, such as Gamestorming or the Nesta Innovation tools.

3) Discussion and voting.

Once you have generated ideas, you need to refine them and focus in on the most likely candidates.

We simply talked about some of the options in more detail, provided a bit of extra explanation and then voted on our favourites. This is when we focused in on neuroevolution and the idea of having an AI learn to play a simple game, in our case “Pong”. We also started to think about what the possible outcomes could be, i.e. a demo of the final system playing the game, but our main objective was still just to see how far we could get during the remaining time.

4) (Just) Do It!

Now it’s time to draw the rest of the owl!

We decided to start from first principles - no frameworks to mask complexities or hinder our understanding of the techniques involved. One of our team had done some work with neuroevolution before so he gave us all a brief intro and then we got down to business.

In addition to experimenting with machine learning, we also gave mob programming a try, as while we often pair program, mobbing is something we don’t get much opportunity to do. It turned out to be a great choice for this experiment as it allowed us all to get up to speed quickly, and the project could continue rolling even when team members had to step away at times.

5) Demo & Disseminate.

One of the key principles behind this sort of experimentation is to disseminate your learnings and experiences, good or bad. How you do that depends a lot on what you’re trying to achieve and the format of your organisation.

As we came towards the end of the week, we had implemented a version of Pong and a basic implementation of the neuroevolution technique that could learn to play the game well, although we hadn’t managed to properly investigate speciation or co-evolution strategies. The team decided to spend some of the remaining time improving the graphical resolution of the Pong game, which resulted in us learning quite a bit about how neuroevolution copes with much larger problem spaces!

We held a showcase and retrospective and invited everyone in the company to attend, so we could share our experiences.

6) Brainstorm next steps.

Now is the time to look at what has been achieved and what can be done next. If you’ve produced something, how can it be further developed? Are there new directions you could go in? Does it warrant more time spending on it? If so, what would the next steps be and how will you go about them?

We held another quick brainstorming session, this time with a focus on producing a demonstrable application, building on what we had learned. Additionally, we wanted to share our knowledge and experiences with the company and a wider audience, so we looked at means of doing that too.

Achievements

Our initial goal was to try something new and have fun doing it, as a means of breaking out of our everyday routines and it’s fair to say that we achieved that. Additionally we learned some practical lessons around the strengths and weaknesses of neuroevolution, along with some ideas around how we could use mob programming more effectively in the future.

From an innovation point of view, the whole process was an experiment itself, and I think it has proven to be a good technique for coming up with and trying out new ideas in a controlled cost/time frame. And from this I’ve already started thinking about how we could run another experiment, and some improvements we could make to the process.

Future Plans

We run two half-day hackathons a year here at Black Pepper, and while fun, there are limits on what can be achieved in that time. This format of a week-long experiment is a great way to try something out in more detail while preventing it from becoming a never-ending series of “what if…” scenarios.

My personal goal for 2019 is to run more of these experiments, maybe on a monthly basis, refining the process as we go. It should be a great way to try out new technologies and learn new techniques in a more in-depth manner than could be achieved by an individual alone. It could become a technique we use on client projects as well to help them discover better solutions to their problems.

The key is to not just do this once, but to keep on experimenting and learning. Making time to experiment and do things out of the ordinary is a great way to come up with new ideas and to be innovative.

This site uses cookies. Continue to use the site as normal if you are happy with this, or read more about cookies and how to manage them.

X