Late September, I gave a guest lecture at Staffordshire University to the School of Computing students on the subject of enthusiasm, and how agile software development helps motivate developers.
During my 15 year career, I've reflected many times on how often external factors have increased, maintained or temporarily reduced my motivation. I've more recently pondered relationships between these factors and the agile principles and practises I now use daily, and whether in turn, they've increased my motivation. I've observed positive changes and logged these informally over a few years.
I had a conversation with Stafford University's Paul Boocock about running an Agile Staffordshire guest lecture some months ago. Along with other ideas, I suggested these observations may provide an interesting subject. Paul agreed. The lecture was recorded and the video embedded below. This blog post aims to summarise these observations.
I started the lecture with an introduction to myself, an overview of my observations, and a brief digression covering the pillars of intrinsic motivation; autonomy, mastery, and purpose. I recommend Dan Pink's 'The puzzle of motivation' as a good introduction to motivation which builds on the work of Herzberg, Maslow and McGregor. Personally, the periods in my career where I've had the greatest levels of energy, and the highest probability of completing a task where those were I was able to influence how I worked, with tools and techniques that I was familiar with, and on products that I perceived to have a high value to the end user. On projects that lacked one or more of factors, my motivation was considerably lower.
I then presented a series of observations made throughout my career where I did suffer lower motivation, and where agile principles did, or could have improved the each situation.
Producing the wrong thing
I've spent the majority of my career collaborating on software products that have provided great value to the end user. Occasionally however, I've perceived this value to be lower than typical, and I've noticed a dip in motivation during these projects. A common trigger for this perception has been suspicion that requirements were not aligned with actual needs of the end user. Sometimes, the users's needs had changed from the original specification. Sometimes their real world usage hadn't been fully taken into account by those responsible for the specification. Either way, I felt the features being worked on would satisfy only contractual compliance rather than provide real value. This is a well documented issue with software procurement with an often cited study that echoes my observations that many developed software features are rarely used.
Embracing change, and ensuring that the most business critical / valuable features are prioritised lies at the heart of agile. Throughout the agile projects on which I have collaborated, I've witnessed ongoing validation that the customer's actual needs are being prioritised. Superfluous requirements are frequently traded out for more valuable features quickly, and with little ceremony before significant effort is wasted on their implementation. The ongoing collaboration with the end user has confirmed and increased my perception of the absolute value to the customer which in turn has increased my perceived values of the features which has significantly improved my motivation.
Lack of measurable progress, or a low MTBA (Mean Time Between Achievements)
I have a poor memory, and rely on task lists. I find using task lists allows me to reduce cognitive load by not having to remember things I must do. This in turn reduces stress and increases my potential energy to get things done. I also enjoy the act of ticking tasks off as it proves I'm progressing, but I only ever tick them off when they're truly and completely done. Ticking off items prematurely with remaining subtasks again leads to the feeling that I've forgotten something. The larger the size of each task, and the less defined, the less convinced I get that I am actually progressing. Furthermore, the typical indirection between waterfall style tasks (detailed design specification, coding, testing) and user features means it's difficult to interpret the ratio of effort expended to that remaining.
By contrast, most agile techniques like Scrum, and Kanban remove this indirection since the units of measurement for progress are commonly user stories. Phases like design and coding are decomposed and incorporated into the delivery of each story. When 7 out of 20 stories are complete, you have a real metric of progress. Furthermore, stories tend to be more numerous but smaller than waterfall phases which means I get to tick things off frequently. Stories are typically done in hours rather than days, and are only ticked off when considered DONE. I get to close the loops which increases my motivation whilst reducing my cognitive load.
Will it all integrate in time, what if it doesn't?
Waterfall norms suggest the component integration phase happen towards the end the project when each component is complete, or close to complete. I've observed three well known problems with this approach, which in combination can cause significant challenges.
- Late integration of components prevents early feedback that components won't integrate well. Assumptions made during development of one component conflict with thoses of other components and completed components have to be re-worked at great cost.
- As integration occurs near the end of a project, contingency to resolve these often architecturally significant issues happen at the worst possible time.
- Finally, many requirements of the system may be met only by successful integration of several components, and thus failed integration risks non delivery of a significant set of requirements.
During my collaboration on projects that use agile practises such as continuous integration, multi-disciplined teams, and one by one delivery of requirements in priority order, these issues have been prevented. Stories rarely 'await integration', and at any stage within the project, the set of highest priority stories are ready for deployment.
Big risky releases
Frequently, I've rolled out large software releases without the confidence I should have. The infrequent, and thus large and typically manual nature of these releases presents significant risks. I've often observed procrastination in releasing software due to lack of confidence.
By contrast, agile projects are typically expected to deliver frequently and within a few weeks of project inception. This frequency provides justification for automation which increases in repeatability and thus reliability and reduced risk . Furthermore, frequent releases typically mean smaller releases, which again reduces risk. With both of these risk reducing techniques in place, I've always been confident in deploying releases. Further, the ongoing collaboration with customers sets expectations as to which stories can be delivered, and by when.
Prematurely knowing all the things
Agile teams generally embrace the idea that knowledge grows during a project, and as such important decisions should be deferred to the latest possible moment when they benefit from the maximum knowledge. This contrasts to the important architectural decisions that I've made in the very early stages of waterfall projects with little knowledge of requirements or the technological challenges that become evident later throughout the project execution.
Through understanding the benefits of deliberately deferring decisions, I've become more confident in making important decisions at the right time, and / or justifying not making them too early in the project. This confidence is an example of the mastery pillar of motivation.
Following these observations, I opened the floor to questions. I was delighted with the number and the quality of the questions posed. It was clear that I'd managed to engage with the audience. Most of the questions covered common concerns and myths of agile development such as lack of documentation and difficultly in contracting fixed price and fixed time projects with variable scope. A great question asked was whether inexperienced developers would find it hard to work within agile teams that definitely benefit from enthusiastic but disciplined developers who can master tools and techniques quickly. My response covered some of the related agile practises such as pair programming, retrospectives and frequent code reviews that provide great support to new members to a team. I hope I reassured him.
I received some great feedback about the session, in particular about Personal Kanban, a personal time management system that I briefly mentioned. Based on this feedback, a joint Agile Staffs, BCS session on the 4th November included an introduction to flow, using Personal Kanban as a tool to demonstrate it's suitability to both personal projects and wider software development..
I'd like to thank Paul Boocock for arranging this lecture, and the students for both engaging and asking some great questions. I hope they enjoyed it as much as I did.