Because it's the best.
"Gather round", Dan North told the attendees at the QCon London 2009 software development conference, "and I'll tell you a story". This story starts with a story-within-a-story.
As a software engineer, I want to attend a conference so that I can become a better engineer.
He started off be describing what he saw as the typical development of an architect. One starts out, fresh from university all keen and eager to build some exciting and cool software. This is the "Enthusiastic Developer" stage. Unfortunately it turns out that what customers want is typically boring, boring, boring - "create a system that allows us to maintain Customers"; "create a system that allows us to maintain Products", and so on. This leads to the "Disillusioned Developer", repeating similar tasks day in and day out. Then, having seen that they are doing something that looks the similar each time, the newly minted architect decides to build a generic solution, a "maintain generic things" engine. Then we can spend just a little time (oh it will only take us a few months to get the framework finished) and then there is only a small amount of specific configuration required, say 20% of the over all total effort. This is the "Enthusiastic Architect".
So they spend 80% of the time building this wonderful generic solution and then then remaining 320% of the project on the rest of it. The problem being that customer's don't really want generic solutions. The form that they want to use to maintain their customers has to behave differently from the one used for products (otherwise they'd use Excel right), it's all those little exceptions that one has to introduce to work around the generic solution, or to extend the generic solution into an all signing and dancing framework that takes the time. We've all been there, well at least all of us who are honest and will admit it. This leads to the "Disillusioned Architect".
He gave a fun little quote: "Some programmers when faced with a problem turn to a generic solution... Now they have two problems."
The key to concurrent software design is immutable data structures. If a data structure is immutable then it must always be in a consistent state. Multiple threads can never see a partial update since the values never change. If one considers the number 42, it never changes, it is always 42. Similarly if we think about a composite value, for example the date "12th March 2009", it also never changes. It makes no sense to set the month to be "January". There is another date, "12th January 2009", but this is distinct from the first one.
So, values are immutable, whether they are primitives or composite values.
Functions in functional programming (and indeed really in any form of software) work on values and derive new values. For example, the function "add" takes two immutable values and evaluates to a third one that is the sum of these two. It always produces the same output for the same input, it doesn't matter when it is called, in what order it is called in relation to other functions, it always does the same thing. It is an ideal construct for building highly concurrent programs.
Guido started his talk by discussing some interesting statistics regarding the costs of traditional outsourcing. He compared the costs of doing the development locally using a "hyperproductive" agile projects verses standard outsourced waterfall development. In the standard model, when one outsources to a very distant company with a different culture and especially in a different timezone, one has to define exactly what one wants in very detailed specifications which lead directly to a waterfall approach.
His figures suggested that to get the return on investment that one needs for a successful outsourcing project over for example a two year period then the cost of the outsourced team must be about 10% of the alternate local agile team's costs, or even less. However, typically one finds that outsourced teams are about 30% of the cost of local teams. Therefore, he suggested, taking a project that could be done locally with an agile team and cost $2 million, if outsourced to a traditional waterfall team, then that same project would cost around $6 million. Very interesting figures indeed and something that I feel is right and empirically sounds right from my experience of customers who have outsources, however I want to do more research before accepting these figures.
Having laid out some of the well known problems of this traditional approach to distributed teams, especially across continents, he described how his project had addressed them.
The premise was essentially that we as a society have been taking caffeine since the beginning of the Industrial Age and that life today is different from that of a 19th Century factory worker so we may want to re-evaluate whether the drug caffeine is still useful to us. I'm not really sure how the contents of the talk, which was all about caffeine, fits the title of the talk or the 'agile' track it was in.
She started by highlighting that before the Industrial Revolution, Europeans tended to have beer for breakfast - since water wasn't really safe to drink. However of course that didn't really lend itself to the regimented and time controlled practices required in a factory at the beginning of the Industrial Age. Tea and coffee became popular at about that time, she posited, because it could help us get up with the clock rather than the sun.
The main part of the talk was about the metabolism and concentration of caffeine in the body and discussing some studies that show that while caffeine is good at helping people concentrate on simple and long running tasks (such as those required in a factory), it has no extra benefit in terms of the sorts of tasks that software developers do. In particular she highlighted some of the issues regarding how caffeine effects are based on concentration in the body, and pointed out that drinks often marketed at children can have high levels of caffeine in them: that we "learn" to drink caffeine at an early age where we would be alarmed at giving children other drugs. This was indeed food for thought and I agree with her that perhaps we as a society ought to be re-evaluating how much caffeine we drink.
He had a very interesting presentation style and I would recommend that if you get a chance to see one his talks that you do.
The initial theme of his talk was about IT commoditisation, how today's cutting edge and hot topics are tomorrow's boredom. That software (and indeed anything) goes through a process from innovation, bespoke implementations, products and finally ubiquitous utility-like services. All business services exist somewhere on the curve from innovation through to commodity.
This lead him to ask the question of how this impacts IT. Well, in order for one to innovate in the first place then you need build on top of stable components. Without ubiquitous Internet access then one couldn't have a Google. So as we see ubiquitous services, such as Infrastructure as a Service, then we will see innovation that can build on that, such as Sales Force as Software as a Service.
Clojure is a lisp dialect designed to run on the JVM.
I was particularly interested in its use of references to manage concurrency. All data structures are immutable, so they can never be in an inconsistent state. However one needs to be able to "update" state as a program runs. Effectively this means replacing the "value" (as complex as that may be) that is "identified" by the reference. So concurrency in this scenario is managing the process where new instances of the data structures are set as the value when multiple clients want to do this at once. Clojure has four types of references:
Refs, these use software transactional memory to ensure co-ordination of updates of the references. Agents, these are asynchronous processes that defer the update until a convenient point in time. Atoms, these synchronously update the references and repeatedly retries in the event of a conflict. Finally, vars, which are isolated so that they are only accessible to the local process.
Dion Almaer and Ben Galbraith, talked initially about a four "up and coming" technologies that could well transform the web "web platform". The first being Canvass that came out of Safari and has now spread to Firefox, Opera and Chrome. It allows a 2D drawing environment on which one can draw, opening the way for pure web applications to take control of the page, and move away from the document based text, image and rectangle model of 20th century web.
They demonstrated the Bespin in browser editor that is being developed by Mozilla Labs to show the power of Canvass, amongst the other technologies. Like Flash and JavaFX being able to draw arbitrary components natively in the browser means that one can move away from the problems of imposed by the document-centric model of today's applications.
Secondly, they discussed Fast JavaScript, with new JavaScript engines on the horizon one can see that web applications can be transformed. One will be able to do things when JavaScript is blindingly fast that one would not even consider today.
The conference opened with a keynote by renowned computer scientist Tony Hoare. He talked about the science of computing and the engineering of software. The main thrust of his talk was that if computing is a science then software must be engineering and vice versa, and of course he posited that both were.
He made some interesting observations about the difference between science and engineering and that there is a continuum from scientist to engineer.
Science is a long term endeavour, where its theories and proofs have long lasting consequences, once discovered a theory is (potentially) applicable for decades. Whereas engineering is confined to shorter term goals, an engineer is focused on solving a particular problem.