Are waterfalls agile?

People talk about agile but what do they really mean by agile?

David Matthewman CIO, OU

David Matthewman CIO, OU

I read a very insightful and interesting interview with David Matthewman, the OU’s newly appointed CIO, in Computing Weekly and I’ve also had a number of discussions with him about development and programming. In the interview he says “As part of a more disciplined approach to market methodologies, Matthewman will be introducing a more prolific use of agile and scrum development, as well as service management standards such as ITIL.”

I think this is a move in the right direction for the OU and for other organisations who similarly have adopted up until recently very traditional waterfall methodologies for enterprise level system development, however agile development methodology on it’s own won’t solve the problem. I read a paper last week commissioned by Hays for example which was about agile development called “The Elephant in the Developers’ Room” – it’s kind of drawing conclusions it wants to make the case for agile, but the headline statistics alone are stunning:-

  • 60% – 80% of project failures can be attributed directly to poor requirements gathering, analysis, and management, costing US businesses $30 billion per annum
  • 50% of major projects (defined as costing >£10m) are cancelled when at least 40% of spend has been incurred
  • 40% of system problems are found by end users
  • 25% of all spending on projects is wasted as a result of re-work
  • Up to 80% of budgets are consumed fixing self-inflicted problems
  • Only 8% of large-scale applications projects (those that cost between £6 million and £10 million) succeed.
  • Just 16% of software development projects close within acceptable constraints of cost, time and quality.
  • Cost overruns of anywhere from 100% to 200% are common in software projects.
  • IT workers spend more than 34% of their time fixing software bugs

Anyway I had a twitter discussions with some developer colleagues, which is by the way the best way to solve the worlds problems, and the conclusions were as follows:-

Agile methods alone wont fix the problem of very large developments failing.

Here are some of the practical reasons why very large projects fail from experience with projects of 60m+ which I been involved in with SUN and Microsoft and others who do this stuff well on the whole:-

  1. Some issues are behavioural to do with the makeup and background of the team
  2. Some are to to do with poor management, not specific project management but people management and lack of ability to think creatively, direct appropriately and react (important as scope changes)
  3. Project scope changes, so some issues are to do with inflexibility, not reviewing scope regularly and adapting
  4. Some are to do with lack of empowerment of developers, making them both understand and grow, giving them challenge and enabling cross working
  5. Some are to do with siloing of activity, “only X knows about that” mentality
  6. lack of ownership of issues “not my problem mate” – it’s everyones problem
  7. Good (critical) people leaving during the project. Perhaps there’s a place for a ‘golden handshake’?
  8. Some are to do with complexity. i.e.not breaking the big complex system build down into smaller manageable chunks.
  9. Some are to do with people not understanding the vision. Everyone must understand it.
  10. Finally by far the best projects I’ve worked on are ones where everyone contributes to the solution, feel tied to the success of it. The reporting, logging and reviewing processes serve a purpose that those in the project understand, i.e. it’s directly relevant to assisting them and their colleagues. The organisational structure is kept light and serves to help development, rather than for MI alone.

As we move into a more commoditized, off-the-shelf, ROI, SLA, cloud-based, shared solution, outsourced, yield enhanced and recession proof world it’s important to remember that the ‘uniquness’ of an organisation is created through the innovations that come from within rather than without. Developers can still add that uniqueness to an organisation by building bespoke services very well and at scale.

We’ve just started a venture to develop the OU Media Player for example which is going to create ‘the worlds’ most accessible media player’. It’s built using existing services but we’ll add the value to make it provide captioning and accessibility services and to link to all OU media materials on a variety of platforms including the VLE. This is a very small team working over the next five months in an agile way. I’ve got 100% confidence in it’s success because it’s a great team, everyone understands how important it is to the OU and they’re being given the freedom to build it iteratively, creatively and well, i.e. serving the OU’s mission in being “open and accessible”.

Convergence v Specialism

I’m very interested in the trend with devices such as XBox 360 towards a convergence of media types and delivery with it’s support of Sky TV through the XBox and broadband via Sky Player – Stephen Nuttall from Sky was quoted as saying: ‘Our partnership with Xbox is a further example of our commitment to put choice and control in the hands of customers.’

I’m particularly interested in the ‘blurring’ or perhaps integration is a better word between the different media types so the idea of interactivity around watching a football match whilst downloading stats and also interacting with other fans is cool, also concepts around adding value to experiences through ‘back channel’ activities is something becoming more prevalent, as is the concept of ‘on demand’ services.

I think the really interesting stuff will be when the boundaries between an interactive TV experience, a gaming experience or an internet experience all disappear to the extent that they become platform neutral and coherent rather than bolt on things. The announcement of the Boxee box earlier this month is a step in the right direction, this really is opening up the rich resources and putting power int he hands of the users. It also means that you no longer need to get content ‘produced’ on a TV channel in order to get your content to a large audience, consumers become producers.

I’m very interested in using gaming technology and interactive TV in more powerful ways to develop engagement and learning, supported with internet they become extremely powerful tools.

Aliens, Wifi and nasties…

Three odd things happened in the past few days (well probably more but three that I’m aware of) – Firstly a report I read in the Independent from a senior medical advisor to the UK government saying that wifi could be as dangerous to people as asbestos. This has sparked a huge debate with many people saying that there’s no proof of any issues. In May the BBC reported that scientists had concluded there were no health risks.

Secondly there is a report out this week proving that there are no short term health issues associated with the use of mobile phones for adults however there may be a risk over a long term (the report didn’t cover children who may be at higher risk) http://news.bbc.co.uk/1/hi/health/6990958.stm

 Thirdly a meteorite fell in Peru and caused a mystery illness http://www.guardian.co.uk/space/article/0,,2171920,00.html

Where does this leave me? – I don’t know what’s killing me. Last year I was told banana’s (one of my favourite foods) cause cancer. I think that almost everything is carcinogenic and that perhaps scientists should spend a lot more time researching things, but more importantly the media shouldn’t pounce on one persons opinion or unpublished papers and tout them as proof. It’s highly irresponsible.

I’m off to my bunker now to escape from all these rays!