A really expensive experiment

I was speaking at a PMI conference in Sweden and took the opportunity to listen to some of the other speakers, one of whom was Henrik Kniberg. Henrik is a guru on agile and lean and he told the story of a rather large public sector project in Sweden. The inadvertent experiment was that they did the same project TWICE!
To start with, Henrik discussed what “success” is and he defined it as being happy customers, happy users and happy development team. With apologies to PMI’s “triple constraint” he said to forget about time, cost and scope . . . actually this bit of the talk mirrored the start of my own talk at that conference, so “Mr Classic PM” and “Mr Agile” are on the same page. Why you undertake work is more important than what or how you deliver.

Henrik's different perspectives on success, over and above the "triple constraint". This thinking matches my thinking in Project Workout.

Henrik’s different perspectives on success, over and above the “triple constraint”. This thinking matches my thinking in Project Workout.

Henrik’s case study was all about improving society by helping the Swedish police force work more effectively. You’ve all read those dark, gloomy Scandinavian crime thrillers, so you get the idea. As we were in Sweden in “Wallender” country, it all seemed appropriate.
So, the first project was like this:

  • Mission Improve society
  • Requirements Unclear
  • Delivery Agile (= Iterative/incremental)
  • User involvement Continuous
  • Team’s influence over technology choices High
  • Technical platform Java

It took two years and comprised a series of releases, starting with the first one after 6 months, which the police hated, then after 18 months a nation-wide release and ending up with something they all really liked. SUCCESS!

Then “someone” had a bright idea to rebuild the whole thing in Oracle Siebel as Oracle said it would be better and cheaper. And as they were in a hurry to save money, they would go “big bang”. After all, they knew the requirements as the project had already been completed. So this is project 2:

  • Mission Reduce maintenance cost
  • Requirements Clear
  • Delivery Big bang
  • User involvement None
  • Team’s influence over technology None
  • Technical platform Oracle Siebel

This turned out to be a train wreck in slow motion:

Stakeholder response: outrage
Cost 200m KR (double that of project 1)
Time 24 months (as opposed to 18 months for project 1
Measured impact: Police blocked from using it several hours a day; error rate increased.

Lesson to be learned

So, the lessons are:

  • Focus on solving user needs not just cost: effective nearly always out-strips efficient;
  • Deliver iteratively and incrementally if you don’t know what you really need;
  • Involve real users.

Beware of standard platforms:

  • They might not be as standard as you think;
  • Listen to your technical people not just to a vendor selling stuff.

How many of you are building major capabilities on the promise of using a “standard” platform? I can certainly think of one I was involved with recently!

How do you think the re-platforming should be handled? . . or even should it be done at all, if what is there isn’t broken?

What about migrating from one to the other? Should we always design this in from the start? Or is it like turkeys voting for Christmas?

So agile is always better, is it?

Oh, and the stuff about ‘agile’ and ‘big bang’? That is just a red herring. You should use whatever delivery approaches are appropriate and proportionate to what you are doing. I expect some people reading this have had senior executives issue an edict: “do all projects agile!” This approach is just as mistaken as not trying agile at all – it underlines the classic IT confusion between a development approach and a project method. It’s not a case of ‘agile’ versus ‘project management’. Nor of thinking ‘waterfall = project management’. ‘Agile’ and ‘waterfall’ are delivery methods. Project management is a wrapper, which can contain any number of delivery methods, depending on the type of deliverables being produced.

Huge scale agile – a case study

Eating off the floor

There is often a debate about how agile delivery fits with other approaches and a conference I went to earlier in the year was no exception. In this conference, CMMI Made Practical, people were asking should we “do CMMI” or “do Agile”. This question mirrors the one I often hear about: “Should we do PRINCE2 or Agile”? It’s a question that actually misleads people. At the panel session for the conference wrap up, one panelist (me!) said that comparing CMMI and Agile or comparing PRINCE2 and Agile is a nonsense; it’s like comparing tables and chairs – they are different; but you can use them together, unless you want to eat off the floor”.

The distinction is very clear:

  • “Agile delivery” is done at work package level, with each sprint a representing a work package;
  • Each “Release” can be represented as a project stage;
  • Classic project management is the governance wrapper.
  • The early project stages are about business need, strategic fit and architecture.
  • The later project stages are about validation and deployment.
  • The agile bit is in the middle.
  • A project may comprise both agile and classic work packages.
  • Some good agile practices and are good project practices.

Taking classic and agile to an extreme

At that conference, I learnt about an organisation which had taken the above concepts to an extreme. I first came across it last November, but only realised who the company was at last week’s conference, when they presented their case study – SITA.

This is not just about software development but truly complex, platform based systems engineering. In other words it is something and large technologu based company might take note of and do some “health checks” on.

Here are some sizing metrics:

  • Distributed workforce in London, India, USA in  18 locations
  • Five primary system domains
  • 52,482 function points
  • Peak staffing: 703
  • Peak scrum teams: 61

So, it’s complex, distributed and off-shored. And in case you doubt it, this system does everything for 90% of airlines around the world before you get to the airport, at the airport, in the air and getting out of the airport, serving both the consumer directly as well as the airlines.

What is their key?

At the core of what they have done is created a classic project management wrap around agile delivery. They told me afterwards, that if anyone thinks they can do “agile” on this scale without without project management, they just prove they haven’t a clue. So what have they done?

At the heart is the Scrum, with its daily inspections, 2 weekly sprints and teams of about 8 people. A classic project stage is made up of 6 sprints (3 month long stages). They have the normal, input of prioritised requirements and an output of working software, giving them an increment in functionality. Within a sprint they do all the expected continuous integration and unit testing. In other words, good scrum stuff.

However because of the size and complexity of the system this is not enough. They have 5 streams of sprints going on at any time, with start and end points synchronized. They need to be able to draw these together and so before any agile development happens, they need to work through the requirements and prepare or update the architecture. Having a strongly managed architecture is key. Interfaces matter.

After the agile teams have delivered “working software”, there are four more project stages:

  • Non-functional tests, integration and performance tests
  • User acceptance test
  • Operational acceptance tests leading to delivery into production.

That is to say, it is a 7 stage project, with the third stage being agile (and sometimes the second also being agile).

The seven project stages.Source Presentation, CMMI Made Practical 2012, Lamri

The seven project stages.
Source Presentation, CMMI Made Practical 2012, Lamri

To make sure that the software coming out of the teams works, each team has a sandbox environment, which replicates the full build at that point in time to check actual interfaces. This means there are fewer defects needing to be trapped in the later formal verification stages. They say this is expensive on environments, but what is the point of writing something that either fails or simply won’t sit on the final hardware with everything else. As you can imagine, change control and configuration management are vital.

They have a core architecture team who continuously tune the architecture as more information emerges. Architecture is designed to minimise system interfaces. Often requirements for particular sprints are driven by this team in order to ensure interfaces are built in a co-ordinated way across sprints and that sets of software can be deployed in a useful way. Working software, in isolation, is useless software, unless it fits into a wider, usable application or system.  Inevitably there is rework resulting from changing architecture, so their sprints have time slots for making sure this happens. They are also clear with their suppliers that rework resulting from architecture changes are different to those resulting from poor workman ship. They expect 20 to 30 % rework but as a result of managing this, defects are trapped far earlier and overall delivery speeds up. They never leave rework for “sprint 6”, their last sprint. The approach of leaving all this until later in their view is counter productive.

Is it a success?

They say it is. By using their approach:

  • Their delivery schedule has reduced by 50%
  • Cost per function point down 43%
  • Defect density reduced by 40%

Sounds good to me; what do you think?  Have you any stories to support or challenge this?

Agile Delivery in Large Enterprises

There is no point in speeding if you are on the wrong road.

There is no point in speeding if you are on the wrong road.

Recently, I went to an “Agile Edge” conference at Valtech to hear Greg Hutchings talk on “Introducing Agile”. Here is what I learned.

You must know WHY you want to “do Agile”
The normal reasons people state for wanting agile are to:
– reduce time to market
– be more flexible
– be more efficient (less cost)
– increase quality.
All very laudable and valid reasons but Greg said there was one, less quoted, which he believed had the greatest leverage  – to increase customer intimacy. By working with customers, you build up a lasting relationship which can survive many of the knocks of corporate life. Business is, at heart, about people working with people. Efficiency is not generally a compelling case for Agile; flexibility and time to market are usually better. If you launch your services early, then your benefits flow earlier, often dwarfing the cost aspects (although costs centre accountants might not look at it that way!).

One thing Greg warned of was not to aim for all those benefits; you’ll just fail. His advice was to choose just one, then aim and focus on it; keep an eye on the others, but don’t let them drive you. He also warned that some things may get worse, but I am sure you “change-savvy” readers know all about that.

Who wants it anyway?
Greg’s key message was, if there is no-one in the Executive (top) level of the company who wants Agile, don’t waste your time. Successful implementations should initially come top down, with the senior leadership team signed up and then management trained on what it’s all about. You can then come bottom up with the training of the practitioners. Why? Agile relies on the right behaviours which, in some organisations, can look very strange or even appear subversive!  (See case study 2, later.)  You also need to make sure your sales force and customer services people are trained, so they can explain effectively to customers what Agile is all about. Finally, traditional contracts may no longer be appropriate as they tend to build in rigidity which works against the flexibility of Agile. Naturally, the customers also have to be involved; if they aren’t, you have no “customer intimacy” and Agile becomes futile!

Big bang or incremental adoption?
Incremental every time! In this way you have the space to “inspect and adapt” your approaches to suit your organisation. Greg did come across a company where they successfully implemented a pilot in three months and the senior leadership team was so impressed, the rest of the company was instructed to roll it out in the next three months. The manager commented that he did it . . . . but it was nowhere near as well done as the first tranche, as he was pushed to hit schedule deadlines rather than make sure it was right. It seems his leadership team had the view that new methods can be turned on and off like a switch. Will we ever learn?

There is no substitute for face to face working
One question from the floor concerned the trend for distributed, remote and home working. Greg was very clear that despite having wikis, teleconferencing, and all that stuff, there is no substitute for face to face work for critical activities. The hidden cost to an organisation of not letting people truly work together can be vast.

Case study 1 – FAILURE
Agile was implemented bottom up on an incremental basis. There was no top-down support. The consultants were removed from the company as what was exposed during the implementation work was too embarrassing for the incumbent Vice President. The implementation had no top down sponsorship nor effective governance. Most of the staff involved were fired. Six months later the Vice President in charge of the area lost his job for covering up some critical business issues. That company now has new leadership and is getting on much better . . . and adopting Agile.

Case study 2 – you’ll die waiting
The second case study is a large, global industrial company. They had executive sponsorship, a core team to implement it but very few people engaged in the outer global reaches of the company. After four years, they decided that Agile was a valid approach to IT development and may be used!

Case study 3 – A large telco – SUCCESS!

This organisation had executive sponsorship, a core team to lead and manage the implementation and good geographic representation. They focussed on product development but were careful to choose which developments in their portfolio would benefit most. They discounted the products at the end of their life-cycle, the cash cows and the highly innovative. They focussed on the others.

. . . . and then Greg ran out of time.

Summing it up
My own view is that I can’t see many customers wanting a fixed timescale and price for a variable output from their contractors. So, going in with the approach with an unconvinced customer may be dodgy. Often, however, a company will have many long term contracts which include “future services” clauses to cover stuff the customer wants but hasn’t a clue as to what the real requirements will be. Surely this is the opportunity space for using Agile?

I want to run an agile project

Agile. Do you remember that it used to be called “RAPID” but senior managers took that too literally and thought that if you want it fast, then use the RAPID approach to software development, regardless of whether it was appropriate. RAPID, soon become rabid.

Then it was called Dynamic Systems Delivery Method (DSDM) or something like that. Very memorable, just like all these acronyms. The sort of branding that makes people say “Eh?”.

Then “Agile” came along with its manifesto. Like the word “flexible” what senior executive or business leader can argue with it? They are not going to say “We want to be an inflexible and un-agile company”, are they? It’s great branding . . . . but do we have the same problems that the RAPID people had . . . perhaps – you let me know.

What happens to people who want to practice Agile in companies that don’t want it . . . or in companies which pretend they want it? Or, where the executive don’t really know what it is, but it sounds trendy? Have a look this video  and enjoy!

It’s a sad tale of a commonly seen situation, but frankly, I think the “Agile delivery manager” in the film, certainly wasn’t using the right approach to engaging his stakeholders. His approach is almost guaranteed to annoy even the most placid manager. At no point does he justify why an agile approach is a good idea in his situation . . . he just goes on and on, like a cracked record. He may have been right, but sometimes you need to be right in the right way!

. . . . . . If you can stand it, there is even a follow-on video on how the “(anti)hero” gets on in a new company, where he was recruited specifically for his agile delivery skills (he still seems to lack the required inter-personal skills though!).

Now, if you are interested in Agile and how it fits with “classic” project management, then let me know in the response and I’ll share some work I’ve done recently. I’ll even challenge to concept of an “agile project”!

The National Audit Office (UK) and Agile

I’d like to draw attention to a report, published by the UK’s National Audit Office on Governance for Agile delivery, which may be found here: http://www.nao.org.uk/publications/1213/governance_for_agile_delivery.aspx. This includes a contribution from a company I work closely with, along with advice from other private sector companies.

This is one of several reports The NAO intends to publish to help the pursuit of the UK government’s IT strategy and in particular, reducing delivery time by 20% by 2014 and using Agile delivery as a technique, in half of their major ICT-enabled change programmes by April 2013.

If you have any comments on this report, add to the discussion below.