Thoughts from the Gartner PPM Summit – risk

One of the key note speakers at the Gartner PPM Summit in London this year was Professor Bent Flyvberg of Oxford University. His talk was about “Why your IT project might be risker than you think”.  In this he summarised the outcomes of some research covering over 4000 projects, from a range of industries, with differing durations and sizes.  The first stunning finding was that, contrary to common belief, IT projects were no more risky that engineering projects – the median for cost overrun is about the same. “Good”, you might think. However that may be a premature conclusion to jump to. What they also found was that if an IT project goes out of control, then it really goes awry – big time. Further, IT projects were 100 times more likely to go out of control. It’s all to do the statistical distribution and “outliers”. Now this sounds more like the story our guts were telling us . . . . but worse.  These outliers are termed “black swans”, on the basis that you rarely see black swans (except if you live in New Zealand, of course).

Now all sectors have “black swan” projects, but the IT sector really seems to have some issues. Despite the projects being far shorter in duration, the IT “black swan projects” have significantly higher costs over runs.  This led the research team to look at a few more hypotheses:

Bigger projects are riskier than smaller ones – WRONG, they found that the risk of “cost” black swans decreased with project size.

Longer projects are riskier than shorter ones – TRUE, longer projects are riskier, especially after 3 years

Lack of benefits management is the single most important deficit in IT project performance management – TRUE, benefits management significantly reduces risk.

Using Agile methods lower project risks – TRUE .  . . but only for schedule risks, not for costs or benefit. (A bit obvious that one, as agile is predicated on a fixed timescale!).

The problem with the above is that the challenge is how to deal with the Black Swans, bearing in mind the significant impact they have. One approach is spot them early, but when is that? When is a “black swan” hatched.  Their research should four situations:

  1. As soon as a vendor has been contracted – watch those contract claims go through the roof! About a fifth are like this.
  2. When the system is being specified, after which the costs stabilise. This covers about a half of the black swan projects.
  3. Just under 10% look great right up until actual development starts and then the escalations come in.
  4. That leaves just under a fifth, which are actually starved to death.

The best early warning indicator is people insisting a project is really unique. “Unique” and “Black Swann” go together very well. Further the research indicates that the best time to look at a project is 15% of the way through; they believe that by this time the die is cast and a review may catch the renegade project before it does too much damage to the organisation.

Here is some advice and references if you want to take this further:

  • Benchmark against your previous projects, your perer industry and best practice.
  • Know your own uncertainty and risks
  • Improve resource allocation and eliminate knck on effects.
  • Scrutinize your plans.
  • Know how likely estimated costs, benefits and schedules are to actually materialise
  • Quantify project viability under different scenarios
  • Reduce the front end bias of your projects.
  • Identify and eliminate delusion, deception and black swan blindness
  • Quantify unknown unknowns
  • Think carefully about reducing scale
  • Reduce technical and social complexity
  • Learn from “master builders” who have a proven track record.

References:

  1. From Nobel Prize to Project management getting risks right. PMJ, 37(3) pp 5-15
  2. “Why your IT project might be riskier than you think” HBR, 89 (9) pp 23-25

About Robert Buttrick
Robert Buttrick is the author of the Project Workout. He has been providing advice and guidance since the publication of the first edition of his best-selling “flagship” book, the Project Workout in 1997 and now its 4th edition. The principles laid out in the publications, take a holistic view, ensuring that culture, systems, processes and accountabilities are mutually compatible. The book has been translated into French, Korean, Chinese, Russian and Romanian . . . but not yet into Latin! Robert received a Distinguished Service Certificate from BSI for services to national and international project management standards; he is a Member of the Chartered Institute of Marketing, Chartered Engineer and an Honorary Fellow of the Association for Project Management. He currently works as a consultant and is a Visiting Teaching Fellow at the University of Warwick.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: