New University of Oxford research shows surprisingly high numbers of out-of-control technology projects

What the customer wanted cartoonResearch conducted by two Oxford academics (Why Your IT Project May Be RiskierThan You Think) has suggested that the private sector has almost as much difficulty managing big software projects as the public sector. It also indicated that some types of projects have put companies’ survival at risk.

Whereas government departments can experience almost permanent revolution, private sector processes, in general, remain fairly stable. So it’s depressing to learn one in six of the projects they studied was a “black swan” – with a cost overruns of 200%.

The causes include: technology that doesn’t work, the difficulty in accommodating the exception cases, managing large teams, changes to the scope of the project, dealing with legacy systems, changes in legislation, and failing to build a system that meets the users’ requirements.

The researchers recommend breaking projects into smaller, more manageable units and using the best possible forecasting techniques.

There’s an additional problem: systems that work technically can still fail. If the user does not understand how to use the system, or if they don’t understand the benefits of using it, your “successful” system can end up under-used. User Assistance (online Help, Getting Started guides, screencasts and so on) mustn’t be forgotten. It’s one of those final steps in a truly successful project.

The over optimistic user

On Dara O’Brien’s Science Club (BBC 2) this week, neuroscientist Dr Tali Sharot explained “Optimism Bias”, suggesting that our brains may be hard-wired to look on the bright side.

Here is her TED presentation on the Optimism Bias:

Nearly everyone is optimistic they will never get divorced, and they are an above average driver, when statistically that’s just not possible. It seems reasonable to infer that users of software are  also over optimistic, believing they are an average or above average user in their ability to use an application.

This has an implication for those developing user documentation and training. It seems likely that most people will believe they don’t need to read the documentation (or receive training) when they actually do.

What word clouds can tell you about your user guides

Someday, some clever person will do a statistical analysis of the frequency of words contained in different user guides to help us gain a better understanding of how to make the best user guide possible. In the meantime, we can use Wordle to give us a glimpse into a few user manuals.

Wordle generates “word clouds” from text that you provide. The clouds give greater prominence to words that appear more frequently in the source text. To get it to analyse a complete guide, your documentation needs to be online and contain a RSS feed. Here are some Word clouds for three companies that provide RSS feeds to their documentation:

Autodesk (Autocad)

Mindtouch

Atlassian (Confluence)


What is surprising is the lack of commonality – very few words seem to be popular across the three examples.

How many cuddles are you putting into your user guides?

From Fast Company:

“Neuroeconomist Paul Zak has discovered, for the first time, that social networking triggers the release of the generosity-trust chemical in our brains.

The essence of affection. The cuddle chemical. In other words, oxytocin. If these changes apply in the world of social media, the implications for business — for every brand, company, and marketer trying to understand the now intimately networked world — could be significant.”

So, how much oxytocin are you putting into your user guides?

Please help us with the Cherryleaf survey

If you received an email to our two Cherryleaf surveys, please take the few minutes needed to complete it. Thanks. The surveys are to help us develop the right business strategy for Cherryleaf in the future, and they should take only a few minutes to complete.

We value your opinion, so your response would be really appreciated.

Getting the balance right between hotline support and user documentation

Many organisations find it hard to know how much time, money and effort to put into supporting their users. There’s a competition, of sorts, between the Technical Support and Technical Publications departments over how much budget they should receive. In some organisations, these departments are also competing, in a way, with content generated by users and others outside the business.

One way of looking to answer this question may be to use Web Analytics from Web based user documentation and call statistics from the Support department to see of there is a self-regulating system in play between all the different forms of user assistance.

Let’s explain what we mean by a self-regulating system, by looking at the ideas of famed scientist James Lovelock. Lovelock argues the Earth is full of many positive and negative feedback systems that lead to a long term balance or equilibrium being achieved. To prove his theory, he created a computer model called daisyworld, to show how these self-regulating systems work:

With the move to Web-based documentation, and the ability to measure its use through Web Analytics, it’s now possible to determine whether there’s a correlation between the amount of documentation and the number of support calls. Where there is a large user base, you may find the number of support calls goes up or down depending on the amount of documentation that is available. You can then decide the most cost effective way for the organisation to allocate its budgets.

It may also be the case that the amount of user generated content developed by users correlates to how much or how little official documentation is available.

Indeed, with enough data, we may be able to identify what level of user assistance, in all its forms, is needed to sustain a typical software application.

How checklists can save your life

Dr Atul Gawande is currently in London, touring the radio stations to promote his book “The Checklist Manifesto“. Dr Gawande is a surgeon in Boston Mass., who has been looking at how to deal with complexity in surgery and elsewhere.

He has discovered that complex systems work, mostly through people using checklists. Furthermore, no matter how expert you were, well-designed checklists could improve outcomes. So, with some assistance from Boeing, he developed a 90 second checklist (download it here)  that reduced surgical deaths and complications in eight hospitals around the world by more than 30%.

In the book, he shows how low-cost checklists actually work, why some make matters worse and why others make matters better. 

According to The Guardian, Dr Gawande argues that the right kind of checklist liberates rather than stifles professional intuition. A ­concise sumary of what might go wrong, and what to do if it does, galvanises groups of professionals into tighter teams. Indeed, one of the key factors, included in the checklist, was to introduce everyone in surgical team to each other: it leads to people having the confidence to speak up.

Michael Scriven, at Western Michigan University, author of a paper called The Logic and Methodology of Checklists commented,

Checklists have long been regarded as beneath the level of serious consideration by methodologists and others interested in the logic of the disciplines. But they are more sophisticated than they appear–and are perhaps the key methodology of those disciplines that really treat theory and practice as equals, e.g., surgery, engineering, neural and public economics, program and product evaluation.

For more on checklists, see:

Guidelines for Checklist Development and Assessment by Daniel Stufflebeam

The Ten Commandments, Constitutional Amendments, and Other Evaluation Checklists by Daniel Stufflebeam

Useability Evaluation Report for the Evaluation Checklist Project Web Site by Barbara Bichelmeyer

The Evaluation Checklist Project: The Inside Scoop on Content, Process, Policies, Impact, and Challenges by Lori Wingate