The Blunders of our Governments by the distinguished academics Anthony King and Ivor Crewe is a deeply fascinating, compelling book, one that deserves to be widely read by politicians, officials, policy wonks, journalists, students and for that matter citizens).

Its claim is that the system of government in the UK (by which they explicitly mean that centred in Whitehall and Westminster) is so flawed that successive administrations preside over policy blunder after policy blunder, to an extent far beyond the experience either of pre-1980 UK governments or many overseas governments. By ‘blunder’ they mean cock-ups that fall far short of the objectives, and greatly over-run the intended cost or fail to deliver expected savings, but they exclude both judgement calls that went wrong and scandals. Examples in the book range from the huge and prominent (the poll tax, Britain’s ERM entry and exit, the Millennium Dome, the infamous NHS IT project) to the smaller and/or less well-known (Individual Learning Accounts, the Asset Recovery Agency, the London Underground ‘public-private partnership’).

The longest section of the book is a series of case studies covering a a dozen blunders from 1979 on. These occurred in both the Thatcher and Blair/Brown governments. The accounts are based on extensive reading and also personal interviews with participants conducted by the authors. It is both highly authoritative and absolutely jaw-dropping. “How could they have been so stupid?” you think.

The final two parts of the book try to answer that question, dividing the explanation into ‘human errors’ and ‘system failures’.

The former include the fact that people in the world of policy making – as any other world – bring their prejudices to their decisions. They are as prone as anybody to group think. Importantly, there is a ‘cultural disconnect’ between people in policy and politics and many other citizens, especially in the field of welfare policy. The clever middle class folk who go to university, read books, take foreign holidays, manage their money, and lead reasonably orderly and law-abiding lives, have no conception of other ways of life in their own land. This is certainly true: I have come across this innocence fairly frequently in my brushes with the policy world, and it is extraordinary that it is possible for people to spend a career in politics or the civil service with no other experience at all. This also helps explain another of the book’s list of  typical errors, namely the absence of any experience relevant to implementing policies, which are developed in an abstract, analytical way with no thought given to how they might be put into practice, in an ‘operational disconnect.’ Implementation is, it seems, scarcely thought of in policy debate. In addition, for various reasons, symbolism and spin have come to play a big part in modern politics.

Among the ‘system failures’ are: the lack of common purpose between Whitehall departments and excessive influence of the Treasury in distorting policies from elsewhere; the massively under-resourced role of the Prime Minister (I hadn’t realised how much of an outlier the UK is in not having a prime minister’s department); the extreme frequency with which ministers change jobs (the Federal Republic of Germany has had 15 ministers of economics since 1949; the UK has had 35); the related absence ways to hold specific invividuals accountable for what happens when policies go wrong; the lack of relevant expertise in Whitehall on project management; no genuine scrutiny role for parliament, and undue speed without proper deliberation of major policy initiatives.

The overall diagnosis is of a government system populated by people who lack relevant expertise, have dysfunctional relationships (especially between civil servants and ministers), where there are no proper checks and balances, and where an extreme fear of being seen to do a u-turn paralyses sensible changes. “The truth is that, looked at close up, British government turns out to be more chaotic than dictatorial,” the authors conclude. The burdens on ministers’ time are intense; they are doing constantly. This makes the loss of trust between ministers and civil servants, on whom they used to lean, all the more damaging.

The book has some suggestions. On the ‘human errors’, there are some obvious steps. People who fancy going into politics could do another job first, in between student politics and their first think tank job or between their think tank and their first constituency. Civil servants could be required to work in local government or the health service or the private sector for several years if they want to get to the top jobs. Ministers could chair their meetings using known techniques for overcoming group think and bias – for example, creating a formal ‘devil’s advocate’ moment when the group is asked to critique a decision. Profs King and Crewe suggest greatly expanding the role for parliamentary scrutiny, especially at the pre-legislative stage. The Project Manager should become a key figure in Whitehall (and many other organisations, public and private, for that matter).

The postscript indicates that the systemic failures continue with the present government. Although the Olympics and Paralympics avoided the errors of the Millennium Dome, we have had the pasty tax and ‘omnishambles’ budget of 2012, and the Universal Credit is showing every sign of shaping up to be a blunder, the authors write.

The thing about system failures is that nobody has a strong incentive to do anything about them. The risks are high, the potential rewards minimal. They are hard to sort out because it takes a long time and involves getting a lot of people to agree to change things in ways that make their lives a bit more difficult. There is no personal risk in continuing with things they way they are now.

British political culture is not conducive to reasoned reform: Punch and Judy are the role models for debate, any sensible change of mind is pilloried as a ‘u-turn’, policy pilots and experiments have to get past the hurdle of accusations of ‘postcode lottery’ and the demand for instant results.  For all that every policy looks like a shambles, whether mini or omni, it is also hard for diligent ministers to get anything done at all – see Chris Mullin’s superb account in the first volume of his diary (A View from the Foothills) of his repeated failure as a junior minister to introduce a modest measure tackling the suburban blight of overgrown leylandii.

So it would be easy to read The Blunders of Our Governments and despair. And yet the latest survey evidence suggests that Britons are divided between anger and boredom (47% fury, 25% boredom, 16% respect, 2% inspiration) when they think about politics. Surely Something Must Be Done? I take some hope from the fact that most of the politicians and officials I’ve met over many years have been sincere in their sense of public service, no matter how clueless or ambitious or plain unpleasant they’ve been. That means there is potential for a coalition for system reform.

Another ray of hope comes from the universal lip service paid to the idea of “what works” or “evidence-based policy.” (This does of course make you wonder what previous policies were based on.) I predict some tumult ahead when politicians discover that the evidence does not in fact support their prejudices, but meanwhile, the system has created some machinery for pointing out when policy proposals are blunders in the making.

I would hope that devolution in the UK could offer another perspective on the system failures. The book notes that officials in Scotland challenged the poll tax, on the basis of how impractical it would be to implement, as their counterparts in England did not – and were ignored. In the devolved arrangements we now have a variety of experience and some natural experiments. Of course, the devolved powers vary in different areas of policy, and the social, cultural and political context of each nation is different; but if comparable officials from each of the four nations could meet to discuss policy problems, what a great forum that could be for testing proposals and sharing experience.

It also seems to me worth thinking about the training we give our politicians and officials. I don’t know how many degree courses in government or economics or public policy contain modules on policy assessment and implementation, or project management skills, but suspect the answer is not enough.

A modest start would be to put this book on all relevant reading lists.