Revellers dressed as Mozart (or the female equivalent) dig up a gas main in central London sometime in the 1930s. Evidence of not much at all.
"Now, what I want is, Facts. Teach these boys and girls nothing but Facts. Facts alone are wanted in life. Plant nothing else, and root out everything else. You can only form the minds of reasoning animals upon Facts: nothing else will ever be of any service to them. This is the principle on which I bring up my own children, and this is the principle on which I bring up these children. Stick to Facts, sir!"
- Thomas Gradgrind, in Hard Times by Charles Dickens (1854)
Several days before I wrote this piece, the British Government suppressed a report it had commissioned on immigration into the United Kingdom. News of this was leaked to the press, and BBC television interviewed a government spokesperson, Mr Vince Cable, Secretary of State for Business, Innovation and Skills and a Liberal Democrat politician. Mr Cable described the report as "one-sided" and was very much at ease with the fact that the Government had had it rewritten twice and was reluctant to issue the final product in any shape or form. The report (which I cannot reference) consisted of a wide-ranging survey of the evidence on immigration to the UK. According to the television news, it concluded that immigration has had a range of positive benefits for British society and economy. It also noted that immigrants use the National Health System less than indigenous Britons and contribute very much to it in terms of their skills and labour. This is in contrast to the Government's legislation to curb "benefits tourism" (a phenomenon that the report notes is largely non-existent).
The reason for mentioning this case here is that the British Government commissioned a review of evidence and then tried to alter and suppress it because the evidence ran counter to its policies. The evidence could not be used to support an anti-European or xenophobic stance, nor to increase the anxieties of voters about multiculturalism and the shortage of jobs.
As it happens, I do believe in "evidence-based practice". Logic demands that we take experience into account and that we consider all relevant knowledge pertaining to a problem before we decide how to solve it. Without such an approach, policy makers risk blundering around in the dark, and their policies risk being, at best, inefficient, and at worst downright injurious. However, there are two main problems with evidence-based practice. One concerns the nature of evidence and the other refers to the way in which it is, or is not, used.
What is evidence?
It is axiomatic that policy and practice should be based on as complete a knowledge of a problem as the evidence will allow. That is why policy formulators use academics and advisors, because they have a wide-ranging knowledge of the problem in question, its connotations and the evidence that, properly interpreted, can lead to a solution.
However, for any problem in society, economy and ecology that begs to be solved, there are at least nine important questions that may well lack an adequate answer. They are as follows.
- What exactly is evidence?
- To what extent is evidence a surrogate for direct experience, or, alternatively, how much evidence should be derived from experience and how much from indirect sources?
- How should evidence be verified?
- Leading on from the previous question, is 'evidence' merely objective data, or does it include subjective experience?
- Evidence of what? To what should the evidence be attributed?
- What is evidence capable of proving or confirming?
- What is the connection between evidence and wisdom?
- Can we do without evidence?
- Lastly, how much evidence is enough before decisions can be made?
Merely trawling for data does not adequately define the process of compiling evidence. On the other hand, the inevitable resort to selectivity risks the introduction of bias into the process of accumulating evidence. Moreover, as risk analysis involves risk perception, and as risk perception has a strong influence on how risks are communicated and managed, then subjective experience is clearly part of the 'evidence' in some way. 'Wisdom' is therefore the process of sifting and selecting evidence in an impartial and even-handed manner. In the modern world, information technology has promoted a massive return to the kinds of inductive science that were common in the times of the Encyclopédistes of the eighteenth century. Computers have taken the hard work out of blind analysis of data, but they have also removed the thinking. In intellectual terms, there is nothing more feeble and pathetic than data mining.
How should evidence be used?
There are three kinds of evidence:-
- precise and decisive
- equivocal, ambiguous and puzzling
- uninterpretable (evidence of what?).
A cautionary tale
The Irish engineer Robert Mallet developed a strong interest in earthquakes. Indeed, he is to some extent the "Father of Seismology". One of his greatest achievements was to compile all the known evidence of earthquakes into a catalogue and map. Mallet knew the location of plate boundaries before anyone knew of the existence of tectonic plates. In December 1857 the Italian region of Basilicata was struck by a major earthquake that killed about 5,000 people. Mallet organised an expedition there and assiduously collected evidence, often at great personal hardship, which he published in two volumes that have become classics of observational science (Mallet 1862). Mallet missed no piece of evidence, however trivial, but he was unable to deduce the cause of earthquakes, which he thought had something to do with subterranean steam. It took John Milne (1850-1913) to do that, in concert with a number of other scientists, using a new and more sensitive kind of seismograph.
Let's ignore the evidence
Here is a more modern case, but one that harks back in its structure to the immigration question with which I started this piece. Since 2006 there has been a set of international regulations that prohibit passengers from taking bottles of liquids larger than 100 ml onto flights. It stems from some assumptions about how liquids could be mixed on board an aircraft to make a bomb. To begin with, the standard size of bottles, at least in the European Union, is 60 an 120 ml. Hence, vast numbers of the latter have had to be thrown away at airports (the container size is what matters, not the amount of liquid inside the bottle). Secondly, in terms of concocting a bomb, 100 ml is definitely not a magic number. I asked a highly experienced counter-terrorism artificer about this and he told me that 25 ml of certain substances would be sufficient. I do not know whether one could buy the relevant substances in the airport pharmacy, having already passed security.
I mention this example because there is virtually never any attempt to evaluate the efficiency and effectiveness of counter-terrorism measures. The evidence is secret, or perhaps merely lacking.
Let's ignore the evidence when it hits us in the face
A report from the UN International Strategy for Disaster Reduction states that "The City of Venice joined the [Safe Cities] Campaign as a role model for cultural heritage protection and climate change adaptation" (UNISDR 2013). Indeed, such an example is Venice that it appeared on the cover of this document. In reality, Venice is severely threatened by the arrival of cruise ships of up to 140,000 tonnes in size, which navigate within one metre of the historical urban fabric, causing damage with their bow waves and creating a massive risk of collision and shipwreck. Despite the example of the Costa Concordia (the world's most expensive shipwreck), in 35 years of debate the city council has refused to legislate on this issue. Protests by Venetian residents have turned violent but there has been no change. Indeed, no attempt was made adequately to regulate the chaotic water transport on the Grand Canal until an eminent German was crushed and drowned in the collision between two boats. Meanwhile, the mayor, Sig. Giorgio Orsoni, has resigned after being arrested in a corruption investigation regarding the city's flood defences. So much for evidence-based practice, both within Venice and looking in.
Conclusion
Evidence-based practice is a good idea providing we are not too naive about it. Any attempt to collect, martial and interpret evidence on a particular problem needs to be transparent, fair and impartial. It must state the criteria by which evidence is included and excluded, and must ensure that an objective, balanced view of the problem is compiled. Besides the fact that they are grossly inefficient, inductive and aductive processes will not automatically ensure this. A 'blind' approach to evidence will not make it objective or comprehensive, because choices inevitably have to be made in the way that evidence is collected.
Lastly, examples described in this essay illustrate the fact that evidence alone does not "shame" policy makers into adopting a better, more objective approach. They are perfectly at liberty to use evidence selectively, or ignore it altogether.
Hence, we need an evidence-based investigation of exactly how and why policy makers ignore or manipulate the evidence.
References
Mallet, R. 1862. Great Neapolitan Earthquake of 1857: The First Principles of Observational Seismology. Chapman and Hall for the Royal Society, London, 2 vols.
UNISDR 2013. Making Cities Resilient: Summary for Policymakers. A Global Snapshot of How Local Governments Reduce Disaster Risk. United Nations International Strategy for Disaster Reduction, Geneva, 20 pp.
Wilson, H.C. 2005. Editorial. Disaster Prevention and Management 14(1).
http://www.emeraldinsight.com/journals.htm?issn=0965-3562&volume=14&issue=1&articleid=17096777&show=html (accessed 21 July 2014).
