I Think You'll Find It's a Bit More Complicated Than That by Ben Goldacre Read Free Book Online Page B

want to show that the level of criminal activity in a group of people who have been arrested, but against whom no further action has been taken, is the same as the level of criminal activity in people who have been arrested and convicted of a crime, or who have accepted a caution.
    On page 30 they explain their methods, haphazardly, scattered about in the text. They describe some people ‘sampled on 1st June 2004, 1st June 2005 and 1st June 2006’. These dates are never mentioned again. I have no idea what their plan was there. They then leap to talking about Table 2. This contains data on people, each from a ‘sample’ in 1996, 1995 and 1994, followed up for thirty months, forty-two months and fifty-four months respectively. Are these anything to do with the people from 2004, 2005 and 2006? I have no idea, and it is impossible to tell.
    In fact, I have no idea what ‘sample’ means – perhaps that was the date on which they were first arrested. I don’t know why they were only followed up for thirty, forty-two and fifty-four months, instead of all the way from the 1990s to 2009. Crucially, I also don’t know what the numbers in the table mean, because that isn’t properly explained. I think it’s the number of people from the original group who have subsequently been arrested again, but there’s no way to tell.
    Then they start to discuss the results from this table. They say that these figures show that arrested non-convicted people are the same as convicted people. There are no statistics conducted on these figures, so there is absolutely no indication of how wide the error margins are, and whether these are chance findings. To give you a hint about the impact of noise on their data, more people are described as having been subsequently re-arrested over the forty-two-month follow-up period than over the fifty-four-month follow-up period, which seems surprising, given that the people in the fifty-four-month group had a much longer period of time in which to get arrested.
    This is before we even get on to the other problems. At a few hundred people, this study seems pretty small for one that is supposed to give compelling evidence that there is no difference between two groups, because to prove a negative like this, you generally need a very large sample, to minimise the chance of missing a true difference in the noise.
    There is no evidence that they have done a ‘power calculation’ to determine the sample size they’d need, and in any case, their comparison group feels a bit rigged to me. In their ‘convicted’ sample they only count people who had a non-custodial sentence, and exclude people who got a custodial sentence, on the grounds that those people would be incapable of committing a crime during their incarceration. This also has the effect, however, of making the ‘criminal’ group really not very criminal, and so consequently a bit more likely to be similar to innocent people.
    I could go on. Table 1 is so thoroughly ‘not as described’ as to be uninterpretable. In the text they talk about different cells on the table which are ‘solid red’, ‘stippled yellow’, and ‘blank’, when in fact the whole thing is just blue.
    This research is incomprehensible and unreadable. Anybody who claims to have been persuaded by the data quoted here is telling you, loudly and clearly in the subtitles, that they don’t need to understand – or possibly even read – a piece of research in order to find it compelling. People like this are not to be trusted, and if research of this calibre is what guides our policy on huge intrusions into the personal privacy of millions of innocent people, then they might as well be channelling spirits.

The Power of Ideas
    The Atheist’s Guide to Christmas, 2009
    I don’t mean to fill your Christmas with Aids and diarrhoea, but there is something awe-inspiring about the power of ideas alone to do great good, and great evil. Diarrhoea will be our happy ending. Aids