How We Know What Isn't So by Thomas Gilovich Read Free Book Online

punishment debate are not the only ones who fail to treat supportive and antagonistic information evenhandedly. Scientists have been known to do the same when evaluating evidence relevant to their fields. The methodological critiques and publication recommendations of peer reviewers, for example, have been shown to be greatly affected by whether the results of a study support or oppose the reviewer’s own theoretical orientation. 9 Every experimental psychologist I know is much more likely to run an additional experiment if the results of an initial study refute a favored hypothesis than if the results support it. More vividly, the history of scientific attempts to relate brain size or body shape to intelligence, personality, and (often by implication) “social worth” is riddled with examples of investigators vigorously challenging and reinterpreting unanticipated results while glossing over similar flaws and ambiguities in more comfortable findings. The French craniologist Paul Broca could not accept that the German brains he examined were on average 100 grams heavier than his sample of French brains. As a consequence, he adjusted the weights of the two brain samples to take into account extraneous factors such as overall body size that are related to brain weight. However, Broca never made a similar adjustment for his much-discussed difference in the brain sizes of men and women. 10 The “criminal anthropologist” Cesare Lombroso supported his thesis about the primitive and animalistic nature of criminals and “lower races” by citing numerous examples of their insensitivity to pain—examples that he construed as courage and bravery when exhibited by a privileged European. 11
    Although the history of science contains numerous examples of an investigator’s expectations clouding his or her vision and judgment, the most serious of these abuses are overcome by the discipline’s insistence on replicability and the public presentation of results. Findings that rest on a shaky foundation tend not to survive in the intellectual marketplace. To a lesser extent, the same is true with regard to beliefs formed in everyday life: Some of our most erroneous beliefs are weeded out by the corrective influence of our peers and society at large (although see Chapter 7 for a discussion of the limits of this phenomenon in everyday life). The biggest difference between the world of science and everyday life in protecting against erroneous beliefs is that scientists utilize a set of formal procedures to guard against the sources of bias and error discussed in this book—a set of procedures of which the average person is insufficiently aware, and has not adequately adopted in daily life. Scientists employ relatively simple statistical tools to guard against the misperception of random sequences discussed in Chapter 2. They utilize control groups and random sampling to avoid drawing inferences from incomplete and unrepresentative data (Chapter 3). They use “blind” observers as one way of eliminating the influence of the biased evaluation processes discussed in this chapter. *
    But perhaps the most fundamental safeguard of the scientific enterprise is the requirement that the meaning of various outcomes be precisely specified (in advance if possible) and objectively determined. If a scientist sets out to test the ability of subliminal self-help tapes to improve the productivity of salespeople, he or she would doubtless focus on actual sales volume, and would ignore the claims of enhanced confidence, improved poise, and increased energy from those who were exposed to the tapes. (If such testimonials were to be used at all, it would be as suggestions for further hypotheses that would themselves be subjected to rigid test.) This kind of precise specification of what constitutes “success” and “failure” is something we rarely do in everyday life, and consequently our preconceptions often lead us to interpret the meaning of various