say that he or she sent out 5,000 surveys, received 1,000 surveys back, and based the results on those responses. You may then think, "Wow, 1,000 responses. That's a lot of data; that must be a pretty accurate survey." Wrong. The problem is, 4,000 people who were selected to participate in the survey chose not to, and you have no idea what they would have said if they had responded. You have no guarantee that the opinions of these 4,000 people are represented by the folks who responded. In fact, the opposite could be true.
Tip
What constitutes a high response rate (that is, the number of respondents divided by the number of surveys sent out)? Some statisticians would settle for nothing less than 70%, but as TV's Dr. Phil would say, statisticians need to "get real." Rarely does a survey achieve that high of a response rate. But in general, the lower the response rate, the less credible the results and the more the results will favor the opinions of those who responded. (And keep in mind that respondents tend to have stronger opinions about the issues they choose to respond to.)
REMEMBER
To watch for fake or missing data, look for information about the study including how many people were chosen to participate, how many finished the study, and what happened to all the participants, not just the ones who experienced a positive result.
Feeling the Impact of Misleading Statistics
How do misleading statistics affect your life? They can affect you in small ways or in large ways, depending on the type of statistics that cross your path and what you choose to do with the information that you're given. The most important way that statistics (good or bad) affect you is in your everyday decision making.
Think about the examples discussed throughout this chapter and how they could affect your decision making. You probably won't stay up at night wondering whether the remaining 14% of those surveyed actually microwave their leftovers. But you may run into other situations involving statistics that can affect you greatly, and you need to be ready and able to sort it all out. Here are some examples:
Someone may try to tell you that four out of five people surveyed agree that taxes should be raised, so you should too! Will you feel the pressure, or will you try to find out more information first? (Were you one of those kids that lived on the phrase "everyone else is doing it"?)
A political candidate sends you a newsletter giving campaign information based on statistics. Can you believe what he/she is telling you?
If you're ever chosen to be on a jury, chances are that somewhere along the line, you'll see a lawyer use statistics as part of an argument. You have to sort through all of the information and determine whether the evidence is convincing "beyond a reasonable doubt." In other words, what's the chance that the defendant is guilty? (For more on how to interpret probabilities, see Chapters 7 and 8 .)
The radio news on the top of the hour says cellphones cause brain tumors. Your spouse uses his or her cellphone all the time. Should you be concerned?
What about those endless drug company advertisements? Imagine the pressure doctors must feel from their patients who come in convinced by advertisements that they need to take certain medications now. Being informed is one thing, but feeling informed because of an ad sponsored by the maker of the product is another.
If you have a medical problem, or know someone who does, you may be on the lookout for new treatments or therapies that could help. The world of medical results is full of statistics that can be very confusing.
In life, you come across everything from honest arithmetic errors to exaggerations and stretches of the truth, data fudging, data fishing (fishing for results), and reports that conveniently leave out information or communicate only those parts of the results that the researcher wants you to hear. While I need to stress that not all statistics are misleading and