Comic Relief
"Frogs when deprived of all legs become deaf.".
There once was this mad scientist. One day, he wanted to determine how far a frog can jump. So he got a frog and commanded it to jump. At once the frog jumped. The scientist measured and wrote in his journal: frog with four legs jumps 8 feet. Then he cut off the two front legs of the frog. He commanded it to jump. The frog struggled a bit but managed to jump. He wrote in his journal: frog with two legs jumps 3 feet. Once more he cut off the other two legs. He commanded the frog to jump but it just sat there. Then the scientist wrote: Frogs when deprived of all legs become deaf.
Lesson learned from the comic:
How should we interpret the study result: frogs failed to jump when deprived of all legs? To us, it is obviously a cause-effect relationship where deprived of all legs is the cause and fail to jump is the effect/outcome. But the mad scient interpret as frogs when deprived of all legs become deaf and therefore could not hear the command, consequently they could not jump. In another word, deaf is the cause of failing to jump. It is clearly a wrong or biased interpretation of the observation. Unfortunately, in many real-life research studies, biased interpretation of the study results are not so obvious to spot, yet it is one of the major cause of error in a research study.
Interpretation Bias
If you have collected, managed, entered, and analyzed your data without introducing error, your statistical results will reflect the reality of your sample. However, you may still introduce error in your interpretation of those results. In some cases, the errors are introduced from weaknesses in the study design. Others occur only when trying to draw conclusions about the true causal connection between the variables of interest. Interpretation bias usually result from:
- Lack of an adequate control group
- History Bias
- Causal direction bias
- Uncontrolled confounding
Let's now discuss each one from the above list.
Lack of an adequate control group
Lack of an adequate control group is a common bias. To understand the role of one variable in causing another, one would like to study subjects who are identical in every way except for the two variables under study. For example, in studying whether swimming in a park pond was responsible for an outbreak of illness, one would like subjects who are identical in every way except some swam and some didn't, and some became ill and some didn't. Similarly, in studying how attitudes about tobacco companies affects smoking behavior, one would like subjects with a wide range of attitudes about tobacco companies and a wide range of smoking behavior, but identical in all other respects. If the subjects are not identical in every other way, then it may be possible that these differences may be the cause of any apparent associations. In fact, most of the remaining biases relate, in one way or another, to lack of an adequate control group.
Of course, having a perfect control group is impossible. Instead, one must judge the extent the control group is inadequate, and the extent to which the researchers used appropriate statistical methods to adjust for this. In order to identify the potential problem of "Lack of an adequate control group", it is very helpful to ask ourselves this question:" is it possible to conclude what happens in the absence of the "causal" factor?".
Now let's look at a few examples and how asking the above question can help us:
Example #1:
"Pickles Will Kill You" - Every pickle you eat brings you closer to death. Pickles are associated with all the major diseases of the body. Eating them breeds wars . They can be related to most airline tragedies. Auto accidents are caused by pickles. Above conclusions are supported by the following facts/statistics:
- Nearly all sick people have eaten pickles.
- 99.9% of all people who die from cancer have eaten pickles.
- 100% of all soldiers have eaten pickles.
- 99.7% of the people involved in air and auto accidents ate pickles within 14 days preceding the accident.
- Of the people born in 1889 who later dined on pickles, there has been a mortality rate of 100%.
- All pickle eaters born between 1908 and 1918 have wrinkled skin, have lost most of their teeth, have brittle bones, and failing eyesight--if the ills of eating pickles have not already caused their death.
Eat orchid petal soup. Practically no one has as many problems from eating orchid petal soup as they do from eating pickles.
Now let's think through the above example. For each of the statistics listed above, let's ask the questions: "Is it possible to conclude what happens if people haven't eaten pickles?", you will find that we either can not conclude anything or the conclusion does not support the interpretation:
- It is impossible to know that people will or will not get sick had they never eaten a pickle;
- It is impossible to know that people will or will not die from cancer had they never eaten a pickle;
- It is impossible to know that soldiers will or will not die had they never eaten a pickle;
- It is impossible to know that people will or will not be involved in an air or auto accident had they not eaten pickles within 14 day of the travel;
- Of the people born in 1889 who never dined on pickles (probably a small number), there are probably a mortality rate of 100%;
- All non-pickle eaters born between 1908 and 1918 probably have wrinkled skin, have lost most of their teeth, have brittle bones, and failing eyesight.
Example #2:
"It is safer to drive under influence" Driving under influence will protect you from injury. Above conclusion is based on the following statistics show below:
From the chart shown above, more than three quarters of injuries happened when drivers are not under the influence and only less than a quarter of the injuries happened when drivers are under the influence.
Now let's ask the questions: "Is it possible to conclude what happens had people not been under the influence?", It is impossible to know whether people will or will not get injured had they not been under the influence.