Use Confidence Intervals in the Health Professionals
Imagine that you work for a city health department. Your city has had a chronic problem with E. coli outbreaks, particularly among children. You suspect one of the public pools (Oak Pool). If Oak Pool is causing the outbreak, you will need to close it.
You are trying to decide which of the following is closest to reality. Possible Realities:
- Swimming at Oak Pool strongly protects you from getting ill
- Swimming at Oak Pool slightly protects you from getting ill
- Swimming at Oak Pool is unrelated to getting ill
- Swimming at Oak Pool slightly increases your risk of getting ill
- Swimming at Oak Pool strongly increases your risk of getting ill
After a recent outbreak, you surveyed people who swam at various public pools. You calculated the RR of E Coli illness for swimming at Oak Pool compared to other pools. Below are five alternative results. For which results would you act to close Oak Pool? For which would you eliminate Oak Pool as a suspected cause?
RR 95% CI____
Result 1 2.3 0.2 - 12.3
Result 2 2.3 0.6 - 7.1
Result 3 2.3 0.9 - 5.3
Result 4 2.3 1.3 - 3.9
Result 5 2.3 2.1 - 2.5
You can see that the RR in each case is the same, 2.3 - suggesting that the pool is the cause and you should close it. However, you recognize that there is uncertainty in estimating the true RR, and you want to avoid making a mistake in your decision to leave the pool open or to close it. In Result 5 (RR in the range of 2.1-2.5), the range of uncertainty is small and the entire range suggests that the pool is the cause. You could be confident in your decision to close it. At the other extreme, Result 1 (RR in the range of 0.2-12.3) has a very wide confidence interval. It includes the possibility that the pool increases your risk of disease, that it is unrelated to disease, and even that the pool protects you from disease. Clearly, you could not be much confident that a decision to close the pool would be a correct decision. You need more data.
Perhaps the toughest situation is Result 3. Here the confidence interval mostly suggests increased risk, but at the lower end it overlaps 1, suggesting that the pool has little or no effect on disease risk. Some researchers argue that if the confidence interval overlaps 1, you should conclude it has no effect - that is, the independent variable is NOT related to the dependent variable. However, this is not a wise approach for health professionals, since this simply increases the likelihood of having a false negative in order to avoid a false positive. Instead, you will have to use professional judgment. What is the potential cost of waiting for more information? What is the potential cost of acting, if you turn out to be wrong (a false positive)? Don't look for simple ways to make a decision in these cases; you must use your professional reasoning.
A final note on confidence intervals. Statisticians refer results 1 - 3 as "statistically not significant" since RRs overlap 1 suggesting it has no effect - that is the independent variables is NOT related to the dependent variables and results 4 - 5 "statistically significant" since RRs don't overlap 1 suggesting it has an effect. As we have pointed out earlier, statistical significance should NOT be the only factor in making professional decisions.