The cure for pseudoscience: alternative medicine
I just saw my colleague Paul Strode, with whom I wrote a book a few years ago. Knowing my interest in pseudoscience, Mr. Dr. Science Teacher (the name of his blog) directed me to his article Acupuncture Study as a Cure for Pseudoscientific Thinking.
The article is, I think, really two articles. The first describes an experiment that his students perform, but he sets it up so that they generally overlook one important variable. The outcome of the experiment is therefore not necessarily useful. The second article, which relies to some extent on the first, is largely about acupuncture, and that seems to me to be where he gets down to brass tacks.
I will only summarize here. Mr. Dr. Strode concludes that acupuncture is better than no treatment but that it is not better than a placebo. He cites a number of studies showing that sham acupuncture, including poking with a toothpick, works as well as “real” acupuncture. He observes that acupuncture can sometimes have deleterious side effects (nocebo effect) and cites a reference to the effect that there have been five confirmed cases of death resulting from acupuncture treatment.
Finally, and perhaps this is really a third article, Dr. Strode describes perusing the website of the National Center for Complementary and Integrative Health; he is impressed primarily by “how tentative each headline is.” Then he singles out one article, not about acupuncture but rather about how “Meditation or Exercise May Help Acute Respiratory Infections, Study Finds.” May help. May help.
I did not read the original article, but Dr. Strode helpfully provides a P-value of 0.054, and an effect size of 0.043. The P-value means that there is only about a 1 in 20 chance that the claimed effect is real the result (or a more extreme result) could have happened by chance. Statisticians often define a study to be statistically significant when the P-value is less than 0.05, so this study is marginal at best.
Effect size is, in the simplest case, the ratio of the difference between the two means and the sample standard deviation. For example, if the mean of the test group and the mean of the control group differ by one standard deviation, then the effect size equals 1. The means in this study differ by 0.043 standard deviation; in other words the two means are virtually the same.
The headline says that meditation or exercise may help acute respiratory infections. Indeed they may. This study has not ruled out the proposition, but to my mind neither has it provided one whit of evidence in its favor.
Dr. Strode claims that meditation is useful anyway, and it differs from acupuncture by being free. He concludes,
In summary, we may be able to cure our students (our future voters) of pseudoscience and pseudoscientific thinking by exposing them to the claims of practices like acupuncture that masquerade as medical science and by helping them identify and unpack the pseudoscientific assertions of these practices and understand why the claims are indeed pseudoscientific.
To which I have nothing more to add.