Regarding the publication of the recent study entitled:
“Assessing the magnitude of reporting bias in trials of homeopathy: a cross-sectional study and meta-analysis.”
A brand-new study (1) published in the BMJ Evidence-Based Medicine has addressed a frequent problem in all types of scientific and medical research: bias. Bias can distort the results of research.
In this study, two types of bias were looked for among a set of homeopathic research studies (2002 to 2021): that of “publication bias” and that of a change in the “primary outcome”. Both are so-called reporting biases.
Let us explain briefly.
Publication bias occurs when certain research is not published in the end. As you can see, this practice can be used to increase the number of studies with positive results while avoiding publishing others.
The “primary outcome” is what a study is trying to measure. For example, the primary outcome for a new cholesterol drug might be a measure of how much the bad cholesterol level falls. If this criterion is changed along the way, there is a bias. This is bad practice.
At the outset, it should be pointed out that this study by Gartlehner & al, only looked at the biases encountered in homeopathic research, without comparing it with other types of research. However, the authors themselves admit that this problem is present everywhere.
In a few lines, what do we discover…
53% of homeopathic studies are not registered. We have no point of comparison, however, with other types of research.
38% of research in homeopathy is not published, compared with 50% in conventional medicine.(2)
Changes in the primary outcome are observed
➢ in only 25% of homeopathic studies
➢ in 43% of studies in conventional medicine (3)
In short, homeopathy appears to be a better performer when compared.
What this study did not do…
Dr. Alexandre Tournier, Executive Director of the Homeopathy Research Institute, sums up what is to be learned and retained from this publication:
“Reporting bias is a well-recognized issue in all areas of medical research; so it is unsurprising that it occurs in homeopathy research. The most interesting finding of this new study, published in BMJ Evidence Based Medicine, is that we now know that homeopathy is out-performing conventional medicine in this respect, with lower levels of reporting bias. “ (4)
Biases in this study?
Clearly, when one takes the trouble to make the comparisons, this conclusion by Gartlehner et al. is only as good as its ill-founded opinion:
“Registration of published trials was infrequent, many registered trials were not published, and the primary outcomes were often altered or changed. This likely affects the validity of the body of evidence in homeopathic literature and may overestimate the true therapeutic effect of homeopathic remedies.” (5)
Gartlehner’s study, as has been observed repeatedly when it comes to homeopathy, reconfirms what Timothy Caulfield concluded back in 2005, namely that “a publication bias against homeopathy (and therefore a negative bias) exists in mainstream journals.” (6)
The point of view of the media here and elsewhere
In Quebec, Olivier Bernard, a pharmacist, made the following analysis in a interview on 98.5 on 17 March 2022:
“Homeopathic companies proceed in a way that is not rigorous, not honest, not transparent and therefore, they will not publish the results when they are negative, for example, or they will not finally register their studies, modify the protocol along the way; so, they attempt all sorts of little tricks to give the impression that it (homeopathy) works when it does not.”
In Europe, several headlines have put forward the conclusion that homeopathic results are likely to be greatly overestimated essentially because of poor research reporting practices.
Main reference and more details on this study: Homeopathy Research Institute
(1) Gartlehner G & al. Assessing the magnitude of reporting bias in trials of homeopathy: a cross-sectional study and metaanalysis. BMJ Evidence-Based Medicine, 2022; eFirst
(2) Goldacre B & al. Compliance with requirement to report results on the EU Clinical Trials Register: cohort study and web resource. BMJ, 2018;362:k3218
3 Shah K et al. Outcome reporting bias in Cochrane systematic reviews: a cross-sectional analysis. BMJ Open,
(6) BMC Complementary and Alternative Medicine 2005, 5:12 doi:10.1186/1472- 6882-5-12