It is necessary to differentiate between fake news, low quality evidence and insufficient evidence to guide decision making in a changing environment with an excess of information (infodemia).
By Juan Víctor Ariel Franco*
In March 2020, the former president of the United States announced that hydroxychloroquine was a “game changer,” a revolutionary, inexpensive and accessible drug to change the course of the pandemic. The basis for this claim was a very low quality study promoted by the French scientist Didier Raoult. The FDA then proceeded to authorize its emergency use for the treatment of COVID-19, which enabled its widespread use around the world.
This drug was subsequently evaluated in the rigorous RECOVERY and Solidarity clinical trials where it was shown to be ineffective, and recent analyses indicate that, including unpublished data (due to the famous “publication bias”: the results of studies with “positive” results are often published and studies with “negative” results are hidden), hydroxychloroquine would increase mortality. Moreover, Raoult’s original study would have been published in an expedited manner, evidencing unethical links between the authors and the journal editors, possibly bypassing the rigorous peer review processes that would have been expected for publication.
Evidence-based medicine (EBM) involves the conscientious, explicit and judicious use of evidence in the care of an individual patient, taking into account values and preferences and other contextual factors. The case of hydroxychloroquine illustrates the flaws in the judicious use of evidence: How certain could we have been at the time that hydroxychloroquine was an effective treatment? How reliable is the source of information? What evidence would be needed to make a better decision? These three questions would allow us to identify categories of information that often lend themselves to confusion:
- Low certainty: the body of available evidence may have deficiencies in its design, execution, precision, consistency, generalizability and availability (due to the publication bias mentioned above). The GRADE approach allows us to identify this low certainty evidence which, due to its characteristics, does not allow us to reach definitive conclusions on the efficacy of treatments.
- Unreliable (close to the concept of fake news): while the available evidence may arise from fabricated data, manipulation and other forms of academic fraud, most often the promotion of unproven treatments arises from opinion leaders based on their own perception of efficacy, often clouded by bias and frequently influenced by financial, academic or ideological conflicts of interest.
- Insufficient: there are international standards for the implementation of treatments and health interventions, based on evidence from clinical trials, observational studies or other sources of information. However, in some cases the evidence is non-existent and there are only theoretical or mechanistic speculations that support their use.
Often these categories are closely related. For example, in the case of hydroxychloroquine, there was evidence of low certainty from unreliable studies, magnified by claims of opinion leaders with a COVID-19 denialist perspective that sought to minimize the magnitude of the pandemic with simplistic solutions. In the case of chlorine dioxide, very popular among certain sectors in Latin America, there is insufficient clinical evidence; but it would not be necessary to produce clinical evidence for this substance, which is toxic to humans. Finally, in the case of ivermectin, unreliable opinions fostered by the interests of laboratories producing this drug were combined with acclaim from opinion leaders, possibly influenced by financial and academic conflicts, to promote the use of this drug based on evidence of very low certainty.
But it is not all bad news. Great efforts are being made to identify, refine and synthesize scientific information to assist decision-making. Thus, the World Health Organization (OMS) in collaboration with a group of renowned researchers in the field of EBM have produced systematic reviews (structured, reproducible and transparent syntheses) and continuously updated (living) clinical guidelines published in The BMJ, British Medical Journal of scientific information to guide decision makers in the exponential proliferation of information related to COVID-19 (the so-called infodemic).
It is important that the general public, including health professionals and other health decision makers, as well as mass media journalists, become familiar with these resources to encourage evidence-informed decision making and combat misinformation.
Suggested readings from BMJ Evidence-Based Medicine (free access)
- Misleading clinical evidence and systematic reviews on ivermectin for COVID-19.
- Publication by association: how the COVID-19 pandemic has shown relationships between authors and editorial board members in the field of infectious diseases.
*Juan Víctor Ariel Franco is the Editor-in-Chief of BMJ Evidence-Based Medicine and Full Professor of the Research Department of the Instituto Universitario Hospital Italiano de Buenos Aires (Argentina).
