A few bad apples have spoiled the meta-studies that first touted ivermectin, the common deworming agent, as a promising treatment for COVID-19. Within weeks of being made available online, some of these clinical trial overviews were found to contain impossible numbers, unexplainable cohort mismatches, inconsistent timelines and substantial methodological weaknesses. One of these preprint analyses has since been withdrawn, whereas another has been revised after it was found to include fraudulent data. Despite the slew of serious mistakes, millions of doses of ivermectin have already been given to COVID-19 patients the world over, while others who haven't caught the virus are taking matters into their own hands and using it as a preventative, potentially endangering their health. Some scientists are now calling for an immediate remediation of the meta-analysis process to stop this from happening again. In a letter published in Nature, the authors argue we should no longer include any studies in a meta-analysis unless we have access to the raw individual patient data (IPD). If the original study authors are not willing or able to provide such detailed information, then the clinical trial should simply be excluded. Such simple standards would have stopped the meta-studies on ivermectin from ever being published, researchers say. "We recognize that by recommending IPD review... we are calling for change to nearly universally accepted practice over many decades," the authors of the letter admit, "but the consequent potential for patient harm on a global scale demands nothing less." According to one of the authors, epidemiologist Gideon Meyerowitz-Katz, the process behind a meta-analysis runs almost entirely on trust. It's simply assumed that no one will ever commit fraud, and so no checkpoints are put into place. Unfortunately, this means some meta-studies are relying on experimental data that may have never been collected. "In the case of ivermectin, we have evidence that quite a few studies in the literature that were included in meta-analyses are potentially or definitely fraudulent, and these have been included into dozens of meta-analyses without the slightest qualm for months," Meyerowitz-Katz told ScienceAlert. "It is only when you review the actual line data that you can detect fraud of this kind, therefore that needs to become standard practice." That's what happened this summer with ivermectin. In July, a few meta-analyses found evidence that the anti-parasite medicine was "very useful for controlling COVID-19 infections", but in the weeks that followed, a closer look dissolved much of the evidence base. Currently, there is no evidence that ivermectin can be used to treat COVID-19, and the wrong dosage can be downright dangerous, as the United States Federal Drug Advisory (FDA) and the Centers for Disease Control and Prevention (CDC) have repeatedly warned since August. Just this weekend, five people were reportedly hospitalized after taking the drug for COVID-19 in the state of Oregon. Not only can ivermectin cause an overdose if taken improperly, the medication can also interact with blood-thinners and cause side effects like nausea, vomiting, diarrhea, low blood pressure, dizziness, seizures, coma and even death. According to the recent Nature letter, the current pandemic "provides fertile ground for even poorly evidenced claims of efficacy to be amplified, both in the scientific literature and on social media." "This context," the authors continue, "can lead to the rapid translation of almost any apparently favorable conclusion from a relatively weak trial or set of trials into widespread clinical practice and public policy." For years now, scientists have been pointing this out, and some have been calling for updated standards to the long-accepted practice behind meta analyses. Oftentimes, one meta analysis is considered better evidence than a single, well-done clinical trial, but this isn't necessarily the case. Ultimately, the validity of a meta-analysis depends on the rigorousness of the studies sampled, and yet not all scientific journals enforce the same quality control. So while a meta-analysis can select only the best trials for inclusion, it can easily include more questionable data. And that can make all the difference. In the case of ivermectin, for instance, several meta-analyses were skewed by only a few studies with falsified or potentially falsified data. Once incorrect information is out there, it's much harder to retract or clarify. Even after certain conclusions have been shown to be baseless, it is hard work changing people's minds, as we know from our experience with vaccines. Stopping false information from leaking out in the first place is crucial, and to do that, some scientists think cracks in the meta-analysis process need to be filled. Others suggest we should do away with meta-studies entirely, as they might not actually contribute all that much too scientific progress and may, in fact, muddy the waters. (Because meta-analyses do not require original lab work, it is suspected that many such studies are done by authors who only want a publishing record.) At the very least, the authors of the recent letter say we should double-check the raw data included in a meta-analysis before we make any sweeping claims. Whether that will ever actually happen is another matter. "Our recommendations are simple, and easily adoptable," Meyerowitz-Katz told ScienceAlert. "I do not think that many people will take them up, but they absolutely could." Source
