May 23, 2023

How major medical reviews can be ‘gold standard’ evidence, yet flawed

Medical decision-making is complex. There are often hundreds, if not thousands, of published studies that may impact how to manage your medical condition

Some studies look at which drug is best in a particular situation, or whether pain is better treated by, say, avoiding exercise or seeing a physio for therapeutic massage.

In this morass of difficult choices, Cochrane reviews stand out as internationally trusted and independent. They are considered the “gold standard” in evidence-based medicine.

They involve teams of researchers looking through all the published academic research on a topic to

In other cases, there is evidence, but not from randomised clinical trials. Then the debate becomes about how much weight to give this evidence, whether and how to include it, and how to draw conclusions based on this data.

This may seem arbitrary, but there are good reasons to be wary of findings based only on observational research. A systematic review of observational trials of hormone replacement therapy led to widespread use in the late 90s for preventative health, until randomised trials showed the therapy had little to no benefit.

This isn’t actually a new problem. Indeed, it’s something Cochrane has been grappling with for years.

For example, a recent Cochrane review into vaping to help people quit smoking included quite a few non-randomised trials. These were not given the same weight as randomised research, but did provide support for the central finding of the review.

Cochrane is OK about being criticised …

There have been many issues raised with Cochrane teams over the years. This includes problems with how reviewers rate trials included in the reviews.

However, the organisation is famously transparent. If you have an issue with a particular review, you can post your comments publicly. I did this, sharing my concerns about a review on using the drug ivermectin to treat COVID.

Cochrane is also good at incorporating criticism. It even has a prize for the best criticism of its work.

… even if reviews take time

There’s a reason so many experts trust Cochrane. The occasional controversy aside, Cochrane reviews are generally the most detailed and rigorous summary of the evidence on any question you can find.

This attention to detail comes at a cost. Cochrane reviews are often the final word on a subject, not just because they are so robust, but because they take a very long time to come out.

Cochrane aims to publish reviews within two years. But more than half take longer to complete. Cochrane reviews are also meant to be updated regularly, but many have not been updated for more than five years.

In a nutshell

Cochrane reviews can be flawed, cannot answer all medical questions and, while comprehensive, can take long to complete.

But there’s a reason that these reviews are considered the gold standard in medical research. They are detailed, lengthy, and very impressive pieces of work.

With more than 9,000 Cochrane reviews so far, these are still usually the best evidence we have to answer a range of medical questions.

Gideon Meyerowitz-Katz, PhD Student/Epidemiologist, University of Wollongong

This article is republished from The Conversation under a Creative Commons license. Read the original article.

---

UOW academics exercise academic freedom by providing expert commentary, opinion and analysis on a range of ongoing social issues and current affairs. This expert commentary reflects the views of those individual academics and does not necessarily reflect the views or policy positions of the University of Wollongong.