Most Clinical Research is Useless - But Not All - Zignifica

Most Clinical Research is Useless – But Not All

Clinical research is crucial for healthcare to improve.

Clinical research concerns us all.

Clinical research costs a lot of money.

Clinical research is most often useless – but not always.

It is Time to improve.

Zignifica Most Clinical Research is Useless

 

We do know how to improve the usefulness of clinical research; the challenge is that it requires a changed mindset not just from healthcare professionals, but also from the industry, politicians, and from users of healthcare – that is you and me. Clinicians and their patients often find themselves frustrated that they do not get expected results from a therapeutic intervention. This relates to the reproduction challenge in clinical research. I June 2005 wrote Professor John Ioannidis from University of Stanford a paper about this with the title “Why Most Published Research Findings Are False”, it has since been the most downloaded technical paper from the journal PLoS Medicine.

In the paper, Ioannidis wrote

There is increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims. However, this should not be surprising. It has been proven that most claimed research findings are false. Here I will examine the key factors that influence this problem and some corollaries thereof.

What he refers to is the kind of research only looking at statistical significance

…ill-founded strategy of claiming conclusive research findings solely by a single study assessed by formal statistical significance, typically for a p-value less than 0.05.

Now Ioannidis has published a new paper with the title Why Most Clinical Research is Useless. I had the pleasure of hearing Ioannidis talk about this paper at the EvidenceLive16 conference just after the paper was published. In this new landmark study Ioannidis argue, that much research that aims to improve outcomes in the clinic falls short, in part, because it isn’t designed to lead to improvements.

In order to be useful, clinical research should be true, but this is not sufficient.

Say Ioannidis in the paper, and follow up by a list of key features of useful clinical research. This include:

Problem base: that the problem is big/important enough to fix Context placement:
Has prior evidence been systematically assessed to inform
Information gain: Is the proposed study large and long enough to be sufficiently informative
Pragmatism: Does the research reflect real life. If it deviates, does this matter?
Value for money: Is the research worth the money
Feasibility: Can the research be done Transparency: Are methods, data, and analyses verifiable and unbiased
Ioannidis’ research has estimated, that following these key features are low, and that less than 1% of published research is relevant regarding “information gain”, “pragmatism”, and “value for money”.

Surprised? You shouldn’t be. Here is Ioannidis conclusion

Conclusion Overall, not only are most research findings false, but, furthermore, most of the true findings are not useful. Medical interventions should and can result in huge human benefit. It makes no sense to perform clinical research without ensuring clinical utility. Reform and improvement are overdue.

Sure “reform and improvement are overdue”, but with conflicts of interests and the need for not just deciding to do better, but for changing the mindset, it is a slow process, however, crucial.

 

The big challenge is to distinguish between useful and useless clinical research

 

You can see Ioannidis presentation from EvidenceLive16 here: