Jerome C. Hill: The EU Trials Tracker: a new tool to help fight publication bias

Publication bias is a topic that has already been covered on Students 4 Best Evidence in a great article here.

The concept is very simple: if all the evidence is not available to search, we cannot make evidence-based decisions regarding patient care. It is well established that trials reporting positive results are more likely to be published than studies that find no relationship.

When unpublished trials are not included in systematic reviews and meta-analysis, results may suggest a positive overall effect of an intervention when in reality there is none. Non-reporting of trials undermines our quest for evidence-based practice and has real life consequences.

Publication bias in real life

Trials of the antidepressant Reboxetine, a drug which was regulated for use due to promising trial results, epitomise these real-life implications. A systematic review in 2010 cast doubt on these promising trial results. Investigators (1) scoured through as much unpublished literature as they could get their hands on in order to produce the most comprehensive review possible. The results were worrying. Of all the studies comparing Reboxetine to either placebo or selective serotonin reuptake inhibitors (SSRIs), 74% of all the study participants in the meta-analysis were from unpublished studies. When all the available data was collated, the evidence showed Reboxetine to be no more effective than placebo, inferior to SSRIs and possibly more harmful. Patients on Reboxetine may not have been receiving the best care possible, because of the non-reporting of trials.

The conclusion: when all the data is available, our view of a drug can drastically change.

In an ideal world where science rules supreme, results from all studies should be published.

EU Trials Tracker

By law, since December 2016, all clinical trials on the European Union Clinical Trials Register (EUCTR) must report the results of their study directly to the EUCTR within one year of completion. This, in theory, should be quick and easy – direct reporting means no delay in peer and editorial review and should ensure full and complete results are produced. Sounds simple. So what’s the problem?

Here’s where the EU Trials Tracker comes in.

In September 2018, the Evidence-Based Medicine (EBM) DataLab at Oxford, headed by doctor and prominent science writer Ben Goldacre, published a paper in the BMJ: “Compliance with requirement to report results on the EU Clinical Trials Register: cohort study and web resource”(2). They demonstrated that there is significant under reporting of trials on the EUCTR. Despite the requirement being enshrined by law, only 49% of Europe’s clinical trials reported results within the year as they are required to.

Alongside this, the EBM DataLab team have released an online audit and feedback resource at EU.trialstracker.net. It keeps track of studies that are yet to report results and who the organisation or sponsor of that study is.

The goal, through auditing, is that universities and pharmaceutical companies can view their own performance, compare it to the gold standard, look at areas in which they can improve and ultimately bring about meaningful change.

And if you excuse the informality – it’s a really cool website. Easy to use, well designed and effective, I implore anyone reading this to go on and explore. You can log on, type in your institutions name, see if they are publishing trial results and whether they have any results outstanding. You can even go as far as viewing the individual trials who have not reported their results.

The EBM DataLab have created various innovative, live tools which also include the FDAAA Trials Tracker, Trials Tracker and COMPare – take a look! Here is a useful S4BE blog written by 2 students (at the time) who were part of the COMPare project.

A word from the creator

I recently got in contact with Ben Goldacre and asked him what he would say to any students who are coming across Trials Tracker for the first time, and why it is such a useful tool…

Trial reporting is a huge problem and has been ongoing for a very long time. We spent a lot of time thinking about the best way to address it with the low resources available to us. We can’t change the law; and we know that compliance with laws is, in any case, poor. We can’t impose fines on those who breach the reporting rules. We decided that public accountability was the most effective route available to us. This is often crudely characterised as “naming and shaming” but I think it’s much more positive than that. It shows those universities and companies with poor trial-reporting performance that it is possible to report all trials. Poor performers can now identify those universities and companies who are doing well, and learn from them. Furthermore, we give data on each individual overdue trial, which is a vitally important service, because trial sponsors can use our data to find the individual overdue trials that they need to get reported. We know that universities are already using our data to get trials reported, because they are contacting us, and thanking us for the service!

The FDAAA Trials Tracker received this feedback from two major US institutions, highlighting the real-world impact that these tools are having.

Take Action

Please go and check out your institutions and universities on the website. Compare against your friends and colleagues – bragging rights are up for grabs! For more information I also recommend reading the original paper for a more in-depth analysis of the results.

Non-reporting of trials is and will continue to be a problem in modern medicine, but modern initiatives such as Trials Tracker may be part of the solution.