- Posted By: Sally Hopewell
“Is what you read the truth, the whole truth, and nothing but the truth?”
This was the title of a great OCTRU / EQUATOR (www.equator-network.org) seminar this week given by Dr Erick Turner a leading psychiatrist, who was visiting from Portland, Oregon, in the USA.
Dr Turner gave a stimulating talk about how the selective publication of clinical trials – and the outcomes within those trials – can lead to unrealistic estimates of a drug’s effectiveness. His talk focused on drugs prescribed for treating psychiatric conditions such as depression, schizophrenia, bipolar disorder and autism. Dr Turner and his team identified 74 clinical trials of 12 antidepressant drugs involving 12,546 patients registered with the FDA (Food and Drug Administration) in the United States between 1987 and 2004 (1). Of these 74 FDA registered trials - 23 (31%) were never published. Whether and how trials were published was associated with the trial outcome. Thirty-seven out of 38 trials viewed by the FDA as having positive results were published. In contrast, 33 out of 36 trials viewed by the FDA as having negative or questionable results were either not published (22 trials) or published in a way that conveyed a positive outcome (11 trials).
Problems of non-publication, or selective publication is not confined to psychiatry and is present across the medical literature. A number of research studies have identified serious reporting shortcomings in published health research. These problems are widespread and range from the failure to report and publish whole research studies depending on the nature of their research findings “publication bias” (2), to the selective publication (or non publication) of specific study outcomes “selective outcome reporting bias” (3). Even when all study outcomes are reported the interpretation of the study results is sometimes distorted by the authors of the primary study (4).
Failure to adequately report research findings distorts the reality of how the research was actually conducted. It prevents clinicians from applying effective interventions and results in vast amounts of time and money invested in health research being wasted. Incomplete or inadequate reporting also has far reaching ethical implications as the individuals who consent to participate in research, and agencies that provide funding support for these investigations, do so with the understanding that the work will make a contribution to the existing knowledge. Clearly, new knowledge that is not disseminated, or knowledge that is disseminated in a biased way, is not making a true contribution.
A major initiative dedicated to addressing the problems of selective publication and reporting of research is the registration of clinical trials in a publicity access database such as: www.clinicaltrials.gov or www.isrctn.com. Since 2005, the International Committee of Medical Journals Editors (ICMJE) requires prospective registration of all clinical trials as a condition of publication. More recently, in 2013, the Health Research Authority in the UK mandated that all clinical trials are registered prior to receiving ethical approval. In the USA this mandate has been taken a step further whereby it is a legal requirement to post trial results on www.clinicaltrials.gov within one year of completion of the trial. Similar legal requirements are being mandated through the new EU Clinical Trial Regulations (www.alltrials.net/find-out-more/all-trials/#results). Whether and how these legal requirements can be enforced and its impact of research reporting remains to be seen.
1. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008 Jan 17;358(3):252-60.
2. Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K. Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev 2009;(1):MR000006.
3. Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004 May 26;291(20):2457-65.
4. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA 2010 May 26;303(20):2058-64.