Menu

Close

Open Science Saves Lives: Lessons from the COVID-19 Pandemic

Lonni Besançon, Nathan Peiffer-Smadja, Corentin Segalas, Haiting Jiang, Paola Masuzzo, Cooper Smout, Maxime Deforet, Clémence Leyrat

Preprint posted on August 14, 2020 https://www.biorxiv.org/content/10.1101/2020.08.13.249847v1

To close the door on a pandemic, make science more open

Selected by Jonny Coates, Gautam Dey

Since January 2020, the world has been gripped by the COVID-19 pandemic. To date, over 34,900,000 cases have been reported with 1,034,000 deaths (1). Although an initial wave was brought under control in many countries, few have been successful in reducing the cases to negligible levels. A “second wave” is now beginning to take hold across much of Europe, with the USA having failed to control the first wave and experiencing new spikes in cases. The disease has negatively impacted economies across the world and caused the second global recession of the last 20 years.

In response to the pandemic, the scientific community has produced tens of thousands of publications, with over one-third of these shared as open-access preprints and many others being temporarily open-access at the discretion of the journal publishers (2,3). Despite this impressive response, there have been some highly questionable and poor quality scientific papers published and a misuse of the science – primarily by right-wing political and conspiratorial groups.

The traditional scientific and publishing processes are not designed to be transparent, rapid, subjected to public scrutiny or amplified by non-experts. Open science principles attempt to change this, bringing scientific publishing more in-line with a modern world. Open science is based on the viewpoint that science ownership belongs to the community and should be transparent and freely accessible. These principles promote transparent, accessible, knowledge that can be widely shared and developed through collaborative networks.

Besançon et al identify some of the challenges and problems with how the COVID-19 literature has been created and shared. They then further suggest how open science principles can be applied to resolve these issues.

Key lessons

  1.     Inappropriate study designs lead to retractions and potentially contribute to a distrust of science

One of the most widely discussed, and shared, COVID-19 papers was a study investigating the efficacy of hydroxychloroquine to treat patients with COVID-19. This study was even picked up and amplified by the president of the United States through social media channels. Unfortunately, this study suffered from serious methodological flaws.

Besançon et al investigated the COVID-19 papers that have been retracted so far. Although only 29 studies have so far been retracted, almost 30% of these were retracted due to poor study design. Moreover, the authors found a large number of duplicated trials, for example there were 218 assessing the efficacy of hydroxychloroquine by the 26th of April. This represents a large waste of research time and funding that could have been better utilised elsewhere. The authors suggest that pre-registering studies would avoid this duplication and poor study design.

  1.     Conflicts of interest and rushed peer-review reduce the trustworthiness of published results

A recent study revealed that between January and April, the median time taken to peer-review a COVID-19 manuscript was 6 days (4). Besançon et al provided an updated look at this and found that, of 8,455 publications for which they could determine peer-review time, 8% had passed peer-review the day of, or day after, submission. This generates serious concern as to how thorough the peer-review process would have been -if it even took place at all.

The authors further investigated 503 articles for editorial conflicts of interest. Approximately 40% had some kind of editorial conflict of interest. This raises important questions over fairness but also over quality. If certain authors are given priority treatment, is their work going to be sufficiently scrutinised? Unfortunately, the solution to this lies largely with journals in ensuring conflicts of interest are appropriately stated and avoided where possible. The scientific process is supposed to be objective but if the publishing process is not, then the scientific method is vulnerable.

Another key concern with many of the more questionable COVID-19 articles is the lack of availability of data. Recent examples from the Lancet and New England Journal of Medicine highlight the importance of independently assessing the raw data (5).

  1.     It is essential that science is communicated more responsibly

Many of the issues discussed in the paper (and elsewhere) ultimately relate to a lack of trust in, or misuse of, science or a lack of proper understanding of the scientific process. The final lesson that Besançon et al discuss is centred around this point. The authors argue, as we ourselves have done (2), that there is an essential requirement for science to be communicated more responsibly. This includes scientists themselves but also journalists, politicians and other policy making bodies or pressure groups and also the general public. This is a highly challenging issue to address although a variety of recommendations have previously been made (6). Despite these recommendations, the pandemic has highlighted the widening gap between scientific reality and the occasional misguided representation.

 

Identified flaw Solution(s) Responsible stakeholder(s)
Methodological and statistical issues Preregistration of studies and protocols Scientists, journals
Expedited reviewing Open & transparent peer-review, including the time taken to review each paper journals
Distrust of published results Sharing of raw data that can be evaluated by a third party

 

Full disclosure of any conflicts of interest

Journals, scientists
Surge of preprints and their misuse More reasonable and appropriate communication of scientific results

 

Specific training for journalists

Scientists, journalists, government, public

 

Importance of this work

The COVID-19 pandemic has devastated the world, claiming over 1 million lives to date and resulting in economic depressions in many countries. The only route out of the pandemic is via high-quality science. In response to the pandemic, the scientific community overall responded rapidly and openly. Unfortunately, the unprecedented level of public attention and scrutiny combined with elements of misuse and an atmosphere of distrust in science, has had profound consequences. Ensuring that we, as scientists, reflect on how to better improve communication and the use of science is fundamental to combatting and restoring trust in the scientific process.

 

Open questions

  1. The COVID-19 pandemic has been characterised by deliberate misinformation from politicians, conspiracy groups and unscrupulous scientists. How can open-science principles combat, and prevent, misinformation?
  2. public health officials, elected politicians and national health organisations. Public health teams seem poorly equipped to keep up to date with the rapidly changing literature in a comprehensive, transparent way – when this is happening it’s not clear if the reports are widely read or shared. How can we, as scientists, help to improve this situation? Are there open science principles that can be applied here?
  3. You end with a call for scientists to co-sign the manuscript as a sign of agreement. Could you report on how many have done this so far?

 

References

  1.     John Hopkins University & Medicine. COVID-19 Map [Internet]. Johns Hopkins Coronavirus Resource Center. [cited 2020 Oct 4]. Available from: https://coronavirus.jhu.edu/map.html
  2.     Fraser N, Brierley L, Dey G, Polka JK, Pálfy M, Nanni F, et al. Preprinting the COVID-19 pandemic. bioRxiv. 2020 Sep 18;2020.05.22.111294.
  3.     Fraser N. nicholasmfraser/covid19_preprints [Internet]. 2020 [cited 2020 Aug 20]. Available from: https://github.com/nicholasmfraser/covid19_preprints
  4.     Palayew A, Norgaard O, Safreed-Harmon K, Andersen TH, Rasmussen LN, Lazarus JV. Pandemic publishing poses a new COVID-19 challenge. Nature Human Behaviour. 2020 Jul;4(7):666–9.
  5.     Piller C, ServickJun. 4 K, 2020, Pm 5:30. Two elite medical journals retract coronavirus papers over data integrity questions [Internet]. Science | AAAS. 2020 [cited 2020 Oct 12]. Available from: https://www.sciencemag.org/news/2020/06/two-elite-medical-journals-retract-coronavirus-papers-over-data-integrity-questions
  6.     McNutt M. Improving Scientific Communication. Science. 2013 Oct 4;342(6154):13–13.

 

Tags: covid-19, open science

Posted on: 20th October 2020 , updated on: 22nd October 2020

doi: https://doi.org/10.1242/prelights.25388

Read preprint (No Ratings Yet)




Author's response

Lonni Besançon shared

  1. The COVID-19 pandemic has been characterised by deliberate misinformation from politicians, conspiracy groups and unscrupulous scientists. How can open-science principles combat, and prevent, misinformation?

Politicians and conspiracy groups fall beyond what Open Science can directly address. However, misinformation from unscrupulous scientists can directly be tackled by Open Science principles. In particular, open peer-review allows other scientists to show the public that the peer-review on fraudulent papers was either not of good quality or that the reviewer highlighted the issues before the publication of the manuscript. Open peer review has also been highlighted in the past to lead to better reviewing quality [A,B,C] so it should also help to prevent acceptance of misleading or fraudulent manuscripts. Post-publication peer-review is also tremendously useful to allow the scientific community to provide feedback on a manuscript and platforms such as Pubpeer have highlighted the flaws in the COVID-19 manuscripts that appear to be fraudulent or misleading as highlighted in our analysis.

In addition, full Open Access would allow everyone to have access to scientific manuscripts to check claims that they can find online. However, one has to consider, of course, that scientific communication is not easy to read and perhaps not accessible to everyone yet and scholars might have to strive to make them more accessible in the future.

  1. It seems that journalists and the public take their cues on how to engage with science from public health officials, elected politicians and national health organisations. Public health teams seem poorly equipped to keep up to date with the rapidly changing literature in a comprehensive, transparent way – when this is happening it’s not clear if the reports are widely read or shared. How can we, as scientists, help to improve this situation? Are there open science principles that can be applied here?

Open Access directly addresses this issue in the sense that it makes all scientific communications available to all for free. Yet this might not be enough to make sure that everyone and even policy makers feel engaged enough to understand the manuscripts that they would have access to. Many scientists (e.g., [D]) have argued that the way we communicate in scientific reports should evolve to be more robust and method-oriented and therefore more transparent. We report about these initiatives in a second version of the manuscript that will be online this week. But this would not address the issue at hand that manuscript might be too difficult for laypeople to comprehend. Should we switch from story-telling methodological submissions to methodology-oriented only manuscript, one could argue that we could then dedicate some time to explain our scientific reports to a wider audience via specific platforms such as wikipedia in particular. However, researchers already have a high workload and adding scientific communication to it might be challenging. Nevertheless, a lay summary should be published along any scientific papers.

 

  1. You end with a call for scientists to co-sign the manuscript as a sign of agreement. Could you report on how many have done this so far?

The call for the changes we argue in this manuscript has been co-signed by 370 researchers (341 of them, or 92.1%, holding a PhD or MD) in the allocated time given by our preregistration: 18 (4.9%) agreed only with the issues we highlighted, 4 (1.1%) agreed only with the solutions we recommended, and the vast majority, 348 (94.1%) agreed with both problems and solutions

1 comment

1 month

Ram

Great article!
(Quick FYI: there is a small correction in question 2)

Have your say

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Sign up to customise the site to your preferences and to receive alerts

Register here
Close