The effect of bioRxiv preprints on citations and altmetrics

Background

Preprints are on the rise in the biological sciences, bolstered by initiatives such as ASAPbio and a backlash against traditional academic publishers (for example, the dispute betweent the University of California and Elsevier). Moreover, there is an impetus, especially amongst early career researchers (ECRs) to embrace the benefits of preprints (1).

Although preprints in the biological sciences are still playing catch-up to the field of physics, there has been a 10-fold increase in the numbers of preprints deposited per month to servers such as bioRxiv since 2013. Indeed, in their preprint Fraser et al provide further evidence to show this. This represents an increasing acceptance of preprints within the biological community and a growing movement to open access science. One of the major benefits to posting preprints is that they shorten the time to public dissemination by approximately 6 months compared to the traditional publication methods, as shown previously by Abdill and Blekhman (2) and preLighted here.

However, a key question remains: Is there a citation advantage to preprints, compared to the traditional scientific publishing route? Fraser et al set out to answer this question with their recent preprint.

 

Key findings

1. Posting preprints does not reduce the quality of the science, despite an increase in the number of deposited preprints

Fraser et al demonstrate that the increase in the number of articles deposited to bioRxiv did not result in a reduction in quality, with the number of preprints subsequently published in peer-reviewed journals remaining steady at approximately 67%. This is a highly important finding as one of the main detractions from participating in preprints is the (false) notion that they reduce the quality of science in the public domain (3).

 

Figure 1. Percent of preprints deposited to BioRxiv between 2013 and 2017 that were subsequently published. Over 50% of papers deposited to BioRxiv are subsequently published. This has remained steady since the inception of bioRxiv in 2013. Reproduced from the preprint (cropped from figure 2) under a CC BY-NC-ND 4.0 license.

 

2. Articles deposited to bioRxiv receive a citation advantage over articles published by traditional methods

The authors analysed citation data obtained from Scopus and found that bioRxiv-deposited articles had a citation advantage of 61% over non-deposited articles published in the same journals (Fig. 2). A similar preprint citation advantage was found by another study which utilised Crossref citation data (4). A recent preprint by Fu & Hughey independently provides further evidence for this citation advantage for bioRxiv-deposited articles (5).

Figure 2. Average monthly citations per paper (Cpp) for articles deposited on BioRxiv and non-deposited articles. Articles posted on bioRxiv receive a citation advantage of over 50% compared to papers published directly in journals. Reproduced from the preprint (figure 3) under a CC BY-NC-ND 4.0 license.

 

The authors investigated this citation advantage thoroughly. They first investigated if the advantage could be a result of authors selectively uploading higher-quality papers to the preprint server (Fig. 3). Using journal impact factors (IF) as a proxy for quality, the authors found that the preprint-citation advantage was present across all IFs. This suggests that the preprint-citation advantage is not due to an abnormally high quality of papers posted to bioRxiv. The authors importantly admit that IF is not necessarily a true measure of quality, but it can be considered as a proxy for the perceived quality of an article.

Figure 3. Preprints have a citation advantage across all impact factors. Reproduced from the preprint (figure 4) under a CC BY-NC-ND 4.0 license.

 

Next, the authors analysed citations of preprints before and after journal publication. Preprint citations rise in the first six months after bioRxiv deposition and fall sharply afterwards. However, unpublished preprints are cited equally, even without publication in a journal. Even after publication in a journal, preprints are still cited 24-months later, although at a low-level (Fig. 4). This is one of the few arguments used against preprints. Importantly though, the data here shows that this continued citing of a preprint after publication is not a common occurrence. Furthermore, this suggests that authors without access to paywalled articles are citing the preprint instead, potentially showing a more successful and wider dissemination of the science. Importantly, this finding may not only explain who is using preprints but how preprints are being used. These are essential metrics for promoting the use of preprint servers and open access science.

 

Figure 4. Preprints continue to be cited even after publication. (A) Preprints are cited more often than traditionally published articles (figure 5a from the preprint). (B) Preprints continue to be cited even after publication (figure 6a from the preprint). Reproduced from the preprint under a CC BY-NC-ND 4.0 license.

 

3. Altimetric data reveals greater dissemination of preprint articles compared to traditionally published articles

Fraser et al used altmeric data analysis using tweets, blogs, media articles, Wikipedia mentions, and Mendeley reads to reveal that bioRxiv-deposited articles are shared more across various platforms, particularly across social networks of scientific colleagues. However, preprints are found less in mainstream news articles, perhaps as a result of journalists relying mostly on press releases. Additionally, this altmetric advantage for bioRxiv-deposited articles persists across all journal IFs, although higher IF articles had higher altmetric counts. This is also independently supported by Fu & Hughey who find that preprints are associated with an altmetric score that is 1.53 times higher (5).

 

The bottom line: bioRxiv preprints help advance science and scientists

There is an increasingly strong movement towards open access in science. Preprint servers such as bioRxiv enable earlier and more accessible dissemination of scientific work. To further support the open access movement and promote the use of bioRxiv, it is important to measure the impact of such a service. Fraser et al demonstrate the latest attempt to assess the impact of bioRxiv and reveal an impressive citation advantage to preprint articles over traditional publication routes, as much as 52%! Many factors are involved in their analysis, but overall, there is a clear advantage to posting preprints on bioRxiv to extend the visibility and impact of new scientific work – especially essential for ECRs!

 

Open questions

1. Where is the citation advantage to preprints coming from?
   1. The authors discuss many possibilities, including research quality and citations after publication. However, these do not satisfactorily answer this question. Altmetric data (figure 7 from Fraser et al) potentially suggest that this may be due to wider dissemination of BioRxiv-deposited preprints but this remains an open question.
2. Why is mainstream media reluctant to report on preprints? Is this due to a lack of knowledge that preprints exist/are newsworthy? Or is it because the final journal name influences the reporter’s choice to cover that story?

 

 

References

1. Sarabipour S, Debat HJ, Emmott E, Burgess SJ, Schwessinger B, Hensel Z. On the value of preprints: An early career researcher perspective. PLOS Biol. 2019 Feb 21;17(2):e3000151.
2. Abdill RJ, Blekhman R. Tracking the popularity and outcomes of all bioRxiv preprints. Pewsey E, Rodgers P, Greene CS, editors. eLife. 2019 Apr 24;8:e45133.
3. Kaiser J, 2017, Am 9:00. Are preprints the future of biology? A survival guide for scientists [Internet]. Science | AAAS. 2017 [cited 2019 Jul 14]. Available from: https://www.sciencemag.org/news/2017/09/are-preprints-future-biology-survival-guide-scientists
4. Serghiou S, Ioannidis JPA. Altmetric Scores, Citations, and Publication of Studies Posted as Preprints. JAMA. 2018 23;319(4):402–4.
5. Fu DY, Hughey JJ. Releasing a preprint is associated with more attention and citations. bioRxiv. 2019 Jul 13;699652.

 

Tags: asapbio, biorxiv, preprints, publishing

Posted on: 18 July 2019

doi: https://doi.org/10.1242/prelights.12267

Read preprint (No Ratings Yet)