Measuring epistemic success of a biodiversity citizen science program: A citation study

governance [5, 8, 9] is well recognized. However, the growth of citizen science practices goes hand in hand with epistemic concerns about whether and how ‘‘the public can make actual contributions to science” [10]. For instance, doubts have been raised about the quality of the data collected in crowdsourcing programs [11, 12], or about the neutrality of the participants which sometimes pursue a political agenda [13]. In this context, a fundamental question for most scientists concerns the ability of citizen science projects to deliver results that can signifi- cantly advance scientific knowledge [14–18]. In response to these concerns, many contribu- tions have been recently proposed, which aim to demonstrate the epistemic value of citizen science. This is notably the case in ecology and environmental science, where participative practices have proved their utility for biodiversity evaluation [19], the knowledge and manage- ment of natural resources [20, 21], or the assessment of ecosystemic services [21]. Given these purported successes, a more general question emerges about the conditions which maximize the scientific ‘‘impact and reuse” of citizen science data and results [22]. In this context, the systematic evaluation of the scientific outputs of citizen science projects is a central challenge in terms of how both to identify good participative practices and to increase the trust of the whole scientific community in the ability of those projects to produce reliable and useful knowledge. However, if the epistemic evaluation of research is a complex and controversial

issue in general [23], it seems to be even trickier in the case of citizen science. Among the rea- sons put forward by [24]: the results produced by citizen science projects are not always pre- sented and disseminated in the form of peer-reviewed scientific publications, which despite limitations, are normally adopted as ‘‘raw material” for scientometric studies aiming to evalu- ate scientific productivity [15]. A further reason that is advocated for is the technical difficulty of gathering large numbers of existing scientific publications produced by citizen science proj- ects, since these are rarely labeled as such in literature database such as Web Of Science or Google Scholar [25]. Recently, [15] collected 143 publications in peer-reviewed journals resulting from 23 dis- tinct citizen science projects in the field of astronomy. He conducted a citation study of these articles over a 15-year period, and found that their citation rate achieves a peak within three years from publication, by which time they have been significantly more cited than the average astronomy papers. This result is interpreted in terms of scientific ‘‘impact” both by the author and by a data-brief published in Nature Index [26]: citizen science publications in astronomy have a strong short-term relative impact, and this impact decreases on the mid and long-term. Of course, the very notion of scientific impact is an abstract construct which can endorse different meanings, and be quantified through different types of measurements, for

instance by using social network analysis [27]. However, the citation rate (the number of cita- tions of a given article in subsequently published papers) is an old tool [28] which is still largely used as a proxy for the impact of scientific production, even if it is well recognized that it con- stitutes a partial and imperfect variable for research evaluation [29], including ecological sci- ence [30]. In this paper, we aim to contribute to the debate about the evaluation of the epistemic value of public engagement in scientific research by conducting a citation study of citizen science- based publications in the field of ecology and conservation science. Following [15], we con- sider the temporal evolution of the citation rate as a proxy for the scientific impact of a given publication. However, while [15] considered an aggregate of heterogeneous citizen science projects, our study draws on one large-scale, long-term citizen science program: the Vigie- Nature network of participative observatories of biodiversity, managed since 1989 by the Muse´um National d’Histoire Naturelle (MNHN) in Paris, France. PLOS ONE A citation study of a biodiversity citizen science program PLOS ONE | https://doi.org/10.1371/journal.pone.0258350 October 11, 2021 2 / 13