Practice makes the expert: The importance of training volunteers in the generation of phenological data from photographs of biodiversity observation platforms

= 45±15; untrained: �x = 34±14). According to the trained raters, only about 30% of the records for L. nepetifolia (K = 0.46, z = 28.5, p < 0.001) and N. glauca (K = 0.54, z = 26, p <0.001) cor- responded to whole plants. Discussion The reliability of volunteers in citizen science databases has been debated in many studies [50], and often the validity of the observation will depend on the complexity of the observed phe- nomena [51,52]. In this study, groups of experts and trained raters showed high reliability in classifying reproductive phenophases, indicated by high agreement among raters. However, the classification of less obvious phenophases, such as early growth, which is observed as less conspicuous and contrasting green shoots than flowers, may require greater detail and/or training time [23]. The level of agreement was significantly higher for reproductive pheno- phases among trained volunteers, compared to untrained volunteers, and for most pheno- phases the agreement rate among trained volunteers was equal to that of experts. This suggests that phenological assessments from photographs conducted by trained volunteers are as reli- able as those performed by expert scientists. Previous studies have shown that citizen scientists can collect high quality field observa- tions, good enough to be used in original analyses contributing to scientific knowledge Table 1. Comparisons and degree of agreement between raters (3 groups) phenophases (three reproductive and two vegetative) and two species (L. nepetifolia and N. glauca).

Comparison Leonotis nepetifolia Nicotiana glauca Percentage difference z-score x2 equal k0 s , Percentage difference z-score x2 equal k0 s , Immature flower Experts vs trained 0.42 -1.07 215.60��� 10.42 2.13� 12.55� Experts vs untrained 57.89 137.28��� 21.24 3.53� Trained vs untrained 58.32 336.38��� 10.82 2.36� Mature flower Experts vs trained 3.69 -0.52 206.55��� 5.04 1.03 52.46��� Experts vs untrained 53.56 7.05��� 34.24 6.02��� Trained vs untrained 57.25 14.26��� 6.83��� 31.2 Dry fruit Experts vs trained 5.72 0.8 31.86��� 12.11 2.47� 7.63� Experts vs untrained 27.77 3.60�� 15.48 2.57� Trained vs untrained 22.04 5.25��� 3.38 0.74 Initial growth Experts vs trained 22.64 3.172� 75.58��� 4.61 -0.94 212.54��� Experts vs untrained 53.03 6.90��� 61.61 10.24��� Trained vs untrained 30.4 7.27��� 66.21 14.50��� Whole plant Experts vs trained 15.61 2.19� 5.41 20.85 4.25��� 21.05��� Experts vs untrained 17.64 2.29� 12.45 2.07� Trained vs untrained 2.04 0.49 8.4 -1.84 All comparisons used Fleiss’ K for raters (expert, trained, and untrained), phenophase and species. Differences between the three rater groups within a phenophase, x2 equal k0 s was used for each phenophase, and differences between pairs of rater groups were tested with z-tests � : < 0.05 � � : < 0.001 � � � : < 0.0001. https://doi.org/10.1371/journal.pone.0282750.t001 PLOS ONE The training of volunteers as a promoter of phenological studies PLOS ONE | https://doi.org/10.1371/journal.pone.0282750 March 7, 2023 10 / 17