Practice makes the expert: The importance of training volunteers in the generation of phenological data from photographs of biodiversity observation platforms

40. Aviña RV, Vargas FD, Leo´ n JFP. Graptopetalum sinaloensis (Crassulaceae), a new species from sina- loa, Mexico. Acta Bot Mex. 2020. 41. Barve V V., Brenskelle L, Li D, Stucky BJ, Barve N V., Hantak MM, et al. Methods for broad-scale plant phenology assessments using citizen scientists’ photographs. Appl Plant Sci. 2020; 8(1):1–10. 42. Fleiss JL. Balanced Incomplete Block Designs for Inter-Rater Reliability Studies. Appl Psychol Meas. 1981. 43. Gamer M, Lemon J, Fellows I, Singh P. irr: Various coefficients of interrater reliability and agreement. R package version 0.84.1. Cran. 2019. 44. Fleiss JL, Levin B, Paik MC. The Measurement of Interrater Agreement. In: Statistical Methods for Rates and Proportions. 2004. 45. Falotico R, Quatto P. Fleiss’ kappa statistic without paradoxes. Qual Quant. 2015. 46. Bolker BM, Brooks ME, Clark CJ, Geange SW, Poulsen JR, Stevens MHH, et al. Generalized linear mixed models: a practical guide for ecology and evolution. Trends in Ecology and Evolution. 2009. https://doi.org/10.1016/j.tree.2008.10.008 PMID: 19185386 47. Hothorn T, Bretz F, Westfall P. Simultaneous inference in general parametric models. Biometrical Jour- nal. 2008. https://doi.org/10.1002/bimj.200810425 PMID: 18481363 48. R Core Team (2020). R: A language and environment for statistical computing. R: A language and envi- ronment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. 2020. 49. Landis JR, Koch GG. An Application of Hierarchical Kappa-type Statistics in the Assessment of Majority

Agreement among Multiple Observers. Biometrics. 1977. PMID: 884196 50. Kosmala M, Wiggins A, Swanson A, Simmons B. Assessing data quality in citizen science. Front Ecol Environ [Internet]. 2016 Dec 1; 14(10):551–60. Available from: https://doi.org/10.1002/fee.1436 51. Feldman RE, Zˇ emaitė I, Miller-Rushing AJ. How training citizen scientists affects the accuracy and pre- cision of phenological data. Int J Biometeorol [Internet]. 2018; 62(8):1421–35. Available from: https:// doi.org/10.1007/s00484-018-1540-4 PMID: 29732472 52. Kremen C, Ullman KS, Thorp RW. Evaluating the Quality of Citizen-Scientist Data on Pollinator Com- munities. Conserv Biol. 2011. https://doi.org/10.1111/j.1523-1739.2011.01657.x PMID: 21507061 53. Cooper CB, Dickinson J, Phillips T, Bonney R. Citizen science as a tool for conservation in residential ecosystems. Ecol Soc. 2007. 54. Lindsay A. Turnbull MR and MJC. Turnbull et al. 1999. J Ecol [Internet]. 1999;87(5):899–912. Available from: https://www.jstor.org/stable/pdf/2648645.pdf?casa_token=jh3Jnd2YXygAAAAA: p0MOrl0iph48MEWcE9fO-LcwDqMLKoUDe-0Ip3T_kKIaqWt6mtvaN7jHu0lXi-O- qG2sT4vWcd8xOPspnEvJzkPvD-yJlCng7HuNXpfD7vTBKCf8bF7z. 55. Taylor SD, Meiners JM, Riemer K, Orr MC, White EP. Comparison of large-scale citizen science data and long-term study data for phenology modeling. Ecology. 2019. https://doi.org/10.1002/ecy.2568 PMID: 30499218 56. Willis CG, Law E, Williams AC, Franzone BF, Bernardos R, Bruno L, et al. CrowdCurio: an online crowd- sourcing platform to facilitate climate change studies using herbarium specimens. New Phytol. 2017.

https://doi.org/10.1111/nph.14535 PMID: 28394023 57. Geoghegan H, Dyke A, Pateman R, West S, Everett G. Understanding motivations for citizen science. UkeofOrgUk. 2016. 58. Phenology Mayer A. and Citizen Science: Volunteers have documented seasonal events for more than a century, and scientific studies are benefiting from the data. Bioscience [Internet]. 2010 Mar 1; 60 (3):172–5. Available from: https://doi.org/10.1525/bio.2010.60.3.3 59. Bonney R, Shirk JL, Phillips TB, Wiggins A, Ballard HL, Miller-Rushing AJ, et al. Next steps for citizen science. Science. 2014. 60. Project Socientize. Green paper on Citizen Science. Citizen Science for Europe: Towards a society of empowered citizens and enhanced research. Socientize. 2013. 61. McDonough MacKenzie C, Gallinat AS, Zipf L. Low-cost observations and experiments return a high value in plant phenology research. Applications in Plant Sciences. 2020. https://doi.org/10.1002/aps3. 11338 PMID: 32351799 62. Schmidt-Lebuhn AN, Knerr NJ, Kessler M. Non-geographic collecting biases in herbarium specimens of Australian daisies (Asteraceae). Biodivers Conserv. 2013. 63. Rich TCG, Woodruff ER. Recording bias in botanical surveys. Watsonia. 1992. 64. Meyer C, Weigelt P, Kreft H. Multidimensional biases, gaps and uncertainties in global plant occurrence information. Ecology letters. 2016. https://doi.org/10.1111/ele.12624 PMID: 27250865 PLOS ONE The training of volunteers as a promoter of phenological studies PLOS ONE | https://doi.org/10.1371/journal.pone.0282750 March 7, 2023 16 / 17