Dataset search in biodiversity research: Do metadata in data repositories reflect scholarly information needs?

good choice. EASE is not a standard yet but could also be considered for future environmental data descriptions. In order to properly describe digital geographic data, ISO19115 is the recom- mended standard of the respective research community. Genome data could be thoroughly described using ISA-Tab. Moreover, ISA-Tab is suitable for the description of a diverse set of environmental and biomedical experiments that employ one or a combination of technologies. For more individual metadata description, but using controlled vocabularies and terminolo- gies, CEDAR could be an option. Our findings reveal that biodiversity data should not only be described from a specific research focus but should contain supplementary information. Apart from observed species, environmental information such as habitat or biome, involved biological, physical and chemi- cal processes, analyzed materials, data parameters measured and geographical information should be considered in metadata creation. PLOS ONE Dataset search in biodiversity research PLOS ONE | https://doi.org/10.1371/journal.pone.0246099 March 24, 2021 30 / 36

Challenges to be addressed in computer science Besides domain-specific metadata standards, the usage of controlled vocabularies is the second important building block for successful dataset retrieval. According to the RDA Data Maturity Model [66], controlled vocabularies ensure interoperability (I2-01M). From computer science perspective, aligned terminologies that ideally provide URIs to concepts in ontologies allow further technical improvements. Computer science research can contribute to improvements for dataset search by develop- ing methods and software tools that facilitate standard-compliant metadata provision ideally at the time of data collection, thus ensuring metadata standards to be actually used by data pro- viders. Such tools for metadata creation should provide the opportunity to use controlled vocabularies for the data description. In a best case scenario, even the raw data contain aligned terminologies facilitating automatic metadata creation. In the Life Sciences, a variety of con- trolled terminologies and terminology providers already exist. We provide a list of ontology and semantic service providers in our GitHub repository (https://github.com/fusion-jena/ QuestionsMetadataBiodiv/blob/master/biodivTerminologyServices.md). For legacy data, data structure changes are less feasable and therefore, other computer sci- ence techniques are necessary to extract additional information and to improve metadata. The text mining community has already developed various taggers and pipelines to extract organ-

isms [64], chemistry items [62] or genes [72] from textual resources. Recently, a tagger has been developed for biodiversity metadata [65] identifying environmental terms, data parame- ters, processes and materials. These annotations could support automatic facet or category cre- ation. In addition, some of these taggers already provide a linkage to the Linked Open Data (LOD) (https://www.lod-cloud.net/) Cloud and controlled vocabularies. Our results of the content analysis reveals that text mining can help to overcome the shortcomings of current metadata and support the enrichment of metadata with additional relevant information for data discovery. However, more investigations are needed to determine how many false positive annotations occur and if there is a need to further adapt existing pipelines for metadata enrichment. Semantic enrichment also allows the integration of schema.org (https://schema.org) or bioschemas.org (https://bioschemas.org/). Driven by Google and the RDA Data Discovery Par- adigm Interest Group, this community initiative aims to provide structured information for HTML web pages such as unique identifiers, persons, locations or time information. That sup- ports external search engines or data providers to crawl the landing pages of search applica- tions provided by the data repositories per dataset. Finally, with controlled vocabularies and ontological enrichment full semantic search sys- tems are possible. First approaches were mainly developed for publications in bio-medicine

and health sciences, e.g., [73–77]. Semantic search systems need controlled vocabularies for proper entity recognition and disambiguation. That enables developers of search systems in data portals to expand a search query on semantically related terms such as synonyms or more specific or broader terms [76, 77] or faceted search [74, 75, 77]. Due to these numerous studies in bio-medicine, large data repositories such as EBI [78] and NCBI [79] already provide semantic techniques in their search systems. As the timelines in Subsection “Results and Met- rics” reveal, Dryad and Figshare also started using semantic formats for metadata but so far do not offer a semantic search. To the best of our knowledge, there are few semantic search approaches focusing on dataset retrieval in biodiversity research [80, 81]. In addition, there are only very few systems that consider semantically related terms or categories in scoring [77]. Hence, as semantic search approaches are still under research in computer science, integration in current data portals has only just started. PLOS ONE Dataset search in biodiversity research PLOS ONE | https://doi.org/10.1371/journal.pone.0246099 March 24, 2021 31 / 36

Conclusion Scholarly search interests are as diverse as data are and can range from specific information needs such as searches for soil samples collected in a certain environment to broader research questions inspecting relationships among species. Our findings reveal that these search inter- ests are not entirely reflected in existing metadata. One problem are general standards that are simple and mainly contain information that support data citation. Important search interests are only be represented if keywords and suitable search terms are provided in general, non- specific fields. Most data repositories utilize these fields to enrich metadata with suitable search terms. However, if search interests are not explicitly given, facet creation, e.g., filtering over species or habitats, is more difficult. Data findability, one of the four FAIR principles [11], at least partially relies on rich meta- data descriptions reflecting scholarly information needs. If the information scholars are inter- ested is not available in metadata, the primary data can not be retrieved, reused and cited. In order to close this gap, we address challenges to overcome the current obstacles for both, the biodiversity research community and computer science. In our future work, we would like to focus on a machine-supported extraction of relevant search categories in metadata as well as an automatic filling of metadata fields from primary data. That will minimize the metadata creation process and will support scholars and data

repositories in producing proper and rich metadata with semantic enrichment. Acknowledgments The authors would like to thank the annotators for their time and valuable comments. Author Contributions Conceptualization: Felicitas Lo¨ffler, Friederike Klan. Formal analysis: Felicitas Lo¨ffler, Friederike Klan. Funding acquisition: Birgitta Ko¨nig-Ries. Methodology: Felicitas Lo¨ffler, Friederike Klan. Project administration: Felicitas Lo¨ffler. Resources: Felicitas Lo¨ffler, Valentin Wesp. Software: Felicitas Lo¨ffler, Valentin Wesp, Friederike Klan. Supervision: Birgitta Ko¨nig-Ries. Validation: Felicitas Lo¨ffler, Friederike Klan. Writing – original draft: Felicitas Lo¨ffler, Valentin Wesp. Writing – review & editing: Felicitas Lo¨ffler, Birgitta Ko¨nig-Ries, Friederike Klan. References 1. idiv. German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig; 2019. https://www. idiv.de, accessed on 11.04.2019. 2. GBIF. GBIF Science Review 2018. 2018. 3. Culina A, Baglioni M, Crowther TW, Visser ME, Woutersen-Windhouwer S, Manghi P. Navigating the unfolding open data landscape in ecology and evolution. Nature Ecology & Evolution. 2018; 2(3):420– 426. https://doi.org/10.1038/s41559-017-0458-2 PMID: 29453350 PLOS ONE Dataset search in biodiversity research PLOS ONE | https://doi.org/10.1371/journal.pone.0246099 March 24, 2021 32 / 36

4. Khalsa S, Cotroneo P, Wu M. A survey of current practices in data search services. Research Data Alli- ance Data (RDA) Discovery Paradigms Interest Group; 2018. 5. Kacprzak E, Koesten L, Iba´ ñez LD, Blount T, Tennison J, Simperl E. Characterising dataset search— An analysis of search logs and data requests. Journal of Web Semantics. 2018; https://doi.org/10.1016/ j.websem.2018.11.003. 6. Parker TH, Forstmeier W, Koricheva J, Fidler F, Hadfield JD, Chee YE, et al. Transparency in Ecology and Evolution: Real Problems, Real Solutions. Trends in Ecology & Evolution. 2016; 31(9):711—719. https://doi.org/10.1016/j.tree.2016.07.002 PMID: 27461041 7. Ramakers J, Culina A, Visser M, Gienapp P. Environmental coupling of heritability and selection is rare and of minor evolutionary significance in wild populations. Nature Ecology & Evolution. 2018;. https:// doi.org/10.1038/s41559-018-0577-4 PMID: 29915341 8. Gregory K, Groth P, Scharnhorst A, Wyatt S. Lost or Found? Discovering Data Needed for Research. Harvard Data Science Review. 2020;. https://doi.org/10.1162/99608f92.30a5e938 9. Kilicoglu H, Ben Abacha A, Mrabet Y, Shooshan SE, Rodriguez L, Masterton K, et al. Semantic annota- tion of consumer health questions. BMC Bioinformatics. 2018; 19(1):34. https://doi.org/10.1186/ s12859-018-2045-1 PMID: 29409442 10. Nentidis A, Bougiatiotis K, Krithara A, Paliouras G, Kakadiaris I. Results of the fifth edition of the BioASQ Challenge. In: BioNLP 2017. Vancouver, Canada,: Association for Computational Linguistics;

2017. p. 48–57. Available from: http://www.aclweb.org/anthology/W17-2306. 11. Wilkinson M, Dumontier M, Aalbersberg Iea. The FAIR Guiding Principles for scientific data manage- ment and stewardship. Scientific Data. 2016; 3(1):160018. https://doi.org/10.1038/sdata.2016.18 PMID: 26978244 12. Manning CD, Raghavan P, Schu¨tze H. Introduction to Information Retrieval. New York, NY, USA: Cambridge University Press; 2008. 13. Croft B, Metzler D, Strohman T. Search Engines: Information Retrieval in Practice. 1st ed. USA: Addi- son-Wesley Publishing Company; 2009. 14. AquaDiva. CRC AquaDiva; 2020. http://www.aquadiva.uni-jena.de/, accessed on 12.01.2020. 15. GFBio. The German Federation for Biological Data; 2020. https://www.gfbio.org, accessed on 12.01.2020. 16. Nature. Scientific Data, Recommended Data Repositories; 2018. https://www.nature.com/sdata/ policies/repositories, accessed on 18.12.2018. 17. Dryad; 2019. https://datadryad.org/, accessed on 16.05.2019. 18. Zenodo; 2019. https://zenodo.org/, accessed on 16.05.2019. 19. Figshare; 2019. https://figshare.com/, accessed on 16.05.2019. 20. Pangaea. Data Publisher for Earth & Environmental Science; 2019. https://www.pangaea.de/, accessed on 30.11.2019. 21. GBIF. Global Biodiversity Information Facility; 2020. https://www.gbif.org/, accessed on 12.01.2020. 22. Chapman A, Simperl E, Koesten L, Konstantinidis G, Iba´ ñez LD, Kacprzak E, et al. Dataset search: a survey. The VLDB Journal. 2019;. https://doi.org/10.1007/s00778-019-00564-x

23. Unger C, Freitas A, Cimiano P. In: Koubarakis M, Stamou G, Stoilos G, Horrocks I, Kolaitis P, Lausen G, et al., editors. An Introduction to Question Answering over Linked Data. Springer International Pub- lishing; 2014. p. 100–140. 24. Jurafsky D, Martin JH. Speech and Language Processing: An Introduction to Natural Language Pro- cessing, Computational Linguistics, and Speech Recognition. 1st ed. Upper Saddle River, NJ, USA: Prentice Hall PTR; 2008. 25. Roberts K, Gururaj AE, Chen X, Pournejati S, Hersh WR, Demner-Fushman D, et al. Information retrieval for biomedical datasets: the 2016 bioCADDIE dataset retrieval challenge. Database. 2017;. https://doi.org/10.1093/database/bax068 26. Chamanara J, Ko¨nig-Ries B, Jagadish HV. QUIS: In-situ Heterogeneous Data Source Querying. Proc VLDB Endow. 2017; 10(12):1877–1880. https://doi.org/10.14778/3137765.3137798 27. Faith D, Collen B, Arturo A, Koleff P, Guinotte J, Kerr J, et al. Bridging the biodiversity data gaps: Rec- ommendations to meet users’ data needs. Biodiversity Informatics. 2013; 8(2). https://doi.org/10. 17161/bi.v8i2.4126 28. Ariño AH, Chavan V, Faith DP. Assessment of user needs of primary biodiversity data: Analysis, con- cerns, and challenges. Biodiversity Informatics. 2013; 8(2). 29. Hersh W, Voorhees E. TREC genomics special issue overview. Information Retrieval. 2009; 12(1):1– 15. https://doi.org/10.1007/s10791-008-9076-6 PLOS ONE Dataset search in biodiversity research PLOS ONE | https://doi.org/10.1371/journal.pone.0246099 March 24, 2021 33 / 36

30. Polychronopoulos D, Almirantis Y, Krithara A, Paliouras G. Expert Team; 2013. D3.1. Available from: http://www.bioasq.org/sites/default/files/PublicDocuments/BioASQ_D3.1-ExpertTeam_final_0.pdf. 31. Thessen AE, Cui H, Mozzherin D. Applications of Natural Language Processing in Biodiversity Science. Advances in Bioinformatics. 2012; 2012(Article ID 391574):17 pages. https://doi.org/10.1155/2012/ 391574 PMID: 22685456 32. Ananiadou S, Friedman C, Tsujii J. Introduction: named entity recognition in biomedicine. Journal of Biomedical Informatics. 2004; 37(6):393—395. https://doi.org/10.1016/j.jbi.2004.08.011 33. White RW, Roth RA. Exploratory Search: Beyond the Query-Response Paradigm, Synthesis Lectures on Information Concepts, Retrieval, and Services. Marchionini G, editor. Morgan & Claypool; 2009. 34. Hearst MA. Clustering versus faceted categories for information exploration. Communications of the ACM. 2006; 49(4):59–61. https://doi.org/10.1145/1121949.1121983 35. Hildebrand M, van Ossenbruggen J, Hardman L. /facet: A Browser for Heterogeneous Semantic Web Repositories. In: Cruz I, Decker S, Allemang D, Preist C, Schwabe D, Mika P, et al., editors. The Semantic Web—ISWC 2006. Berlin, Heidelberg: Springer Berlin Heidelberg; 2006. p. 272–285. 36. Dakka W, Ipeirotis PG. Automatic Extraction of Useful Facet Hierarchies from Text Databases. In: 2008 IEEE 24th International Conference on Data Engineering; 2008. p. 466–475. 37. Xu B, Zhuge H. Automatic faceted navigation. Future Generation Computer Systems. 2014; 32:187— 197. https://doi.org/10.1016/j.future.2012.12.003

38. Moreno-Vega J, Hogan A. GraFa: Scalable Faceted Browsing for RDF Graphs. In: Vrandečić D, Bontcheva K, Sua´ rez-Figueroa MC, Presutti V, Celino I, Sabou M, et al., editors. The Semantic Web— ISWC 2018. Cham: Springer International Publishing; 2018. p. 301–317. 39. Feddoul L, Schindler S, Lo¨ffler F. Automatic Facet Generation and Selection over Knowledge Graphs. In: Acosta M, Cudre´ -Mauroux P, Maleshkova M, Pellegrini T, Sack H, Sure-Vetter Y, editors. Semantic Systems. The Power of AI and Knowledge Graphs. Cham: Springer International Publishing; 2019. p. 310–325. 40. Google; 2019. https://developers.google.com/search/docs/guides/intro-structured-data, accessed on: 20.02.2019. 41. RDA Data Discovery Paradigm Interest Group; 2019. https://www.rd-alliance.org/groups/data- discovery-paradigms-ig, accessed on: 20.2.2019. 42. Michel F, Community TB. Bioschemas & Schema.org: a Lightweight Semantic Layer for Life Sciences Websites. Biodiversity Information Science and Standards. 2018; 2:e25836. https://doi.org/10.3897/ biss.2.25836 43. Pfaff CT, Eichenberg D, Liebergesell M, Ko¨ nig-Ries B, Wirth C. Essential Annotation Schema for Ecol- ogy (EASE)—A framework supporting the efficient data annotation and faceted navigation in ecology. PLOS ONE. 2017; 12(10):1–13. https://doi.org/10.1371/journal.pone.0186170 PMID: 29023519 44. Taylor CF, Field D, Sansone SA, Aerts J, Apweiler R, Ashburner M, et al. Promoting coherent minimum reporting guidelines for biological and biomedical investigations: the MIBBI project. Nature Biotechnol-

ogy. 2008; 889. https://doi.org/10.1038/nbt.1411 PMID: 18688244 45. Sansone SA, McQuilton P, Rocca-Serra P, Gonzalez-Beltran A, Izzo M, Lister AL, et al. FAIRsharing as a community approach to standards, repositories and policies. Nature Biotechnology. 2019; 37(4):358– 367. https://doi.org/10.1038/s41587-019-0080-8 PMID: 30940948 46. Cook B, Michener W, Vieglais D, Budden A, Koskela R. DataONE: A Distributed Environmental and Earth Science Data Network Supporting the Full Data Life Cycle. In: EGU General Assembly 2012, held 22-27 April, 2012 in Vienna, Austria., p.11863; 2012. 47. DataONE. Indexer Documentation; 2019. https://github.com/DataONEorg/indexer_documentation, accessed on 20.11.2019. 48. DataONE. Quantifying FAIR: metadata improvement and guidance in the DataONE repository network; 2019. https://www.dataone.org/webinars/quantifying-fair-metadata-improvement-and-guidance- dataone-repository-network. 49. Diepenbroek M, Glo¨ ckner F, Grobe P, Gu¨ ntsch A, Huber R, Ko¨ nig-Ries B, et al. Towards an Integrated Biodiversity and Ecological Research Data Management and Archiving Platform: GFBio. In: Informatik 2014; 2014. 50. Krallinger M, Rabal O, Leitner F, Vazquez M, Salgado D, Lu Z, et al. The CHEMDNER corpus of chemi- cals and drugs and its annotation principles. Journal of Cheminformatics. 2015; 7(1):S2. https://doi.org/ 10.1186/1758-2946-7-S1-S2 PMID: 25810773 51. Lo¨ ffler F, Pfaff CT, Karam N, Fichtmu¨ ller D, Klan F. What do Biodiversity Scholars Search for? Identify-

ing High-Level Entities for Biological Metadata. In: Algergawy A, Karam N, Klan F, Jonquet C, editors. Proceedings of the 2nd Semantics for Biodiversity Workshop held in conjunction with ISWC2017. PLOS ONE Dataset search in biodiversity research PLOS ONE | https://doi.org/10.1371/journal.pone.0246099 March 24, 2021 34 / 36

Vienna, Austria: October 22nd, 2017; 2017. Available from: http://ceur-ws.org/Vol-1933/poster-paper- 10.pdf. 52. Fleiss JL. Measuring Nominal Scale Agreement Among Many Raters. Psychological Bulletin. 1971; 76 (5):378–382. https://doi.org/10.1037/h0031619 53. Gwet KL. Computing inter-rater reliability and its variance in the presence of high agreement. British Journal of Mathematical and Statistical Psychology. 2008; 61(1):29–48. https://doi.org/10.1348/ 000711006X126600 PMID: 18482474 54. Quarfoot D, Levine RA. How Robust Are Multirater Interrater Reliability Indices to Changes in Fre- quency Distribution? The American Statistician. 2016; 70(4):373–384. https://doi.org/10.1080/ 00031305.2016.1141708 55. Landis JR, Koch GG. The Measurement of Observer Agreement for Categorical Data. Biometrics. 1977; 33(1):159–174. https://doi.org/10.2307/2529310 PMID: 843571 56. The International Organization for Standardization. ISO 23081.1—s3 Terms and Definitions; 2017. https://www.iso.org/obp/ui/#iso:std:iso:23081:-1:ed-2:v1:en, accessed on 20.09.2020. 57. re3data; 2020. https://https://www.re3data.org, accessed on 15.09.2020. 58. RDA Metadata Standards Catalog, version 2; 2020. https://rdamsc.bath.ac.uk/, accessed on 15.09.2020. 59. M M. (ed.) San Diego CA: FORCE11, Data Citation Synthesis Group: Joint Declaration of Data Citation Principles.; 2014. 60. Fenner M, Crosas M, Grethe JS, Kennedy D, Hermjakob H, Rocca-Serra P, et al. A data citation road- map for scholarly data repositories. Scientific Data. 2019; 6(1):28. https://doi.org/10.1038/s41597-019- 0031-8 PMID: 30971690

61. Gaiji S, Chavan V, Ariño AH, Otegui J, Hobern D, Sood R, et al. Content assessment of the primary bio- diversity data published through GBIF network: Status, challenges and potentials. Biodiversity Informat- ics. 2013; 8(2). https://doi.org/10.17161/bi.v8i2.4124 62. Cunningham H, Tablan V, Roberts A, Bontcheva K. Getting more out of biomedical documents with GATE’s full lifecycle open source text analytics. PLoS computational biology. 2013; 9(2):e1002854– e1002854. https://doi.org/10.1371/journal.pcbi.1002854 PMID: 23408875 63. Cunningham H, Maynard D, Bontcheva K, Tablan V. GATE: A Framework and Graphical Development Environment for Robust NLP Tools and Applications. In: Proceedings of the 40th Anniversary Meeting of the Association for Computational Linguistics (ACL’02); 2002. 64. Naderi N, Kappler T, Baker CJO, Witte R. OrganismTagger: detection, normalization and grounding of organism entities in biomedical documents. Bioinformatics. 2011; 27(19):2721–2729. https://doi.org/10. 1093/bioinformatics/btr452 PMID: 21828087 65. Lo¨ ffler F, Abdelmageed N, Babalou S, Kaur P, Ko¨nig-Ries B. Tag Me If You Can! Semantic Annotation of Biodiversity Metadata with the QEMP Corpus and the BiodivTagger. In: Proceedings of The 12th Lan- guage Resources and Evaluation Conference. Marseille, France: European Language Resources Association; 2020. p. 4557–4564. Available from: https://www.aclweb.org/anthology/2020.lrec-1.560. 66. RDA—FAIR Data Maturity Model WG. FAIR Data Maturity Model, Specification and Guidelines 2020; 2019-2020. https://www.rd-alliance.org/system/files/FAIR.

PLOS ONE | https://doi.org/10.1371/journal.pone.0246099 March 24, 2021 35 / 36

74. Ernst P, Siu A, Milchevski D, Hoffart J, Weikum G. DeepLife: An Entity-aware Search, Analytics and Exploration Platform for Health and Life Sciences. In: Proceedings of ACL-2016 System Demonstra- tions. Berlin, Germany: Association for Computational Linguistics; 2016. p. 19–24. Available from: https://www.aclweb.org/anthology/P16-4004. 75. Hu W, Qiu H, Huang J, Dumontier M. BioSearch: a semantic search engine for Bio2RDF. Database. 2017; 2017. https://doi.org/10.1093/database/bax059 PMID: 29220451 76. Faessler E, Hahn U. Semedico: A Comprehensive Semantic Search Engine for the Life Sciences. In: Proceedings of ACL 2017, System Demonstrations. Vancouver, Canada: Association for Computa- tional Linguistics; 2017. p. 91–96. Available from: https://www.aclweb.org/anthology/P17-4016. 77. Ernst P, Terolli E, Weikum G. LongLife: a Platform for Personalized Searchfor Health and Life Sciences. In: Sua´ rez-Figueroa MC, Cheng G, Gentile AL, Gue´ ret C, Keet M, Bernstein A, editors. Proceedings of the ISWC 2019 Satellite Tracks (Posters & Demonstrations, Industry, and Outrageous Ideas) co- located with 18th International Semantic Web Conference (ISWC 2019) (pp. 237-240); 2019. 78. The European Bioinformatics Institute (EMBL-EBI); 2020. https://www.ebi.ac.uk/, accessed on 15.09.2020. 79. The National Center for Biotechnology Information; 2020. https://www.ncbi.nlm.nih.gov/, accessed on 15.09.2020. 80. Lo¨ ffler F, Opasjumruskit K, Karam N, Fichtmu¨ ller D, Schindler U, Klan F, et al. In: Blomqvist E, Hose K,

Paulheim H, Ławrynowicz A, Ciravegna F, Hartig O, editors. Honey Bee Versus Apis Mellifera: A Semantic Search for Biological Data. Cham: Springer International Publishing; 2017. p. 98–103. 81. Klan F, Faessler E, Algergawy A, Ko¨ nig-Ries B, Hahn U. Integrated Semantic Search on Structured and Unstructured Data in the ADOnIS System. In: Proceedings of the 2nd International Workshop on Semantics for Biodiversity co-located with 16th International Semantic Web Conference (ISWC 2017); 2017. PLOS ONE Dataset search in biodiversity research PLOS ONE | https://doi.org/10.1371/journal.pone.0246099 March 24, 2021 36 / 36