Impact of deep learning and post-processing algorithms performances on biodiversity metrics assessed on videos

PLOS One | https://doi.org/10.1371/journal.pone.0327577 August 11, 2025 3 / 14 according to a user-defined criterion. This criterion can be based on a confidence score returned by the models [29] or on the size of the bounding box [30]. Computer vision models are designed to process images [31]. Hence, to process videos the first step is to extract a subset of the video frames after choosing a frame rate (e.g., 1 image per second among the 25 or 30 images per second recorded). Then, models will analyze these images one by one. The selected frame processing rate affects the number of images analyzed hence the processing time as well as the potential to accurately assess biodiversity. Increasing the pro- cessing frame rate is expected to reduce false negatives from the detection model (more frames will increase the chance to catch an individual) and hence allow censusing more species and more individuals. Meanwhile, an increasing process- ing rate is also expected to increase the number of false positives from both detection and classification models resulting in biased species richness and species abundances. However, the impact of frame processing rate on quality of detection and identification outputs is yet to be assessed. Furthermore, the ultimate goal of such automated video processing is to compute biodiversity metrics, such as total abundance, species richness, and species diversity, based on outputs from the models on the sequence of images from

the video. In more detail, the species richness will be accurately estimated as long as at least one individual of each spe- cies is correctly detected and classified in at least one frame of the video, with the condition that none of the detections are classified as a species that does not appear in the video. Total abundance estimated using the maximum number of individuals present simultaneously in a single image (i.e. maxN) can be accurately estimated only if for each spe- cies all individuals are detected and identified on at least one of the images where the maximum number is visible [32]. This metric is highly sensitive to detection, as only a few frames typically contain the maximum number of individuals, hence false negatives prevent from censusing all individuals. It is also sensitive to classification, since misclassifications (i.e., false negative) in those key frames decrease the number of individuals from a given species seen simultaneously. Hence, some biodiversity metrics are by definition more sensitive to model errors, yet such differences remain unevalu- ated to date. In the present study, we evaluate the influence of video processing rates, the performance of detection and iden- tification models, and the use of post-processing thresholds on the accuracy of biodiversity metrics estimation using simulations of remotely censused communities. We achieve this using simulated remote videos of fish communities and simulated 14,406 automated processing pipelines, and calculating error rates and biodiversity metrics. Materials and methods

Simulating video-based community surveys We considered a hypothetical case study with a regional pool of 30 species (close to the richness observed for verte- brates in terrestrial and aquatic temperate ecoregions) [24,33,34]. The 30 species were divided into 8 groups with contrasting abundance and temporal abundance variation in a site (Table 1). More precisely, groups were characterized by varying: (i) abundance, defined as the maximum number of indi- viduals seen simultaneously (ranging from 1 to 20); (ii) relative temporal occupancy, or the proportion of time they were present (ranging from 0.5% to 90%); and (iii) the clustering of their presence across time, represented by the number of time slots (from 1 to 15) when a species was visible. These groups were designed to represent the variability of abundance, mobility (i.e., average moving speed and pro- portion of time spent stationary for feeding or resting), and gregariousness observed in most animal communities sur- veyed with remote cameras (e.g., solitary fox vs gregarious mouflon, gregarious surgeonfishes vs solitary barracuda). We note that the scarcest and most elusive species is represented by a single individual recorded for only half a second over 10 minutes while the most abundant species is represented by 20 individuals visible for 2 minutes over 10 minutes and the most present species is represented by 2 individuals visible for 9 minutes over 10 minutes (S1 Fig). The number of species in each group varied from 3 to 5 species (Table 1).