Impact of deep learning and post-processing algorithms performances on biodiversity metrics assessed on videos

PLOS One | https://doi.org/10.1371/journal.pone.0327577 August 11, 2025 11 / 14 species pairs that are confused more than others (e.g., species with similar shapes and colors, or species with the most images in the training set). In such cases the final error in species richness assessment would depend on the number and co-occurrence of these confounded species: if they are often co-occurring (i.e., present in the same habitats), confusion will not lead to bias in richness spatial patterns, but if they have different habitats, the species richness will be inflated in all habitats (i.e., the 2 species will be erroneously detected everywhere). Therefore, as there were a large number of false positives all species were eventually erroneously estimated to be present. If classification of false positives from the detection tend to be towards a small subset of species (e.g., to cryptic species because of their similarity with benthos), such overestimation of species occurrence could be lower, but will still be an issue for biodiversity monitoring. One way to limit errors for these species is to apply a filter based on the size of the detected objects. While our study may not fully represent all the real-world complexities, such as non-random error patterns or unique community structures, the restric- tion in computational resources limited the scope of scenarios we could explore. However, the simulation code is openly available and can be adapted to test additional or more complex cases.

In addition to biases in species occurrence, and hence richness of communities, detection and classification models can lead to biases in species abundance. Here we found that total abundance was overestimated by 160% when video was processed at 1 image per second by detection and classification models with low performance (recall and precision of detection both <0.8 and classification accuracy <0.8). Even with a near perfect detection model (recall and precision both 0.99) and a near perfect classification model (classification accuracy = 0.99), abundance was still overestimated by at least 128% due to false positives. Similarly, species diversity, which accounts for the evenness of species abundances, was overestimated by all models although less than species richness. This lower sensitivity of abundance-based metrics to model errors resulted from two potentially cumulative patterns. First, a species’ abundance is overestimated only if a false positive of this species is added to one of the video frames where this species was the most abundant. Second, false negatives due to detection or classification models could com- pensate for false positives. Importantly, our simulations revealed that post-processing outputs from the identification model markedly improved estimation of the three biodiversity metrics (Fig 3, S3 Fig). A detection model with moderate performance (precision and recall at 0.6) and a moderate classification model (accu- racy at 0.6) with a post-processing threshold at 0.99, which overestimated species richness by 160%, outperformed

a near-perfect detection model (precision and recall at 0.99) and a near-perfect classification model without post- processing, which overestimated species richness by 200%. Furthermore, the pipeline with models having moderate performance and a strict post-processing provided more accurate estimates of total abundance and species diversity, with error of 38% and 82% respectively, than the pipeline with high-performance models without post-processing (error of 95% and 74%) (Fig 3). This comparison highlights the importance of implementing post-processing of model outputs based on a confidence threshold. For all detection models with a given precision, the gain with post-processing was the same (Fig 4). Indeed, for a model with a recall of 0.8, Bray Curtis similarity increased from 0.83 to 0.93 with post-processing, whereas for a recall of 0.9, Bray Curstis similarity increased from 0.83 to 0.97 with post-processing. We therefore recommend improving precision rather than recall for a detection model when using a post-processing on the outputs of the classification model. This aligns with previous works (e.g., [40]), which showed that conventional metrics like classification error may not fully cap- ture a model’s ability to reliably estimate key ecological indicators. The post-processing method used in this study is similar to the one proposed by Villon et al. [26] which requires only splitting the image dataset into 2 subsets, one for training the model and one for setting the confidence threshold. Given

our findings, we recommend setting the threshold to 0.99 to minimize the number of false positives and misclassifica- tions. This aligns with previous works (e.g., [41]), which showed that applying threshold on classification outputs improve