Predicting genetic biodiversity in salamanders using geographic, climatic, and life history traits

important predictor. However, latitude was not included in the list of predictor variables that were likely to be important (Fig 4). Predictive modeling as a tool to address the Linnean shortfall Recently, Parsons et al. [35] used publicly available genetic barcoding data to develop a predic- tive framework to identify mammalian clades most likely to contain hidden species and deter- mine specific trait complexes that indicate where hidden mammal diversity is likely to exist. We adopted a similar approach to evaluate undescribed genetic lineages in the clade Caudata, a group which differs from mammals in several key aspects, including species richness and sampling intensity. We focused on a lower taxonomic level so there are fewer recognized spe- cies of salamanders (<1000; [57]) compared to the mammal dataset, making the ability to pro- duce robust predictive models more challenging. Additionally, there was a smaller proportion of available data for salamanders than mammals (~10% compared to 60% of described spe- cies). However, these smaller datasets might be more realistic in that they are more representa- tive of the type of data most likely to be available for the taxonomic groups that are in greatest need of attention from taxonomists. While the random forest models generated in this study actually have a higher overall accu- racy than those used in Parsons et al. [35] (see Table 3), relying on this metric alone to evaluate the performance of predictive models can be misleading [87–89]. For classification models,

model accuracy depends on how well the predicted classifications match the observed classifi- cations. While seemingly straightforward, accuracy does not account for other model charac- teristics that may be influencing model behavior, such as the class frequencies of the underlying dataset [87]). In cases where one class occurs at a much higher frequency than the other, a predictive model can attain a high accuracy by simply always predicting the higher class. Therefore, an important benchmark to consider when interpreting overall model accu- racy is the frequency at which the majority class occurs, the no information rate (NIR) [88]. If a model’s accuracy is not significantly higher than the NIR (i.e., p-value [Accuracy > NIR]), it can remain unclear whether the model is making meaningful decisions. In our models, the overall accuracy was found to be high, but the 95% confidence intervals for the accuracy values are very wide for most of the models. In addition, because the dataset is skewed towards spe- cies classified as containing hidden diversity, the p-value [Accuracy > NIR] was found to be significant in only one model. This is important to point out because even though there are large datasets available, choosing the right analytical tools can remain challenging depending on the use of the predictive models. Beyond analytical tools, it’s also important to consider your dataset, and how the characteristics of your dataset are affecting the results you obtain. Considering the scale of not only the dataset, but also the analytical methods used and the

Table 3. Summary of results of mammal random forest classification models presented in Parsons et al. (Parsons et al., 2022 [35]). Model metrics for each random forest classification model generated using data from the class Mammalia are shown. Mammal Models ABGD COI ABGD cytb GMYC COI GMYC cytb consensus Accuracy 0.737 0.68 0.6429 0.6517 0.781 Accuracy (95% CI) (0.6802, 0.7885) (0.6333, 0.7241) (0.5821, 0.7004) (0.6014, 0.6996) (0.7273, 0.8285) No Information Rate 0.7222 0.6235 0.6128 0.5488 0.6533 Pos Pred Value 0.56667 0.6304 0.17271 0.6624 2.85E-06 Neg Pred Value 0.75833 0.6937 0.5571 0.6345 0.807 P-Value [Acc > NIR] 0.32 0.008792 0.6735 3.00E-05 2.85E-06 https://doi.org/10.1371/journal.pone.0310932.t003 PLOS ONE Predicting hidden lineages in salamanders PLOS ONE | https://doi.org/10.1371/journal.pone.0310932 October 18, 2024 14 / 20