Predicting genetic biodiversity in salamanders using geographic, climatic, and life history traits

our predictor variables (S3 Fig in S1 File). Due to the presence of strong correlations between several of the geographic and environmental variables in our dataset we performed multiple random forest models with progressive sets of correlated variables removed at different cutoff values (i.e., |correlation coefficient| > 0.75; 0.85; 0.9). The results of these random forest mod- els are presented below (Table 1). All random forest models were found to have high predictive accuracy, with the majority rules and all agree models achieving accuracies of 75–85% and 87–93%, respectively, in identi- fying nominal species likely to contain hidden diversity. Although these results may initially seem to suggest that all our models are able to make meaningful predictions, further examina- tion of additional model evaluation metrics reveals potential overfitting and inflation of pre- dictive power. For example, despite the high accuracy of the models, the 95% confidence intervals for these values are broad with an average length of nearly 40% for most of the models (Tables 1 and 2). Additionally, the no information rates (NIRs), a measure of prediction signif- icance based on the underlying dataset that needs to be exceeded in order for model results to be significant, are particularly high for the all agree consensus models, where the class frequen- cies are more skewed towards species predicted to harbor hidden diversity. The high NIR val- ues combined with wide confidence intervals result in a p-value [Accuracy > NIR] greater

than 0.05 in all models, except for the majority rules consensus using a correlation cutoff of 0.90. While all our models show high accuracy, when the additional model evaluation metrics are considered only one has strong predictive power. Therefore, we only used the majority rules consensus using a correlation cutoff of 0.90 for interpreting variable importance of our data. Evaluation of variable importance We extracted variable importance measurements from each random forest classification using the variable importance metrics MDA and Gini. While there was some overlap of top predictors between different models (Fig 4; S4 Fig in S1 File), no specific predictors were Table 1. Results of majority rules consensus random forest models. Model metrics for each random forest predictive model generated using the majority rules consen- sus classifications are shown. Majority Rules Models Original |Correlation| > 0.75 |Correlation| > 0.85 |Correlation| > 0.90 Accuracy 0.75 0.75 0.75 0.8333 Accuracy (95% CI) (0.5329, 0.9023) (0.5329, 0.9023) (0.5329, 0.9023) (0.6262, 0.9526) No Information Rate 0.625 0.625 0.625 0.625 Pos Pred Value 0.7368a 0.8 0.7647 0.7895 Neg Pred Value 0.8 0.6667 0.7143 1 P-Value [Acc > NIR] 0.1453 0.1453 0.1453 0.02435 https://doi.org/10.1371/journal.pone.0310932.t001 Table 2. Results of all agree consensus random forest models. Model metrics for each random forest predictive model generated using the all agree consensus classifica- tions are shown.

All Agree Models Original |Correlation| > 0.75 |Correlation| > 0.85 |Correlation| > 0.90 Accuracy 0.8667 0.9333 0.8667 0.8667 Accuracy (95% CI) (0.5954, 0.9834) (0.6805, 0.9983) (0.5954, 0.9834) (0.5954, 0.9834) No Information Rate 0.8 0.8 0.8 0.8 Pos Pred Value 0.8571 0.9231 0.8571 0.8571 Neg Pred Value 1 1 1 1 P-Value [Acc > NIR] 0.398 0.1671 0.398 0.398 https://doi.org/10.1371/journal.pone.0310932.t002 PLOS ONE Predicting hidden lineages in salamanders PLOS ONE | https://doi.org/10.1371/journal.pone.0310932 October 18, 2024 9 / 20