Mapping climate discourse to climate opinion: An approach for augmenting surveys with social media to enhance understandings of climate opinion in the United States

Model development & assessment. We use a random forest model [27] to map CCO on topic distributions. Random forest is a tree-based, non-parametric model. To make predic- tions, the random forest algorithm builds b decision trees and averages the estimates of all of the trees to create a final estimate of the response variable (CCO) as a function of the topic dis- tribution observed in the county of interest. This is mathematically expressed as: ^f ðxÞ ¼ 1 B X B b¼1 TbðxÞ where Tb is an individual regression tree and x represents the input data. The random forest algorithm is effective in modeling data that exhibits complex, nonlinear relationships due to its flexibility and lack of assumptions about the underlying structure of the data [27]. The focus of this analysis is characterize the regional differences in the relationships between tweet topic distribution and survey response. A significant amount of prior literature has demonstrated that beliefs in climate change vary significantly throughout the country [1, 28] and a natural extension of previous results is to develop models for each region of the con- tinental US. If beliefs in climate change vary by region, it follows that the relationship between social media topic portfolios and CCO would also vary regionally. Furthermore, the develop- ment of regional models can help extract nuanced relationships that may otherwise be masked in a larger, national model. Specifically, we develop a model for the Northeast, Midwest, South- east, Southwest, Pacific, and Rocky Mountain regions.

A key component of building each of the final six regional models is variable selection, as including too many variables in a model can lead to overfitting or inaccurate inferences on the relationships between predictors and response [29]. Here, we employed the Variable Selection Using Random Forests (VSURF) algorithm [30]. This is a data-driven algorithm that ranks the importance of every candidate variable in making a prediction. VSURF defines importance as the mean reduction in mean square error observed by including a variable in the model. A var- iable which leads to a greater reduction in error is ranked as more important. To determine the ranking, VSURF repeatedly builds models and averages the variable importance from each iteration. Beginning with the most important variables, models are built by step-wise addition of variables in order of decreasing importance. Once the improvement in model accuracybe- comes negligible, the process ends and the most important variables are returned. We devel- oped final models for all regions using the key variables identified by the VSURF algorithm. In addition to reducing overfitting, variable selection also improves model interpretability [29]. Though random forest models are non-parametric, there are a wide variety of tools avail- able to characterize the relationship between predictors and response. To characterize the rela- tionship between predictors and response for each model, we use partial dependence plots. These plots show the effect a predictor variable on the response while the effects of the other

variables in the model are accounted for [31]. Mathematically, the relationship of the predictor on the response is given as: ^fj ðxjÞ ¼ 1 n X n i¼1 ^fj ðxj; x j;iÞ ð1Þ where ^fj represents the trained model, n is the number of observations in the training set, and x−j is all variables other than xj in the training set. In order to evaluate the generalization per- formance of each regional model}, we conduct randomized partitioning of the data into train- ing and test sets such that 80% of the data is used for training, and 20% for testing. The regional models are then built using the training set and validated by making predictions PLOS ONE An approach for augmenting surveys with social media to understand climate opinion in the United States PLOS ONE | https://doi.org/10.1371/journal.pone.0245319 January 14, 2021 8 / 16