Mapping climate discourse to climate opinion: An approach for augmenting surveys with social media to enhance understandings of climate opinion in the United States

using the test set. Model performance is evaluated using normalized root mean square error. Model performance results are presented in the Table 2 below. These results show that the models perform better on in-sample data as compared to the out-of-sample data, which is to be expected as the models were trained on the in-sample data. In this study, we are primarily interested in in-sample performance as we are seeking to explain the available data rather than generalize to new use cases. However, including the out- of-sample performance is important as it demonstrates that the models don’t overfit the data and possess reasonable predictive power. Additionally, each model was compared to the null (mean-only) model as a benchmark for the performance of statistical regression models. The null model does not rely on leveraging a statistical model (based on supervised learning) to estimate the response variable as a function of the independent variables; instead it assumes that the best predictions can be achieved by simply calculating the historical average value of the response variable. Table 2 shows that in sample, the models outperform the null model by an average of 59.5% and outperform the null model by an average of 27.4%, indicating that they are successfully identifying relationships in the data. Results Regional topic portfolios In order to investigate how the attributes associated with climate change opinions differ across regions of the US, we develop statistical models for each region of the US relating Twitter top-

ics to Consolidated Climate Opinion (CCO). As previously described, we identify the subset of Twitter topics which best predict CCO and construct models for each region using only that subset. The topics included in the model and the regional breakdown are depicted in Fig 2, which illustrates the regional differences in topic portfolios used to predict CCO. In addition to reaffirming previous research that opinions on climate change vary regionally, this analysis highlights that the topics which are important to predicting CCO in one region are not neces- sarily important in predicting CCO in a different region. Many of the topic portfolios reflect regionally distinct trends contributing to climate opin- ions. One such topic is Climate Impacts in the Pacific as compared to the Southeast. These regions have recently experienced a disproportionate number of natural disasters [32, 33] which has been previously shown to influence climate change beliefs of a region [4, 5]. Simi- larly, Environmental Justice emerges as an important topic in the Rockies, Southeast, South- west, and Northeast. We hypothesize that the importance of this topic in these regions is driven by the concerns surrounding the Keystone and Atlantic pipelines raised by activists and Table 2. Model performance based on Normalized Root Mean Square Error (NRMSE) and R2 for each regional model. Region In Sample Out of Sample NRMSE R2 Impr. Over Null (%) NRMSE Correlation Impr. Over Null (%) Midwest 39.7 0.96 60.2 73.1 0.73 26.4 Southwest 42.6 0.96 57.1 72.1 0.77 26.5

Southeast 40.4 0.96 59.5 78.7 0.62 20.8 Northeast 35.0 0.96 64.9 61.1 0.80 38.2 Pacific 39.9 0.96 59.9 70.4 0.77 27.6 Rockies 44.2 0.95 55.5 73.1 0.73 24.8 Models with lower NRMSE and higher correlation have more predictive power. Additionally, we show the percent improvement over the mean-only, or null model. A higher percent improvement indicates increased predictive power. https://doi.org/10.1371/journal.pone.0245319.t002 PLOS ONE An approach for augmenting surveys with social media to understand climate opinion in the United States PLOS ONE | https://doi.org/10.1371/journal.pone.0245319 January 14, 2021 9 / 16