Learning from climate change news: Is the world on the same page?

scores and most informative features. The majority of experiments were executed on the Pere- grine HPC cluster of the University of Groningen, and some additional runs were performed on Hábrók, its successor (rug.nl/society-business/centre-for-information-technology/ research/services/hpc/facilities/peregrine-hpc-cluster). The accuracy and macro F1-scores yielded by all 18 experiments can be seen in Table 8 in S5 Appendix. These models were trained on Train data and evaluated on the Dev set. For some final experiments, word embeddings were incorporated into the feature set. While this type of feature is not useful in answering the research questions at hand, embed- dings have proven to be powerful predictors in text classification [44]. Therefore, the impact of implementing them (or conversely, avoiding their use) was evaluated. The 300-dimensional word embeddings vectors from Google’s word2vec project (code.google.com/archive/p/ word2vec/) were used here as averaged document vectors. Performance improved only slightly or not at all when these vectors were added to the feature set, which justified the use of solely textual features for further model development, given that explainability was favored over per- formance in this context. Taking all experiments into account, the configuration with word 1–3-grams and the full preprocessing strategy (Experiment 15) was chosen as the final configuration for all three aspects. Tables 3–5 show the classification reports for the final model. This was trained using

this optimal configuration on Train-Dev data, and then evaluated on the Test set. This led to scores that were only slightly deviant from those in the experimental phase, where training was performed on Train and evaluation on Test data. Regression Architecture. Apart from classification, an implementation of regression was developed for the COP meeting labels. These labels are numeric and have a temporal nature. Thus, it is better to predict a meeting closer to the real label than one that is several years off. This is not taken into account when using classification with F1-scores for evaluation, where a prediction can be either wrong or right. A regression model is therefore considered a more suitable way to predict the COP label. This type of model leads to a continuous number and uses the differ- ence from the correct value for evaluation, called the error. Regression was implemented using the LinearRegression (scikit-learn.org/stable/ modules/generated/sklearn.linear_model.LinearRegression) class from Scikit Learn. This was Table 3. Classification report of the final model for country labels. precision recall f1-score Australia 0.89 0.89 0.89 Canada 0.94 0.88 0.91 United Kingdom 0.88 0.92 0.90 Macro average 0.90 0.89 0.90 https://doi.org/10.1371/journal.pone.0297644.t003 Table 4. Classification report of the final model for political orientation labels. precision recall f1-score left-center 0.79 0.90 0.84 right-center 0.84 0.68 0.75 Macro average 0.81 0.79 0.80 https://doi.org/10.1371/journal.pone.0297644.t004 PLOS ONE

Learning from climate change news PLOS ONE | https://doi.org/10.1371/journal.pone.0297644 March 20, 2024 12 / 27