Short text classification with machine learning in the social sciences: The case of climate change on Twitter

randomly sampled (with replacements). A decision-tree model is then fitted to each of these training sets. The predictions from the models are combined, e.g., using majority voting over decision trees in the forest. This improves the model’s generalizability by reducing the variance of the classifier, without increasing its bias. The RF approach is an extension of bag- ging that also randomly selects subsets of features used in each data sample. Fig 3 illustrates an RF classifier in action. A total of B decision trees are formed by considering various sub- sets of available features. Each decision tree makes its own prediction on its own subset of the training data. Then majority vote defines the final prediction label of the random forest classifier. The RF classifier has the advantages of good interpretability, possibility to assess the impor- tance of input features, and efficient avoidance of overfitting with many trees. The main disad- vantage of this method is its computational complexity and slow speed with a large number of trees, which limits its applicability for real-time operation. For our empirical analysis, we adjusted the maximum depth of the decision trees and the number of estimators in order to optimize the RF classifier’s performance. k nearest neighbors. The k nearest neighbors (KNN) algorithm [42] is a simple classifier which has been widely used by researchers for text categorization (see, e.g., [43, 44]). In con- trast to other ML algorithms, KNN does not have a training phase. Instead the training set is

simply stored and the actual computation is deferred to the prediction phase. In a nutshell, given an unlabeled observation x, the algorithm calculates the distances d(x, x(i)) to training observations x(i), for i = 1, . . ., N, in the feature space and finds k nearest neighbors to x. The classes of these neighbors are used to weigh the available classes for the given observation. That is, majority voting is performed on the observations within the set of k nearest neighbors. Fig 4 illustrates a KNN classifier in action on synthetic data. Fig 3. Illustration of a random forest classifier. The classifier consists of a collection of decision trees, each making its own prediction regarding the class of an unseen data point. The predictions are then used in a majority voting for producing the final prediction. https://doi.org/10.1371/journal.pone.0290762.g003 PLOS ONE Short text classification with machine learning in the social sciences PLOS ONE | https://doi.org/10.1371/journal.pone.0290762 September 29, 2023 9 / 26