# Toward a Standard Approach for Echo Chamber Detection: Reddit Case Study
## 4 Reddit: Politics, Gun Control, and Minorities Discrimination
### Users' Ideology Inference  
Once we have gathered data, the second step consists of inferring users' ideology on the controversy. As stated in Section 3.2, we model the task of predicting the political alignment of users' posts as a text classification problem. Since we consider two sides of the controversy in this case study (i.e., Pro-Trump and Anti-Trump), we model the text classification task as a binary problem. Among the suitable NLP approaches discussed in Section 2.2, we test two different Deep Learning models (i.e., LSTM and BERT) which have been widely used to predict the political leaning both of OSN contents [53; 54; 55] and news articles [56; 57]. Concerning LSTM, we have already carried out some experiments in a preliminary work [58]. In the following, for both models, we discuss the updated experimental setup, the model evaluation phase, and the prediction results on sociopolitical topics. **Experimental Setup. ** To train and test both models, we rely on the Polarized Ground Truth dataset defined in Section 4.1. To create the training and validation sets, we randomly select 80% of the whole dataset (242,762 posts) in such a way to guarantee the balancing between the Pro-Trump and Anti-Trump classes. The remaining 20% (60,000 posts) is used as the test set. During model selection, we perform a 3-fold Cross-Validation trying different hyper-parameters configurations of both models. For LSTM, we varied the number of LSTM units [32; 64; 128] as well as the type of word embeddings with a fixed dimension of 100, i.e., GloVe pre-trained word embeddings and embeddings directly learned from the texts. In both scenarios, we vectorize each input submission creating a lexicon index based on word frequency, where 0 represents padding: indexes are assigned in descending order of frequency. Moreover, since the Out-of-Vocabulary (OOV) rate is rather low (i.e., 1.36%), we mark OOV tokens with a reserved index. In all settings, we use a dropout regularization of 0.3, adam optimizer, the sigmoid activation function, and as loss function, the binary cross-entropy. We obtain the best performances on the validation set using GloVe word embeddings and 128 LSTM units, obtaining an average accuracy of 82.9%. For BERT, we leverage the pre-trained model \(BERT_{BASE}\) presented by [PERSON] et al. [39]. Specifically, we rely on the PyTorch implementation publicly released by Hugging Face [59]. Then, we varied the length of the input [64; 128; 512; 56], the learning rate [\(2\times 10^{-5},3\times 10^{-5},5\times 10^{-5}\)], and the type of text pre-processing, i.e., with or without punctuation. We obtain the best results on the validation set, leaving punctuation, 512-token input (note that 512 tokens is also the maximum input length supported by BERT; nevertheless, in this case study, we deal with OSN posts that tend to be relatively short, i.e., only 0.4% of total posts are longer than the 512 limit), and a learning rate of \(2\times 10^{-5}\), reaching an average accuracy of 86.3%. Comparing the two models' validation results, we notice that BERT reaches a definitely higher accuracy than LSTM. Thus, we select such a model for this case study. **Model Evaluation. ** We assess BERT performances on the test set, obtaining an accuracy of 85.6%. Then, to verify to what extent the model is able to generalize on less polarized posts, we further evaluate it on the sociopolitical datasets. However, in this scenario, we do not have any labeled data with respect to [PERSON]'s beliefs. Thus, we search for users belonging to our ground truth datasets, label them accordingly, and then apply the model to their posts. Table 3 shows model evaluation results for the test set and the three topics. Even if the model suffers from the domain change, it can generalize quite well, reaching an accuracy greater than 70% among all sociopolitical datasets.