SlowGuess/NAACL_2022_OCR huggingface dataset:
Kiran Garimella, Gianmarco De Francisci Morales, Aristides Gionis, and Michael Mathioudakis. 2018. Quantifying controversy on social media. ACM Transactions on Social Computing, 1(1):1-27.
Jesse Graham, Jonathan Haidt, and Brian A. Nosek. 2009. Liberals and conservatives rely on different sets of moral foundations. Journal of Personality and Social Psychology, 96(5):1029-1046.
Jesse Graham, Brian A. Nosek, Jonathan Haidt, Ravi Iyer, Spassena Koleva, and Peter H. Ditto. 2011. Mapping the moral domain. Journal of Personality and Social Psychology, 101(2):366-385.
Jon Green, Jared Edgerton, Daniel Naftel, Kelsey Shoub, and Skyler Cranmer. 2020. Elusive consensus: Polarization in elite communication on the COVID-19 pandemic. Science Advances, 6:eabc2717.
Ted Grover and Gloria Mark. 2019. Detecting potential warning behaviors of ideological radicalization in an alt-right subreddit. In International AAAAI Conference on Web and Social Media (ICWSM) 13.
Jonathan Haidt and Jesse Graham. 2007. When morality opposes justice: Conservatives have moral intuitions that liberals may not recognize. *Social Justice Research*, 20(1):98-116.
Jonathan Haidt and Craig Joseph. 2004. Intuitive ethics: How innately prepared intuitions generate culturally variable virtues. *Daedalus*, 133(4):55-66.
William Hamilton, Jure Leskovec, and Dan Jurafsky. 2016. Diachronic word embeddings reveal statistical laws of semantic change. In Annual Meeting of the Association for Computational Linguistics (ACL) 54.
Zihao He, Negar Mokhberian, Antonio Camara, Andres Abeliuk, and Kristina Lerman. 2021. Detecting polarized topics in COVID-19 news using partisanship-aware contextualized topic embeddings. In Findings of the Association for Computational Linguistics: EMNLP 2021.
Itai Himelboim, Stephen McCreery, and Marc Smith. 2013. Birds of a feather tweet together: Integrating network and content analyses to examine cross-ideology exposure on Twitter. Journal of Computer-Mediated Communication, 18(2):40-60.
Valentin Hofmann, Janet B. Pierrehumbert, and Hinrich Schütze. 2021. Dynamic contextualized word embeddings. In Annual Meeting of the Association for Computational Linguistics (ACL) 59.
Valentin Hofmann, Hinrich Schütze, and Janet B. Pierrehumbert. 2022. The Reddit politosphere: A large-scale text and network resource of online political discourse. In International AAAI Conference on Web and Social Media (ICWSM) 16.
Matthew Honnibal, Ines Montani, Sofie van Landeghem, and Adriane Boyd. 2020. spaCy: Industrial-strength natural language processing in Python.
Mohit Iyyer, Peter Enns, Jordan Boyd-Graber, and Philip Resnik. 2014. Political ideology detection using recursive neural networks. In Annual Meeting of the Association for Computational Linguistics (ACL) 52.
Simon Jackman. 2001. Multidimensional analysis of roll call data via Bayesian simulation: Identification, estimation, inference, and model checking. Political Analysis, 9(3):227-241.
Kristen Johnson and Dan Goldwasser. 2018. Classification of moral foundations in microblog political discourse. In Annual Meeting of the Association for Computational Linguistics (ACL) 56.
Diederik P. Kingma and Jimmy L. Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR) 3.
Thomas N. Kipf and Max Welling. 2016. Variational graph auto-encoders. In NIPS Bayesian Deep Learning Workshop.
Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR) 5.
Justin H. Kirkland. 2013. Hypothesis testing for group structure in legislative networks. *State Politics & Policy Quarterly*, 13(2):225-243.
Vivek Kulkarni, Junting Ye, Steven Skiena, and William Y. Wang. 2018. Multi-view models for political ideology detection of news articles. In *Conference on Empirical Methods in Natural Language Processing* (EMNLP) 2018.
Srijan Kumar, William Hamilton, Jure Leskovec, and Dan Jurafsky. 2018. Community interaction and conflict on the web. In The Web Conference (WWW) 27.
Mirko Lai, Cristina Bosco, Viviana Patti, and Daniela Virone. 2015. Debate on political reforms in Twitter: A hashtag-driven analysis of political polarization. In International Conference on Data Science and Advanced Analytics (DSAA) 2015.
George Lakoff. 2008. The political mind: Why you can't understand 21st-century politics with an 18th-century brain. Viking, New York, NY.
Taeku Lee and Christian Hosam. 2020. Fake news is real: The significance and sources of disbelief in mainstream media in Trump's America. Sociological Forum, 35:996-1018.
Jeffrey B. Lewis, Keith Poole, Howard Rosenthal, Adam Boche, Aaron Rudkin, and Luke Sonnet. 2021. Voteview: Congressional roll-call votes database.
Qimai Li, Zhichao Han, and Xiao-Ming Wu. 2018. Deeper insights into graph convolutional networks for semi-supervised learning. In Conference on Artificial Intelligence (AAAI) 32.
Antonis Matakos, Evimaria Terzi, and Panayiotis Tsaparas. 2017. Measuring and moderating opinion polarization in social networks. Data Mining and Knowledge Discovery, 31(5):1480-1505.
Maxwell E. McCombs and Donald L. Shaw. 1972. The agenda-setting function of mass media. The Public Opinion Quarterly, 36(2):176-187.
Miller McPherson, Lynn Smith-Lovin, and James M. Cook. 2001. Birds of a feather: Homophily in social networks. Annual Review of Sociology, 27:415-444.
Julia Mendelsohn, Ceren Budak, and David Jurgens. 2021. Modeling framing in immigration discourse on social media. In Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HTL) 2021.
Joanne M. Miller, Jon A. Krosnick, and Leandre R. Fabrigar. 2017. The origins of policy issue salience: Personal and national importance impact on behavioral, cognitive, and emotional issue engagement. In Jon A. Krosnick, I-Chant A. Chiang, and Tobias H. Stark, editors, Political psychology: New explorations, pages 125-171. Routledge, New York, NY.
Pushkar Mishra, Marco del Tredici, Helen Yannakoudakis, and Ekaterina Shutova. 2019. Abusive language detection with graph convolutional networks. In Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HTL) 2019.
Negar Mokhberian, Andrés Abeliuk, Patrick Cummings, and Kristina Lerman. 2020. Moral framing and ideological bias of news. In International Conference on Social Informatics (SocInfo) 12.
Kenton Murray and David Chiang. 2015. Auto-sizing neural networks: With applications to  $n$ -gram language models. In Conference on Empirical Methods in Natural Language Processing (EMNLP) 2015.
Vinod Nair and Geoffrey E. Hinton. 2010. Rectified linear units improve restricted Boltzmann machines. In International Conference on Machine Learning (ICML) 27.
Thomas E. Nelson and Zoe M. Oxley. 1999. Issue framing effects on belief importance and opinion. The Journal of Politics, 61(4):1040-1067.
Mark Newman. 2018. Networks. Oxford University Press, Oxford, UK.
Neal Parikh and Stephen Boyd. 2013. Proximal algorithms. Foundations and Trends in Optimization, 1(3):123-231.
Keith Poole and Howard Rosenthal. 1985. A spatial model for legislative roll call analysis. American Journal of Political Science, 29(2):357-384.
Keith Poole and Howard Rosenthal. 1997. Congress: A political-economic history of roll call voting. Oxford University Press, New York, NY.
Daniel Preotiuc-Pietro, Ye Liu, Daniel J. Hopkins, and Lyle H. Ungar. 2017. Beyond binary labels: Political ideology prediction of Twitter users. In Annual Meeting of the Association for Computational Linguistics (ACL) 55.
Shamik Roy and Dan Goldwasser. 2020. Weakly supervised learning of nuanced frames for analyzing polarization in news media. In Conference on Empirical Methods in Natural Language Processing (EMNLP) 2020.
Peter H. Schonemann. 1966. A generalized solution of the orthogonal procrustes problem. Psychometrika, 36(1).
Itamar Shatz. 2017. Fast, free, and targeted: Reddit as a source for recruiting participants online. Social Science Computer Review, 35(4):537-549.
Qinlan Shen and Carolyn Rose. 2019. The discourse of online content moderation: Investigating polarized user responses to changes in Reddit's quarantine policy. In Workshop on Abusive Language Online 3.
Robert Tibshirani. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, 58(1):267-288.
Oren Tsur, Dan Calacci, and David Lazer. 2015. A frame of mind: Using statistical models for detection of framing and agenda setting campaigns. In Annual Meeting of the Association for Computational Linguistics (ACL) 53.
Aman Tyagi, Anjalie Field, Priyank Lathwal, Yulia Tsvetkov, and Kathleen M. Carley. 2020. A computational analysis of polarization on Indian and Pakistani social media. In International Conference on Social Informatics (SocInfo) 12.
Vorakit Vorakitphan, Marco Guerini, Elena Cabrio, and Serena Villata. 2020. Regexit or not regexit: Aspect-based sentiment analysis in polarized contexts. In International Conference on Computational Linguistics (COLING) 28.
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S. Yu. 2021. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1):4-24.
Mason Youngblood. 2020. Extremist ideology as a complex contagion: the spread of far-right radicalization in the United States between 2005 and 2017. *Humanities and Social Sciences Communications*, 7(1):310.
Ming Yuan and Yi Lin. 2006. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society, 68(1):49-67.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Coterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender bias in contextualized word embeddings. In Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HTL) 2019.
A.1 Data Statistics
Table 5 provides summary statistics of Reddit Politosphere (Hofmann et al., 2022). We compute average shortest path length as
$$
\mu_ {\pi} = \sum_ {i, j \in \mathcal {V}} \frac {\pi (i , j)}{| \mathcal {V} | (| \mathcal {V} | - 1)},
$$
where  $\pi (i,j)$  is the shortest path from subreddit  $i$  to subreddit  $j$ . We compute density as
$$
\rho = \frac {2 | \mathcal {E} |}{| \mathcal {V} | (| \mathcal {V} | - 1)}.
$$
We compute modularity as
$$
Q = \frac {1}{2 | \mathcal {E} |} \sum_ {i, j \in \mathcal {V}} \left(\mathbf {A} _ {i j} - \frac {d _ {i} d _ {j}}{2 | \mathcal {E} |}\right) \delta (i, j),
$$
where  $\delta (i,j) = 1$  if  $i$  and  $j$  are in the same community, else  $\delta (i,j) = 0$ . The maximum  $Q$  values are indicative of the level of polarization in the graph.  $Q > 0.3$  for all years, which is a typical cutoff value to determine polarized networks (Garcia et al., 2015). Notice we use the standard definitions of the three measures (Newman, 2018).
A.2 Details on Moral Subspaces
For  $\mathbf{e}(v_i, c_j)$ , we extract the mean-pooled embedding if the concept is split into multiple WordPiece tokens and sample a maximum of 100 occurrences per subreddit and concept. For  $\mathbf{e}(m_k)$ , we sample 1,000 occurrences per word.
It is important to notice that  $p_k(v_i, c_j)$  is impacted by two different factors. On the one hand,  $p_k(v_i, c_j)$  captures the association of concepts with moral foundations due to intrinsic lexical-semantic
Year	|D|	|V|	|E|	μd	μπ	ρ	Q
2013	6,306,458	108	324	6.00	3.08	.056	.560
2014	6,664,567	132	335	5.08	3.86	.039	.663
2015	9,230,022	168	493	5.87	3.87	.035	.672
2016	34,801,075	255	1,318	10.34	3.14	.041	.603
2017	38,278,685	295	1,572	10.66	3.14	.036	.585
2018	40,222,627	316	1,604	10.15	3.17	.032	.584
2019	46,590,000	412	2,536	12.31	3.20	.030	.603
Table 5: Dataset statistics.  $|\mathcal{D}|$ : number of comments;  $|\mathcal{V}|$ : number of nodes (subreddits);  $|\mathcal{E}|$ : number of edges;  $\mu_{d}$ : average node degree;  $\mu_{\pi}$ : average shortest path length;  $\rho$ : density;  $Q$ : maximum modularity.
properties, which can be seen by examining the variation of  $p_k(v_i, c_j)$  across different concepts. Thus, computing  $\frac{1}{|\mathcal{V}|} \sum_{v_i \in \mathcal{V}} p_k(v_i, c_j)$  for all concepts and moral foundations (i.e., the average value of  $p_k(v_i, c_j)$  across subreddits), we find that the lexical semantics of concepts with the highest values are directly related to the moral foundations (e.g., patriot and revolution for loyalty/betrayal).
On the other hand,  $p_k(v_i, c_j)$  also captures the association of concepts with moral foundations that is due to extrinsic cooccurrence patterns caused by ideological framing, which can be seen by examining the variation of  $p_k(v_i, c_j)$  across different contexts and subreddits (i.e., sets of contexts). To check this empirically, we use the 20 highest-ranked words per moral foundation from the Moral Foundations Dictionary (Frimer et al., 2017) and compute for each subreddit  $v_i$ , concept  $c_j$ , and moral foundation  $m_k$  the proportion of occurrences in which at least one  $m_k$  word is found in a context window of 10 words around  $c_j$ , which is similar to traditional ways of measuring ideological framing (e.g., Fulgoni et al., 2016). We then create for each concept  $c_j$  and moral foundation  $m_k$  (i) a set  $\mathcal{T}_k(c_j)$  containing the  $d$  subreddits with the largest proportion of moral context words and (ii) a set  $\mathcal{B}_k(c_j)$  containing the  $d$  subreddits with the smallest proportion of moral context words. We set  $d = 5$ , but results are robust with respect to the exact selection of  $d$ . Comparing the average value of  $p_k(v_i, c_j)$  of subreddits in  $\mathcal{T}_k(c_j)$  and  $\mathcal{B}_k(c_j)$  for all concepts, we find it to be consistently higher for  $\mathcal{T}_k(c_j)$  than for  $\mathcal{B}_k(c_j)$  (Table 6). The fact that this result holds for all years and moral foundations suggests that the extent to which the concepts cooccur with certain moral frames is indeed captured by the projections of contextualized embeddings into the moral subspaces. Crucially, while  $p_k(v_i, c_j)$  in principle captures both types of factors, only
Set	Fairness/cheating	Sanctity/degradation
2013	2014	2015	2016	2017	2018	2019	μ±σ	2013	2014	2015	2016	2017	2018	2019	μ±σ
Tk(cj)	.074	.074	.074	.075	.076	.076	.076	.075±.001	.067	.068	.067	.070	.069	.068	.067	.068±.001
Bk(cj)	.068	.068	.068	.070	.071	.070	.070	.069±.001	.065	.064	.064	.067	.066	.065	.064	.065±.001
Table 6: Comparison of average  $p_k(v_i, c_j)$  values for  $\mathcal{T}_k(c_j)$  (large proportion of moral context words) and  $\mathcal{B}_k(c_j)$  (small proportion of moral context words). The table shows the values for fairness/cheating and sanctity/degradation, but the trend is consistent across all moral foundations. Higher value per column in gray.
Model	2013	2014	2015	2016	2017	2018	2019	μ±σ
SF-SGAE	.857	.893	.911	.921	.923	.913	.921	.906±.022
S-SGAE	.833	.868	.872	.864	.883	.865	.904	.870±.020
F-SGAE	.832	.880	.863	.861	.884	.868	.894	.869±.019
SF-SLAE	.712	.812	.772	.771	.778	.729	.748	.760±.031
SF-GAE	.852	.887	.910	.935	.939	.926	.943	.913±.031
Table 7: Dev performance (AUC). SF-SGAE outperforms S-SGAE, F-SGAE, and SF-SLAE. It performs similarly to or better than SF-GAE despite using only a fraction of concepts. Best score per column in gray.
the extrinsically-driven variation due to ideological framing is expected to be valuable for predicting the social network structure.
The input layer of the model has 1,000 dimensions (which are sparsified during training), the first hidden layer 100 dimensions, and the second hidden layer 10 dimensions. We perform grid search for the number of epochs  $e \in \{1, \dots, 1000\}$ , the learning rate  $r \in \{1 \times 10^{-4}, 3 \times 10^{-4}, 1 \times 10^{-3}, 3 \times 10^{-3}\}$  and the regularization constant  $\lambda \in \{1 \times 10^{-4}, 3 \times 10^{-4}, 1 \times 10^{-3}, 3 \times 10^{-3}\}$ .
All experiments are performed on a GeForce GTX 1080 Ti GPU (11GB). The total number of trainable parameters is 107,110 for SF-SGAE, SF-SLAE, and SF-GAE, 101,110 for S-SGAE, and 106,110 for F-SGAE.
A.4 Dev Performance
Table 7 provides the dev performance for all models considered in Section 5 of the paper.
A.5 Sparsity Threshold
Figure 6 presents the results of the experiment varying the sparsity threshold described in Section 5 of the paper for all years.
Figure 6: Impact of sparsity threshold  $\theta_{|\mathcal{C}|}$  on performance (AUC). SF-SGAE performs better than any other model in the sparse regime ( $\theta_{|\mathcal{C}|} \leq 200$ ), showing that it better captures polarization.