SlowGuess/NAACL_2022_OCR huggingface dataset:
We observe that the model suffers from catastrophic forgetting since finetuning on  $\mathcal{T}_{\text{target}}$  reduces the performance on  $\mathcal{T}_{\text{source}}$ . The forgetting is most evident when finetuning on unbalanced datasets such as DAV than balanced datasets such as BLM. In fact,  $\mathcal{C}(\text{finetune}, \text{source})$  leads to only slightly worse results than  $\mathcal{C}(\text{source}, \text{source})$  in BLM (decrease of  $2\%$  in  $m$  and  $7\%$  in  $M$ ), with the difference being largest in DAV (decrease of  $15\%$  in  $m$  and  $25\%$  in  $M$ ).
Figure 2 shows that the finetuning paradigm ensures good performances on  $\mathcal{T}_{\text{target}}$  even when the model is trained on a small portion of  $\mathcal{T}_{\text{target}}$ . Next, we evaluate catastrophic forgetting in the same setting, comparing  $\mathcal{C}(\text{source}, \text{source})$  and  $\mathcal{C}(\text{finetune}, \text{source})$  when the model is trained with increasing portions of  $\mathcal{T}_{\text{target}}$  (10%, 25%, and 50%) as shown in Figure 3.
Figure 3:  $\mathcal{C}$  (source, source) and  $\mathcal{C}$  (finetune, source) results trained with increasing portions of  $\mathcal{T}_{\text {target }}$
Figure 3 indicates that catastrophic forgetting worsens as the model is trained with a larger portion of  $\mathcal{T}_{\text {target }}$ .  $\mathcal{C}$  (finetune, source) trained with  $10\%$  of  $\mathcal{T}_{\text {target }}$  leads to a decrease of  $4\%$  in  $m$  and  $7\%$  in  $M$  compared to  $\mathcal{C}$  (source, source) (evident by comparing the source flat blue line to the
first red finetune square in Figure 3). Further,  $\mathcal{C}(\text{finetune}, \text{target})$  trained with  $10\%$  of  $\mathcal{T}_{\text{target}}$  leads to an increase of  $7\%$  in  $m$  and  $6\%$  in  $M$  compared to  $\mathcal{C}(\text{source}, \text{target})$  (evident by comparing the average  $\mathcal{C}(\text{source}, \text{target})$  in Table 4 to the first red finetune square in Figure 2). These results show the tradeoff between the advantage of transfer learning and the impact of forgetting, even when finetuning with a small portion of  $\mathcal{T}_{\text{target}}$ .
4.5 Misclassification Errors
We reported  $F_{1}$ -scores to provide an overview of the model performance in different training settings. Next, we investigate the behavior of the model through the lens of the MFT. We inspect (1) the confusion between morally loaded and non-moral tweets, and, (2) the mistakes among and within moral foundations since moral foundations are differentially manifested in language (Kennedy et al., 2021). We highlight the following four types of misclassification errors (which add up to  $100\%$
Error I A tweet labeled with one (or more) values is classified (by the model) as nonmoral.
Error II A tweet labeled as nonmoral is classified with one (or more) values.
Error III A tweet labeled with a value is classified with values from other foundations.
Error IV A tweet labeled as a vice/virtue is classified as the opposite virtue/vice of the foundation.
Table 5 shows the distribution of errors, averaged over the seven datasets.
Table 5: Distribution of errors per setting (in percentage)
Setting	Err. I	Err. II	Err. III	Err. IV
Csource, source)	25.8	34.3	36.3	3.5
C(target, source)	41.8	24.4	32.0	1.8
C(finetune, source)	38.7	27.5	31.3	2.5
C(all, source)	25.9	34.3	36.3	3.4
Csource, target)	34.7	32.3	30.2	2.8
C(target, target)	31.5	27.6	38.5	2.4
C(finetune, target)	36.0	28.6	32.6	2.8
C(all, target)	30.8	33.0	33.1	3.1
Generalizability In  $\mathcal{C}$  (target, source), Error I occurs largely more often than the other errors, indicating that, when generalizing from one to several domains, labeling value-laden tweets as non-moral is the most common mistake. In contrast, in  $\mathcal{C}$  (source, target), when generalizing from several to one domain, Error I is less prominent, indicating that the model attempts to classify moral rhetoric in the novel domain.
Transferability Error III is more prevalent in  $\mathcal{C}$  (target, target) than  $\mathcal{C}$  (finetune, target). Thus, the confusion among moral values reduces when a model is pretrained on the source domain.
Catastrophic Forgetting Error I occurs largely more often in  $\mathcal{C}(\text{finetune}, \text{source})$  than  $\mathcal{C}(\text{source}, \text{source})$ , indicating that the major type of catastrophic forgetting is missing moral rhetoric in the source dataset.
Finally, Error IV occurs seldom, suggesting that the models generally learn to not confuse between virtues and vices of the same moral foundation.
4.6 Annotators Agreement
We analyze the correspondence between the model predictions and the annotators agreement. Each tweet in the MFTC was annotated by at least three and at most eight different annotators (Hoover et al., 2020, Table 1). More than  $99\%$  of the tweets were annotated by three to five annotators and  $84\%$  by three or four annotators. As described in Section 2, the majority agreement was selected for training and evaluation—that is, only values annotated by at least  $50\%$  of the annotators were retained as correct labels. However, given the subjectivity in value annotation, values labeled by a minority of annotators ought to be considered too.
Tables 6 and 7 show the percentage of annotators that agree with the model predictions considered as errors and accurate, respectively, averaged over the seven datasets. The columns indicate the percentage of annotators agreeing with the model prediction. For instance, if one out of the four workers who annotated a tweet agrees with the model prediction, we record a  $25\%$  agreement.
Table 6: Distribution (in percentage) of classification errors and annotators agreement percentage
Setting	0	(0, 25]	(25, 34]	(34, 50)
Csource, source)	26.1	22.3	45.0	6.6
C(target, source)	49.5	18.0	28.5	3.9
C(finetune, source)	38.5	20.2	36.1	5.2
C(all, source)	26.3	22.2	45.0	6.5
C.source, target)	40.2	23.2	30.4	6.2
C(target, target)	19.7	30.7	40.6	8.9
C(finetune, target)	21.2	30.5	39.9	8.4
C(all, target)	25.6	27.5	39.0	7.9
First, we analyze the classification errors in Table 6. We observe that the sum of the last three columns is always larger than  $50\%$ . This indicates that, in all settings, more than half of
Table 7: Distribution (in percentage) of correct predictions and annotators agreement percentage
Setting	[50, 66)	[66, 75)	[75, 100)	100
Csource, source)	16.9	24.4	20.9	37.7
C(target, source)	16.8	20.0	20.2	43.1
C(finetune, source)	17.0	22.7	20.9	39.4
C(all, source)	17.0	24.5	20.9	37.7
C.source, target)	15.0	27.5	18.5	39.0
C(target, target)	15.0	27.7	18.8	38.5
C(finetune, target)	15.8	28.5	18.7	37.0
C(all, target)	15.7	28.4	18.8	37.2
the model classification errors are not severe in that at least one human annotator agrees with the model prediction. Then, we notice that the settings with the highest incidence of 'bad' classification errors (i.e., where no annotators agree with the model prediction) are those employed to evaluate generalizability ( $\mathcal{C}$  (target, source) and  $\mathcal{C}$  (source, target)) and catastrophic forgetting ( $\mathcal{C}$  (finetune, source)). These results are explained by the harder challenge represented in these settings (refer to Sections 4.2 and 4.4 for a more in-depth discussion). Finally, we observe that there is a small percentage of errors with agreement between  $34\%$  and  $50\%$ . For the agreement to be in this range, a tweet must have been annotated by at least 5 annotators. However,  $84\%$  of the tweets in the MFTC have been annotated by four annotators or less, thus resulting in a smaller agreement in the last column.
Second, we analyze the correct predictions in Table 7. We notice, in all settings, a high correspondence between  $100\%$  agreement among annotators and correct model predictions—that is, tweets annotated with consistent agreement reliably lead to correct predictions. Further, we observe that the distributions of agreement and correct predictions are consistent across different settings.
We review closely related works on value estimation from text, and on cross-domain classification in NLP subfields relevant to value classification.
5.1 Value Estimation from Text
Value estimation has been addressed from both unsupervised and supervised approaches. Unsupervised methods exploit value lexicons to identify values in text. Value lexicons are generated manually (Graham et al., 2009), via semi-automated methods (Wilson et al., 2018; Rezapour et al., 2019; Araque
et al., 2020; Hopp et al., 2021), or expanded from an initial seed via NLP techniques (Ponizovskiy et al., 2020; Araque et al., 2021). Value lexicons are used to identify values in text through word count software (Pennebaker et al., 2001) or similarity in embedding space (Garten et al., 2018; Shen et al., 2019; Bahgat et al., 2020). However, adapting a lexicon to a novel domain is a significant additional effort as it requires identifying words that are relevant and removing words that are not relevant in the novel domain.
Supervised methods employ the classification paradigm (Lin et al., 2018; Mooijman et al., 2018; Hoover et al., 2020; Alshomary et al., 2022; Kiesel et al., 2022). A textual dataset is annotated with values belonging to a value taxonomy, and the labels are used to train a supervised model. This approach is akin to the one we use in this paper. However, in the reviewed literature, no emphasis is put on the effect of cross-domain training. Further, several of the works mentioned above (Lin et al., 2018; Mooijman et al., 2018; Hoover et al., 2020) use binary classification to independently predict the presence of a value in text. That is, given  $N$  values,  $N$  classifiers are employed (one per value). However, it has been shown that modeling relationships among values (and additional contextualizing information such as actors) helps improve downstream performances (Johnson and Goldwasser, 2018; Roy et al., 2021). Thus, we train a multi-label value classifier, similarly to Alshomary et al. (2022) and Kiesel et al. (2022). Furthermore, our objective is not to compare binary and multi-label value classification but to evaluate the cross-domain capabilities (generalizability, transferability, and catastrophic forgetting) of a multi-label value classifier.
5.2 Datasets with Moral Content
The recent success of NLP models has sparked a surge of research in constructs akin to moral values, e.g., moral norms, ethical judgments, and social biases. Researchers have collected large datasets annotated with the related implicit components of human language similar to the MFTC (Section 2). Forbes et al. (2020) introduced SOCIAL-CHEM-101, a corpus of almost 300,000 rules-of-thumb aimed at learning social and moral norms. Sap et al. (2020) collected the Social Bias Inference Corpus with the intent of modeling the way in which people project social biases onto each others. Hendrycks et al. (2021) proposed the ETHICS dataset to as
sess basic knowledge of ethics through well-studied theories of normative ethics (such as deontology and utilitarianism). Lourie et al. (2021) introduced SCRUPLES, a dataset composed of 625,000 ethical judgments over 32,000 real-life anecdotes. Finally, Emelin et al. (2021) presented Moral Stories, a crowd-sourced collection of contextualized narratives with the intent of investigating grounded, goal-oriented social reasoning.
These datasets offer an unprecedented opportunity for studying the social and moral aspects of language. In our research we employ the MFTC as the same moral value theory is used to annotate data in seven different domains, allowing for a direct cross-domain comparison.
5.3 Cross-Domain NLP Classification
Cross-domain classification is gaining attention (Aji et al., 2020; Nguyen et al., 2021; Rongali et al., 2021; Bornea et al., 2021; Markov and Daelemans, 2021). Ruder (2019) provides an overview of the basic terminology, including generalizability, transferability, and catastrophic forgetting.
Cross-domain classification has been investigated in NLP tasks such as sentiment analysis (AlMoslmi et al., 2017; Qu et al., 2019; Du et al., 2020), fake news detection (Fung et al., 2021; Silva et al., 2021; Yuan et al., 2021), and argument mining (Al-Khatib et al., 2016; Daxenberger et al., 2017; Thorn Jakobsen et al., 2021). These tasks are similar to value classification in that they aim to classify high-level constructs (such as sentiments and arguments). However, value classification stands out for its multi-label and domain-specific nature. Also, cross-domain classification is particularly important for values because reasoning about values (Pomeranz et al., 2012) and generating value-annotated datasets is very difficult.
6 Conclusions and Directions
We perform a comprehensive cross-domain evaluation of a multi-label value classifier, by comparing a deep learning model (BERT) in seven domains with four cross-domain training scenarios. Our aim is to support practical applications of moral rhetoric classification, e.g., the detection of radicalism through the study of moral homogeneity (Atari et al., 2021), the prediction of violent protests (Mooijman et al., 2018), the identification of moral concerns of citizens (Mouter et al., 2021; Siebert et al., 2022), and the extraction of moral
rhetoric supporting both stances and arguments (Draws et al., 2022; van der Meer et al., 2022). Our findings inform both computer scientists and social scientists on training value classifiers. However, we do not provide a fixed recipe since the right model and approach depend on the time, resources, and data available.
We show that a value classifier generally exhibits the ability to classify moral values across domains. However, the results are highly dependent on the distribution of moral rhetoric in a domain.
Our experiments support the following key findings. First, a value classifier can generalize to novel domains, especially when trained on multiple domains. However, its performance on the novel domain improves even when trained with a small portion of data from the novel domain. Second, pretraining a value classifier with data from different domains has three benefits when finetuning the classifier. It yields (1) better performances on the novel domain than other settings, (2) good performances even when little training data is available in the novel domain, and (3) smaller confusion among moral values, especially among those less frequent in the novel domain. Third, finetuning on a novel domain causes catastrophic forgetting of the domain it was pretrained with, even when finetuning on a small portion of data from the novel domain. Thus, the tradeoff between benefits of transferability and adverse effects of forgetting must be considered in choosing the extent of finetuning. Finally, despite the challenging nature of cross-domain value classification, the majority of classification errors are not severe in that, in all evaluation settings, at least one annotator agrees with the model prediction.
Our investigation opens avenues for additional experiments with advanced methods to improve transfer learning (Howard and Ruder, 2018; Jiang et al., 2020; Nguyen et al., 2021) and mitigate catastrophic forgetting (Kirkpatrick et al., 2017; Li and Hoiem, 2018; Thompson et al., 2019). Further, based on the analysis of classification errors, we suggest incorporating the annotators (dis-) agreement into the training of the model, e.g., by employing the full distributions of annotations, as opposed to the current majority approach (Uma et al., 2021).
We discuss three ethical considerations relevant to our work. First, the MFTC is composed of mono
lingual tweets about US-centric topics. Whether or not our conclusions hold for results across different languages and cultures is yet to be evaluated. This limitation may cause the perpetuation of Western biases and values (Mehrabi et al., 2021). However, we believe that our experimental setup offers a systematic approach to studying such cultural influences when pertinent data is available.
Second, the MFTC has low annotator agreement (Hoover et al., 2020, Table 6), potentially caused by the subjectivity and complexity of annotating values. Selecting the majority label as golden label may perpetuate the 'tyranny' of the majority, which is especially dangerous when dealing with values. We expose the impact of the annotator agreement in Section 4.6 and identify an avenue for addressing it as a future direction in Section 6.
Finally, the importance of understanding moral values has been recognized by computer scientists (Russell et al., 2015) and designers (Friedman et al., 2008). However, we recognize that value classification can be misused, especially, when sensitive attributes such as gender and race are attached to the data. For instance, authorities could use it to automatically identify and suppress liberal minorities in non-liberal countries. Additional research is necessary for addressing such problems, e.g., by devising techniques that mitigate bias and unfairness by design (Kleinberg et al., 2018; Dinan et al., 2020; Vargas and Cotterell, 2020).
This research was (partially) funded by the Hybrid Intelligence Center, a 10-year programme funded by the Dutch Ministry of Education, Culture and Science through the Netherlands Organisation for Scientific Research. Furthermore, we thank Florentin Arsene for his contribution in previous iterations of the project.
Alham Fikri Aji, Nikolay Bogoychev, Kenneth Heafield, and Rico Sennrich. 2020. In Neural Machine Translation, What Does Transfer Learning Transfer? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL '20, pages 7701-7710, Online. Association for Computational Linguistics.
Nirav Ajmeri, Hui Guo, Pradeep K. Murukannaiah, and Munindar P. Singh. 2020. Elessar: Ethics in norm-aware agents. In Proceedings of the 19th Conference
on Autonomous Agents and MultiAgent Systems, AAMAS '20, pages 16-24, Auckland. IFAAMAS.
Zeynep Akata, Dan Balliet, Maarten de Rijke, Frank Dignum, Virginia Dignum, Guszti Eiben, Antske Fokkens, Davide Grossi, Koen Hindriks, Holger Hoos, Hayley Hung, Catholijn J. M. Jonker, Christof Monz, Mark Neerincx, Frans Oliehoek, Henry Prakken, Stefan Schlobach, Linda van der Gaag, Frank van Harmelen, Herke van Hoof, Birna van Riemsdijk, Aimee van Wynsberghe, Rineke Verbrugge, Bart Verheij, Piek Vossen, and Max Welling. 2020. A Research Agenda for Hybrid Intelligence: Augmenting Human Intellect With Collaborative, Adaptive, Responsible, and Explainable Artificial Intelligence. Computer, 53(8):18-28.
Khalid Al-Khatib, Henning Wachsmuth, Matthias Hagen, Jonas Kohler, and Benno Stein. 2016. Cross-Domain Mining of Argumentative Text through Distant Supervision. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL '16, pages 1395-1404.
Tareq Al-Moslmi, Nazlia Omar, Salwani Abdullah, and Mohammed Albared. 2017. Approaches to Cross-Domain Sentiment Analysis: A Systematic Literature Review. IEEE Access, 5:16173-16192.
Milad Alshomary, Roxanne El Baff, Timon Gurcke, and Henning Wachsmuth. 2022. The Moral Debater: A Study on the Computational Generation of Morally Framed Arguments. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, ACL '22, pages 1-16, Dublin, Ireland. Association for Computational Linguistics.
Oscar Araque, Lorenzo Gatti, and Kyriaki Kalimeri. 2020. MoralStrength: Exploiting a Moral Lexicon and Embedding Similarity for Moral Foundations Prediction. Knowledge-Based Systems, 191:1-29.
Oscar Araque, Lorenzo Gatti, and Kyriaki Kalimeri. 2021. The Language of Liberty: A preliminary study. In *Companion Proceedings of the Web Conference* 2021, WWW '21 Companion, pages 1-4, Ljubljana, Slovenia. Association for Computing Machinery.
Mohammad Atari, Aida Mostafazadeh Davani, Drew Kogon, Brendan Kennedy, Nripsuta Ani Saxena, Ian Anderson, and Morteza Dehghani. 2021. Morally Homogeneous Networks and Radicalism. Social Psychological and Personality Science, 12:1-11.
Mohamed Bahgat, Steven R. Wilson, and Walid Magdy. 2020. Towards Using Word Embedding Vector Space for Better Cohort Analysis. In Proceedings of the International AAAI Conference on Web and Social Media, ICWSM '20, pages 919-923, Atlanta, Georgia. AAAI Press.
Mihaela Bornea, Lin Pan, Sara Rosenthal, Radu Florian, and Avirup Sil. 2021. Multilingual Transfer Learning for QA Using Translation as Data Augmentation.
In Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI '21, pages 12583-12591, Online.
Francisco Charte, Antonio J. Rivera, María J. del Jesus, and Francisco Herrera. 2015. Addressing Imbalance in Multilabel Classification: Measures and Random Resampling Algorithms. Neurocomputing, 163:3-16.
Kinzang Chhogyal, Abhaya Nayak, Aditya Ghose, and Hoa K. Dam. 2019. A Value-Based Trust Assessment Model for Multi-Agent Systems. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI '19, pages 194-200.
Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated Hate Speech Detection and the Problem of Offensive Language. In Proceedings of the 11th International Conference on Web and Social Media, ICWSM '17, pages 512-515.
Johannes Daxenberger, Steffen Eger, Ivan Habernal, Christian Stab, and Iryna Gurevych. 2017. What is the Essence of a Claim? Cross-Domain Claim Identification. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP '17, pages 2055-2066.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL '19, page 4171-4186.
Emily Dinan, Angela Fan, Ledell Wu, Jason Weston, Douwe Kiela, and Adina Williams. 2020. Multi-Dimensional Gender Bias Classification. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP '20, pages 314-331.
Tim Draws, Oana Inel, Nava Tintarev, Christian Baden, and Benjamin Timmermans. 2022. Comprehensive Viewpoint Representations for a Deeper Understanding of User Interactions With Debated Topics. In Proceedings of the 2022 ACM SIGIR Conference on Human Information Interaction and Retrieval, CHIIR '22, pages 135-145, Regensburg, Germany. Association for Computing Machinery.
Chunning Du, Haifeng Sun, Jingyu Wang, Qi Qi, and Jianxin Liao. 2020. Adversarial and Domain-Aware BERT for Cross-Domain Sentiment Analysis. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL '20, pages 4019-4028.
Denis Emelin, Ronan Le Bras, Jena D. Hwang, Maxwell Forbes, and Yejin Choi. 2021. Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language