What does a Text Classifier Learn about Morality? An Explainable Method for Cross-Domain Comparison of Moral Rhetoric (2023)

Abstract: Moral rhetoric influences our judgement. Although social scientists recognize moral expression as domain specific, there are no systematic methods for analyzing whether a text classifier learns the domain-specific expression of moral language or not. We propose Tomea, a method to compare a supervised classifier’s representation of moral rhetoric across domains. Tomea enables quantitative and qualitative comparisons of moral rhetoric via an interpretable exploration of similarities and differences across moral concepts and domains. We apply Tomea on moral narratives in thirty-five thousand tweets from seven domains. We extensively evaluate the method via a crowd study, a series of cross-domain moral classification comparisons, and a qualitative analysis of cross-domain moral expression.

## introduction moral narratives play a fundamental role in stance taken on controversial social issues @xcite . recognizing moral narratives helps understand the argumentation around important topics such as vaccine hesitancy @xcite , violent protests @xcite , and climate change @xcite . language reveals deep psychological constructs, including moral values @xcite . thus, language is an important avenue for analyzing moral expression. in particular, supervised text classification models have been showing promising results on morality prediction @xcite @xcite . these models leverage the wisdom of crowds (via annotations of moral expression) to attain a descriptive understanding of morality. however, the supervised learning paradigm can lead to black-box models @xcite . understanding what these models learn is crucial, especially for the morality classification task, which is likely to be used in sensitive applications like healthcare @xcite . moral expression is context dependent @xcite @xcite , where context refers to factors such as actors, actions, judges, and values @xcite . for a text classifier, the domain from which the training data is sourced represents the context. for example, in the context of recent iranian protests, tweets tagged #mahsaamini can form the training domain. we expect this domain to have a different moral expression than the training domain of #prolife tweets, representing a different context. recent works @xcite analyze the out-of-domain performance of morality classifiers. however, what leads classifiers to perform differently across domains has not been systematically explored. such an insight is essential for understanding whether classifiers can learn a domain-specific representation of morality. we propose tomea (from the greek τ oµ´ α, meaning "domain") to compare a text classifier's representation of morality across domains. tomea employs the shap method @xcite to compile domain-specific moral lexicons, composed of the lemmas that the classifier deems most predictive of a moral concept in a domain, for each moral concept and domain. through such moral lexicons, tomea enables a direct comparison of the linguistic cues that a classification model prioritizes for morality prediction across domains. we employ tomea to compare moral rhetoric across the seven social domains in the moral foundation twitter corpus (mftc) @xcite . then, we perform a crowdsourced evaluation to assess the agreement between the human intuition and the automatically obtained results of tomea. we show that this agreement is consistent across domains but varies across moral concepts. further, we find a strong correlation between the results of tomea and the out-of-domain performance of the models used for obtaining the moral lexicons. in addition, we perform qualitative analyses of the moral impact of specific lemmas, unveiling insightful differences in moral concepts and domains. tomea allows to inspect and compare the extent to which a supervised classifier can learn domain-specific moral rhetoric from crowdsourced annotations. tomea can guide computer scientists and practitioners (e.g., social scientists or policymakers) in the responsible use of transfer learning approaches. in transfer learning, large datasets are used to pre-train language models, which are then finetuned with data collected in the domain of interest. such pre-training typically helps in improving performance in the finetuning domain. however, increased performance may come at the cost of critical mistakes which may hinder the usage of the model, especially when the finetuning domain concerns minority groups @xcite . tomea can assist in the qualitative comparison of pre-training and finetuning domains by unveiling potential critical differences and guiding practitioners in judging the appropriateness of using a morality prediction model in an application. ## related works we introduce the theoretical background and review related works in morality classification in text, domain dependency in nlp models, and explainability in nlp. ## the tomea method tomea 1 is a method for comparing a text classifier's representation of morality across domains. tomea takes as input two dataset, classifier pairs, where, in each pair, the classifier is trained on the corresponding dataset. since tomea intends to compare moral expressions across domains, the two datasets input to it are assumed to be collected in different domains. tomea's output is a qualitative and quantitative representation of the differences in moral expressions between the two input domains. figure 1 shows the two key steps in the method. first, we generate moral lexicons capturing the classifiers' interpretable representations of the moral elements specific to their domains. then, we compare the moral lexicons in two ways. (1) we compare the moral lexicons generated for the same moral elements in different domains. (2) we combine the moral lexicons generated for the same domains and provide a single measure of moral rhetoric similarity between two domains. ## experiment design we evaluate tomea on mftc @xcite . using tomea, we generate moral and domain lexicons for the seven mftc domains and perform pairwise comparisons, obtaining 10 mdistances and one d-distance per comparison. the m-distances and d-distances are intended to compare the classifiers' representation of moral rhetoric across domains. we perform two types of evaluation to inspect the extent to which these distances capture the differences in moral expression across domains. we also perform a qualitative analysis to find fine-grained differences across domains. ## results and discussion first, we describe the pairwise comparisons resulting from tomea. then, we describe the results from the evaluations. finally, we perform a qualitative analysis to provide fine-grained insights. ## conclusions and directions tomea is a novel method for comparing a text classifier's representation of morality across domains. tomea offers quantitative measures of similarity in moral rhetoric across moral elements and domains. further, being an interpretable method, tomea supports a fine-grained exploration of moral lexicons. tomea is generalizable over a variety of classification models, domains, and moral constructs. the similarities computed by tomea positively correlate with human annotations as well as the out-of-domain performance of morality prediction models. importantly, tomea can shed light on how domain-specific language conveys morality, e.g., the word 'brotherhood' has a high impact on moral elements in the alm domain, whereas the word 'treason' has a high impact in the blm domain. tomea can be a valuable tool for researchers and practitioners. it can be used to study how a text classifier represents moral rhetoric across personal, situational, and temporal dimensions, and across different types of moral values @xcite . tomea can support societal applications such as modeling stakeholders' preferences on societal issues @xcite @xcite , analyzing the impact of events like the covid-19 pandemic @xcite , and predicting violent protests @xcite . finally, tomea can assist nlp researchers in generating morally aligned text @xcite that is domain specific. a key direction to improve tomea is incorporating refined explanations, e.g., by rule-based inferences @xcite . additional distance metrics and normalization procedures may also provide a more accurate lexicon comparison. finally, the qualitative analysis that we performed could be systematized as a methodology for analysts.