Assessing Cognitive Linguistic Influences in the Assignment of Blame (2021)

Abstract: Lab studies in cognition and the psychology of morality have proposed some thematic and linguistic factors that influence moral reasoning. This paper assesses how well the findings of these studies generalize to a large corpus of over 22,000 descriptions of fraught situations posted to a dedicated forum. At this social-media site, users judge whether or not an author is in the wrong with respect to the event that the author described. We find that, consistent with lab studies, there are statistically significant differences in uses of first-person passive voice, as well as first-person agents and patients, between descriptions of situations that receive different blame judgments. These features also aid performance in the task of predicting the eventual collective verdicts.

## introduction dyadic morality theory proposes that the harm one party causes another is an important component in how other people form judgments of the two parties as acting morally or not. under this framework, perpetrators (agents) are perceived as blameworthy, whereas victims (patients) are not @xcite . this effect appears to transfer to how active (agentive) a party is described to be, even if the activity was in the past -a phenomenon described by @xcite paper titled, "to escape blame, don't be a hero -be a victim". the online forum https://reddit.com/ r/amitheasshole collects first-person descriptions of (purportedly) real-life situations, together with commentary from other users as to who is blameworthy in the situation described; two examples are shown in figure 1 . (additional examples may be found in @xcite this data allows us to evaluate findings from dyadic morality theory on a corpus involving over 22,000 events and 685,000 passed judgments. the research questions we address with this data in this paper include: the first question is motivated by @xcite , who found that using passive voice, by placing someone who was actually a victim in subject position (e.g., "x was threatened by y"), causes the victim to seem more responsible for the event. ## data the subreddit from which we draw our data is selfdescribed as follows: a catharsis for the frustrated moral philosopher in all of us, and a place to finally find out if you were wrong in an argument that's been bothering you. tell us about any non-violent conflict you have experienced; give us both sides of the story, and find out if you're right, or you're the @xcite . it has served as the basis of prior computational analysis of moral judgment by @xcite and @xcite . (the moral stories dataset @xcite would have been an interesting alternative corpus to work with. it also draws some of its situations from the same subreddit.) since the scruples dataset @xcite , also based on the aforementioned subreddit, does not include corresponding full comments, which we wanted to have as an additional source of analysis, 1 we scraped the subreddit ourselves. our dataset (henceforth aita) includes posts from the same timeframe as scruples: november 2018-april 2019. the winning verdict of each post is determined, according to the subreddit's rules, by the verdict espoused by the top-voted comment 18 hours after submission. we aim to only include posts with meaningful content, so we discard posts with fewer than 20 comments and fewer than 6 words in the body, as manual appraisal revealed that these were often uninformative (e.g., body is "as described in the title"). for simplicity, we only consider situations with the yta (author in the wrong, other party in the right) and nta (author in the right, other party in the wrong) verdicts, although other verdicts (such 1 for each post, up to 100 of the most "upvoted" comments were also retrieved; these were not used here but could be useful for further cognitive theory reinforcement or nuanced controversy analysis. ## methodology passive subject identification to model the use of passive voice in moral situations, a dependency parser is used to match spans of passive subjects in sentences. we use spacy's matcher object to extract tokens tagged nsubjpass. cases where the extracted passive subject is in first person (1p) are also tracked, as indication of the author being referred to passively. some examples include: for manual evaluation, we randomly selected 500 posts, containing a total of 675 uses of passive voice. the tagger achieved 0.984 precision on these posts. among the 199 first-person passive subjects tagged, the precision achieved was 0.971. because @xcite and @xcite find that passive voice is associated with greater perception of victims' causal responsibility, we hypothesize that situations with the yta verdict may have higher rates of 1p passive subject usage. thematic role identification to approximate moral agents vs. patients, semantic role labelling (srl) is used to extract agents and patients. semantic, or thematic, roles express the roles taken by arguments of a predicate in an event; an agent is the volitional causer of the event, while the theme or patient is most affected by the event @xcite . the allennlp bert-based semantic role labeller @xcite is employed to extract spans that are tagged arg0 for agents and arg1 for patients. we also tag uses of 1p-agents and patients. here are two examples: as a sanity check, we manually evaluated a subset of 579 verb frames, corresponding to 15 posts, identified by the srl tagger. the tagger achieved a precision of 0.934 on all verb frames. the precision on the 193 verb frames in this subset that contained a first-person arg0 or arg1 was 0.891. examples where the tagger failed include sentence fragments (e.g. "made my mma debut today.") and use of first-person pronouns to describe other parties (e.g. "everyone we know"). @xcite concluded that "it pays to be a [patient] when trying to escape blame. ## statistical analysis due to the post length discrepancy between verdicts, we attempt to control for length in the analysis by assessing significance at the sentence level. while nta posts average approximately 50 words more than yta posts, sentences from nta posts average only 0.5 words more than yta sentences (17.0 vs. 16.6 words respectively). we assess statistical significance as follows. we use a simple binomial test: let r be the rate of the given feature of interest (say, 1p-passive voice) over the entire collection of posts. we then compute the probability according to the r-induced binomial distribution -i.e., the null hypothesis that there is no difference between the yta posts and the body of posts overall -of the observed number of occurrences of the feature in just the yta posts. similarly, we compute this probability for just the nta posts. ## task results to give the imbalanced labels equal importance, we evaluated macro-average scores. weighted average scores were usually around 1% higher than the macro-average scores. overall, the non-linear model achieves higher f1 scores for each of our feature sets, though the linear model does better with tf-idf features (see tables 5 and 6 ). ## discussion despite noting the significant difference in firstperson passive voice usage between verdicts, the feature set of length+passive yields slightly lower performance than the length baseline for both models. this could be due to not having enough instances of passive voice, as each post has on aver-age 1.39 counts of passive voice, of which 30.5% are first-person. regex searches confirmed that the dependency-parser did not simply have poor recall, though the methods for passive voice extraction are not exact. thus, the passive features may be acting as noise. length+srl builds off of more srl instances per post, so these features provide less noisy information. this feature set's performance beats that of the length baseline for both models, suggesting that srl features do play a role in making moral judgments. the srl features do not store lexical information, which helps remove the influence of the content of the posts. length+passive+srl performance likely suffers from the additional passive features' noise. a notable difference is the non-linear model's tendency to favor nta and the linear model's preference for yta. a possible explanation for this is that the features corresponding to yta situations are more linearly separable than those corresponding to nta situations. comparing scores for the length baseline, we see that the random forest has a 3.8% improvement in f1 score over logistic regression. this may suggest that post length is not a linear feature, which would account for nuances such as long yta and short nta posts (see figure 1b for an example of a short nta post). ## conclusion and future work we introduce findings from moral cognitive science and psychology and assess their application to a forum of user-generated ethical situations. statistical tests confirm that there are significant differences in usage of first-person passive voice along with first-person agents and patients among situations of different verdicts. incorporating these differences as features in a verdict prediction task confirms the linkage between first-person agents and patients with assigned blame, though passive voice features appear too sparse to yield meaningful results. from this study, we conclude that the manner in which a situation is described does appear to influence how blame is assigned. in the forum we work with, people seem to be judged by the way they present themselves, not just by their content, which aligns with previous cognitive science studies. future endeavors in ethical ai could incorporate such theories to promote interpretability of models that produce moral decisions. there are several areas of this project that could be refined and pursued further. we can repeat these experiments with the other verdicts, incorporating situations where all parties or no parties are blamed. we can use stricter length control than the sentencelevel comparison, since the average sentence length still differs between posts of different verdicts. we should also incorporate validation that the srl methodology effectively extracts the moral agents and patients we are trying to analyze. another direction we would like to pursue, and one also mentioned by a reviewer, is to group situations by topic to try to control for other confounds in the moral situations. finally, we hope to be able to incorporate the range of votes from the comments accompanying each post to allow for more nuanced verdict prediction, as done with scruples in @xcite .