Modeling Appropriate Language in Argumentation (2023)

Abstract: Online discussion moderators must make ad-hoc decisions about whether the contributions of discussion participants are appropriate or should be removed to maintain civility. Existing research on offensive language and the resulting tools cover only one aspect among many involved in such decisions. The question of what is considered appropriate in a controversial discussion has not yet been systematically addressed. In this paper, we operationalize appropriate language in argumentation for the first time. In particular, we model appropriateness through the absence of flaws, grounded in research on argument quality assessment, especially in aspects from rhetoric. From these, we derive a new taxonomy of 14 dimensions that determine inappropriate language in online discussions. Building on three argument quality corpora, we then create a corpus of 2191 arguments annotated for the 14 dimensions. Empirical analyses support that the taxonomy covers the concept of appropriateness comprehensively, showing several plausible correlations with argument quality dimensions. Moreover, results of baseline approaches to assessing appropriateness suggest that all dimensions can be modeled computationally on the corpus.

## introduction people have varying degrees of sensitivity to controversial issues and may be triggered by different emotional responses dependent on the issue and the opponents' arguments @xcite . this often makes it hard to maintain a constructive discussion. in competitive debates, a moderator ensures that participants argue appropriately. debating culture, dating back to the 18th century, demands appropriate behavior, such as staying on topic and avoiding overly emotional language @xcite . ac-"there is scientific evidence that shows having a mother and a father is the healthiest way for a child to progress physically and mentally. so a lousy father is better than none (that is of course assuming that he is not abusive in any way). also, people change. who says that he will be lousy forever. there are family therapy sessions you can attend to help." ## related work the notion of appropriateness has been explored in several sub-disciplines of linguistics. in communicative competence research, @xcite considered the knowledge about cultural norms as a requirement to produce appropriate speech, which is a central part of acquiring communicative competence. defining sociolinguistics, @xcite linked appropriateness to the notion of politeness that is required in various social settings. later, @xcite argued that appropriateness is a more salient notion than politeness as it explicitly accounts for the context. some of these cultural speech properties were identified as linguistic etiquette by jdetawy and hamzah (2020), including correct, accurate, logical, and pure language. regarding the discussion of controversial issues, debating culture has required participants since its origins to stay on topic and to avoid offensive and overly emotional formulations @xcite . @xcite differentiate between good and bad bias in argumentation, where the latter exhibits close-mindedness, distortion of the conversation, or an imbalance of pro and con arguments. similarly, walton (1999) introduced the concept of dialectical bias, explicitly addressing the context in which an argument is judged to be appropriate. this perspective on argumentation is also described by @xcite as "[...] making appropriate choices in light of situation and audience." as a sub-dimension of argument quality, appropriateness was first studied in nlp by @xcite , a significant inspiration for our work. the authors derived appropriateness as one of the rhetorical argument quality dimensions based on the work of aristotle @xcite . while several of the quality dimensions they proposed were addressed explicitly in previous work, the appropriateness dimension has not been systematically assessed until now. @xcite only provided a relatively shallow definition of appropriateness that requires a simultaneous assessment of three properties, namely the creation of credibility and emotions as well as proportionality to the issue. in contrast, we model these properties individually (in addition to several other dimensions) to better understand what exactly impacts appropriateness. computationally, only @xcite tried to predict appropriateness alongside all the other quality dimensions of @xcite . however, their models relied on a rather small sample of 304 arguments. in comparison, our corpus consists of 2191 arguments spanning three argumentative genres, providing deeper insights into the appropriateness of an argument. related to this notion is the convincingness of arguments studied by @xcite which correlates with appropriateness @xcite , as well as the effectiveness of arguments @xcite . in the context of appropriateness, walton (2010) explored the notion of emotional fallacies in reasoning, some of which were later assessed computationally @xcite @xcite . although we consider some of these fallacies in our work, we also consider other dimensions and exclude some irrelevant to appropriateness (i.e., logical fallacies) because of their more technical nature. we model toxic emotions based on the emotional fallacies identified by walton (2010): ad populum, ad misericordiam, ad baculum, and ad hominem. we merged these four into a single sub-dimension called emotional deception based on the results of a pilot annotation study (section 4). additionally, we define a sub-dimension excessive intensity to address overly intense emotions. in particular, our analysis revealed the presence of a subset of propaganda errors, including loaded language, flagwaving, repetition, exaggeration, and minimization da san martino et al. (2020). ## modeling appropriateness this section explains how we established the relevant dimensions of appropriateness by systematically analyzing research on argument quality. ## the appropriateness corpus this section details the data acquisition and annotation process of our appropriateness corpus and provides statistics of the collected annotations. statistics of our corpus split by argument source are found in appendix f. ## analysis building on existing corpora on theoretical and practical argument quality, we now report the cor-relations of our proposed dimensions and the quality dimensions of @xcite and @xcite . correlations with @xcite are found in appendix e (only one dimension is directly related to appropriateness). ## experiments the corpus from section 4 is meant to enable the computational treatment of inappropriate language in argumentation. as an initial endeavor, this section reports baselines for classifying all 14 dimensions in the taxonomy from section 3. ## conclusion online discussions of controversial topics mostly turn out fruitful only, when the participants argue appropriately, a dimension of argumentative language that has received no systematic investigation so far. therefore, we have presented a taxonomy of 14 dimensions to model inappropriate language in argumentation, derived from rhetoric and argumentation theory. to enable computational research on appropriateness, we compiled a corpus of 2191 arguments from three genres, carefully annotated for all dimensions. our extensive corpus analyses confirm correlations with both theoretical and practical dimensions of argument quality from the literature. the taxonomy covers inappropriateness comprehensively according to human annotators. while a debertabased baseline already comes rather close to human performance in classifying inappropriate language, our corpus allows for developing more sophisticated models in future work that may serve an automatic (or semi-automatic) content moderation. to make content moderation successful and accepted, we think that providing clear reasons supporting the moderation is important, so the participants can better frame their arguments in online discussions. the defined taxonomy dimensions lay out how such reasons may look like.