Towards Argument Mining for Social Good: A Survey (2021)

Abstract: This survey builds an interdisciplinary picture of Argument Mining (AM), with a strong focus on its potential to address issues related to Social and Political Science. More specifically, we focus on AM challenges related to its applications to social media and in the multilingual domain, and then proceed to the widely debated notion of argument quality. We propose a novel definition of argument quality which is integrated with that of deliberative quality from the Social Science literature. Under our definition, the quality of a contribution needs to be assessed at multiple levels: the contribution itself, its preceding context, and the consequential effect on the development of the upcoming discourse. The latter has not received the deserved attention within the community. We finally define an application of AM for Social Good: (semi-)automatic moderation, a highly integrative application which (a) represents a challenging testbed for the integrated notion of quality we advocate, (b) allows the empirical quantification of argument/deliberative quality to benefit from the developments in other NLP fields (i.e. hate speech detection, fact checking, debiasing), and (c) has a clearly beneficial potential at the level of its societal thanks to its real-world application (even if extremely ambitious).

## introduction considering argument mining (am) for social good implies a strong conceptual shift: the discourse exchange is not to be interpreted as a competition to be won by the most persuasive contribution 1 , but rather as a cooperative endeavor in which 1 in this paper, we use the term "contribution" to refer to a turn in a discourse exchange; more concretely a contribution is a textual unit in a discourse contex, e.g., a post in a forum, a tweet in a discussion thread; a speech in a parliamentary debate). building on this assumption, we review recent developments in the field of am from the perspective of its application in socially relevant contexts. our survey has a strong interdisciplinary perspective, putting the focus on the collaboration between nlp and the social sciences and, more specifically, in argumentation targeted at decision-making (deliberation). deliberative discourse historically characterizes parliamentary debates; however, it pervades, more and more frequently, discussions in digital democracy forums and, beyond that, specific strands of discussions in "generalistic" social media. looking at argumentation through the lens of deliberation has a 2-fold benefit. from a purely nlp perspective, the insights gained through modeling deliberative features can in turn be employed in applications targeting discourse in deliberative forums and social media more broadly, allowing systems to be more adaptable to real-world discourse settings. social sciences, in turn, can enormously benefit from the possibility of scaling up to a larger public with the support of nlp methods. the novelty of this survey with respect to literature @xcite is precisely in its interdisciplinary focus, which leads us to a novel formulation of the widely debated notion of argument quality @xcite , which we put in direct comparison to deliberative quality @xcite . the take-home message of this comparison is that the quality of a contribution to an argument cannot only be quantified in terms of its textual (linguistic/logical) properties and the relation to the preceding contributions (as commonly done in argument quality), but also the relation to the "cooperation challenge" needs to be brought in the picture. in other words, a good contribution is one that ensures the discourse to unfold productively. 2 we conclude the survey by defining the conceptual coordinates and the practical challenges of (semi-)automatic moderation, a highly integrative application of am for social good which represents a natural testbed for the integrated definition of quality discussed above. we propose to implement moderation as a form of discourse optimization, and spell out the objective of such optimization -that is to say, the desiderata for an nlp-based moderator. we discuss the concrete challenges related to the tasks of an nlp moderator, and review existing work that, albeit not targeted at nlp moderation directly, can be brought in as part of a puzzle which is both ambitious and worthwhile to pursue. ## argument mining argument(ation) mining (am) is a field encompassing varying tasks that deal with the automated analysis of arguments from natural language text. @xcite defines am as "the general task of analyzing discourse on the pragmatics level and applying a certain argumentation theory to model and automatically analyze the data at hand". the progress in the field of nlp in recent years has also influenced this research area: automatic recognition and identification of arguments has been enabled in various domains and different models for the analysis and representation of argumentative structure have been developed. furthermore, there is a growing research interest in other aspects of am, such as argument quality. ## argument quality: an integrated definition the second stage in the framework of am is defined as relation assignment (c.f. section 2.1); a complex task that aims to predict the relations holding between the arguments defined in the first stage. being able to model the relations between arguments and components within the structure, for example in argument graphs @xcite , allows us to actually work with the argumentative text in an applicationbased setting, understand the stance and context of arguments, and develop a story for the consequential impact of arguments on the discourse, among other things. generally speaking, we can use this task as an approach to analyze argument quality (aq). however, within the am community, an open question concerns the adequate definition and operationalization of the notion of aq. despite this, to move forward with the task of aq analysis and to create large corpora with crowd-sourced annotations, some approaches rely on the relative assessment of quality: given two arguments, which is more convincing? @xcite @xcite thus the natural way of quantifying the success of an argument is in terms of its persuasiveness. indeed, plenty of previous work has explored the many factors which contribute to the persuasiveness of a message: the linguistic features employed by the authors @xcite , the semantic type of claims and premises @xcite , the different sources of evidence produced to support an argument (addawood and bashir, 2016), the effects of the personality traits and prior beliefs on persuasiveness @xcite @xcite , the interaction with other participants @xcite , the use of argument invention when debating about unknown topics @xcite , the structure of the arguments @xcite , and the effect of the style of the text in achieving persuasion @xcite . persuasiveness is, however, not the only way to define whether an argument is good -at least not from a deliberation point of view. a good contribution to a debate is one which uncovers a previously unnoticed aspect of a problem, thus generating a perturbation in the discourse (controversies can be productive!). or else, a good contribution is one that settles an issue, by stating the differences between opposing views and allowing the discourse to stabilize in a series of clusters (convergence on just one position is not necessarily a good outcome). most recent research projects @xcite aim to address the challenge of redefining the notion of aq, away from persuasiveness and towards a more "situated" definition which has to do with the needs of argumentation in a real-world scenario. this new definition has been the basis for the creation of new corpora from different domains @xcite , where feature-based @xcite and neural models were tested for automatic prediction @xcite . other aspects of aq have become the subject of am research such as the relevance and impact of arguments @xcite , the verifiability @xcite , local acceptability @xcite and the best "deliberative move" @xcite . we argue that this shift is necessary for two reasons: (1) working with real-world applications of am naturally forces us into the more heterogeneous realm of data structures, such as social media, in which language, structure, and content are less uniform and confined to the classic notion of logical debate; and (2) in order to encourage deliberation from an open audience of citizens, we need to redefine our concept of aq and productive discourse such that there is equal worth and participation granted to each contributor of the argument. ## grounding aq in deliberation: moderation as a real-world application grounding aq in a discourse perspective which quantifies "team-playing" and its impact on discourse dynamics is a clear challenge, both theoretically, in the social sciences and argumentation theory, and concretely, as the empirical quantification of discourse-grounded aq will require large annotation efforts, real-time implementations, and thorough evaluation strategies. we propose to make a first step in tackling this challenge by mapping it into a concrete application: (semi-)automatic moderation implemented as a form of discourse optimization, or, as it is commonly referred to in the social sciences, facilitation @xcite . to illustrate the dynamics of moderation, let us start from concrete examples from a deliberation platform, regulationroom. this discussion forum has been employed by public institutions to gather citizens contributions on discussions targeting very heterogeneous issues (more details can be found in appendix). let us consider the following example from a discussion on the distracted driving by commercial vehicle operators (e.g., truckers and bus drivers). the posts we selected (arrows indicate comment nesting) are from the discussion are we proposing that they should be punished, for the inabilities of others? for people who spend much of their time in the car, this time might be their only chance to communicate with loved ones, do business, or make important calls. if they are physically capable to use their phones safely while driving, why restrict their freedoms? → → moderator: it's true that any distraction can cause an accident. the agency decided that texting was particularly unsafe, in part on the basis of the vtti study that we reference lower on the page. click the graphic to get a sense of the safety risks associated with different activities. a question: do you think that this rule imposes an undue burden on personal communication? what alternative restrictions on texting, if any, would you propose to impose on professional drivers? the example involves two users who clearly differ in their argumentation style and position. user 1 has a clear position on the topic (claim in bold: not just texting, but all cellphone interactions should be banned), which she/he supports with personal reports (underlined text) an emotional tone, and a style which is typical of social media text. user 2 replies, opening the post on a sarcastic note, which serves as the first premise to her/his (implicit) claim which is encoded in three rethorical questions (in bold): there should be no restrictions at all, because imposing them would be unfair. this is the case because (premises underlined): any distraction can cause an accident, some people are capable of using their phone while driving, people who spend lot of time in the car for professional reasons still need to communicate with loved ones. a moderator then joins the discussion to (a) provide a clarification as to why the focus is on texting and a link to further information on the matter, and (b) ask user 2 to elaborate on the personal communication issue, and to propose alternatives. in the appendix we report another example from the same topic and thread, where the user acts as a problematizer, challenging the scope and definition of the rule under discussion and the moderator acts as a "discourse traffic director", pointing out that the user should read and contribute to different threads in the discussion. the guidelines for human moderators in reg-ulationroom have been defined in advance in a 'moderator protocol' @xcite which reflect the moderator actions mentioned in the examples. in the protocol the moderator roles were divided into two main classes. supervision functions include general moderator actions that do not necessarily target the specific content of the posts, e.g., greeting participants, monitoring compliance with netiquette (policing), or helping with technical difficulties. substantive moderator functions aim to improve the quality of comments and promote fruitful discourse. as the examples above clearly show, this can both mean that the moderator encourages exchanges between discourse participants and participation in other posts (broadening the scope of the discussion), or helping users to improve the content of their posts (requests for clarification, focusing on one topic, substantive reasoning, sharing personal experiences). regulationroom represents an excellent example of the beneficial role of the moderator in maintaining productive argumentation from participants. however, to the best of our knowledge, there is little to no nlp work targeting moderation modeling. @xcite used data from regula-tionroom and conducted an annotation study to empirically categorize the types of moderator interventions specified in the moderator protocol. classification experiments were conducted using svm to predict the type of action a moderator would perform, given the previous comment. however this work is limited as it only focuses on two types of moderator interventions (broadening the scope of the discussion, improving argument quality) and as it does not predict whether the moderator should intervene, building on the assumption that a given comment has already been flagged as "in need for moderation". besides the concrete example of regulation-room, moderation and discourse facilitation have been, and still are, a crucial topic in digital democracy. 6 the know-how of digital democracy experts is an invaluable starting point for the application of am to moderation, as current research targets both the integration of digital solutions to facilitate online campaigns, and a critical reflection of the effects of such innovations on the deliberation outcomes. digital innovation supporting deliberation argument maps @xcite are widely employed to support online discussions, as an emerging optimization of the deliberation. given a specific topic, for example possible reactions to climate change, users who wish to contribute to the discussion are requested to structure their contribution by producing an item in a conceptual map and optionally writing an accompanying post. their contribution to the argument maps is often reviewed by a moderator. so in a sense, the argument map for a given deliberation process is the outcome of a process that comes both from below (the user) and above (the moderator). thanks to argument maps, the overall discourse picture can be overviewed and it is easier for the group of contributors to express support for one (or many) of the available options, without having to read a large number of long posts. an example of this approach is represented in deliberatorium foot_6 , an e-deliberation platform which has been extensively employed in many reference studies on the effect of digital innovation on deliberation @xcite . another example of a digital deliberation platform which integrates argument maps and offers an option for moderation is colagree @xcite . among the studies testing the impact of such digital platforms on online deliberation, @xcite tests the effect of deliberatorium's argument maps on an online discussion among the supporters of the italian democratic party concerning the desired features of electoral law to be proposed by the party to the parliament. this study compared the discussion of users employing deliberatorium and a control group using a traditional forum format which was then encoded into argument maps. the comparison showed that the argument map modality did not discourage participation, and while it appeared to make users less creative (fewer new ideas as compared to the traditional forum), it also reduced the rate of claims without further discussion. yet, the need for trained moderators tends to be a significant bottleneck (both in terms of time and of costs) in digital deliberation. moreover, empirical research on the effect of moderation on deliberation has uncovered the risks of biased moderation. for example, the experiment in @xcite tests the extent to which moderators can influence participants' behavior by expressing their views during the moderation process. ## conclusion while there are clear social drawbacks to working with data and approaches to am that limit the participation of the argumentation/deliberation, opening the floodgates to unregulated, evenly weighted contribution of all arguments also presents a dilemma. we present an interdisciplinary formulation of the notion of argument quality, which is more apt to work with heterogeneous data and platforms, such as discussion forums and social media. with the goal of ensuring a productive development of the discourse, we propose nlp-supported moderation to facilitate argumentation and deliberation in digital democracy.