Dimensions of Online Conflict: Towards Modeling Agonism (2023)

Abstract: Agonism plays a vital role in democratic dialogue by fostering diverse perspectives and robust discussions. Within the realm of online conflict there is another type: hateful antagonism, which undermines constructive dialogue. Detecting conflict online is central to platform moderation and monetization. It is also vital for democratic dialogue, but only when it takes the form of agonism. To model these two types of conflict, we collected Twitter conversations related to trending controversial topics. We introduce a comprehensive annotation schema for labelling different dimensions of conflict in the conversations, such as the source of conflict, the target, and the rhetorical strategies deployed. Using this schema, we annotated approximately 4,000 conversations with multiple labels. We then train both logistic regression and transformer-based models on the dataset, incorporating context from the conversation, including the number of participants and the structure of the interactions. Results show that contextual labels are helpful in identifying conflict and make the models robust to variations in topic. Our research contributes a conceptualization of different dimensions of conflict, a richly annotated dataset, and promising results that can contribute to content moderation.

## introduction conflict is everywhere online. from political protests to spirited debates over the latest tiktok trend, these conflicts are simultaneously celebrated as promoting democracy and condemned for fostering polarization and undermining public institutions. conflicts-ideas, arguments, or attitudes that oppose each other-are also central to platform moderation and monetization: from the amplification of certain controversies to provoke user engagement, to content takedowns to comply with a platform's terms of use or national laws @xcite @xcite @xcite . not all conflicts are equal, and the confusion over the political and social value of conflict partly stems from its diverse nature. conflict exists on a sliding scale ranging from antagonistic conflict between enemies (which is often silencing, undemocratic, and hateful because it is focused on delegitimizing opponents' rights and status) to agonistic conflict between adversaries (which has productive potential for the emergence of democratic dialogue, dissent, and trust in the public and political sphere, since the struggle is over interpretation and not legitimacy to speak) @xcite . as many across the political spectrum have noted, democracy needs agonism to widen voices and prevent public spaces from becoming totalitarian or meaningless consensus hubs. but how do we know which conflicts are agonistic and which hateful? how do we know what kinds of counterspeech effectively facilitate dialogue and which destroy it? which follow the spirit of platforms' terms of use and which do not? to begin answering these questions, we present an in-depth exploration of conflict online, using the platform once known as twitter (now "x") as a case study. our work builds upon related natural language processing (nlp) research fields such as abusive language, persuasion, and constructive comments. we use twitter because, unlike the controlled environment of a priori conflictual discussions like those in reddit's 'change my view' @xcite , it offers a more organic setting. to determine the nature of a conflict, context is central @xcite @xcite . currently, though, automatic content moderation mainly focuses on a single utterance rather than on the ongoing conversation. whether a message fosters agonism depends on various dimensions, including which groups are participating, how many participants, their level of interaction, and their relative power differences, all subject to change over time. we introduce a methodology for collecting and curating a dataset of english twitter conversations embodying various aspects of dimensions of conflict. we collected conversations about trending events, as these often serve as catalysts, prompting individuals to interrogate their stance on current issues and deliberate their self-conception in relation to their views. we then annotated the conversations following our own coding schema, created with antagonistic and agonistic conflict in mind. these annotations took context into consideration rather than focusing on an isolated tweet. next, we trained logistic regression and transformer models (bert and gpt-3) on this dataset, to predict dimensions of conflict. the models are trained on human annotations of the conversations, enhanced with a) previously proposed labels for online conversation: constructiveness and toxicity; and b) contextual aspects such as cardinality (participant counts) and topology (interaction structure). we show that all models are sensitive to the specific words in the conversation, making them less generalizable across topics and domains. incorporating conversational context, however, makes the models more robust, showing that cardinality and topology are important dimensions in the prediction of online conflict. the potential use cases of this work include the measurement of productive (agonistic) versus unproductive (antagonistic) conflicts, providing insights into where learning and constructive discourse can be fostered. this work ties in with attempts at reflective content moderation, where the goal is not simply to delete harmful content or make it less visible @xcite , but also to identify and promote content that can be constructive and productive towards democratic goals @xcite @xcite . this kind of analysis has implications for detecting early signs of scapegoating and unproductive disputes-cases where patterns of discourse do not necessarily break the terms of service, but can nonetheless bring harm over time. our findings will be instrumental in shaping online discourse, aiming to harness conflict as a driver of democratic conversation (agonism) rather than as a destructive silencing element (antagonism). our main contributions are: a) a conceptualization of conflict online on a scale between antagonism and agonism; b) a process to retrieve conflict-ual conversations on twitter; c) a detailed schema to annotate conversations for various dimensions of conflict and the resulting annotations, with good inter-annotator agreement; and d) a set of experiments that show the usefulness of contextual information in predicting online conflict. ## conflict predictive models for now, we focus only on finding conflict, leaving the issue of whether the conflict is agonistic or not for future work (although we make some suggestions in section 5). our starting hypothesis is that we can find signals of conflict in the data. the target variable of interest in this paper is the presence of conflict. predictive features include: words in the text (bag of words model); constructiveness and toxicity labels; and context from the conversation, namely cardinality (number of participants) and topology (structure of participant interactions). by conversational topology we refer to the multithreaded nature of online conversations, which have been described as polylogues @xcite . let us examine it with an example, represented in figure 2 . amal sends out a message about a new mural in their city. boróka angrily replies that this mural is a waste of tax-payer money. deniz replies to boróka with a meme making fun of amal, and boróka sends the laugh emoji back to deniz in response. carlu tells amal that a section of the mural is controversial and divisive and shouldn't have been publicly-funded. eryl replies to deniz clarifying that the section of the mural they are referring to was not actually funded by the city. eryl then says the same thing to carlu. each directed edge embeds information about who received a notification from whom. if there is a path from c to a, that means that a received a notification from c. for the above example, deniz's message to boróka sends a notification to both amal and boróka, but not to carlu. intuitively, it should help to know the directed graph's structure of a conversation. it seems important to know, for example, that an utterance is part of a larger conversation sending out a notification to five different people versus a back-and-forth conversation between just two people. for the experiments, we encoded this feature as a binary 'has_bidirectionality' feature (i.e., is there back-and-forth interaction), but the dataset we will release has a full representation of this dynamic. for instance, the conversation in figure 2 is represented as: [(b, a), (d, b), (b, d), (c, a), (e, d), (e, c)], which is also converted to a matrix form in the annotated dataset. ## approximating agonism we can attempt to approximate agonism as well as other categories of these conversations given the three dimensions shown in figure 3 by defining p a as the possibly agonism score, p u as the possibly unproductive score, and p s as the possibly small-talk score as follows: using these proxy scores, we can compare the ratios of unproductive versus agonistic conversations across different trending topics over time, shown in figure 3 . part of our future work involves a qualitative analysis of all the conversations in that green zone, to investigate whether they have traces of agonism. furthering our analysis, we sampled the top 100 bidirectional conversations from the highest p u and p a scores. a different set of annotators was then tasked with categorizing each conversation as either agonistic, antagonistic, or neither. this resulted in an inter-annotator agreement quantified by κ = 0.44, indicating a moderate level of agreement. the distribution of resulting labels in agreement is shown in figure 3 : conversations plotted by conflict, constructiveness, and toxicity likelihoods. we are postulating the green volume as the 'zone of agonism'. ## conclusions and future work the long-term goal of our program of research is to identify agonistic conflict and distinguish it from less productive and democratic forms of conflict. the work presented here contributes a definition of agonism and its operationalization in a coding schema, an important step in approaching content moderation as a task of fostering agonistic dialogue, a productive form of conflict that is essential to democratic dissent. we introduce a richly annotated dataset of online conversations containing conflict. using this data, we test methods that can identify conflict from conversations, crucially incorporating contextual information. we experiment with dimensions of the context that we believe can be proxies for agonistic conflict, including the presence of constructiveness and toxicity, the number of participants, and the topology of the conversation, which includes the level and direction of interaction. we show that the contextual information is key to identifying conflict, especially because it helps the model remain topic-agnostic. this contextual approach can be helpful not just in identifying conflict and agonism, but also in detecting abusive language, as it provides a wider view of the conversation, rather than whether an utterance is abusive or not in isolation. we have made data and code available in a repository. 6 this includes: the dataset of 4,022 conversations with annotations, the code to collect conversations, the labelstudio annotation scheme, and the code for all the experiments described. the appendices in this paper include detailed information about the data collection process, the coding schema, and multiple examples of annotated conversations from the dataset. the repository also contains links to raw data, a larger dataset of conversations we collected but have not annotated (the entire dataset contains 220,626 conversations). we will, additionally, make available a demo web application linked in the repository to experiment with the model results. the next steps in our research program involve deploying the other dimensions in the data (us vs. them conflict; individual/group conflict, etc.). we also plan to perform a qualitative analysis of conversations with high conflict, high constructiveness, and low toxicity, which we have defined as a potential zone of agonism. further experiments will ex-tend this model to other topics in our larger dataset, to test whether it can be used for semi-automatic annotation. we also plan to gather additional topics that generated discussion by querying keywords from monthly snapshots of the wikipedia portal:current_events and the top_25_report through the wayback machine. defining and identifying agonism in conversations is a difficult task. we also acknowledge the difficulty of fostering the kinds of spaces that are conducive to agonistic debate, both online and offline, which is why an interdisciplinary approach with both quantitative and qualitative approaches such as the one presented here is needed.