Argument Mining on T witter: A Case Study on the Planned Parenthood Debate (2021)

Abstract: Twitter is a popular platform to share opinions and claims, which may be accompanied by the underlying rationale. Such information can be invaluable to policy makers, marketers and social scientists, to name a few. However, the effort to mine arguments on Twitter has been limited, mainly because a tweet is typically too short to contain an argument — both a claim and a premise. In this paper, we propose a novel problem formulation to mine arguments from Twitter: We formulate argument mining on Twitter as a text classification task to identify tweets that serve as premises for a hashtag that represents a claim of interest. To demonstrate the efficacy of this formulation, we mine arguments for and against funding Planned Parenthood expressed in tweets. We first present a new dataset of 24,100 tweets containing hashtag #StandWithPP or #DefundPP, manually labeled as SUPPORT WITH REASON, SUPPORT WITHOUT REASON, and NO EXPLICIT SUPPORT. We then train classifiers to determine the types of tweets, achieving the best performance of 71% F1. Our results manifest claim-specific keywords as the most informative features, which in turn reveal prominent arguments for and against funding Planned Parenthood.

## introduction the goal of argument mining is to automatically extract arguments-typically defined as consisting of both a claim and at least one premise supporting the claim-from text in various domains. by analyzing the argumentative structure, we can not only identify claims, but also gain a deeper understanding of the evidence and reasons behind the claims @xcite @xcite @xcite . and the domains for argument mining have quickly expanded to include less formally written text on social media @xcite @xcite @xcite . yet, the effort to mine arguments from twitter has been limited due to a rather obvious reasontweets are often too short to contain an entire argument, i.e., a claim and a premise @xcite . for this reason, existing approaches to argument mining on twitter typically focus on identifying claims, evidence, or either, but not both at the same time @xcite @xcite . this is not ideal, since the underlying rationale can be as important as the claim. to mine full arguments from twitter, we propose a novel problem formulation based on an observation-some trendy hashtags serve as placeholders for claims, which may or may not be supported by the contents of the tweets containing them. in the case of the planned parenthood debate, the two opposing sides use hashtags #standwithpp and #defundpp to specify their respective claims; only a subset of the tweets serve as premises supporting the given claim. for instance, example 3 in to demonstrate the efficacy of the proposed formulation, we mine arguments for and against funding planned parenthood expressed on twitter: we first present a new dataset of 24,100 tweets containing hashtag #standwithpp or #defundpp, each manually labeled as support with rea-son, support without reason, or no ex-plicit support. we then train several classifiers and test them on 30% of the dataset held-out in advance. we find that fine-tuned bert performs the best, achieving 71% f 1 . we also show that claim-specific words serve as the most important features for this task, which in turn reveal important arguments for and against funding planned parenthood. why planned parenthood? the planned parenthood debate is multi-faceted, involving issues like the personhood of fetuses, women's rights, and health services to people of various socioeconomic status. a major benefit of automatically extracting arguments from twitter is that it provides an easy access to arguments people have made. this is especially helpful for complex topics like planned parenthood, where unique but noteworthy arguments can be lost in the midst of others. from a practical perspective, each side of the planned parenthood debate has a dominantly used hashtag, allowing us to target two specific hashtags and gain a holistic view of the debate. our main contributions are threefold: ## premise-tweet identification argument mining consists of several subtasks, such as identifying argumentative spans of text, determining argumentative units-e.g. premise and claim-within the arguments, and recognizing the argumentative structure connecting the units. in this work, however, the claim is easily identifiable, as we assume that it takes the form of a hashtag, i.e., claim-hashtag, that is known in advance. thus, the core of our approach to mining arguments on twitter is deciding whether or not a given tweet is a premise-tweet for a given claim-hashtag. to tackle the task, we train fine-tuned bert, cnn, ## limitations and implications there are two main limitations of this work that need to be considered in future research. first, labeled training data is required to train a premisetweet classifier for each claim of interest. the most informative features for our classifiers are specific to the claim-hashtag they are trained for; even though #standwithpp and #defundpp are on the same topic, the informative features are drastically different. this suggests that a classifier trained for a claim-hashtag likely is not effective for identifying premise-tweets for other claims-hashtags. in fact, this was confirmed through cross-domain testing, i.e., we fined-tuned bert on the #standwithpp training set and tested on the #defundpp test set, and vice versa. there was a significant drop in the f1 score from 71% to 56% in both scenarios. to determine how much training data is necessary, we fine-tuned bert on randomly selected subsets of the training sets. we tested training set sizes in increments of 1k as shown in figure 3 . the same pattern can be observed for both claimhashtags: there is a drastic improvement in performance after fine-tuning with even a small training set of size 1k; and the performance plateaus after increasing the size to about 3k to 4k. based on the result, we suggest that a labeled dataset of at least 3k tweets is prepared to train a premise-tweet classifier for a claim-hashtag of your interest. second, our results are based on experiments with tweets containing two specific claim-hashtags. future work should consider a more diverse set of claim-hashtags. it will not only test the generalizabilty of this approach, but may also reveal informative features that are claim-independent. manually compiling a list of diverse claimhashtags can be laborious, however. to alleviate this issue, we have identified a class of claimhashtags that can be automatically recognized. these hashtags represent so called policy propositions, meaning they suggest policies, or courses of action to be taken @xcite . they typically take the form of an imperative-starting with a verb and ending with a noun, e.g. #stand-withpp, #defundpp, #fightfor15, #legalizemarijuana, and #banguns. hashtags do not contain spaces, but the camelcase capitalization can be used for tokenization-a capitalized letter marks the beginning of a new word, unless several capitalized letters appear in succession to denote a proper noun. the repetitive use of hashtags in tweets is helpful in this regard, as it is very likely that at least one variation of a given hashtag is in camelcase. thus, to identify a diverse set of claim-hashtags, we suggest the method of identifying trending hashtags representing policy propositions. ## conclusion twitter is a popular platform to share opinions, which may be accompanied by the underlying rationale. however, the effort to automatically extract arguments from twitter has been limited, mainly due to tweets typically not containing both a claim and a premise. the brevity renders it difficult to apply argument mining techniques designed for other domains, where claims and premises can be extracted together. in this paper, we proposed a novel problem formulation to mine arguments from twitter: we formulated argument mining on twitter as a text classification task to identify tweets serving as premises for hashtags that represent claims. we demonstrated the efficacy of this formulation by mining arguments for and against funding planned parenthood expressed on twitter. we achieved the best performance of 71% f 1 with fine-tuned bert. we also showed that domain specific words serve as the most important features, which in turn reveal prominent arguments in support of the given claim. in future work, we would like to continue the effort addressing the issues discussed in section 6.