Exploiting Personal Characteristics of Debaters for Predicting Persuasiveness (2020)

Abstract: Predicting the persuasiveness of arguments has applications as diverse as writing assistance, essay scoring, and advertising. While clearly relevant to the task, the personal characteristics of an argument’s source and audience have not yet been fully exploited toward automated persuasiveness prediction. In this paper, we model debaters’ prior beliefs, interests, and personality traits based on their previous activity, without dependence on explicit user profiles or questionnaires. Using a dataset of over 60,000 argumentative discussions, comprising more than three million individual posts collected from the subreddit r/ChangeMyView, we demonstrate that our modeling of debater’s characteristics enhances the prediction of argument persuasiveness as well as of debaters’ resistance to persuasion.

## introduction persuasion is a primary goal of argumentation @xcite . it is often carried out in the form of a debate or discussion, where debaters argue to persuade others to take certain stances on controversial topics. several studies have examined persuasiveness in debates by probing the main factors for establishing persuasion, particularly regarding the role of linguistic features of debaters' arguments @xcite , the interaction between debaters @xcite , and the personal characteristics of debaters @xcite . while the impact of debaters' characteristics on persuasiveness has been observed in online debates, the exploitation of these characteristics for predicting persuasiveness has been done based on explicit characteristics-related information in users' profiles or on questionnaires. for example, @xcite performed a personality trait test for selected people and asked them for their stances on specific topics to estimate their beliefs. also, @xcite used the information in users' profiles in an online forum, where their stances on controversial topics are explicitly stated, as a proxy of their beliefs. such a means of exploitation limits the applicability of predicting persuasiveness, as the characteristics of debaters are usually not explicitly available in online debates, and it is not practicable to survey every debater. the paper at hand studies how the characteristics of debaters can be modeled automatically and utilized successfully for predicting persuasiveness. to this end, we propose a new approach of various features that capture the beliefs, interests, and personality traits of debaters on the subreddit "changemyview" based on the debaters' previous activity on the reddit.com platform. we apply this approach to the tasks of predicting argument persuasiveness and predicting debater's resistance to persuasion. our experiments show that incorporating debater characteristics improves the prediction effectiveness of the two tasks over previous approaches which rely primarily on linguistic features. interestingly, personality traits alone were the most predictive feature for resistance to persuasion, outperforming the linguistic features of the post itself. the contribution of this paper is three-fold: the prediction of argument persuasiveness has been investigated in several studies (e.g., @xcite , @xcite , @xcite , and @xcite . to mitigate the lack of annotated data, @xcite proposed a light supervision model for persuasiveness scoring by explicitly modeling errors that negatively impact the persuasiveness of an argument. @xcite built an annotated corpus of concessions in cmv discussions using expert annotations and automatic classification. they observed that concessions are equally distributed among persuasive and non-persuasive threads and that they do not play any significant role as a means of persuasion. studying the effect of argument sequencing, @xcite provided evidence that the order in which arguments are presented plays a crucial role in persuasion. considering the importance of linguistic features, @xcite studied debater skill as it improves over time due to prolonged interaction with other debaters. combining linguistic features such as length of turns, and co-occurrence of hedges and fighting words, they developed a strong estimator of debaters' persuasive skill over time. apart from content-based features, modeling the audience is crucial for predicting persuasiveness. @xcite studied the interaction of social media argument types with audience factors, to compare the belief change that results from social media dialogs to that from professionally curated monologic summaries. participants were profiled for prior beliefs and personality types-neutral and balanced arguments were successful at changing the beliefs of all participants. in contrast, an entrenched audience was convinced by more emotional dialogs. @xcite further explored the role of prior beliefs by predicting the success of debaters with explicitly stated religious and political ideologies, and found that readers were more likely to be convinced by a debater with the same ideology. @xcite examined linguistic features of debates together with audience features such as demographic information, prior beliefs, and debate platform behavior. they found that for a priori undecided users, audience features were prominent in predicting persuasiveness. for decided users, stylistic features of the argument were more effective. closely related to our work, @xcite explored the effects of debaters' language, their prior beliefs and traits, and social interactions with other users on the ddo (debate.org) platform. the social interaction features were crucial in predicting the success of a debater, and combining them with features capturing debaters' language performed best. ddo explicitly provides information on personal traits of debaters, including demographics such as gender, ethnicity, and user's beliefs. our data source lacks this information, which increases the difficulty of modeling users. recently, @xcite modeled the interplay of comments to study their cumulative influence on persuading the audience. they proposed a sequential model that captures the interplay as local and non-local dependencies and outperforms studies focusing only on lexical features. the "changemyview" subreddit (cmv) has been exploited for argument persuasiveness in many studies. for example, @xcite , @xcite , and @xcite used cmv as a source of real-world persuasive discourse. in this paper, we address the two persuasiveness tasks that have been proposed by @xcite : we use reddit.com as a source of debates. this platform comprises a variety of user-generated content, organized within communities called "subreddits". the subreddit "r/changemyview" (cmv) focuses on organized debates. as shown in figure 1 , contributors to cmv make an original post (op) stating their stance on a debate topic of their choice. other reddit users may post opposing comments in response, to which the submitter of the op may respond in turn, and award a "delta" to any comment that successfully changed their stance. the cmv setting allows deriving gold standard labels for the two studied persuasiveness tasks. in original post title: "cars should be equipped with both angry and apologetic horns" comment #1 ... a standard and widely used method of conveying thanks and apology already exists: blinking your hazards a few times. ... comment #2 ... you jump a big step here, because you don't explain why the traditional light wave/hand under mirror gesture isn't effective. i see it pretty much every time ... ah, really? didn't know that! i give you delta! ∆ specific, we can assume that the comments that receive deltas are persuasive compared to those which do not. here, an individual comment, made by a reddit user in response to a cmv post, can be regarded as an argument. this is a simplification, but it is reasonable considering the characteristics of cmv @xcite . also, if the user who submits the op gave a delta to any response, we can assume that their stance is malleable. following the assumptions above, cmv has been crawled and the comments there have been labeled for persuasiveness, resulting in the cmv corpus of @xcite , which covers the complete set of posts and comments until 2015. to extend this corpus, we collected all available cmv posts and comments from the foundation of the subreddit in 2005 until september 2017. ## modeling debater characteristics we develop features to capture the interests, prior beliefs, and personality traits of debaters, and compute the similarity between two debaters based on these features. ## experiments and results we evaluate our approach against the tasks described in section 3. the tasks are predicting ar7 the liwc api recommends input to be at least 300 words and up to 1000 words long, hence we exclude debaters with less than 300 words of posting activity. ## conclusion this paper proposes a new approach for modeling the personal characteristics of debaters including interests, prior beliefs, and personality traits for predicting both argument persuasiveness and debaters' resistance to persuasion. we hypothesize that these characteristics can be induced automatically from the history of debaters' activity such as their earlier texts. based on this hypothesis, we develop a set of various features to capture debaters characteristics using the reddit.com platform. applying these features on persuasiveness corpora derived from the subreddit r/changemyview, we accomplish a fair improvement on the effectiveness of tackling the studied persuasiveness tasks, particularly in predicting the debaters' resistance to persuasion. in the future, we plan to consider the ethos mode of persuasion by exploring how debaters strengthen their credibility in debates.