Does Social Pressure Drive Persuasion in Online Fora? (2021)

Abstract: Online forums such as ChangeMyView have been explored to research aspects of persuasion and argumentative quality in language. While previous research has focused on arguments between a view-holder and a persuader, we explore the premise that apart from the merits of arguments, persuasion is influenced by the ambient social community. We hypothesize that comments from the rest of the community can either affirm the original view or implicitly exert pressure to change it. We develop a structured model to capture the ambient community’s sentiment towards the discussion and its effect on persuasion. Our experiments show that social features themselves are significantly predictive of persuasion (even without looking at the actual content of discussion), with performance comparable to some earlier approaches that use content features. Combining community and content features leads to overall performance of 78.5% on the persuasion prediction task. Our analyses suggest that the effect of social pressure is comparable to the difference between persuasive and non-persuasive language strategies in driving persuasion and that social pressure might be a causal factor for persuasion.

## introduction recent interest in analyzing online discussion forums has led to growing body of work on exploring aspects such as argument quality and persuasion in comments, posts, and tweets @xcite . in particular, @xcite 's seminal work used linguistic features of comments and their interplay with a view-holder's stance to predict persuasion. @xcite investigate differences in persuasion between sociomoral vs non-sociomoral topics. other work has explored the role of grammatical @xcite and argument structure @xcite @xcite in persuasive language. finally, @xcite cmv : in certain forms of motorsport @xmath0. we explore the premise that comments from the ambient social community influence whether the op is persuaded by a rr. in this work, we investigate the ties between social pressure and persuasion. social influence and peer pressure are highly pervasive in online discussions @xcite . thus, apart from the merits of a discussion, peer pressure can be a strong force that can make one susceptible to change @xcite . @xcite explore social influence in the context of viewing the attention a comment receives by using discussiontree-based features from the community. we focus on the task of predicting the persuasiveness in the changemyview (cmv) subreddit by also incorporating content from the community's comments. towards this, we design a structured model, which can dynamically model the evolution of community opinion and sentiment towards a discussion. our experiments show that social features are surprisingly effective at predicting persuasion. in fact, strong predictive performance can be achieved using community-based features even without looking at the language content of a persuasion-seeking post. in §2, we introduce terminology and describe the data and task. §3 describes our method and features. in §4, we present the results and qualitative discussion on the observations. we also present some analysis suggesting that social pressure might be a causal factor for persuasion. following previous work, we consider the op's original post and subsequent comments made along the op-rr discussion path (but, crucially, also including the community comments) to predict persuasion. we construct a dataset that builds on previous data compiled by @xcite by following a strategy similar to @xcite . our data consists of positive and negative instances of discussion trees (initiated by rr's but also contains comments from the rest of the community) that end in a ∆ being awarded by the op (or not, for negative examples). we remove all posts that receive less than 50 comments to ensure sufficient discussion. for discussions that receive a ∆, we only consider comments until the ∆ was received, and for the unsuccessful attempts, we consider the discussion till the last comment from the op. positive examples consist of instances where a ∆ is received, we store the corresponding path in the discussion tree along with community comments in the subtrees of path nodes. we sample negative examples by following a path (not leading to a ∆) in a depth-first search of the discussion tree such that the path length is similar to those of the positive examples. ## method given a post from the op and a reply from the rr, one can extract features from both to learn a predictive model ( similar to tan et al. (2016) ). to explore the social community's role, this can be extended to also include features based on other parts of the discussion tree (i.e. posts other than from the op and the rr). however, such an approach would be insensitive to the sequential structure of arguments in the discussion, and the evolution of social sentiment towards the op and the rr. therefore, we model this task of persuasion using a structured model that considers the discussion's temporal structure. to model social dynamics and the pressure that it exerts, we look at comments from the rest of the community and use features based on these to model the community's influence. we extract the sequence of interactions between the op and the rr along the path that leads to a ∆ (or the longest discussion path) as described earlier. the comments on this path are ordered by time. this sequence of comments serves as an input to our model. at each step in this sequence, we maintain a social vector representing the overall state of the social community at that point, modeled using comments from the community seen till the current time. our structured model, shown in figure 2 , consists of an lstm network. at each step, the model takes as input the content of an op/rr comment (represented with base-features, or an embedding representation, as described next) along with a vector of social features (see figure 2 ). the final hidden state of the rnn is fed to a feed-forward classifier, which predicts a ∆/no ∆. next, we describe three sets of features that are computed at each step, which we use for our task. base features: base features use the content of a comment to provide information about its provenance, agreement, and argument qualities. we performed n-gram analysis on comments that were successful/unsuccessful and observed that words foot_0 that express agreement or contrast had significantly different proportions. foot_1 some of these features draw inspiration from @xcite , who model interactions through frames sensitive to agreement/disagreement expressed in comments. 1. author type: indicating if the author of a comment/post is the op, the rr or community. 2. # contrast words: count of words that are used to express contrast selected from a set of 13 words ("but", "however", etc.). 3. # agreement words: count of words that express agreement selected from a set of 10 words. 4. # disagreement words: count of words that express disagreement from a set of 3 words. 5. # question words: count of question words selected from a set of 8 wh words. 6. # links: number of hyperlinks in the comment. 7. # words: number of words in the comment. ## experiments and results in this section, we evaluate the performance of our approach on the persuasion prediction task. we first perform a comparative analysis of our structured model variants using different feature families and their combinations. next, to understand the role of the evolution of community social features, we compare our structured models with unstructured models that use the same sets of features. further, we discuss some analysis that indicates a causal relationship between social pressure and persuasion. finally, we consider counterfactual explanations of the learned models, which indicate the highly significant role of social features foot_2 . feature ablation: with this model, we get the overall best performing model with accuracy of 78.5%. figure 3 shows the t-sne(van der maaten and hinton, 2008) reduction of the learned social vectors, showing clear clustering that segregates the ∆ cases from the no-∆ cases. this substantiates an underlying relation between the community comments and persuasion. role of structure: to clearly delineate the role of structure, we also explore unstructured models (logistic regression and multi-layer perceptron) trained on the averaged feature embedding of all the comments and the final social vector. volume of discussion v/s social features: in constructing the dataset, we try to match the number of back-and-forth interactions between the op and rr and include all comments from the community in this time frame. the average sequence length for the two classes ( relative importance of persuasive language vs social features: significant previous work has studied the efficacy of content-based features in predicting persuasion @xcite @xcite @xcite . we perform an analysis to quantify the relative effect-sizes of language-usage and social pressure in influencing persuasion. for this experiment, our content-based features consist of bert representations 5 of the op's initial statement and rr's opening comment alone, and do not incorporate any information about the subsequent interactions between the op and the rr. the social features are represented by their averaged value over all the community comments, as in previous experiments. this partition ensures that there is no conflation between content-based and social features as they are computed from non-overlapping parts of the discussions. the results are shown when ip was not considered 5 since bert-based representation perform the best among our content-based representations in ## conclusion we present a model that explores the role of social pressure in affecting persuasion. by incorporating structural information in discussions and simple social features, the approach shows surprisingly high predictive performance. we also perform ex-