Status Biases in Deliberation Online: Evidence from a Randomized Experiment on C hange M y V iew (2022)

Abstract: Status is widely used to incentivize user engagement online. However, visible status indicators could inadvertently bias online deliberation to favor high-status users. In this work, we design and deploy a randomized experiment on the ChangeMyView platform to quantify status biases in deliberation online. We find strong evidence of status bias: hiding status on ChangeMyView increases the persuasion rate of moderate-status users by 84% and decreases the persuasion rate of high-status users by 41% relative to the control group. We also find that the persuasive power of status is moderated by verbosity, suggesting that status is used as an information-processing heuristic under cognitive load. Finally, we find that a user’s status influences the argumentation behavior of other users they interact with in a manner that disadvantages low and moderate-status users.

## introduction fair and equitable deliberation facilitates consensus among decision makers with diverse viewpoints @xcite . deliberation increasingly takes place online, on platforms such as wikipedia, reddit, and github @xcite . such platforms typically incentivize user engagement by rewarding active users with some form of visible status @xcite . while status is a powerful incentive @xcite , it could also act as an informationprocessing heuristic under cognitive overload @xcite , and bias deliberation to favor highstatus users by virtue of its persuasive power. status effects have been reported in the context of collaborative software development @xcite , organizational communication (pingwest, 2020), knowledge curation @xcite , social media discussions @xcite , among other settings. ## research context -changemyview changemyview is an online deliberation platform hosted on reddit @xcite . posters on changemyview share posts that are subsequently attacked by challengers seeking to persuade the poster to change their view. at any point in the conversation with a challenger, the poster may explicitly indicate that their view has changed using the ∆ symbol (or equivalent alternatives), which awards the challenger a ∆ point. the total number of ∆ points earned, if non-zero, is displayed next to each challenger's username (illustrated in figure 1 ); we term this their status. we only consider challengers that respond directly to posts, and not users that comment on challenger responses but do not respond to the post itself. the availability of explicit indicators of persuasion is unique to changemyview and has been widely used by research on computational argumentation @xcite . we use these persuasion indicators to measure the persuasion rate of each challenger in a time period (the fraction of posts they challenged in that time period which led to the poster changing their view). this is the outcome in our randomized experiment. the possibility of status biasing deliberation has been raised by changemyview users in the past foot_1 , but not led to any subsequent policy changes. a key reason for this is the difficulty of interpreting a disparity in persuasion rates between low and high-status challengers as a status bias. a disparity could arise from high status challengers being more skilled at debating than low status challengers, for example. however, a status bias is the disparity caused by status visibility, and not by other confounders. by randomizing the challengers for whom we hide status, we eliminate the impact of all confounders to quantify this causal effect. ## randomized experiment design in this section, we use the potential outcomes framework @xcite to formalize our causal inference setup. ## quantifying status bias we first examine the average persuasion rates of treated and untreated challengers within each stratum. figure 2 shows that the impact of hiding status is heterogeneous. among low-status challengers (stratum 1), there is a negligible difference in the average persuasion rates of treated and untreated challengers. among moderate-status challengers (stratum 2-4), treated challengers have a higher average persuasion rate than untreated challengers. among high-status challengers (stratum 5-7), treated challengers have a lower average persuasion rate than untreated challengers. based on this heterogeneity, we partition challengers into 3 groups (as shown in figure 2 ) and report estimates of the unconditional status bias along with estimates of the conditional status bias by group in in contrast, the status bias for moderate-status challengers is statistically significant (@xmath3.010); hiding the status of moderate-status challengers increases their persuasion rate by 1.2 percentage points, which corresponds to an 84% increase over the control group average persuasion rate for moderate-status challengers. the status bias for high-status challengers is also statistically significant (@xmath4.043); hiding the status of high-status challengers decreases their persuasion rate by 2.3 percentage points, which corresponds to a 41% decrease over the control group average persuasion rate for high-status challengers. separate t-tests show that the difference between the estimated conditional status bias in each pair of challenger groups is also statistically significant (@xmath5.05). we further decompose the status effect heterogeneity by estimating the conditional status bias in each stratum. we use a linear regression model similar to equation (4) that includes interactions between the treatment indicator t i and stratum indicators i[s @xmath6, . . . , 7 (instead of the group indicators i[g @xmath7. the estimates in ## the moderating effect of verbosity we now examine a possible mechanism underlying the persuasive power of status. psychological theory suggests that status could be used as an information-processing heuristic under cognitive overload @xcite . if this theory holds, we expect status to have a larger impact when the information being processed exerts a greater cognitive load. hence, we use challengers' average response length in characters as a proxy for the average cognitive load they exert on posters. we partition challengers based on their average response length quartile and estimate the conditional status bias by quartile separately for challengers in stratum 2 and 7 (based on the estimated status effects in to estimate the conditional status bias by quartile, we use a linear regression model similar to equation ( 4 the estimates in ## the impact of status on transactivity in this section, we further explore how status affects persuasion by quantifying the impact of status on argumentative behavior. specifically, we quantify the impact of challengers' status on the degree of transactivity @xcite in posters' replies to challengers. transactivity corresponds to conversational behavior wherein speakers build on each other's ideas @xcite . while several types of transactivity exist @xcite , we focus on the strongest form of transactivity exhibited by posters: when they counter parts of challengers' responses. ## related work our work is related to several directions of research on status, persuasion, and causal inference. status in online platforms. extensive prior literature finds that status has powerful effects on user behavior online @xcite . status-conferring awards have been shown to increase the retention rate of wikipedia editors @xcite , to increase the production of user-generated content on reddit @xcite , to "steer" user behavior on stackoverflow @xcite , and to correlate with coordination @xcite , politeness @xcite , and other linguistic behavior online @xcite . our work extends this literature by quantifying the impact of status on persuasion and on conversational transactivity. persuasion and argumentation online. extensive prior literature has studied various aspects of persuasion online @xcite . this literature primarily relies on observational or quasi-experimental studies, and focuses on correlations between content factors and persua-sion without considering the role of status or other source factors. recently, @xcite examine (with an observational study) how persuasion is correlated with author identity on wikipedia, and @xcite examine (with a quasiexperimental study) the impact of status on persuasion in changemyview. our work extends the literature on source factors and persuasion with a randomized experiment to establish causality in a more credible manner than prior studies. causal inference and natural language processing. our work contributes a case study and dataset to the nascent literature on causal inference and natural language processing @xcite @xcite @xcite , specifically in the settings where text is a moderator (section 5) and where text is an outcome (section 6). ## conclusion and future work we design and deploy a randomized experiment on the changemyview online deliberation platform to quantify the causal impact of status visibility on persuasion, and hence, the existence of status bias. we find that moderate-status users are disadvantaged and high-status users benefit from their status visibility. we also find that the persuasive power of status is moderated by verbosity, suggesting that status is used as an information-processing heuristic by posters to alleviate the cognitive load of verbose (cognitively costly) challenger responses. we also find evidence for increased transactivity (crossexamination and critique) in the replies to low and moderate-status challengers that is caused by their status visibility, suggesting a possible mechanism via which status affects persuasion. in future work, we plan to address violations of the consistency identifiability assumption by deriving bounds on the treatment effect @xcite . we also plan to examine the moderating effect of linguistic factors on the persuasive power of status, such as politeness @xcite and the usage of emotional appeals.