Eliciting Uncertainty in Chain-of-Thought to Mitigate Bias against Forecasting Harmful User Behaviors (2024)

Abstract: Conversation forecasting tasks a model with predicting the outcome of an unfolding conversation. For instance, it can be applied in social media moderation to predict harmful user behaviors before they occur, allowing for preventative interventions. While large language models (LLMs) have recently been proposed as an effective tool for conversation forecasting, it’s unclear what biases they may have, especially against forecasting the (potentially harmful) outcomes we request them to predict during moderation. This paper explores to what extent model uncertainty can be used as a tool to mitigate potential biases. Specifically, we ask three primary research questions: 1) how does LLM forecasting accuracy change when we ask models to represent their uncertainty; 2) how does LLM bias change when we ask models to represent their uncertainty; 3) how can we use uncertainty representations to reduce or completely mitigate biases without many training data points. We address these questions for 5 open-source language models tested on 2 datasets designed to evaluate conversation forecasting for social media moderation.

## introduction conversation forecasting -where a model predicts the outcome of a partial conversation -is useful across many domains, e.g., see research on negotiation dynamics @xcite , mental health monitoring @xcite , and social media moderation @xcite . for instance, in online moderation, the forecasting task may be to predict whether a harmful behavior (like digital bullying) will eventually occur in an unfolding conversation, allowing moderators to intervene to prevent these behaviors. recently, @xcite demonstrate pre-trained language models are relatively effective conversation forecasters, setting themselves apart because they do not require copious amounts of domain-specific training data figure 1 : two difficult social media moderation examples. both instances appear as if they may derail, leading to harmful user behaviors. yet, only one does. these are real examples from the moderation corpora we study, identified using this online tool. indeed, the data used in common instructiontuning algorithms -e.g., rlhf @xcite and dpo @xcite -are designed to align language models with human values, and subsequently, avoid any propagation of harm. meanwhile, the motivating tasks of this paper draw a fine line between propagation and prediction. surely, "predicting" a harmful outcome is not "speaking into existence" but it's unclear whether this distinction is lost on "aligned" language models. or, if it is not lost, whether underlying data bias (i.e., against harmful outcomes) predisposes language models to propagate this bias when forecasting harmful outcomes. while the role of alignment mechanisms in producing model bias is difficult to confirm, foot_0 our own empirical results, and those of previous work @xcite , indicate current language models are indeed biased against predicting harmful out-comes. aptly, this paper is interested in mitigating these biases, and we approach this task using uncertainty estimation. by its nature, conversation forecasting is a highly uncertain task. for instance, two seemingly similar conversations can end with opposite outcomes (e.g., a personal attack vs. an amicable resolution, as in figure 1 ). while modeling this uncertainty has independent motivations besides the study of bias @xcite , we are specifically interested in how considering uncertainty effects the "reasoning" process of language models. foot_1 indeed, neuroscience (both cognitive and computational) recognizes the role uncertainty plays in human decision-making, wherein the brain is understood to both predict and process different forms of uncertainty @xcite . we hypothesize language models may benefit from utilizing similar patterns of reasoning, having learned these (statistical) patterns from the human-generated text on which they are trained. in particular, we hypothesize elicitation of uncertainty can mitigate bias in model predictions. in studying this broader hypothesis, we focus on three central research questions: ## experiments in general, we use hoeffding's inequality to test statistical significance at level α = 0.05. it provides a versatile (albeit, conservative) confidence interval with limited assumptions, making it applicable to accuracy (acc) and statistical bias (sb). ## conclusions this paper studies three research questions about the interaction between uncertainty estimation and forecast bias for social media moderation using language models. briefly, our findings show how asking language models to represent their uncertainty when forecasting personal attacks can reduce bias and increase accuracy, especially if a small amount of data is available to fine-tune these inferences. one interesting point, which we are unable to address, is the root cause of the biases observed. we speculate this is a result of alignment mechanisms biasing language models against predicting the harmful outcomes we wish to forecast (i.e., personal attacks). yet, more transparency in language model training is needed to investigate this issue.