Bias Neutralization in Non-Parallel Texts: A Cyclic Approach with Auxiliary Guidance (2023)

Abstract: Objectivity is a goal for Wikipedia and many news sites, as well as a guiding principle of many large language models. Indeed, several methods have recently been developed for automatic subjective bias neutralization. These methods, however, typically rely on parallel text for training (i.e. a biased sentence coupled with a non-biased sentence), demonstrate poor transfer to new domains, and can lose important bias-independent context. Toward expanding the reach of bias neutralization, we propose in this paper a new approach called FairBalance. Three of its unique features are: i) a cycle consistent adversarial network enables bias neutralization without the need for parallel text; ii) the model design preserves bias-independent content; and iii) through auxiliary guidance, the model highlights sequences of bias-inducing words, yielding strong results in terms of bias neutralization quality. Extensive experiments demonstrate how FairBalance significantly improves subjective bias neutralization compared to other methods.

## introduction avoiding bias is a cornerstone of many heavily relied upon resources. examples include wikipedia @xcite , scholarly articles @xcite , many news sources @xcite , as well as emerging large language models @xcite . in particular, subjective bias is the use of biased language in presenting objective information with an implied proposition or conclusion @xcite . biased language can manipulate our perception of reality and intensify social conflicts @xcite @xcite . ## problem statement we assume we have a corpus x, composed of biased statements x 1 , x 2 , ..., x m , and an additional corpus y containing neutral statements y 1 , y 2 , ..., y n . these corpora exhibit nonparallelism, with no direct correspondence between their respective elements, such as x 1 and y 1 . our goal is to develop a model that corrects the subjective bias in the text and generates a neutraltone output while preserving the bias-independent content of the original text. for simplicity, let's consider that x represents an input biased text and y represents a transformed neutral text. the subjective bias in text x can be modeled as a function f (x), which we aim to reduce or eliminate while generating the neutral text y. for any given x, the model should generate y such that the following conditions are satisfied: therefore, the objective function is to minimize f (y), subject to: g(y) ≈ g(x) and h(y) > h(x). hence, we need to establish mapping functions between these biased x and neutral statements y , and reciprocally, by only leveraging unpaired samples drawn from the distributions p d (x) and p d (y). ## method: fairbalance in this section we present fairbalance, which enables the training of a subjective bias neutralization model in the absence of parallel data. the goal of fairbalance is to effectively transform biased text into a neutral form, showcasing both fluency and diversity. more importantly, it achieves this subjective bias transfer while preserving the underlying semantic content of the original biased text. ## experiment setup this section provides an overview of the datasets, the baseline bias neutralization models, and the detailed evaluation results. ## conclusion in this paper, we propose a new subjective bias neutralization model trained using non-parallel data through an auxiliary guided cycle consistent gan. fairbalance consists of a bias tagger to identify subjective words in the text, a cross-domain bias classifier that accurately detects whether the given text is subjectively biased or not and a cyclic network that trains the neutralization model with a combination of adversarial, cycleconsistency and identity mapping loss. the combination of auxiliary guidance, pre-trained clas-