Reinforced Sequence Training based Subjective Bias Correction (2023)

Abstract: Subjective bias is ubiquitous on news sites, social media, and knowledge resources like Wikipedia. Many existing methods for subjective bias correction have typically focused on making one-word edits and have been trained over a single (often, noisy) domain. In contrast, we propose a novel reinforced sequence training approach for robust subjective bias correction. Three of the unique characteristics of the approach are: (i) it balances bias neutralization with fluency and semantics preservation through reinforcement learning, to broaden the scope to bias beyond a single word; (ii) it is cross-trained over multiple sources of bias to be more robust to new styles of biased writing that are not seen in the training data for a single domain; and (iii) it is used to fine-tune a large pre-trained transformer model to yield state-of-the-art performance in bias text correction task. Extensive experiments show that the proposed approach results in significant improvements in subjective bias correction versus alternatives.

## introduction objective writing is essential for many important communication venues like news, encyclopedias, scientific publications, and more. and yet, bias is seemingly ubiquitous whether due to malice or unintentional habits of the writer. this subjective writing not only expresses the writer's preferences and personal interpretations, but also can influence the reader's viewpoints on the topic @xcite . hence, much like modern spelling and grammar checkers, there is a need for effective methods to detect and neutralize biased language. the goal of this subjective bias correction is to rewrite a source sentence s into a neutral sentence t that is clear, objective, and stereotype-free. furthermore, the rewritten sentence t should preserve the original meaning of the source text s. enabling such subjective bias correction is difficult. many ## reinforced bias corrector in this section, we introduce the design of the proposed subjective bias correction approach (illustrated in figure 1 ). given a biased statement x that describes a facts f using a set of words (x 1 , x 1 , ..., x n ), where n is the length of x. the aim of a bias correction system is to generate a statement @xmath0, y 2 , ...., y m ), where m is the length of y. the generated statement y is not only expected to have a neutral tone, but also preserve the original semantic content expressed in statement x. additionally, y is also expected to present the fact f in a grammatically correct and fluent language that is easy for a reader to comprehend. ## rl rewards and policy extensions in our subjective bias correction task, the key testing objectives are: (i) reduce subjective bias; (ii) preserve content expressed in the source text; and (iii) the generated text needs to be grammatically correct fluent. based on the above objectives, the designed reward function r(x, ŷ) consist of three rewards: neutrality reward. the central objective of this first reward is to remove any subjective bias that is contained in the input sentence x. the neutrality reward r n (ŷ) computes the presence or absence of biased tone in the output sentence ŷ. it is not a relative score that is computed between the input text x and output text ŷ, rather an absolute score that is computed purely on the output text ŷ. because of which, there is a possibility that the model might remove a chunk of text that is biased and output a partial sentence that is neutral. but the combination of neutrality reward r n and semantic similarity reward r s ensures the corrected text carries all the information expressed in an unbiased or neutral tone. to compute the neutrality reward, a roberta based binary classifier is used that takes the sentence as input and produces a probability score that the input text is biased or not @xcite . more details about this classifier is available on our previous work on detecting linguistic bias @xcite . ## experimental setup this sections provide an overview of the datasets, the baseline bias correction models, the evaluation metrics, and the details of their implementation. ## experimental results we present the results of the experiments through both automated and human judgement evaluation. ## conclusion and future work we proposed a hybrid method to improve the performance of seq2seq and transformer-based bias correction models by incorporating semisupervised training strategy that uses a supervised pre-training using noisy data and reinforced finetuning using high-quality cross-domain data. the proposed training method is able to successfully alleviate the exposure bias in mle optimized sequence models and address the in-stability issue in reinforcement learning methods. it also shows that by carefully designing the reward function with respect to the testing objective, high quality results can be obtained. specifically, our method was able to generate text with high neutrality. the text generated by our method is more fluent and retains more semantic information relative to previous methods. in this study, we explored only a limited range of rewards using simple aggregations, but a greater range of rewards and scaling could be explored for better domain adaptation.