Bias Neutralization in Non-Parallel Texts: A Cyclic Approach with Auxiliary Guidance
Authors: Karthic Madanagopal
James Caverlee
Venue: EMNLP 2023 Main (2023)
Keywords: Bias Correction
Subjective Bias
Generative Adversarial Networks
Unsupervised Learning
Auxiliary Guidance
TL;DR: A Cyclic Adversarial Network-based approach for automatic subjective bias correction in non-parallel text.

Abstract: Objectivity is a goal for Wikipedia and many news sites, as well as a guiding principle of many large language models. Indeed, several methods have recently been developed for automatic subjective bias neutralization. These methods, however, typically rely on parallel text for training (i.e. a biased sentence coupled with a non-biased sentence), demonstrate poor transfer to new domains, and can lose important bias-independent context. Toward expanding the reach of bias neutralization, we propose in this paper a new approach called FairBalance. Three of its unique features are: i) a cycle consistent adversarial network enables bias neutralization without the need for parallel text; ii) the model design preserves bias-independent content; and iii) through auxiliary guidance, the model highlights sequences of bias-inducing words, yielding strong results in terms of bias neutralization quality. Extensive experiments demonstrate how FairBalance significantly improves subjective bias neutralization compared to other methods.

Bias Neutralization in Non-Parallel Texts: A Cyclic Approach with Auxiliary Guidance
Karthic Madanagopal, James Caverlee Texas A&M University, College Station, USA karthiclle amu. edu, caverleeftam,, edu
Abstract
Biased Statements
The "winners" are chosen by a group of aca- demics, activists, distinguished businessmen. and trade unionists.
Objectivity is goal for Wikipedia and many news sites, as well guiding principle of many large language models. Indeed, sev- eral methods have recently been developed for automatic subjective bias neutralization. These methods, however, typically rely on parallel text for training (i.e. a biased sen- lence coupled with a non-biased sentence), demonstrate poor transfer to new domains, can lose important bias-independent con- text. Toward expanding the reach of bias neutralization, we propose in this paper new approach called FairBalance. Three of its unique features are: i) a cycle consistent adversarial network enables bias neutraliza- tion without the need for parallel text; ii) the model design preserves bias-independent tent; and íii) through auxiliary guidance, the model highlights sequences of bias-inducing words, yielding strong results in terms of bias neutralization quality. our evaluations volving seven models comprising of adversar- ial and -on-saderalrial models, the FairBal- ance method showed notable improvement in bias neutralization based on subjective human judgment when compared to other techniques.
Released on May 16, 2002, Attack of the Clones was generally perceived as a slight improve- ment upon the feeble The Phantom Menace, though not at all on par with the original Star Wars trilogy.
Only tiny proportion of these companies have so far grown into multinationals: ARM, Autonomy Corporation, and AVEVA are the most obvious examples, and more recently CSR has rapid growth due to the uptake of Bluetooth.
Table Sample of biased statements extracted from Wikipedia. Spans highlighted are the output our bias tagger by type such Epistemological Bias, Framing Bias and Demographic Bias.
(Greenstein and Zhu, 2012; Liu et al., 2021; Re- casens et al., 2013). To illustrate, Table high- lights sentences from Wikipedia displaying epis- temological bias (i.e. casting doubt), framing bias (i.e. one-sided frames), and demographic bias (i.e. unequal representation based on demo- graphic characteristics). Encouragingly, a number of studies have identified efficient methods de- tect biased language in text (Recasens al., 2013; Bhosale et al., 2013: Mísra and Basak, 2016; Hube and Fetahu, 2018; Zhong, 2021; Madanagopal and Caverlee, 2022) and a few studies have began explore automatic subjective bias neutralization methods (Pryzant et al., 2019; Liu et al., 2021; Madanagopal and Caverlce, 2023). These ap- proaches, however, face a number of key chal- lenges:
1 Introduction
Avoiding bias a cornerstone of many heavily re- lied upon resources. Examples include Wikipedia (Greenstein and Zhu, 2012), scholarly articles (Politzer-Ahles et al., 2020), many news sources (Liu et al., 2021), as well as emerging large lan- guage models (Patel and Pavlick, 2021). In par- ticular, subjective bias the use of biased lan- guage in presenting objective information with an implied proposition or conclusion (Wiebe, 1994). Biased language can manipulate our perception of reality and intensify social conflicts (Greenstein and Zhu, 2012, 2014; Beukeboom and Burgers, 2017).
Reliance on Parallel Training Data. The major- ity existing subjective bias neutralization mod- els rely on supervised methods trained over a
While many communication venues strive for objectivity, subjective writing can still arise

---

dataset of paired examples (Pryzant et al., 2019; Liu et al., 2021; Madanagopal and Caverlee, 2023), consisting of biased sentence X (e.g., "John is a great actor") and its neutral version Y (e.g.. "John is an actor"). These datasets are chal- lenging and expensive prepare, and simply do not exist for many domains.
bias neutralization results.
In summary:
We propose cyclo-consistent adversarial network for subjective bias neutralization, which can be trained effectively in the ab- sence of parallel data and efficiently pre- serves the bias-independent characteristics of text during the ncutralization process.
Inefficient Domain Adaptation. Even with mod- els trained in one domain, there is often poor adap- tation to other domains. For example, words like "affordable health" or "abortion" may be highly charged in political speeches (Sim et al., 2013; Misra and Basak, 2016; Liu et al., 2021), but of lit- tle salience in other domains. Since Wikipedia is the primary training dataset for existing subjective bias neutralization methods (Pryzant et al., 2019; Zhong, 2021), patterns of bias there may not gen- eralize.
The approach effectively handles longer in- put sequences and produce text that is both semantically coherent and fluent by using a transformer-basee model as generator.
We improve the training process and make the results more consistent, by incorporating an auxiliary guidance mechanism that guides the generator on which parts of the text are bi- ased and need to be rephrased or removed. It also efficiently address multi-occurrence bias a single sentence.
Content Preservation. Besides neutralization, subjective bias correction aims to preserve the bias- itaeenddeenn information from the original text in the new neutral text. Existing methods use n-gram precision metrics like BLEU to refine the performance content preservation by com- paring the generated neutral text against human- generated reference text (Pryzant et al., 2019; Zhong, 2021). The absence of such reference (par- allel) text motivates the need for training methods that naturally incorporate the content preservation objective.
We use pre-trained cross-domain bias clas- sifier as a self-supervised signal during adver- sarial training, eliminating the need for ex- ternal labels. The cross-domain nature also helps to efficiently adapt to other domains, making it more practical for real-world ap- plications.
Through seven models (3 baselines and 4 ad- versarial) along with human judgement evalua- tion, we demonstrate that the proposed approach outperforms conventional text style transfer tech- niques through a combination of subjective and objective evaluation metrics. Furthermore, both quantitative and qualitative evaluations on seman- tic similarity and fluency benchmarks demonstrate good preservation of semantic content of the orig- inal text, as well coherent generated text.
To address these challenges, we propose Fair- Balance, a content preserving subjective bias neu- tralization approach that is designed to mitigate subjective bias in text without the need for parallel data. Our approach employs a cyclic adversarial network architecture, which includes two sequen- tial generative adversarial networks (GAN) with a pre-trained discriminators. The first GAN takes a biased text as input and generates neutral text and the second GAN takes the output of the first GAN as input to generate a biased text. By com- puting the cyclic loss between the original biased text with the generated biased text, the network ensures the bias independent content preserved in the subjective neutralization process. Another key property of FairBalance is an auxiliary guid- ance mechanism with the help of a bias tagger that guides the generator toward identifying biased portions of the text and providing instructions on what should he eliminated or rephrased. By in- tegrating this approach, we accelerate the training process and achieve more consistent and reliable
2 Related Work
2.1 Subjective Bias Neutralization
Efforts to address bias neutralization in text often focus on demographic bias such as gender bias (Manzini et al., 2019; Zhao et al., 2017, 2019; Bordia and Bowman, 2019; Wang et al., 2018). Pryzant et. al (Pryzant et al., 2019) was the first to address generic linguistic bias correction us- ing two types of models. Their modular model first detects bias and later performs the rewrite, whereas the concurrent model performs bias neu- tralization as an end-to-end task. Both methods achieve reasonable performance, but are restricted to single word bias (e.g., removing single adjec- tive). Additionally, both models resemble pointer

---

Compate emantic
Contan
Biased Statement
Ncutral Statement
Biased Statement
He respected singer- songwriter in country music, westem music (North America), and popular mmasic.
He a singer-songwrier country music, westem music (North America), and popular music.
He is tamous singer- songwriter in country music, western music (North America), and popular music.
Neutraliration
De De-neutralization
Check Biased Language
Compare Semantic
Content
Figure 1: Overview of our proposed framework. Neutralization ensures the subjective bias is removed and the de-neutralization ensures the semantic content is preserved.
networks that attempt to maximise the occurrence of the same words in the source sentence, result- ing in generated sentences that can lack fluency and diversity. Liu et al. (Liu et al., 2021) worked on depolarizing political text. In both these ap- proaches, text segments (words or sentences) that are subjective or polarizing are first identified and then replaced with those that are semantically similar but less subjective. Madanagopal et al. (Madanagopal and Caverlee, 2023) proposed a re- inforcement learning-based method to improve the performance of supervised seq2seq models. They also addressed the problem of multi-word bias. is worth noting that all existing models for bias neutralization have been supervised models that rely on labeled datasets for single-word bias neu- tralization. In contrast, this paper targets self- supervised bias neutralization while also address- ing wordi-word bias, with an emphasis on fluency and diversity in the generated sentences.
style (Liu et al., 2020; Yi et al., 2021; Jin et al., 2020). However, these studies have observed that the overlap between the generated and ground- truth data is very low, indicating that the autoen- coder model tends to discard too much informa- tion from the original text (Hu et al., 2017; Bow- man et al., 2015). While these methods focus mainly on changing the attribute style of the text, they do not necessarily preserve the content. Our proposed method draws inspiration from attribute style transfer techniques. However, we specifi- cally focus on addressing content-leaking that oc- curs during training for subjective bias neutraliza- tion using non non-part con-parllll data.
2.3 Generative Adversarial Networks
GANs have shown remarkable results in image style transfer with non-parallel data (Goodfellow et al., 2020). One limitation of using GANs in the text domain is the non-differentiable nature of discrete word tokens. Despite these challenges, GANs have been effectively applied to text style transfer (Yang et al., 2018; Liu et al., 2021). Liu et al. paired a transformer-basee language model with a GAN network to neutralize and reverse the polarity of news articles (Liu et al., 2021), while Yang et al. employed a conditional GAN-based framework towards diverse paraphrase generation of text (Yang et al., 2018). Fedus et al. developed MaskGAN, integrating GAN with actor criticism to complete missing text based on the context (Fee dus et al., 2018). 8). Zhu et al. proposed CycleGAN, GAN variant to learn image-to-image translation by leveraging two GAN networks in sequence to improve content preservation, functioning on the principle of cycle-consistent adversarial networks (Zhu et al., 2017). Wang et al. evaluated cycle training- based methods for generating text from structured data and showed the performance on- par with supervised trained models (Wang et al.,
2.2 Text Style Transfer
Bias ncutralization shares many similarities with text style transfer. Typically in text style trans- fer, an input text is reconstructed so that linguis- tic attributes are transferred from one value to an- other while preserving content (Jin et al., 2022; Lample et al., 2018). Supervised methods on language style transfer typically use a sequence- to-sequence (seq2seq) neural machine translation model to transfer attribute styles using parallel corpora (Briakou et al., 2021; Madaan et al., 2020; Prabhumoye et al., 2018; Pryzant et al.. 2019). In the absence of parallel corpus, text style trans- fer relies on disentanglement, prototype editing, and pseudo-paralle! corpus construction (Jin et al., 2022). With disentanglement, the content and lin- guistic attributes of interest such as sentiment are separated using variants of autoencoders, and the disentangled content rewritten with the target

---

Identity
Identity
loss
loss
Reconstructed Biased Text
Auxiliary Guidance
Restrict injecting new bias
Neutral Text
Restrict injecting new bias
Generator
Generator
Biased Text
Encoder
Decoder
Encoder
Decader
Bias-to-Neutre Neutral Sentence Generator
Neutral to Blas Sentence Generator
Tagger
Cycle loss
Enhance Conten: Preservation
Adversarial
D
Adversarial
Dr
loss
loss
Address Dias Neutralization
Address Bias Neutralization
Discriminator
Discriminator
Figure 2: Illustration of the proposed cycle-consistent training based neutralization approach for improved content preservation. The cycle consistency loss ensures the preservation of bias-independent content, adversarial loss works neutralize bias, and the identity loss prevents over-corection that could introduce new bias.
2023). The core of our proposed approach in- spired by this cycle-consistent adversarial train- ing approach. Our approach incorporates auxil- iary guidance for better performance and reduced training time, well as novel prompt variation thus minimizing content loss and improving sub- jective bias neutralization accuracy.
Therefore, the objective function is minimize (v), subject to: g g(x) and h(y) > h(x)). Hence, we need establish mapping functions be- tween these biased X and neutral statements Y, and reciprocally, by only leveraging unpaired sam- ples drawn from the distributions pa(a)) and pa(y)-
Method: FairBalance
3 Problem Statement
In this section present FairBalance, which en- ables the training of a subjective bias neutraliza- tion model in the absence of parallel data. The goal of FairBalance is effectively transform bi- ased text into neutral form, showcasing both flu- ency and diversity. More importantly, it achieves this subjective bias transfer while preserving the underlying semantic content of the original biased text.
We assume have corpus X, composed of biased statements 31, T.. and additional corpus Y containing neutral state- ments y1, 12. Un- These corpora exhibit non- parallelism, with no direct correspondence be- tween their respective elements, such x1 and V1. Our goal is develop model that corrects the subjective bias in the text and generates neutral- tone output while preserving the bias-indes ilu-iindependent content of the original text. For simplicity, let's consider that represents input biased text and y represents transformed neutral text. The sub- jective bias in text can be modeled as function (x), which aim to reduce climinate while generating the neutral text y.
4.1 Overview
Given our primary objective of performing bias style transfer with non-parallel data (Section 3), GANs are a natural fit. However, they often suf- fer from limited and repetitive outputs, resulting in a lack of diversity the generated text. This phenomenon mode collapse occurs when the generator successfully deceives the discriminator to such extent that it fails to provide adequate feed- back, resulting in a failure of training process. To mitigate mode collapse, we adopt a cyclic consis- tent generative adversarial approach for bias neu- tralization.
For any given æ, the model should generate such that the following conditions are satisfied:
The bias in y should be minimal. f(y) Az 0
The bias independent content from & should be retained in y. g(x) represents func- tion that extracts the bias independent content from v, then we want g(x) Re g(y)
Such a cyclic approach consists two sequen- tial GAN networks Gxy X converts biased text to neutral text, and Gyx Y-x performs the inverse conversion. Each generator is accom-
The generated text y should be fluent and di- verse. If (xx)) represents the fluency of the text æ. then we want h(y) > h(x))

---

panied by discriminator. The goal is learn the mapping function between the source distribution X and target distribution Y that minimizes f()), while also preventing mode collapse. To address mode collapse and preserve bias independent con- tent (a))) x ((v)), such cyclic approach intro- duces eycle consistency loss that is computed by comparing the original input biased text with the reconstructed biased text (second network) (Zhu al., 2017). Additionally, the text generated by both the generators are validated for its neutral- ity by subjective bias classifier. Consequently, we generate more diverse and realistic sam- ples (h(y) h(z)) without suffering from mode collapse.
4.3 Auxiliary Guidance
Naively applying such a cyclic GAN architecture to bias ncutralization can be challenging however. These methods, while sophisticated, can take a long time to converge. Further, the high dimen- sional output space and larger vocabulary make it computationally challenging to converge on large datascts, leading to elongated convergence times and, certain instances, preventing models from reaching cquilibrium entirely. Additionally, the sequential nature text required the models to capture long-range dependencies between words, which further leads to model instability.
To address these issues, we introduce an aux- iliary guidance as a mask that informs the gen- erator what part of the sentence needs to be cor- rected. However, identifying what part of the text needs to be corrected is a challenging task. Con- cretely, we use the WikiBIAS dataset and train a bias tagger using a seq2seq model as detailed (Zhong, 2021). This model identifies the section of text that biased, based on the contextual us- age, and highlights it for neutralization. This ap- proach is able to identify multiple instances bias within single sentence, demonstrating strong performance with an accuracy of 95% with a re- call of 92% on the validation set. This approach, leveraging auxiliary guidance, helps in maintain- ing control over the quality of neutralization and enhances the robustness and dependability of the niataauutra model.
4.2 Training Objectives
The generators and two discriminators trained simultaneously with the following set of losses:
Adversarial Loss responsible for distinguishing real data from fake data generated by the genera- tor. The adversarial loss for both networks can be expressed as:
1 Σ (1 - (G(()))
Teado (G, Dy: X)= =
m7
Lodu (F, (F, Y) 1 (a
Cycle Consistency Loss enforces mapping from the source domain to the target domain and back, so that the final text generated by the second GAN network is as close as possible to the original. The cycle consistency loss can be expressed as:
4.4 Cross-domain Discriminator Learning
For GAN-based bias neutralization model to de- tect and correct subjective biases across multiple domains, the discriminator must be able to accu- rately identify many forms of bias and compute losses accordingly. Since our training data is de- rived solely from one domain (Wikipedia), train- ing discriminator with it can affect the model's 's performance and generalizability across other do- mains. Hence, we propose to pretrain the dis- criminator using a cross-domain bias classification dataset (Madanagopal and Caverlee. 2022). The cross-domain bias classification dataset contains a collection of subjectively-biat biased and neutral state- ments collected from various domains such as pol- itics, entertainment and e-commerce. The cross- domain data helps the discriminator learn to iden- tify patterns and features that are common across various domains. The use an external classi- fier as discriminator helps to leverage its gained knowledge in the GAN and reduces the model
1 [(F(((()) xz)) x.)) m
Leyete(G, (,,F,X,Y)
+(G(F())) - y4)
Identity Mapping Loss helps to regularize the generator to get close to an identity mapping when the original input text is already the neutral form (target domain). The identity mapping loss can be formulated as:
Lident (G,en FG, = Paosa (w) [[] -yll]]
+ + + (x) [x)) - =1]1]]
By utilizing an adversarial loss, the generator gen- erates neutral text that validated by the discrim- inator and the cycle consistency loss ensures the bias independent content is preserved.

---

During the initial stages of the training process, we noticed that the generator was generating text with repeated words, which allowed the discrim- inator to easily beat the generator. In order to bootstrap our generator to generate reasonable text even from the beginning, we pretrained it as an au- training time significandly.
Book Corpus (IBC) (Sim al., 2013). A total of 412, sentences curated for training cross- domain bias classifier.
4.5 Model Training
We use a transformer model for our generator net- work and pre-trained a subjective bias classifier for the discriminator. Since the bias classifier is pre-trained completely, the weights of the model are frozen when used as discriminator and only the generator weights are updated during the train- ing process. All three losses in the previous sec- tions are important for the bias neutralization task. To produce a high-quality bias neutralization, all three losses need to be combined with a good bal- ance before using it to update the generator algo- rithm. To make it flexible to test various configu- rations of the losses, the loss weights (eyycl and Aidentity) are made configurable by the user.
The non-parallel dataset for GAN training was derived by analyzing the edit histories of Wikipedia articles that were tagged for Neutral- point-of- (NPOV) issues. We followed the data harvesting approach similar (Pryzant et al., 2019; Recasens al., 2013) with modifications to support muldi-wor, multi-occurrence bias. We had total of 201,175 biased sentences and 296,504 neutral sentences for building subjective bias neutralization model. More details of the data preparation steps are available in Appendix A.
5.2 Baseline Models
In the following, we present the baseline mod- els that are developed to evaluate the perfor- mance of the proposed auxiliary guided, cycle- consistent training- subjective bias neutral- ization method.
LGenerator
Lado (G, Dy: X,
Ladv F, Dz, Y, X)
+ +hayele * Leyele (G, F, X)
+Xident Lident (G,
Delete Biased Word(s) uses a biased word lexicon that curated from various studies to identify bi- ased word in the input sentence and removes them based on certain linguistic rules.
Control-gen Model is neural text generation model developed by (Hu et al., 2017) that uses a variational autoencoder as the generator and dis- criminator for conditioning the generated text.
toencoder where the generator is expect to produce an output that is similar to the input text. Both the forward and inverse generators were trained using a similar approach.
We also explored diverse set of GAN-based models aiming to gain insights into their respective approaches for subjective bias neutralization:
Also, in order to prevent the model from chang- ing drastically from iteration to iteration, the dis- criminators were fed history of generated text, rather than just the ones produced by the latest ver- sions of the generator. To do this we keep storing the 50 most recently generated text. Based on this technique we reduce the model oscillation well model overfitting
GAN: This model employs a transformer-based generator along with a pre-trained bias classifier as the discriminator. It serves as a fundamental GAN architecture for our experiments.
MaskGAN: The MaskGAN-based model incor- porates a bias tagger to identify and mask word tokens that are considered subjectivity indicators (Fedus et al., 2018). Similar to the GAN model, utilizes a transformer-base generator and a pre- trained bias classifier as the discriminator.
5 Experiment Setup
This section provides an overview of the datasets, the baseline bias neutralization models, and the detailed evaluation results.
CycleGAN: Designed as cycle-consistent ad- versarial network, the CycleGAN-based model adopts transformer-basee architectures for both generators (Yang et al., 2018). It employs the same pre-trained classifier with an inverse probability score as the discriminator.
5.1 Datasets
For training the discriminator, we used a cross- domain bias dataset that was constructed from diverse sources, including Wikipedia, and other subjectivity rich domains such as MPQA Opin- ion corpus (Wiebe et al., 2005) and Ideological
5.3 Experimental Results
In this section, we present the results of the exper- iments through both automated and human judge-

---

Transformer-on
MaskGAN
CycleGAN
FairBalance
G
D
G
D
G
G
G
B + N
B > N
D
D
D
- N
N N
B
- 8
Figure 3: Illustration of the various GAN- based models experimented to gain insight into their respective ap- proaches for bias neutralization. B + N represents converting biased text neutral and N + B represents converting neutral text to biased. M represents the masking that highlights biased spans in the input sentence.
Model
Neutrality BLEUt
BERT.
PPL
Score
Source Copy
12.88
100.00
100.00
35.14
Delete Biased Word
39,45
79.71
92.23
48.20
Control-gen (Hu al., 2017)
69.18
41.64
78.31
29.47
GAN
59.85
30.81 *
91.58*
27.17*
MaskGAN
67.26*
23.61 *
93.36*
27.92*
CycleGAN
66.94
40.30
95.65*
25.61
FairBalance
69.78*
51.38
96.42*
26.48*
Table 2: Performance comparison Subjective Bias Neutralization models using various rule-based and neural GAN-based style transfer models. For quantitative metries, with asterisks (*) significantly different than the preceding row. f /4 means higher/lower score is preferred for the corresponding metric.
ment evaluation. The details of evaluation metrics used are available in Appendix E.
mance and training performance of the GAN- based models. The vanilla GAN-based model, which employs transformer model as its gener- ator and simple RoBERTa based discriminator took long time to train (weeks) and the gener- ated text contained lots of repeated words. With the introduction of our pre-training approach, the Munsformer-GAN model started producing coher- ent text. For simple framing biases, such as of weasel words, the transformer-GAN model was able to identify the biased terms in sentence and neutralize it by removing it replacing it with other words. However, it struggled neu- tralize sentences that with quotes and uncertain- ties (epistemological bias). The masked GAN- based model, which marks the biased words in the input sentence, allowed for faster training (days) and achieved neutrality score of 67.26 which is 8 points better than vanilla-GAN models. When sentence length exceeded 40 words, the masked transformer model began to lose significant origi- nal content from the input text.
5.3.1 Automated Evaluation
Table 2 shows the performance of our proposed model and its variants, in comparison with the selected baseline models. Initially, we evaluated the performance of our non-GAN based mod- els (baselines) to comprehend the strengths and weaknesses of our baseline models. Since more than 30% of the biased words in our corpus were adjectives, the method of deleting biased words proved more effective by eliminating the adjec- tives and neutralizing the sentences. However, this approach resulted in grammatical issues, such as incorrect usage of determiners and articles.
Alternately, the neural network based control- gen model developed by Hu et al. (Hu et al., 2017) neutralized the sentences more effectively than the delete biased word method. This can be mainly attributed to the discriminator used to con- trol the generation process. Despite the control- gen method's superior neutralization performance and the fluency of the output text, there was a sig- nificant loss content. Sometimes, also added additional content to the neutralized text. While the generated text retained the overall theme of the input, some important facts were omitted.
Both the CycleGAN and FairBalance models aimed to balance neutralizing the input text with retaining portions of the original semantic con- tent. Both models were able to neutralize long sentences and generate more fluent and grammat- ically correct sentences. The FairBalance model
Next, we analyze the neutralization perfor-

---

had 2 point improvement in neutrality score rel- ative other models and also it took less time it to converge. It was also capable of addressing multi- ple instances of bias within a single sentence.
the MaskGAN with Transformer model had a best bias score of -0.894 (See Table 4). Interestingly the masked transformer GAN model had second best of 0.664. This shows the masking approach works better than other methods in making the text neutral. In terms of fluency, most of the mod- els were in the same range of 1.1-0.12. FairBal- ance had the best fluency score of 0.115. In terms of content preservation, text generated by Cycle- GAN based models were preferred by the users with the FairBalance with transformer model hav- ing the best content preservation score of 1.394. Overall, both masking based models were scored high in the human judgement. Given both neu- tralization and content preservation are important, FairBalance is the best performing model.
Interestingly, the FairBalance- models did not introduce new bias into the input text, whether it originally neutral biased. This aspect of neutralization is significant and the combination of identity loss and discriminator loss introduced in the FairBalance along with auxiliary guidance is responsible for this important neutralization aspect in addition its improved content preservation.
5.3.2 Multi-Occurence Bias Evaluation
When it comes to addressing multi-occurrence bias, models incorporating auxiliary guidance or masking (MaskGAN and FairBalance) showed su- perior performance. The masking done with the token-wise bias tagger aids in identifying biased sections of text and instructs the generator to ad- dress them. Other GAN models such as the GAN and CycleGAN with transformer, primarily ad- dressed single occurrence of bias and retained other subtly biased chunks. Even though both models with auxiliary guidance performed well, the CycleGAN model with auxiliary guidance re- tained the bias independent content in the gener- ated text better than the MaskGAN.
5.4 Domain adaptation
To evaluate the domain adaptation of the pro- posed FairBalance model on domains outside of Wikipedia, collected 25 statements from each of the following domains: Academics, News, and Polities. This resulted in total of 75 sentences, which were then presented to five human judges for qualitative assessment. Three criteria were evaluated: neutrality, content preservation, and fluency. In terms of neutrality, the FairBalance model exhibited robust performance across all domains, outperforming the second-best model, MaskGAN (See Table 5). The FairBalance model demonstrated particularly strong performance in the Academics domain. With respect to fluency, all models performed reasonably well. Exclud- ing the Academics domain, FairBalance showed strong performance across the remaining domains. As for content preservation, there was a marked improvement in the performance of FairBalance relative to the other models. The human evaluation revealed that the FairBalaace model outperformed other models across different domains.
Model
Neutrality
% biased
GAN
61.19
30.57
MaskGAN
69.26
23.94
CycleGAN 64.09
29.54
FuirBalance
70.48
22.16
Table 3: Performance evaluation of addressing multi- occurrence bias. % biased represents the percentage of sentences that contains at least one instance of biased chunk after bias neutralization (lower the better).
5.3.3 Human Evaluation
In the presence of parallel data, it is relatively easy to assess the performance of neutralization models using automatic metrics such as BLEU and BERTScore. In the absence of non-parallel data, we chose to evaluate the performance of our neutralization models through human evalu- ations to ensure unbiased feedback. Therefore, a blind heads-up comparison was used to evaluate the quality of samples generated by various GAN- based bias neutralization models. Three aspects text were evaluated using human judgement: bias, fluency and content preservation. For every judge, we presented the original text and neutral- ized text and asked them to rate the quality of neu- tralized text in the scale of -2 to 2. In terms of bias,
Conclusion
In this paper, we propose a new subjective bias neutralization model trained using non-parallel data through an auxiliary guided cycle consis- tent GAN. FairBalance consists of a bias tag- ger to identify subjective words in the text, a cross-domain bias classifier that accurately detects whether the given text is subjectively biased or not and a cyclic network that trains the neutralization model with a combination of adversarial, cycle- consistency and identity mapping loss. The com- bination of auxiliary guidance, pre-trained clas-

---

Model
Bias.,
Fluency|
Content
GAN
-0.066
0.108
2.165
MaskGAN
-0, 0.6664
0.106
1.898
CycleGAN
-0.648
0.112
1.501
FairBalance
-0.894
0.115
1.394
Table 4: Human Evaluation of Subjective Bias Neutralization models using four different GAN-based text style transfer models. The rating scale for bias ranges from -2 to 2. fluency ranges from -2 to 2, and content preservation ranges from 0 to 4. f //4 means higher/lower score is preferred for the corresponding metrie.
Academies
News Fluency
Politics Fluencyt
Model
Bias,
Fluencyf
Content
Bias
Content
Bias.
Content
GAN
-0.617
0.085
2.97
-0.489
0.105
2.17
40.427
0.092
2.03
MaskGAN
-0.866
0.089
2.25
-0.715
0.110
2,09
-0.524
0.102
2.48
CycleGAN
-0.828
0.103
1.45
-0.684
0.109
1.58
-0.503
0.097
1.16
FairBalance
-0,901
0.102
0.93
-0.746
0.112
1.53
-0.584
0.105
1.04
Table 5: Human evaluation of subjective bias neutralization performance across 3 different domains. The rating
scale for bias ranges from -2 to 2. fluency ranges from -2 to 2, and content preservation ranges from 0 to 4. 1 means higher/lower score is preferred for the corresponding metric.
sifier and cycle-consistent network yielded sig- nificant improvement in performance when com- parison with other GAN-based models. Further, the FairBalance model also performed well on other domains which is evident through the cross- domain bias evaluation.
Additionally, this study focuses on specific forms of subjective bias observed at the sentence level. However, considering the context (para- graph) in which statement is made could poten- tially alter the perspective. that while the FairBalance approach is evaluated on real-world subjective bias neutralization task in this study, further testing is necessary to encom- pass more challenging bias types and explore other target model architectures.
important to note
While the proposed model performs well, there is still scope for improvement. For example, in- corporating more contextual information such as paragraphs or preceding sentences can improve the detection and neutralization performance. Ad- ditionally, the performance of FairBalance de- pends on the bias tagger performance. By incor- porating more subtle forms of subjective biases in refining the bias tagger improved the performance of proposed method. In future studies, we plan to extend our proposed framework model work on long sentences and more subtle forms of biases. We will also explore paragraph level bias neutral- ization make the generated text more consistent and accurate by using more contextual data.
The work presented in this paper introduces a promising approach for developing bias neutral- ization models in domains like news and polities where parallel data is not available, which is a field of great significance yet remains under ex- plored. We hope that by evaluating these models across different domains, will stimulate further research in developing robust, nuanced, and fair bias neutralization models.
Ethics Statement
In our exploration of machine learning models for neutralizing linguistic bias, we acknowledge and emphasize the inherent limitations and challenges. Specifically, our FairBalance model bases its anal- ysis on individual sentences without accounting for the broader context in which they appear. This absence context might lead to instances where the model misidentifies a sentence as either neu- tral or biased. We recognize that this potential inaccuracy could influence its operational utility. Readers and practitioners should be cautious and considerate of this limitation when interpreting or deploying the results from our model.
7 Limitations
FairBalance shares certain limitations with previ- ous research (Kim et al.. 2017; Yi et al., 2017) on cyclic adversarial-based learning methods. One of the major limitations is that the mappings learned are deterministic and the cycle consistency forces the mappings to be inverses of each other (Alma- hairi et al., 2018). Instead of capturing true, structured conditional distribution, these cyclic adversarial- -based models arbitrarily pick one-to- one mapping in situations when faced with com- plex source-target relationships.

---

References
Interac- Zhiting Hu, Zichao Yang. Xiaodan Liang, Ruslan Salakbutdinov, and Eric Xing. 2017. Toward con- trolled generation of text. In International con- ference on machine learning, pages 1587-1596. PMLR.
2022. Good arttices..
Amjad Almahairi, Sai Rajeshwar, Alessandro Sordoni, Philip Bachman, and Aaron Courville. 2018. Aug- mented cyclegan: Learning many-to-many map- pings from unpaired data. In International confer- ence on machine learning, pages 195 104. PMLR.
Christoph Hube and Besmik Fetahu. 2018. Detecting biased statements in wikipedia. In Companion Pro- ceedings of the The Web Conference 2018, pages 1779-1786.
Camiel J Beukeboom and Christian Burgers. 2017. Linguistic bias. In Oxford research encyclopedia of communication.
Di Jin, Zhijing Jin, Zhiting Hu, Olga Vechtomova, and Rada Mihalcea. 2022. Deep learning for text style transfer: A survey. Computational Linguistics, 48(1):155-205.
Shruti Bhosale, Heath Vinicombe, and Raymond Mooney. 2013. Detecting promotional content in wikipedia. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Pro- cessing, pages 1851-1857.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, Lisa Orii, and Pe- ter Szolovits. 2020. Hooks in the headline: Learning to generate headlines with controlled styles. arXiv preprint arXw::0004001990.
Shikha Bordia and Samuel R. Bowman. 2019. Iden- tifying and reducing gender bias word-level lan- guage models. In Proceedings of the 2019 Confer- ence of the North American Chapter of the Asso- ciation for Computational Linguistics: Student Re- search Workshop. pages 7-15, Minncapolis, Min- nesota. Association for Computational Linguistics.
Tacksoo Kim, Moonsu Cha, Hyunsoo Kim, Jung Kwon Lec, and Jiwon Kim. 2017. Learning to discover cross-domain relations with generative adversarial networks. In International conference on machine learning, pages 1857-1865. PMLR.
Guillaume Lample, Sandeep Subramanian, Eric Smith, Ludovic Denoyer, Mare' Aurelio Ranzato, and Y- Lan Boureau. 2018. Multiple-atrribute rewrit- ing. International Conference on Learning Rep- resentations.
Samuel R Bowman, Luke Vilnis, Oriol Vinyals, An- drew M Dai, Rafal Jozefowicz, and Samy Ben- gio. 2015, Generating sentences from a continuous space. arXiv preprint arXiv:1511.06349
Dayiheng Liu, Jie Fu, Yidan Zhang, Chris Pal, and Jiancheng Lv. 2020. Revision in continuous space: Unsupervised text style transfer without adversarial learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34. pages 8376-8383
Eleftheria Briakou, Di Lu, Ke Zhang, and Joel Tetreault. 2021. Olá, bonjour, salve! xformal: A benchmark for multilingual formality style transfer. In Proceedings of the 2021 Conference of the North American Chupter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 3199-3216.
Ruibo Liu, Chenyan Jia, and Soroush Vosoughi. 2021. A transformer based framework for neutralizing and reversing the political polarity of news articles. Pro- ceedings of the ACM on tion, 5(CSCW 1):1-26..
William Fedus, Ian Goodfellow, and Andrew M Dai. 2018. Maskgan: better text generation via filling in the arXiv preprint arXiv: 180100736.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer. and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv: 7907.1 16692.
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative ad- versarial networks. Communications of the ACM, 63(1 139-144.
Fuli Luo, Peng Li, Jie Zhou, Pengcheng Yang, Baobao Chang. Zhifang Sui, and Xu Sun. 2019. A dual rein- forcement learning framework for unsupervised text style transfer. arXiv preprint arXiv: 10060.
Shane Greenstein and Feng Zhu. 2012. Is wikipedia biased? American Economic Review, 102(3):343- 48.
Aman Madaan, Amrith Setlur, Tanmay Parekh, Barn- abas Poczos, Graham Neubig, Yiming Yang. Ruslan Salakhutdinov, Alan W Black, and Shrimai Prabhu- moye. 2020. Politeness transfer: A tag and generate approach. arXiv preprint arXiv:2004. 142577.
Shane Greenstein and Feng Zhu. 2014. Do Ex- perts Or Collective Intelligence Write with More Bias?: Evidence frode Encyclopediaa Britannica and Wikipedia. Harvard Business School.
Karthic Madanagopal and James Caverlee. 2022. Im- proving linguistic bias detection in wikipedia us- ing cross-domain adaptive pre-training. In Compan- ion Proceedings of the Web Conference 2022, pages 1301-1309.
Junxian He, Xinyi Wang, Graham Neubig, and Taylor Berg- Kirk-Kiikattrcc.... 2020. A probabilisticc formulation of unsupervised text style transfer. arXiv preprint arXiv:2002.03912

---

Karthic Madanagopal and James Caverlee. 2023. Re- inforced sequence training based subjective bias cor- rection. In Proceedings of the 17th Conference of the European Chapter of the Association for Com- putational Linguistics, pages 2577-2590.
Janyce Wiebe, Theresa Wilson, and Claire Cardie. 2005. Annotating expressions of opinions and emo- tions in language. Language resources and evalua- rion, 39:165-210.
Janyce M Wiebe. 1994. Tracking point view in nar- rative. arXiv preprini cmp-lg/9407019.
Thomas Manzini, Yao Chong Lim, Yulia Tsvetkov, and Alan W Black. 2019. Black is to criminal as cau-
casian is to police: Detecting and removing mul- ticlass bias in word embeddings. arXiv preprint arXiv:900.00404..
Zichao Yang, Zhiting Hu, Chris Dyer, Eric P Xing, and Taylor Berg- Kirkpatrick. 2018. Unsupervised text style transfer using language models as discrimina- tors. Advances in Neural Information Processing Systems, 31.
Arkajyoti Misra and Sanjib Basak. 2016. Political bias analysis.
Xiaoyuan Yi, Zhenghao Liu, Wenhao Li, and Maosong Sun. 2021. Text style transfer via learning style in- stance supported latent space. In Proceedings of the Twenty-Ninth International Conference Interna- tional Joint Conferences on Artificial Intelligence, pages 3801-3807.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic eval- uation of machine translation. In Proceedings of the 40th Annual Meeting of Association for Com- putational I Linguistics, pages 31 1-318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
Zili Yi, Hao Zhang, Ping Tan. and Minglun Gong. 2017. Dualgan: Unsupervised dual learning for image-to-inas to-age translation. In Proceedings of the IEEF international conference on computer vision, pages 2849-2857.
Roma Patel and Ellie Pavlick. 2021. "was it "stated" was "claimed"?: How linguistic bias affects gener- ative language models. In Proceedings of the 2021 Conference Empirieal Methods in Natural Lan- guage Processing, pages 10080-10095.
Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Eval- vating text generation with bert. In International Conference on Learning Representations.
Stephen Politzer-Ahles, Teresa Girolamo, and Saman- tha Ghali, 2020. Preliminary evidence of linguistic bias in academic reviewing. Journal of English for academic purposes, 47:100895.
Jieyu Zhao, Tianlu Wang. Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplifica- tion using corpus-level constraints. arXiv preprint arXiv:1707.09453
Shrimai Prabhumoye, Yulia Tsvetkov, Ruslan Salakhutdinov, and Alan W Black. 2018. Style
transfer through nack-translation. arXiv preprint 30Xi::/804.0000.
Reid Pryzant, Richard Diehl Martinez, Nathan Dass, Sadao Kurohashi, Dan Jurafsky, and Diyi Yang. 2019. Automatically neutralizing subjective bias text. arXiv preprint ArXiv:1911.0977
Wei Zhao, Liang Wang, Kewei Shen, Ruoyu Jia, and Jingming Liu. 2019. Improving grammatical ror correction via pre-training copy-augmented architecture with unlabeled data. arXiv preprint arXiv:1903.00138.
Marta Recasens, Cristian Homescc--iiulessu--a and Dan Jurafsky. 2013. Linguistic models for an- alyzing and detecting biased language. In Proceed- ings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa- pers), pages 1650-1659.
Yang Zhong. 2021. WIKIBIAS: Detecting Multi-Span Subjective Biases in Language. Ph.D. thesis, The Ohio State University.
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Pro- ceedings of the IEEE international conference on computer vision, pages
Yanchuan Sim, Brice DL Acree, Justin H Gross, and Noah A Smith. 2013. Measuring ideological pro- portions in political speeches. In Proceedings of the 2013 conference on empirical methods in nat-
ural language processing, pages 91-101.
A Dataset Preparation
Tianlu Wang, Jieyu Zhao, Kai-Wei Chang. Mark Yatskar, and Vicente Ordonez. 2018. Adversarial removal of gender from deep image representations. arXiv preprini 1edi::1811.08400 3.
Our data harvesting approach was similar to Re- casens et. al (Recasens et al., 2013) and Pryzant et. al (Pryzant et al., 2019), but with some mi- nor changes such as not restricting to single word edits. Sentences that had NPOV tags before the revision were considered as biased sentences and the sentences whose NPOV tags were removed af- ter edits were considered as unbiased sentences.
Zhuoer Wang. Mareus Collins, Nikhita Vedula, Si- mone Filice, Shervin Malmasi, and Oleg Rokhlenko. 2023. Faithful low-resource data-to-text gen- eration through cycle training. arXiv preprint arXiv:2305. 14793.

---

We ignored revisions that were related to missing references, misspellings and punctuation. Addi- tionally, we downloaded Wikipedia pages that are tagged "Good Articles" and included them in the neutral sentences corpus (Wik, 2022).
criminator. Moreover, we employed bias tagger for the purpose of masking, which further aligned these models with our dataset's specific require- ments. By adapting these models with modern components and techniques, we aimed to bridge the gap between their original designs and the de- mands of our bias neutralization task. This ap- proach enabled us harness the advantages of both established and cutting-edge methodologies. ensuring that evaluation as comprehensive and effective as possible.
Since the objective of this research is cor- rect bias that is induced by single word and mul- tiword, we expanded the corpus by modifying the harvest function of pryzant et. al. (Pryzant et al., 2019) Also, our method uses the latest dump from Wikipedia which contains new biased sentences that were not considered in the previous study. Ad- ditionally, some data cleanups were done to make this model not sensitive to proper nouns in the text. We replaced all proper nouns with generie names, but retained honorifies because some of the gender biases were introduced through honorifies. The numbers mentioned in the text were also replaced with NUM tag.
C Implementation Details
All the GAN models were implemented in Py- Torch (Python 3.7.11) and trained using a Tesla V100 with CUDA 11.3. For discriminator training, pre-trained contextualized language models is used to efficiently incorporate sen- tence semanties in performing text classifications (Madanagopal and Caverlee, 2022). The pre- trained language model, RoBERTa was down- loaded and further trained using cross-domain dataset (Liu et al., 2019). We used ADAM opti- mizer with a learning rate of 2e-4 and the trained discriminator had a precision of 89% with Fl Score of 87%. The input sentence is first en- coded into fixed length vector h using pre- pre-trained RoBERTa model, which captures the contextual information of the sentence. Using fully con- nected layer with softmax function, the encoded vector is transformed to probability distribution over the possible labels which is given by:
B Baseline Model Selection
Our study aimed to comprehensively evaluate both disentanglementt-aaeee and adversarial-bart basedd models in the context of our bias neutralization dataset. This upproach was motivated by the need to explore the strengths and weaknesses of dif- ferent techniques in addressing bias neutraliza- tion using non-parallel corpus. In the realm of disentanglement based models, our investigation led us to the discovery that the control-gen method (Hu et al., 2017) exhibited the most promising per- formance for bias neutralization within our bias neutralization dataset. This result came as sur- prise, particularly given the prevailing expectation of better performance from more recent models (Yi et al., 2021; Jin et al., 2020). That's why we used only the control-gen method for deeper analysis on disentanglement methods. Since our method was adversarial-basse based, we choose one dis- entanglement method (best) and more than one ad- versarial model for our evaluation.
p(yls) softmar [h h b))
(1)
where W is the weight matrix of the fully con- nected layer and b is the bias vector.
Model
Precision
Recall
Fl-score
CLSGlove
81.81
80.24
81.02
CLSBERT
83.57
79.26
81.36
For adversarial-based models, we recognized that while CycleGAN (Yang et al., 2018) and MaskGAN (Fedus et al., 2018) were relatively older references, we did not simply adopt these models as-is for our study. Instead, we took steps to enhance their relevance and applicability. We incorporated state-of-the-art transformer models to leverage advancements in natural language pro- cessing, ensuring that our adversarial-based mod- els were equipped with the latest language under- standing capabilities. We used RoBERTa mod- els for training the pre-trained classifier for dis-
CLSRoBERTA
89.41
85.94
87.64
Table 6: The CLSROBERT trained with cross- dossin dataset showed significant improvement in classifying biased statement. Bold indicates best results.
The generator model used in all four GAN mod- els consists of three transformer layers with three attention heads. In order to capture the underlying semantics of the input text, we pretrained the gen- erators as autoencoders. The maximum length of the text to be generated was set to 128. To opti-

---

mize the networks, we employed the Adam opti- mizer with learning rate of 0.0002, betas of (0.9, 0.98), and epsilon value of le-9. To improve the stability and convergence, penalty using gradient norm with weight À 10 and added it the discriminator loss during the GAN training. For the encoder and decoder, dropout rate of 0.33 applied. During the train- ing of the generator, we utilized 25 epochs, while for the FairBalance training, used 100 epochs. Each training iteration was performed with batch size of 16. To assign appropriate importance to different components of the loss function, the loss weights as Acyele 8 and Aidentity 5. These weights help in balancing the impact of cy- cle consistency and identity preservation during the training process.
BLEU: A set of 100 biased sentences were se- lected for this evaluation and corresponding neu- tral sentences were manually created with human experts who have good English language skills. Using this reference dataset, n-gram precision counting metric (BLEU) was computed to mea- sure the similarity of machine corrected text and human reference correction through n-gram preci- sion counting (Papineni et al., 2002).
computed gradient
BERTScore: Uses contextual language models and computes semantic distance between candi- date and reference sentences (Zhang" al., 2020) using contextualized language model like BERT. PPL: To evaluate the grammatical correctness and fluency of the machine generated text, we puted the perplexity score (PPL) using the large pre-trained language model GPT-2. The perplex- ity score PPL is computed directly the gener- ated text with no reference text.
D Bias Tagger
The bias tagger is a token classification model used to analyze input sentences and identify bi- ased spans of text. It is developed in PyTorch using RoBERTa for the token classification trans- former model. The Adam optimizer with a learn- ing rate of 3e-5 was used. The model is trained using the WikiBIAS dataset developed by Zhong et al. [cite: zhong2021 wikibias]. The trained bias (nnng22iiiibbisss tagger model achieved 95% accuracy in identify- ing biased spans of text with a recall of 92%. In the FairBalance model, the input sentence is first fed to the bias tagger to identify biased spans of text. This approach helps identify multi-word bi- ased spans as well as multiple occurrences of bi- ased text, The output of the bias tagger is then used to add [M [SK] tokens around biased words, which are subsequently fed as input to the first GANs generator network.
F Human Judgement
A set of 50 biased sentences were selected and processed using the experimented GAN models. This combination of biased and neutral sentences was used for human evaluation by panel of 10 judges. Every time a biased sentence and its cor- responding neutralized sentence from one of the model is presented to the user and asked to rate the following three aspects separately:
Neutrality: Rate the neutrality of the neutral- ized sentence on scale of -2 2. A rating of indicates that the original text is more biased than the neutralized text, rating of O means both texts have the same level of bias, and rating of 2 means the neutralized text is more biased.
Content Preservation: Rate whether the bias independent content is retained in the neutralized sentence on a scale of 4 with 0 means the content is very different and 4 means very similar.
Evaluation metrics
To assess the performance of our proposed bias neutralization model, we used the following au- tomatic evaluation metrics.
Fluency: Rate the grammatical correctness and fluency of the neutralized sentence with respect to the original text on a scale of 2 with -2 mean the original text is more flu- ent and 2 means the neutralized text is more fluent.
Neutrality: Similar to (Luo et al., 2019; He et al., 2020), We used a subjective bias classifier to com- pute the neutrality score for an input sentence. The subjective bias classifier is a BERT-hased model that was trained using subjective statements de- rived various domains such as political speeches and product reviews (Madanagopal and Caverlee, 2022). On a human reference dataset, this bias classifier had an accuracy of 89% (Madanagopal and Caverlee, 2022)

---

Examples of Bias Neutralizations
Source
Unfortumately Terry Fox could not finish his courageous run.
GAN
Unfortunately, Terry fox could finish.
MaskGAN
Sadly, Terry fox could not finish.
CycleGAN
Courageous Terry Fox could not finish his run.
FairBalance
Terry fox couldn't finish his run.
Source
Critics believe that some or all of these exclusionary policies are wrong and amount to discrimination.
GAN
Critics think that some or all of the policies are wrong.
MaskGAN
Some critics believe that these discriminatory policies are wrong-
CycleGAN
Critics believe that some of the policies are wrong and could lead discrimination.
FairBalance
According to critics, some or all of these policies are wrong and discriminatory.
Source
Carleton University a prestigious non-denominationall, co-educational university Ottawa, Ontario, Canada
GAN
Carleton University is public university.
MaskGAN
Carleton University, Canada is a non-denominationall, co-educational university.
CycleGAN
Carleton University non-denominational,, co-educational university in Ottawa, Ontario, Canada.
FairBalance
The Carleton University is a non-denominntiioal, co-educational university located in Ottawa, Ontario, Canada.
Source
Joe is respected singer-songwriter country music, western music (North America), and popular music.
GAN
Joe sings in country music, western music, and popular music.
MaskGAN
Singer Joe specializes country music, western music (North America), and popular music.
CycleGAN
Joe singer-songwriter country music, western music (North America), and popular music.
FairBalance
Joe is regarded as a leading singer-songwriter in country music, western music (North America), and popular music.
Source
Redstone's son has sued him and his daughter regarding the managing of National Amusements.
GAN
Surprisingly, Redstone's son sued him and his daughter for managing National Amusements.
MaskGAN
Redstone was sued by his son and daughter over the handling of National Amuse- ments.
CycleGAN
Redstone's son has sued him and his daughter over managing National Amusements.
FairBalance
A lawsuit has been filed against Redstone and his daughter by their son over the management of National Amusements.
Table 7: Example outputs from GAN-based neutralization models. Biased words are highlighted in itulic.


Reviews:
[{"review_id": "qSZ3ar3df9", "reviewer": "Reviewer_THyJ", "summary": "", "questions": "1. There is no information about human judges, even though that might be very relevant. How was their education, age, etc.? This information is increasingly included in recent works about bias and relates to their labels. \n\n2. The study focuses on specific forms of subjective bias observed at the sentence level. However, considering the context (paragraph) in which a statement is made could potentially alter the perspective. You say one sentence on this, but do you already have ideas on how it will be implemented in your work?\n\n3. The referenced bias-neutralizing papers are not always completely clear in which kind of bias they deal with. However, bias (from linguistic bias to sentiment bias and bias by word choice and other concepts) is very complex. Can you clarify the definition of bias that you apply?\n\n4. There seems to be no link to all data and code (anonymized). Is this a mistake? The paper is not reproducible as it stands.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the detailed feedback. We have added comments to both \"Reasons to Reject\" and \"Questions to Author\". The questions and comments are highlighted in bold and the response following it.\n\n**The authors do not compare the performance of the FairBalance model with many other state-of-the-art models for subjective bias neutralization. Including such comparisons would provide a better understanding of the model's effectiveness and potential improvements. I feel it would have made the model easier to compare to the existing (and more limited) models, such as by Liu, Madanagopal, and Pryzant.**\n\nPryzant's Join embedding model (Pryzant et al., 2019) and Madanagopal's Reinforced sequence training models (Madanagopal and Caverlee, 2022)  were both supervised training models. However, due to the nature of our dataset being non-parallel, we were unable to train these models using our dataset as they require parallel data. Instead, we trained Pryzant's model and Madanagopal's model using their respective datasets and subsequently evaluated their performance against our dataset.\n\n|             | Neutrality | BERTScore | PPL   |\n|-------------|------------|-----------|-------|\n| Pryzant     | 48.79      |     81.54 | 33.52 |\n| Madanagopal | 67.83      |     92.11 | 24.19 |\n| FairBalance | 69.78      |     96.42 | 26.48 |\n\t\nPryzant et al.'s model was primarily designed for single word bias neutralization, and its limitations are evident in its lower bias neutralization score of 48.79 as 70% of the data points in our dataset have multi-word bias. We also observed that Pryzant’s joint embedding model mostly addresses the first instance of bias in a sentence and retains most of the other bias instances following it. In contrast, the reinforced sequence trained models by Madanagopal et al. demonstrated effective performance in bias neutralization, achieving a score of 67.83, which is comparable to our proposed FairBalance model. Notably, the reinforced model also exhibited a better perplexity (PPL) score than our FairBalance model. However, when it comes to content preservation, the FairBalance model outperformed the reinforced sequence trained models significantly. These relative close values in performance might be attributed to the common domain of Wikipedia across these models. It's important to highlight that our FairBalance method offers flexibility due to its ability to operate with non-parallel datasets, enabling its application in diverse domains without the need for parallel data.\n\n---------------------------------------\n\n**The paper has some wording issues. For example, I don't favor word choices such as \"the models returned satisfactory results\", \"both models had a good balance\", extensive experiments, or \"the best way to evaluate the performance (...) is to conduct unbiased human evaluations\". Similar wordings happen occasionally throughout the paper, and I feel descriptions should be specific and the judgment left to the reader.**\n\t\nWe appreciate your comments on using a neutral tone and letting the reader form their own judgments based on the presented results. In response to your comments, we have made several changes. Following are some examples:\n\n(Before) In the absence of non-parallel data, the best way to evaluate the performance of our neutralization models is to conduct unbiased human evaluations\n\n(Now) In the absence of non-parallel data, we chose to evaluate the performance of our neutralization models through human evaluations to ensure unbiased feedback.\n____________\n\n(Before) Both the CycleGAN and FairBalance models had a good balance of neutralized input text and preserving semantic content from the original input text.\n\n(Now) Both the CycleGAN and FairBalance models aimed to balance neutralizing the input text with retaining portions of the original semantic content\n\n_____________\n\n(Before) Extensive experiments demonstrate how FairBalance significantly improves subjective bias neutralization compared to other methods.\n\n(Now) In our evaluations involving seven models consisting of adversarial and non-adversarial models, the FairBalance method showed a notable improvement in bias neutralization based on subjective human judgment when compared to other techniques. \n\n__________\n \n(Before) Through extensive experiments, we demonstrate that the proposed approach outperforms conventional text style transfer techniques through a combination of subjective and objective evaluation metrics. \n \n(Now) Through seven models (3 baselines and 4 adversarial) along with human judgment evaluation, we demonstrate that the proposed approach outperforms conventional text style transfer techniques through a combination of subjective and objective evaluation metrics \n\n_____________\n\n**Bias neutralization has ethical implications. However, the paper does not provide any ethics statement, even though such a statement does not affect the page limits. Not providing one is a major drawback.**\n\nThank you for highlighting the absence of an ethics statement in our study. Initially, we presumed that testing the models in the Wikipedia domain wouldn't pose sensitivity concerns. However, considering the broader implications you've aptly brought to our attention, we have now incorporated an ethics statement into our paper.\n“In our exploration of machine learning models for neutralizing linguistic bias, we acknowledge and emphasize the inherent limitations and challenges. Specifically, our FairBalance model bases its analysis on individual sentences without accounting for the broader context in which they appear. This absence of context might lead to instances where the model misidentifies a sentence as either neutral or biased. We recognize that this potential inaccuracy could influence its operational utility. Readers and practitioners should be cautious and considerate of this limitation when interpreting or deploying the results from our model.”\n\n___________\n\n**Questions For The Authors:**\n**There is no information about human judges, even though that might be very relevant. How was their education, age, etc.? This information is increasingly included in recent works about bias and relates to their labels.**\n\nA total of 10 judges were used for our human judgment and the details are given below:\n4 Undergraduate students (under 20)\n1 non-native English speaker\n2 Linguistic Experts\n5-10 years’ experience (master’s degree)\n15-20 years’ experience (PhD in Computer Science)\n4 people (30-50+ years)\n1 Person (30-40 years) (master’s degree)\n2 people (40-50 years) (1 master’s Degree and 1 PhD)\n1 Person (50+) (bachelor’s degree)\nA set of 50 biased sentences were selected and processed using the experimented GAN models. This combination of biased and neutral sentences was used for human evaluation by a panel of 10 judges. Every time a biased sentence and its corresponding neutralized sentence from one of the models is presented to the user and asked to rate the following three aspects separately: Neutrality, Content Preservation and Fluency\n\n**The study focuses on specific forms of subjective bias observed at the sentence level. However, considering the context (paragraph) in which a statement is made could potentially alter the perspective. You say one sentence on this, but do you already have ideas on how it will be implemented in your work?**\nThank you for highlighting the potential influence of context on bias detection and neutralization. While the current bias neutralization model operates solely on biased sentences, broadening the scope to include contextual factors like article titles, section headers, paragraphs, and neighboring figures could significantly enhance its robustness. Even the presence or absence of references could provide valuable insights. Specifically, we can use an additional auxiliary signal for highlighting a sentence in the paragraph that is biased along with detailed biased words in the sentence. Our next research phase will involve incorporating these expanded contextual elements to perform bias neutralization, which we believe will contribute to a more comprehensive understanding of how contextual cues influence bias mitigation.\n\n**The referenced bias-neutralizing papers are not always completely clear in which kind of bias they deal with. However, bias (from linguistic bias to sentiment bias and bias by word choice and other concepts) is very complex. Can you clarify the definition of bias that you apply?**\nWe appreciate your insightful question that prompts us to provide a clear definition of the bias we focus on in our research. The type of bias that is focused on in our paper is linguistic bias, which is a type of judgmental language used in presenting facts that are intended to persuade, argue, or otherwise present an opinion. Primarily, we focused on three types of linguistic bias as identified by Recasens et. al.\nFraming bias is an explicit form of bias that reveals the authors’ stance on a topic using one-sided or subjective words. Framing bias is particularly concerned with highlighting a topic in a positive or negative manner. It’s mostly seen in argument situations, where the speaker takes one side and expresses an opinion strongly opposing it or supporting it.\nEpistemological bias is an implicit and a subtle form of bias that tends to cast a doubt in the expressed information. Epistemological bias is linguistically realized using factive verbs, entailments, assertive or hedges. Identifying epistemological bias can be extremely difficult because if the expressed uncertainty in the statement is universally accepted, then the fact is unbiased.\nDemographic Bias is defined as a systematic asymmetry in word choice that reflects the authors’ social-categorical belief towards a described group or individuals on a topic. It roots from the common criticism of Wikipedia is that it is a male-dominated community. As a result, certain controversial topics like marriage, feminism, and abortion may display gender bias.\n\n**There seems to be no link to all data and code (anonymized). Is this a mistake? The paper is not reproducible as it stands.**\nOne of the libraries that is used in our code has a dependency on proprietary modules that’s why our code is not shareable at its current state. However, we intend to address this by actively working on eliminating that dependency and making the code publicly available in the future. In the revised version, we will include the dataset utilized for training the discriminator, bias tagger and neutralization components. Open review is not allowing us to add attachments to the submission. To enhance reproducibility, we've also appended additional implementation details in the appendix.\n\nWe already made changes to the paper, but open review doesn't allow us to upload the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper addresses the challenge of preserving bias-independent information while neutralizing subjective bias. By incorporating a cycle-consistent network, the proposed approach ensures that the bias-independent content of the original text is preserved in the generated neutral text. This is an essential strength as it maintains the integrity and meaning of the original text. Also, the application of the method towards bias neutralization is novel and exciting to follow up on. \n\n2. In total, the paper proposes an application of a model in a new domain and setting, introduces an auxiliary guidance mechanism that highlights sequences of bias-inducing words, and evaluates the results thoroughly. \n\n", "reasons_to_reject": "1. The authors do not compare the performance of the FairBalance model with many other state-of-the-art models for subjective bias neutralization. Including such comparisons would provide a better understanding of the model's effectiveness and potential improvements. I feel it would have made the model easier to compare to the existing (and more limited) models, such as by Liu, Madanagopal, and Pryzant. \n\n2. The paper has some wording issues. For example, I don't favor word choices such as \"the models returned satisfactory results\", \"both models had a good balance\", extensive experiments, or \"the best way to evaluate the performance (...) is to conduct unbiased human evaluations\". Similar wordings happen occasionally throughout the paper, and I feel descriptions should be specific and the judgment left to the reader. \n\n3. Bias neutralization has ethical implications. However, the paper does not provide any ethics statement, even though such a statement does not affect the page limits. Not providing one is a major drawback.\n\n", "questions_to_authors": "1. There is no information about human judges, even though that might be very relevant. How was their education, age, etc.? This information is increasingly included in recent works about bias and relates to their labels. \n\n2. The study focuses on specific forms of subjective bias observed at the sentence level. However, considering the context (paragraph) in which a statement is made could potentially alter the perspective. You say one sentence on this, but do you already have ideas on how it will be implemented in your work?\n\n3. The referenced bias-neutralizing papers are not always completely clear in which kind of bias they deal with. However, bias (from linguistic bias to sentiment bias and bias by word choice and other concepts) is very complex. Can you clarify the definition of bias that you apply?\n\n4. There seems to be no link to all data and code (anonymized). Is this a mistake? The paper is not reproducible as it stands.", "paper_topic_and_main_contributions": "The paper addresses the issue of subjective bias in text, which can manipulate our perception of reality and intensify social conflicts. It highlights the importance of avoiding bias in resources like Wikipedia, scholarly articles, and news sources. Existing methods for bias neutralization rely on parallel text and struggle with domain transfer and content preservation. Mainly, the paper introduces a new approach called FairBalance, which aims to expand the reach of bias neutralization. It utilizes a cycle-consistent adversarial network to neutralize bias without needing parallel text. The model design focuses on preserving bias-independent content and auxiliary guidance.", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 920, the sentence \"We replaced all proper nouns with generic names ,\" has an unnecessary space before the comma.\nLine 505, the sentence: Since more than  30% of the biased words in our corpus were adjectives, the method of deleting biased words proved more effective by eliminating them and neutralizing the sentences. It is not entirely clear that \"them\" means adjectives. \nSome sources have issues, e.g., the paper in line 711 by Shikha Bordia and Samuel R Bowman is published by now. See https://aclanthology.org/N19-3002/. Sources should be thoroughly checked. \nThe authors seem to not have added DOIs to the references. I'm not fully aware whether this is an issue of the document to review only, but if they are not added, they need to be. "}}, {"review_id": "dET2FcTzYw", "reviewer": "Reviewer_htDX", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**FairBalance's performance depends on the quality of the bias tagger    used to identify biased language. If the bias tagger is not accurate,    the performance of FairBalance will be affected.**\n\nThe key factor contributing to the enhanced performance of the FairBalance model is the integration of the bias tagger. In the absence of the auxiliary signal provided by the bias tagger, the neutrality performance of the CycleGAN model was 66.94. However, with the inclusion of the bias tagger, we achieved an improved performance of 69.78. It's noteworthy that training a bias tagger is a relatively straightforward process, and the FairBalance model's adaptability to new domains is facilitated by leveraging this component. A significant advantage of employing the bias tagger lies in the control it affords over the bias mitigation process. Unlike the less controllable black box nature of adversarial learning, which doesn't offer precise control over specific linguistic aspects to address, the bias tagger provides a more manageable and controllable approach. This controllability empowers us to precisely pinpoint and address the aspects of language that require mitigation, contributing to the model's effectiveness in bias reduction.\n\n_________________________________\n\n**The experiments conducted in the paper are limited to a specific dataset and domain, and it is unclear how well FairBalance would perform on other datasets and domains.**\n\nEven though our proposed FairBalance model was trained on Wikipedia data, we have performed experiments on its domain adaptability on other domains such as academics, news and politics and provided the results in Table 5. \n\n_________________________________\n\n**The paper does not provide a detailed analysis of the computational complexity of the proposed approach, which could be a concern for large-scale applications.**\n\nWe value your feedback about the computational complexity of our proposed approach. While our primary focus in this paper was on bias neutralization using a non-parallel corpus, we noted the training durations of our models.\nFor context:\n - The RoBERTa-based discriminator training required approximately 3 hours on a V100 GPU with a batch size of 32.\n - The sequence2sequence model for bias tagger finished training in around 8 hours under the same GPU and batch size conditions.\n\nAll the adversarial models were trained for 200 epochs with a batch size of 16 on V100 with 16GB shared memory and following are the training times for the adversarial models:\n - CycleGAN - 18 days 8 hours. \n - Vanilla GAN model - 15 days 20 hours\n - MaskGAN - 9 days 6 hours\n - FairBalance model - 13 days 2 hours.\n\nThough we did not have details into theoretical computational complexities, these real-world training times should offer a practical perspective on the time efficiency of our models. We recognize the importance of this time complexity for large-scale applications, and will consider a more detailed analysis in our future works.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. FairBalance can be trained effectively in the absence of parallel data, which is a significant advantage since parallel data is often difficult to obtain.\n2. The model design preserves bias-independent content, which is important for maintaining the original meaning of the text.\n3. FairBalance produces text that is both semantically coherent and fluent, which is important for readability and comprehension.\n4. The auxiliary guidance mechanism incorporated in FairBalance helps to improve the training process and makes the results more consistent and reliable.\n5. FairBalance yields strong results in terms of bias neutralization quality, as demonstrated by extensive experiments.", "reasons_to_reject": "1. FairBalance's performance depends on the quality of the bias tagger used to identify biased language. If the bias tagger is not accurate, the performance of FairBalance will be affected.\n2. The experiments conducted in the paper are limited to a specific dataset and domain, and it is unclear how well FairBalance would perform on other datasets and domains.\n3. The paper does not provide a detailed analysis of the computational complexity of the proposed approach, which could be a concern for large-scale applications.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper is about bias neutralization in non-parallel texts. It proposes a new approach called FairBalance that uses a cycle consistent adversarial network to enable bias neutralization without the need for parallel text. The model design preserves bias-independent content and through auxiliary guidance, the model highlights sequences of bias-inducing words, yielding strong results in terms of bias neutralization quality. The paper also discusses the importance of avoiding bias in many heavily relied upon resources such as Wikipedia, scholarly articles, news sources, and emerging large language models. \n\nFairBalance is a new approach proposed in this paper for bias neutralization in non-parallel texts. It has three unique features:\n\n1. A cycle consistent adversarial network enables bias neutralization without the need for parallel text.\n2. The model design preserves bias-independent content.\n3. Through auxiliary guidance, the model highlights sequences of bias-inducing words, yielding strong results in terms of bias neutralization quality.\n\nFairBalance uses a transformer-based model as a generator to produce text that is both semantically coherent and fluent. It also incorporates an auxiliary guidance mechanism that guides the generator on which parts of the text are biased and need to be rephrased or removed. This approach improves the training process and makes the results more consistent.\n\nThe paper also discusses the limitations of FairBalance and suggests future directions for improvement, such as incorporating more contextual information and refining the bias tagger by incorporating more subtle forms of subjective biases.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "UiIwhY3OBK", "reviewer": "Reviewer_fXMK", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**I have a main concern on the baseline models used. All the baseline models are kind of old. MaskedGAN and CycleGAN, the two main baseline models used in this work, are introduced in 2018 and 2017, respectively. Are there no recent models that can be used to compare? In the related work sections, there were multiple recent works that were described - the proposed approach should compare with them, or at least some of them. Without comparison with more recent models, the evaluation seemed to be unfair.**\n\nWe appreciate your perspective on the selection of models that were a little outdated and would like to elaborate on the justifications for our approach based on the findings from our study. Our study aimed to comprehensively evaluate both disentanglement-based and adversarial-based models in the context of our bias neutralization dataset. This approach was motivated by the need to explore the strengths and weaknesses of different techniques in addressing bias neutralization using non-parallel corpus.\n\nIn the realm of disentanglement-based models, our investigation led us to the discovery that the control-gen method exhibited the most promising performance for bias neutralization within our bias neutralization dataset. This result came as a surprise, particularly given the prevailing expectation of better performance from more recent models. That’s why we used only the control-gen method for deeper analysis. Since our method was adversarial based, we choose one disentanglement method (best) and more than one adversarial model for our evaluation.\n\nComing to the adversarial-based models, we recognized that while cycleGAN and MaskGAN were relatively older references, we did not simply adopt these models as-is for our study. Instead, we took steps to enhance their relevance and applicability. We incorporated state-of-the-art transformer models to leverage advancements in natural language processing, ensuring that our adversarial-based models were equipped with the latest language understanding capabilities. We used RoBERTa models for training the pre-trained classifier for discriminator. Moreover, we employed a bias tagger for the purpose of masking, which further aligned these models with our dataset's specific requirements.\n\nBy adapting these models with modern components and techniques, we aimed to bridge the gap between their original designs and the demands of our bias neutralization task. This approach enabled us to harness the advantages of both established and cutting-edge methodologies, ensuring that our evaluation was as comprehensive and effective as possible.\n\nWe are committed to transparency in our approach and have taken your feedback to heart. To ensure clarity in our paper, we will include a section in the methodology that explicitly discusses our choice of baseline models and our reasons for not comparing with recent disentanglement models. By doing so, we hope to provide a clear explanation for our approach and to address any concerns related to the choice of baselines. A new appendix section called Baseline Selection will be added to the revised version of our paper. \n\nWe already made changes to the paper, but open review doesn't allow us to upload the revised version.\n_____________________________\n\n**Missing References:\nIn section 5.2 (description of baseline models), there were no citations and references for the baseline models, MaskGAN and CycleGAN. I noted that references have been made in related work but it should also be re-referenced in the main section that describes the baseline models.**\n\nThank you for highlighting our oversight in not including references to MaskGAN and CycleGAN in the baseline models section. We've now added the references as you suggested.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The work is interesting and the proposed approach aims to tackle several limitation of existing works.", "reasons_to_reject": "I have a main concern on the baseline models used. All the baseline models are kind of old. MaskedGAN and CycleGAN, the two main baseline models used in this work, are introduced in 2018 and 2017, respectively. Are there no recent models that can be used to compare? In the related work sections, there were multiple recent works that were described - the proposed approach should compare with them, or at least some of them. Without comparison with more recent models, the evaluation seemed to be unfair.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper is on subjective bias neutralization on text without the use of parallel text corpus. The paper first discuss the 3 limitation of existing works on subjective bias neutralization, namely 1) the reliance of parallel text corpus, where the resource is limited and expensive, 2) the limited adaptation capability of domains from existing works, and 3) the challenge of preserving the content while removing the bias. The proposed approach is on using a cyclic adversarial network to handle both subjective bias neutralization and content preservation. The experimental results with baseline models show that the approach is promising.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "In section 5.2 (description of baseline models), there were no citations and references for the baseline models, MaskGAN and CycleGAN. I noted that references have been made in related work but it should also be re-referenced in the main section that describe the baseline models."}}]

Author rebuttal:

Meta-review: This paper introduces a new method for reducing subjective bias in text without the need for parallel data using a cycle-consistent adversarial network. The methodology proposed is novel and the method is both conceptually useful and demonstrates an improved performance over prior methods. The paper is well-written. There are minor points that could have been improved -- the method was only trained on Wikipedia; hence, it is unclear how it would fair on other domains. The method has though been tested on a variety of domains. Moreover, the reviewers raise the point that the baselines that the authors compare to are quite old. This is though because there are no more recent baseline methods which operate on non-parallel data. It does make it easier to propose a method which outperforms prior work though.
Decision: Accept-Main