Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper presents an approach to detoxifying and debiasing the outputs for language models at inference time with per-sample-based parameter-efficient fine-tuning. The paper takes two slightly different approaches to debiasing and detoxification but motivates them as being part of a unified Mutual Information Minimization framework between a generated token "x" and an attribute "a" (ex: male/female for debiasing or toxic/non-toxic for detoxification). 

Debiasing takes an adaptive token-level intervention approach wherein parameter-efficient fine-tuning is done to the model when there is a large margin between p(x|c) and p(x|a,c). Optimization is done to minimize this gap based on a MI minimization objective. Detoxification, in contrast, takes an approach similar to PPLM (Dathathriet al., 2020), but adapts only a small fraction of the top layers of the network.
strength_and_weaknesses: Strengths:
1. The paper seeks to solve an important problem, and is well written and motivated until about Section 3.3.
2. The approach, although quite expensive at inference time, avoids re-training the network.
3. The token-level attribute classifier for debiasing based on similarities with the first principal components of attribute-specific terms is clever.
4. Results on detoxification against DExperts which is a fairly strong baseline are impressive.

Weaknesses:
The main weakness of this paper is its presentation. The paper's intro, related work, and motivation around the use of mutual information minimization as a tool to do both debiasing as well as mitigate toxic responses are great, but the paper after that seems to lose focus and branches off into two fairly different approaches for debiasing and detoxification. These are my concerns about the presentation and structure

1. Reading appendix section D.2 was critical to my understanding of how the "redo" approach works - I would recommend moving it to the main body of the paper.

2. Having a pseudo-code block detailing the overall approach after all of the sections (after 3.5) will help the presentation. Right now it's not clear how each component comes together. Specifically what components do detoxification and debiasing share and what is different? Is one a token-level intervention but the other adopts an approach that re-generates things?

3. It is unclear how your initial motivation around minimizing mutual information I(x;a) ties into the final formulation for both section 3.4 and 3.5. There seems to be a disconnect in the presentation between sections 3.2, 3,3, and 3.4. 

4. The paper seems to be missing clear definitions of $p_{\theta}$ and $p_{\omega}$ and their parameterizations.

Clarification:
In Section 3.3, shouldn't we be comparing P(x|c) and P(x|a,c) rather than P (x|c) and P(x|a)? Why is context conditioning missing when conditioning on an attribute? You mention an approach for context-aware rectification but you do not seem to use it in the subsequent formulation.
clarity,_quality,_novelty_and_reproducibility: Clarity - Clarity of the paper is good until Section 3.2
Quality - The paper is of reasonably high quality barring presentation.
Novelty - The approach is not extremely novel, but is novel enough and borrows interesting ideas from parameter-efficient training
Reproducibility - The paper alone certainly lacks the details needed to reproduce this work.
summary_of_the_review: The paper has interesting ideas around the use of parameter-efficient tuning at inference time only, but would significantly benefit from re-organization and better presentation of the main ideas.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper presents a decoding algorithm that reduces (racial, gender) bias and toxicity in the generated outputs of language models. The proposed algorithm consists of token-level and context-dependent components that modify the bias terms of selected or all the layers of a language model at decoding time to suppress or encourage desired behaviors measured by attribute classifiers. Applied to reduce bias between different pairs of prompts and toxicity in output text, the author report improvements over standard evaluation measures for control and quality of the output text. 
strength_and_weaknesses: Strengths:
1. Substantial empirical improvements over baselines, especially in the debiasing setup which are confirmed by human evaluation.
2. Most individual losses proposed are simple and straightforward to implement. 

Weaknesses:
1. The definition of bias is not clear. In 3.1, bias is defined as the difference between f(.) value of two outputs xi and xj generated by different prompts ci and cj. It is not clear if this property is being measured per example or distributionally (the latter makes more sense but it is not clarified in the writing). Additionally, why should it to be two different prompts with biased outputs that should be considered. Shouldn't the goal be given a single prompt, the output should equally likely predict outputs related to either class (gender or race)?
2. In 3.5, a threshold TH is defined to control for perplexity. This threshold in practice is seems difficult to define properly in realistic settings since given the context length and the output sequence length, the perplexity can vary a lot. 
3. The writing is confusing in many places. Multiple definitions of bias is being used without clarification (social bias vs the bias term in neural networks). confusing notations with $x$ used to refer to both whole sequences and just tokens.

Questions:
1. What is the intuition behind using Hellinger distance instead of other distances?
2. For evaluating global bias, how are ci and cj pairs selected? Are there any restrictions placed on whether the prompts talk about the same topic/content?
clarity,_quality,_novelty_and_reproducibility: There are clarity issues in many places as described in the previous section

The method appears novel as far as I am aware of this work. And seems like the results can be reproduced given hyperparameter details. 
summary_of_the_review: The presented algorithm for debiasing and detoxifying language model outputs is simple to implement and has sound motivations. There are some clarity issues which make the paper hard to follow, especially in section 3. This should be addressable in future versions of this paper, and I am willing to revise my score if my questions are answered and clarifications provided.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper presents a unified framework for simultaneously detoxifying and debiasing language models. They consider protected groups as well as toxicity as an attribute of the generated text which can be controlled during the decoding time during an attribute classifier. Authors propose to use an embedding representation based attribute classifier for debiasing and a toxicity classifier for detoxifying tasks. In order to make debaising and detoxifying task parameters efficient, they propose to update only bias terms. Experiments show that the proposed method is computationally efficient and leads to minor degradation in the generation quality. 
strength_and_weaknesses: ### Strengths: 
- To the best of my knowledge, this is the first framework to unify debiasing and detoxifying tasks for open-ended generation.
- Authors have provided clear motivation and intuitions for most of the design choices. Authors have conducted ablation studies to show the effectiveness of their approach.   

### Weaknesses: 

- While the authors emphasize on unification, they use two different versions of attribute classifiers. In particular, for debiasing, they use embedding vectors’s principal component but for detoxifying, they use PPLM’s classifier. It’s not clear 1) why do we need two classifiers?  2) what gains do we get if we use PPLM’s classifier for both tasks.  
- Authors suggest that their framework updates bias terms in only a few layers. However, for debiasing, they update bias terms across all layers. While they frame it as they do it because their framework is efficient, selecting a few bias terms to update is a non-trivial problem. If we select top-k layers for bias update, we might get different results. Authors don’t have any ablation study to show the effectiveness of selecting bias terms for a few layers. 
- Authors mainly rely on perplexity as a proxy to measure quality of generations which is troublesome because 1) perplexity can’t be trusted as a generation quality metric. 2) authors use perplexity as a threshold. Authors present human evaluation results on a subset but do not show correlation with their automated metrics which makes it hard to understand how trustworthy the results are?   
- Multiple critical details are missing from the paper which makes it very difficult for an individual to reproduce the results presented in the paper. 
  - Authors construct their own dataset using publicly available BOLD dataset.  The exact dataset used for the experiments is not available in public which makes it difficult to reproduce the results presented in the paper.  
  - A word can be splitted into multiple subwords, it’s not clear how it’s being handled at attribute classifier level. 
  - Hyperparams used for generations are not defined. Recent works [1, 2] show that decoding algorithm hyperparameters alone play a big role on generated text fairness and quality so it’s important to explicitly define these and ideally use a range of hyperparameters instead of a fixed set of hyperparmas.  

References: 
-  [1] Akyürek, A. F., Kocyigit, M. Y., Paik, S., & Wijaya, D. (2022). Challenges in Measuring Bias via Open-Ended Language Generation. arXiv preprint arXiv:2205.11601.
-  [2]  Dhamala, J., Kumar, V., Gupta, R., Chang, K. W., & Galstyan, A. (2022). An Analysis of the Effects of Decoding Algorithms on Fairness in Open-Ended Language Generation. arXiv preprint arXiv:2210.03826

clarity,_quality,_novelty_and_reproducibility: Clarity, Quality:
Paper cites the relevant literature. However, as pointed out in the weaknesses section, some of the experiments are missing.

Novelty:
The proposed framework to unify detoxifying and debiasing is novel. 

Reproducibility:
As pointed out in the weakness section, some critical details to reproduce the results are missing. 


summary_of_the_review: Debiasing and Detoxifying are very important problems for NLG community. While building solution to tackle bias and toxicity, researchers often treat these as two separate problems. I think this paper is a good contribution towards unification of debiasing and detoxifying solutions.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper presents an approach to detoxifying and debiasing the outputs for language models at inference time with per-sample-based parameter-efficient fine-tuning. The paper takes two slightly different approaches to debiasing and detoxification but motivates them as being part of a unified Mutual Information Minimization framework between a generated token "x" and an attribute "a" (ex: male/female for debiasing or toxic/non-toxic for detoxification). Reviewers had mostly clarification and presentation issues which the authors have addressed in their response. It is an interesting paper and should be presented at the conference. 

This paper was flagged for ethical review, and the recommendation from the ethics committee was that the authors should explicitly recongize their assumptions with respect to the terms they employ (e.g., bias, race and toxicity) and be explicit about the limitation of the research. We expect the authors to respect these recommendations. 
justification_for_why_not_higher_score: I think the topic is interesting yet slightly marginal, this is why I am recommending Acceptance as a poster. 
justification_for_why_not_lower_score: Very good author response, reviewers comments were mostly positive.
Decision: Accept: poster