Balancing the Scales: Reinforcement Learning for Fair Classification
Authors: Leon Eshuijs, Shihan Wang, Antske Fokkens


Reviewer paper summaries: This paper proposes several reinforcement learning approaches for fair classification of text data. The authors present several learning algorithms for optimizing a contextual multi-armed bandit formulation of the fair classification problem. The authors propose reward-scaling methods for promoting fairness. Extensive empirical results are presented on two text classification datasets with protected group labels.
Summary of strengths: * The paper is very well-written.
* The positioning of the paper in past related work is great.
* The experiments are thorough and well executed.
Summary of weaknesses: * The results do not align with the proposed approaches or the conclusions. The RL methods do not seem to provide any benefit based on the results over the supervised training with equal opportunity scaling. The authors claim they provide evidence in favor of such RL methods, but these results are not evident in the paper.
* The proposed approach seems quite overpowered for such a narrow problem. 
* Results on only two datasets are presented. It is not clear why these datasets are the best datasets to evaluate the proposed approach.
* There has been a lot of work for many years on fair classification, yet there does not seem to be a practical benefit for such work. * Furthermore, this work seems to provide no additional benefit over the previous work given the results.
The context vectors used are generated by quite outdated models 5 and 7 years old respectively. Additionally, these models could be biased, but the authors choose to just apply a small de-biasing model on top of a frozen model.
Comments, suggestions and typos: * RL seems like an over-powered tool for such a problem. It might be necessary in some situations, but does not seem to have much of a practical benefit over previous approaches.
* What are some practical applications where fair classification will be used? Why is RL a better approach than other supervised approaches?
* RL tends to require quite a large amount of data and compute and does not provide much of a benefit in such scenarios. Is this large amount of labeled data available in practical applications of fair classification?
* Bias in contextual encoder? Could we use RL to potentially de-bias the internal representations of an LLM? How would this work?

Minor nits / typos:
* Line 208: “The” -> “These”?
* Font size in Figures 2 & 3 is very small
* Inconsistency in subscript and superscript for reward scales $\mathcal{W}$
Overall assessment scores: 2