Kati8/ICLR_2023_2025 huggingface dataset:
authors: Niloy Purkait
authors: Emmanuel Keuleers
authors: Henry Brighton
reviews: 
Comments: The paper proposes a method to guarantee differential privacy (DP) and (approximate) group fairness in learning. The approach is based on adversarial debiasing: while the main model is learning to predict the labels, an adversarial model tries to predict the sensitive attributes given access to the final layer outputs of the main model. The main model is then jointly trained also to minimize the success of the adversarial model, which encourages the learned model to be less dependent on the sensitive features. In the current paper, adversarial debiasing is combined with other techniques: fairness and privacy (from Jagielski et al. 2019) and multi-attribute debiasing (from Kang et al. 2022).
Strengths: * Combining privacy and fairness is a timely topic, in the sense that there has been growing interest on it during the last years.

* Some parts of the paper are clear to read (see weaknesses for some comments on the other parts).
Weaknesses: * There are parts in the paper that could be made clear; most importantly related Alg.1, writing out in equations the projection, clipping and noising, the subtractions (lines 11-12 in Alg.1), etc. As it currently stands, with text description and some math, especially with the details like projections or DP, I have an uneasy feeling that I cannot exactly tell what is going on in the algorithm.

* I have some reservations on the originality of the proposed method: it seems like a fairly obvious combination of existing techniques, that to me does not seem to be any bigger than the sum of its parts.
Recommendation: 3: reject, not good enough
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 2 fair
Clarity: 2 fair
reviews: 
Comments: This work presents an enhancement to the adversarial debiasing approach, enabling it to account for multiple sensitive attributes while upholding a privacy-conscious learning paradigm. Extensive empirical results are conducted to demonstrate the ability of debiasing while providing privacy protection of the proposed method.
Strengths: - Extensive empirical results on benchmark datasets are conducted to evaluate the models.
- Using well-known metrics for Fairness evaluations
Weaknesses: - Marginal contribution regarding novelty.
- Unclear privacy guarantee since there's no theoretical analysis w.r.t privacy.
- The presentation of the work is not at the level of an academic publication.
Recommendation: 3: reject, not good enough
Reviewer confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness/correctness: 2 fair
Clarity: 1 poor
reviews: 
Comments: This paper studies fair classification under the constraint of differential privacy. Existing work on adversarial debiasing, in which an adversary tries to predict a sensitive attribute via gradient reversal, can debias only a single sensitive attribute. This paper proposes a modification to the adversarial debiasing approach which can debias multiple sensitive attributes.
Strengths: The intersection of privacy and fairness is a very interesting and important line of research! Being able to extend fairness from single attributes to multiple sensitive attributes seems like a practical and necessary direction. The authors also seem very knowledgeable on the background and related work.
Weaknesses: I find that the presentation style has some weaknesses…The writing style is very high-level which makes it hard to dig into the technical details. Mathematical notations are introduced informally and I don’t know if I’d be able to implement Algorithm 1 correctly based on the pseudo-code. Furthermore, the privacy guarantee for Algorithm 1 is not stated in the paper.

For an empirical paper, the experiments seem weak. Only three datasets are evaluated; only one privacy regime per dataset is studied; and it is difficult to compare the DP results across datasets because the $\epsilon$ value is different for each dataset. I also don’t see a strong takeaway message from the experimental results and the only baseline is a basic classification model.
Recommendation: 3: reject, not good enough
Reviewer confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 2 fair
Clarity: 2 fair
reviews: 
Comments: The paper enhances the adversarial debiasing approach, enabling it to account for multiple sensitive attributes while upholding a privacy-conscious learning paradigm.
Strengths: 1. The paper presentation is clear.
2. The paper is well motivated.
Weaknesses: 1. The proposed method is incremental and lacks novelty. It is a simple combination of Kang et al. (2022) with DPSGD.  
2. No proper baselines or related comparison justification for the paper. There are a number of works that share the same or similar setting as the paper (see the survey at https://arxiv.org/pdf/2305.06969.pdf. However, the paper does not provide a comparison with those methods.
3. Experiments only show the main results without additional in-depth analysis and ablation study. For example, how different parameter changes would affect the performance of utility and fairness trade-offs.  

Minor
1. Typos: “??” in the paper.
Recommendation: 3: reject, not good enough
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 3 good
Clarity: 3 good