Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This work shows an interesting approach to introduce fairness to machine learning. The introduction and presenation of the approach are well written. The introduction provides a valuable overview and characterization of existing work and motivates the approach proposed in this work. Sections 2 through 5, together with the supplementary material provide a comprehensible description and derivation of the approaches to classification and clustering, together with the corresponding algorithms.
However, experimental section leaves open a number of questions. First of all, for readers not familiar with the task used here, a more detailed description would be desirable, including moving Supplementary Section D into the main text. In Suppl. Sec. D also a succinct definition of the classification task should be provided. What exactly are the classiffication tasks you solve for the Bank and Adult dataset, i.e. what is the set (and number) of classes in both cases?
Figs. 1 (a) and (b): you should provide a legend defining the red and blue curves.
Fig 1 (c): without knowing the other approaches cited here, some comment on the differences in the classifier models used would be helpful to evaluate the differences in test error. Effectively, the reader would like to get an indication, what the differences in test error are induced by: difference in the regulatory fairness terms, or also differences in the classifier models.
Overall, the three datasets used to test the approach seem to be fairly limited, as are the corresponding classifier models used. It would be interesting to get an indication of how the approach would perform on/scale to much larger tasks with (much) deeper classification models?
Completely missing is a final overall conclusion section.
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This paper proposes a method to use renyi correlation to improve fairness of ML models by reducing the dependence of outputs wrt sensitive inputs such as gender. Previous models either use a linear dependence measure or use more complex optimization objectives; this paper improves upon them. They first build a min-max objective where the goodness-of-fit and fairness are jointly optimized. This non-convex objective is difficult to optimize and the authors propose a reformulation of the renyi correlation for discrete random variable case where it reduces to finding second largest eigenvalue. Based on this, they reformulate the objective which can be optimized more efficiently. They show the performance of their model for supervised and unsupervised learning problems on 4 different dataset by comparing to standard correlations such as Pearson.

Overall the paper is clearly written and I liked the idea of using renyi correlation which also has a nice theoretical formulation allowing to be optimized more efficiently. But, the experimental results are a bit weak since the only model they experiment with is logistic regression. Given that their main motivation is to capture non-linear dependencies, I think some results with neural networks is necessary. Since their main focus is on discrete case, the authors can show if training a word embedding model with renyi regularization helps improve fairness of word embeddings.

I have several question regarding the paper:

The datasets that authors use have a predefined feature space. Can you show if the sensitive feature is really important for high accuracy? Can we get the same performance without the sensitive features?
Since the model is trained with gradient descent, how would a more simple baseline where the gradient of the sensitive feature is penalized work?
How would this algorithm generalize to larger problems such as language models since Q_{theta} is regenerated at every iteration? 
review_title: Official Blind Review #2

final_score: rating: 8: Accept
review_content: This paper proposes a novel approach to fair inference/learning by using the Renyi correlation in place of the standard correlation measures (e.g. Pearson correlation) as a measure of "unfairness" expressed as a form of dependence between an outcome and a sensitive variable (which includes a number of standard group fairness definitions including demographic parity and equality of odds). This is motivated by the limitation of standard notions of correlation that they only capture linear correlations. 
The proposed approach is a formulation using the Renyi correlation as a regularization term, which leads to a min-max formulation due to the definition of Renyi correlation as the maximum correlation between any functionals of the variables in question.
The paper then describes in detail how the computation can be done by making use of the fact that Renyi correlation can be computed using singular values of the relevant matrix, and provides computation time analyses for a couple of different assumptions. 
The authors also apply the analogous idea to (k means) clustering to derive what they call the Renyi fair clustering. 
In the experimental evaluation section, they evaluate the performance of the proposed methods with respect to the accuracy fairness trade-off, using 3 publicly available real world data sets and with the 2 standard definition of group fairness. The experimental results show quite convincingly that the trade of is improved by Renyi correlation as compared to the use of standard correlations.  They also evaluate the Renyi fair clustering method using a fairness metric they propose for evaluating the uniformity of clustering results.
Overall the paper presents a crisp new idea, develops sound theoretical analysis and algorithms, and validate their superiority with satisfactory experiments. 
review_title: Official Blind Review #3

Meta-review: The paper addresses the problem of fair representation learning. The authors propose to use Rényi correlation as a measure of (in)dependence between the predictor and the sensitive attribute and developed a general training framework to impose fairness with theoretical properties. The empirical evaluations have been performed using standard benchmarks for fairness methods and the SOTA baselines -- all this supports the main claims of this work's contributions. 
All the reviewers and AC agree that this work has made a valuable contribution and recommend acceptance. Congratulations to the authors! 

Decision: Accept (Poster)