Venue: ICLR 2018 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Strengths: 
1. This paper proposes a novel framework for ensuring fairness in the classification pipeline. To this end, this work explores models that learn to defer. 
2. The work is conceptually very interesting. The idea of learning to defer (as proposed in the paper) as a means to fairness is not only novel but also quite apt. 
3. Experimental results demonstrate that the proposed learning strategy can not only increase predictive accuracy but also reduce bias in decisions. 

Weaknesses: 
1. While this work is conceptually quite novel and interesting, the technical novelty and contributions seem fairly minimal. 
2. The proposed formulations are essentailly regularized variants of fairly standard classification models and the optimization also relies upon standard search procedures.
3. Experimental analysis on deferring to a biased decision maker (Section 7.3) is rather limited. 

Summary: This paper proposes a novel framework for ensuring fairness in the classification pipeline. More specifically, the paper outlines a strategy called learn to defer which enables the design of predictive models which not only classify accurately and fairly but also defer if necessary. Deferring a decision is used as a mechanism to ensure both fairness and accuracy. Furthermore, the authors consider two variants depending on if the model has some information about the decision maker or not. Experimental results on real world datasets demonstrate the effectiveness of the proposed approach in building an end to end pipeline that ensures accuracy and fairness.  

Novelty: The main novelty of this work stems from the idea of introducing learning to defer mechanisms in the context of fairness. While the ideas of learning to defer have already been studied in the context of classification models, this is the first contribution which leverages learning to defer strategy as a means to achieve fairness. However, beyond this conceptual novelty, the work does not demonstrate a lot of technical novelty or depth. The objective functions proposed are simple extensions of work done by Zafar et. al. (WWW, AISTATS 2017). The optimization procedures being used are also fairly standard. Furthermore, the authors do not carry out any rigorous theoretical analysis either.   

Other detailed comments:
1. I would strongly encourage the authors to carry out a more in-depth theoretical analysis of the proposed framework (Refer to "Provably Fair Representations" McNamara et. al. 2017)
2. Experimental evaluation can also be strengthened. More specifically, analysis in Section 7.3 can be made more thorough. Instead of just sticking to one scenario where the decision maker is extremely biased (how are you quantifying this?), how about plotting a graph where x axis denotes the extent of bias in decision-maker's judgments and y-axis captures the model performance?
3. Overall, the paper is quite well written and is well motivated. There are however some typos and incorrect figure refernces (e.g., Section 7.2 first line, Figure 7.2, there is no such figure). 



review_title: Conceptually very interesting, but lacks technical novelty

final_score: rating: 5: Marginally below acceptance threshold
review_content: The proposed method is a classifier that is fair and works in collaboration with an unfair (but presumably accurate model). The novel classifier is the result of the optimisation of a loss function (composed of a part similar to a logistic regression model and a part being the disparate impact). Hence, it can be interpreted as a logistic loss with a fairness regularisation.

The results are promising and the applications are very important for the acceptance of ML approaches in the society. However, I believe that the model could be made more general (than a fairness regularized logistic loss) and its theoretical properties studied.
Finally, this paper used uncommon vocabulary (for the machine learning community) and it make is difficult to follow sometimes (for example, the use of a Decision-Maker entity).

When reading the submitted paper, it was unclear (until section 6) how deferring could help fairness. Hence, the structure of the paper could maybe be improved by introducing the cost function earlier in the manuscript (as a fairness regularised loss).

To conclude, although the application is of high interest and the numerical results encouraging, the methodological approach does not seem to be very novel.

Minor comment : 
- The list of authors of the reference “Machine bias : theres software…” apperars incorrectly (some comma may be missing in the .bib file) and there is a small typo in the title.

Possible extensions :
- The proposed fairness aware loss could be made more general (and not only in the case of a logistic model) 
- It could also be generalised to a mixture of biased classifier (more than on DM).

Edited :
As noted by a fellow reviewer, the paper is a bit out of the scope of ICLR and may be more in line with other ML conferences.
review_title: This paper proposes a method to address a highly important problem in Machine learning (fairness) as regularised cost (which it not so novel)

final_score: rating: 4: Ok but not good enough - rejection
review_content: I like the direction this paper is going in by combining fairness objectives with deferment criteria and learning.  However, I do not believe that this paper is in the scope defined by the ICLR call for papers, as there is nothing related to learned representations in it.

I find it quite interesting that the authors go beyond 'classification with a reject option' to learning to defer, based on output predictions of the second decision maker.  However, the authors do not make this aspect of the work very clear until Section 6.  The distinction and contribution of this part is not made obvious in the abstract and introduction.  And it is not stated clearly whether there is any related prior work on this aspect.  I'm not sure there is, and if there isn't, then the authors should highlight that fact.  The authors should discuss more extensively how the real-world situation will play out in terms of having two different training labels per sample (one from the purported DM and another used for the training of the main supervised learning portion).  Typically it is DM labels that are the only thing available for training and what cause the introduction of unfairness to begin with.

Throughout the paper, it is not obvious or justified why certain choices are made, e.g. cross-entropy.

In the related work section, specifically Incorporating IDK, it would be good to discuss the work of Wegkamp et al., including the paper http://www.jmlr.org/papers/volume9/bartlett08a/bartlett08a.pdf and its loss function. Also, the work of https://doi.org/10.1145/2700832 would be nice to briefly discuss.

Also in the related work section, specifically AI Safety, it would be good to discuss the work of Varshney and Alemzadeh (https://doi.org/10.1089/big.2016.0051) --- in particular the 'safety reserves' and 'safe fail' sections which specifically address reject options and fairness respectively.  

The math and empirical results all seem to be correct and interesting.
review_title: not in scope of call for papers

Meta-review: This work is proposing an approach for ensuring classification fairness through models that encapsulate deferment criteria. On the positive side, the paper provides ideas which are conceptually interesting and novel. On the other hand, the reviewers find the technical contribution to be limited and, in some cases, challenge the practicality of the method (e.g. requirement for second set of training samples). After extensive post-rebuttal discussion, the consensus is that the above issues make the paper fall below the threshold for acceptance – even if the “out-of-scope” issue is not taken into account.
Decision: Invite to Workshop Track