  As machine learning algorithms have been widely deployed across applications,
many concerns have been raised over the fairness of their predictions,
especially in high stakes settings (such as facial recognition and medical
imaging). To respond to these concerns, the community has proposed and
formalized various notions of fairness as well as methods for rectifying unfair
behavior. While fairness constraints have been studied extensively for
classical models, the effectiveness of methods for imposing fairness on deep
neural networks is unclear. In this paper, we observe that these large models
overfit to fairness objectives, and produce a range of unintended and
undesirable consequences. We conduct our experiments on both facial recognition
and automated medical diagnosis datasets using state-of-the-art architectures.
