Title: Benchmarking Stochastic Approximation Algorithms for Fairness-Constrained Training of Deep Neural Networks
Primary area: datasets and benchmarks
Keywords: Fair Machine Learning, stochastic approximation, Augmented Lagrangian, Sequential Quadratic Programming, benchmarking

Abstract: The ability to train Deep Neural Networks (DNNs) with constraints is instrumental in improving the fairness of modern machine-learning models. Many algorithms have been analysed in recent years, and yet there is no standard, widely accepted method for the constrained training of DNNs. In this paper, we provide a challenging benchmark of real-world large-scale fairness-constrained learning tasks, built on top of the US Census (Folktables, Ding et al, 2021). We point out the theoretical challenges of such tasks and review the main approaches in stochastic approximation algorithms. Finally, we demonstrate the use of the benchmark by implementing and comparing three recently proposed, but as-of-yet unimplemented, algorithms both in terms of optimization performance, and fairness improvement. We will release the code of the benchmark as a Python package after peer-review.

TL;DR: We provide a benchmark for comparing stochastic approximation algorithms, based on real-world fairness-constrained learning problems.