  Machine learning systems are often trained using data collected from
historical decisions. If past decisions were biased, then automated systems
that learn from historical data will also be biased. We propose a black-box
approach to identify and remove biased training data. Machine learning models
trained on such debiased data (a subset of the original training data) have low
individual discrimination, often 0%. These models also have greater accuracy
and lower statistical disparity than models trained on the full historical
data. We evaluated our methodology in experiments using 6 real-world datasets.
Our approach outperformed seven previous approaches in terms of individual
discrimination and accuracy.
