  The human voice conveys unique characteristics of an individual, making voice
biometrics a key technology for verifying identities in various industries.
Despite the impressive progress of speaker recognition systems in terms of
accuracy, a number of ethical and legal concerns has been raised, specifically
relating to the fairness of such systems. In this paper, we aim to explore the
disparity in performance achieved by state-of-the-art deep speaker recognition
systems, when different groups of individuals characterized by a common
sensitive attribute (e.g., gender) are considered. In order to mitigate the
unfairness we uncovered by means of an exploratory study, we investigate
whether balancing the representation of the different groups of individuals in
the training set can lead to a more equal treatment of these demographic
groups. Experiments on two state-of-the-art neural architectures and a
large-scale public dataset show that models trained with
demographically-balanced training sets exhibit a fairer behavior on different
groups, while still being accurate. Our study is expected to provide a solid
basis for instilling beyond-accuracy objectives (e.g., fairness) in speaker
recognition.
