  End-to-end Automatic Speech Recognition (ASR) models are commonly trained
over spoken utterances using optimization methods like Stochastic Gradient
Descent (SGD). In distributed settings like Federated Learning, model training
requires transmission of gradients over a network. In this work, we design the
first method for revealing the identity of the speaker of a training utterance
with access only to a gradient. We propose Hessian-Free Gradients Matching, an
input reconstruction technique that operates without second derivatives of the
loss function (required in prior works), which can be expensive to compute. We
show the effectiveness of our method using the DeepSpeech model architecture,
demonstrating that it is possible to reveal the speaker's identity with 34%
top-1 accuracy (51% top-5 accuracy) on the LibriSpeech dataset. Further, we
study the effect of two well-known techniques, Differentially Private SGD and
Dropout, on the success of our method. We show that a dropout rate of 0.2 can
reduce the speaker identity accuracy to 0% top-1 (0.5% top-5).
