# Self Attention Networks in Speaker Recognition
## 5 Experimental Setup  
The VoxCeleb datasets [37; 60; 61] are used in all of our experiments. They consist of speech segments extracted from a wide range of YouTube videos, encompassing diverse scenarios such as clean studio interviews, red carpet interviews, outdoor environments with background noise, and situations involving multiple speakers. The videos comprise a mixture of professionally recorded content as well as footage captured using handheld or unrefined recording devices, without any editing. The VoxCeleb datasets were obtained through an automated pipeline that relied on computer vision techniques. The noise, reverberation, compression, and other artifacts in the corpus have not been removed from the original recorded speech. Since speech signals are extracted from interviews, the segments are usually short and the speech is conversational. The segments come in different duration and may include background speech from other identities. There may be several speech signals per speaker available in the corpus. For all the experiments the systems are evaluated on the fixed-condition VoxSRC 2019 training restrictions [62]. The fixed training condition mandates training exclusively on the VoxCeleb2 dev dataset [37], which includes 1,092,009 utterances from a total of 5994 speakers. During the test time for the VoxCeleb1 protocol, it is evaluated over the VoxCeleb1 test pairs. The VoxCeleb1-E test protocol consists of a collection of 581,480 random pairs extracted from the complete VoxCeleb1 dataset, which includes both the development and test subsets. These pairs encompass 1251 different speakers. The VoxCeleb1-H test protocol involves a set of 552,536 pairs sampled exclusively from the VoxCeleb1 dataset. All pairs within this set share the same nationality and gender. The list comprises a total of 18 combinations of nationality and gender, with each combination containing a minimum of 5 individuals. A small subset of about 10% of the data is reserved as a validation set for the best model selection. For all the trained systems in our study, SpeechBrain [63] is employed for the whole training/test pipeline. It is used to extract the 128-dimensional (unless otherwise stated) Mel-Frequency Cepstral Coefficients (MFCCs) with the standard 25 ms window size and 10 ms shift to represent the speech signal. To form the input for the networks, the deltas and double deltas of the MFCCs are combined, resulting in a 384-dimensional representation. Before training, all features undergo sentence-level Cepstral mean normalization. The models are then trained using speech feature chunks consisting of 300 frames. During testing, embeddings are extracted from the entire speech signal. For the models trained with data augmentation, a total of 5 extra samples are generated for each utterance. The first two augmentations are provided using speed perturbation by slightly speeding up or slowing down an audio signal. This is accomplished via resampling the audio signal at a rate that is similar to the original rate, to achieve a slightly slower or slightly faster signal [64]. The remaining three augmentations are generated by adding a mix of noise and reverberation using the RIR dataset provided in [65]. During the training process, all models are trained using the cyclical learning rate technique, which varies between \(1\times 10^{-8}\) and \(1\times 10^{-3}\). The learning rate follows a triangular policy as described in [66]. The Adam optimizer [67] is utilized. Each cycle consists of 130,000 iterations. Additive Angular Margin softmax (AAM-softmax) [68; 69] is employed with a margin of 0.2 and softmax prescaling of 30. To mitigate overfitting, a weight decay of \(2\times 10^{-6}\) is applied to all model weights. The mini-batch size for training is 64. All models are trained for 150 epochs and the best model is chosen according to the error rate of the validation data. To compute singular values we take advantage of the linear algebra routine of the NumPy package [70].