Introduction
Speaker Recognition is an essential task with applications in biometric authentication, identification, and security among others BIBREF0 . The field is divided into two main subtasks: Speaker Identification and Speaker Verification. In Speaker Identification, given an audio sample, the model tries to identify to which one in a list of predetermined speakers the locution belongs. In the Speaker Verification, the model verifies if a sampled audio belongs to a given speaker or not. Most of the literature techniques to tackle this problem are based on INLINEFORM0 -vectors methods BIBREF1 , which extract features from the audio samples and classify the features using methods such as PLDA BIBREF2 , heavy-tailed PLDA BIBREF3 , and Gaussian PLDA BIBREF4 .
Despite the advances in recent years BIBREF5 , BIBREF6 , BIBREF7 , BIBREF8 , BIBREF9 , BIBREF10 , BIBREF11 , Speaker Recognition is still a challenging problem. In the past years, Deep Neural Networks (DNN) has been taking place on pattern recognition tasks and signal processing. Convolutional Neural Networks (CNN) have already show that they are the actual best choice to image classification, detection or recognition tasks. In the same way, DNN models are being used combined with the traditional approaches or in end-to-end approaches for Speaker Recognition tasks BIBREF12 , BIBREF13 , BIBREF14 . In hybrid approaches, it is common to use the DNN model to extract features from a raw audio sample and then encode it on embedding vectors with low-dimensionality which samples sharing common features with closer samples. Usually, the embedding vectors are classified using traditional approaches.
The difficult behind the Speaker Recognition tasks is that audio signals are complex to model in low and high-level features that are discriminant enough to distinguish different speakers. Methods that use handcrafted features can extract more human-readable features and have a more appealing approach because humans can see what the method is doing and which features are used to make the inference. Nevertheless, handcrafted features lack in power. In fact, while we know what patterns they are looking for, we have no guarantee that these patterns are the best for the job. On the other hand, approaches based on Deep Learning have the power to learn patterns that humans may not be able to understand, but usually get better results than traditional methods, despite having more computational cost to training.
A promising approach to Speaker Recognition based on Deep Learning is the SincNet model BIBREF16 that unifies the power of Deep Learning with the interpretability of the handcrafted features. SincNet uses a Deep Learning model to process raw audio samples and learn powerful features. Therefore, it replaces the first layer of the DNN model, which is responsible for the convolution with parametrized sinc functions. The parametrized sinc functions implement band-pass filters and are used to convolve the waveform audio signal to extract basic low-level features to be later processed by the deeper layers of the network. The use of the sinc functions helps the network to learn more relevant features and also improves the convergence time of the model as the sinc functions have significantly fewer parameters than the first layer of traditional DNN. At the top of the model, the SincNet uses a Softmax layer which is responsible for mapping the final features processed by the network into a multi-dimensional space corresponding to the different classes or speakers.
The Softmax function is usually used as the last layer of DNN models. The function is used to delimit a linear surface that can be used as a decision boundary to separate samples from different classes. Although the Softmax function works well on optimizing a decision boundary that can be used to separate the classes, it is not appropriate to minimize the distance from samples of the same class. These characteristics may spoil the model efficiency on tasks like Speaker Verification that require to measure the distance between the samples to make a decision. To deal with this problem, new approaches such as Additive Margin Softmax BIBREF15 (AM-Softmax) are being proposed. The AM-Softmax introduces an additive margin to the decision boundary which forces the samples to be closer to each other, maximizing the distance between the classes and at the same time minimizing the distance from samples of the same class.
In this paper, we propose a new method for Speaker Verification called Additive Margin SincNet (AM-SincNet) that is highly inspirited on the SincNet architecture and the AM-Softmax loss function. In order to validate our hypothesis, the proposed method is evaluated on the TIMIT BIBREF17 dataset based in the Frame Error Rate. The following sections are organized as: In Section SECREF2 , we present the related works, the proposed method is introduced at Section SECREF3 , Section SECREF4 explains how we built our experiments, the results are discussed at Section SECREF5 , and finally at Section SECREF6 we made our conclusions.
Related Work
For some time, INLINEFORM0 -vectors BIBREF1 have been used as the state-of-the-art feature extraction method for speaker recognition tasks. Usually, the extracted features are classified using PLDA BIBREF2 or other similar techniques, such as heavy-tailed PLDA BIBREF3 and Gauss-PLDA BIBREF4 . The intuition behind these traditional methods and how they work can be better seem in BIBREF18 . Although they have been giving us some reasonable results, it is clear that there is still room for improvements BIBREF18 .
Recently, neural networks and deep learning techniques have shown to be a particularly attractive choice when dealing with feature extraction and patterns recognition in the most variety of data BIBREF19 , BIBREF20 . For instance, CNNs are proving to produce a high performance on image classification tasks. Moreover, deep learning architectures BIBREF21 , BIBREF22 and hybrid systems BIBREF23 , BIBREF24 , BIBREF25 , BIBREF26 , BIBREF27 are higher quality results on processing audio signals than traditional approaches. As an example, BIBREF28 built a speaker verification framework based on the Inception-Resnet-v1 deep neural network architecture using the triplet loss function.
SincNet BIBREF16 is one of these innovative deep learning architecture for speaker recognition which uses parametrized sinc functions as a foundation to its first convolutional layer. Sinc functions are designed to process digital signals just like audio, and thus the use of them as the first convolutional layer helps to capture more meaningful features to the network. Additionally, the extracted features are also more human-readable than the ones obtained from ordinary convolutions.
Besides, the sinc functions reduce the number of parameters on the SincNet first layer because each sinc function of any size only have two parameters to learn against INLINEFORM0 from the conventional convolutional filter, where INLINEFORM1 is the size of the filter. As a result, the sinc functions enables the network to converge faster. Another advantage of the sinc functions is the fact that they are symmetric, which means that we can reduce the computational effort to process it on INLINEFORM2 by simply calculating half of the filters and flipping it to the other side.
The first layer of SincNet is made by 80 filters of size 251, and then it has two more conventional convolutional layers of size five with 60 filters each. Normalization is also applied to the input samples and the convolutional layers, the traditional and the sinc one. After that, the result propagates to three more fully connected layers of size 2048, and it is normalized again. The hidden layers use the Leaky ReLU BIBREF29 as the activation function. The sinc convolutional layer is initialized using mel-scale cutoff frequencies. On the other hand, the traditional convolutional layers together with the fully connected layers are initialized using INLINEFORM0 scheme. Finally, a Softmax layer provides the set of posterior probabilities for the classification.
Additive Margin SincNet
The AM-SincNet is built by replacing the softmax layer of the SincNet with the Additive Margin Softmax BIBREF15 . The Additive Margin Softmax (AM-Softmax) is a loss function derived from the original Softmax which introduces an additive margin to its decision boundary.
The additive margin works as a better class separator than the traditional decision boundary from Softmax. Furthermore, it also forces the samples from the same class to become closer to each other thus improving results for tasks such as classification and verification. The AM-Softmax equation is written as: DISPLAYFORM0 DISPLAYFORM1
In the above equation, W is the weight matrix, and INLINEFORM0 is the input from the INLINEFORM1 -th sample for the last fully connected layer. The INLINEFORM2 is also known as the target logit for the INLINEFORM3 -th sample. The INLINEFORM4 and INLINEFORM5 are the parameters responsible for scaling and additive margin, respectively. Although the network can learn INLINEFORM6 during the optimization process, this can make the convergence to be very slow. Thus, a smart choice is to follow BIBREF15 and set INLINEFORM7 to be a fixed value. On the other hand, the INLINEFORM8 parameter is fundamental and has to be chosen carefully. On our context, we assume that both INLINEFORM9 and INLINEFORM10 are normalized to one. Figure FIGREF1 shows a comparison between the traditional Softmax and the AM-Softmax.
The SincNet approach has shown high-grade results on the speaker recognition task. Indeed, its architecture has been compared against ordinary CNNs and several other well-known methods for speaker recognition and verification such as MFCC and FBANK, and, in every scenario, the SincNet has overcome alternative approaches. The SincNet most significant contribution was the usage of sinc functions as its first convolutional layer. Nevertheless, to calculate the posterior probabilities over the target speaker, SincNet applies the Softmax loss function which, despite being a reasonable choice, is not particularly capable of producing a sharp distinction among the class in the final layer. Thus, we have decided to replace the last layer of SincNet from Softmax to AM-Softmax. Figure FIGREF4 is a minor modification of the original SincNet image that can be found in BIBREF16 which shows the archtecture of the proposed AM-SincNet.
Experiments
The proposed method AM-SincNet has been evaluated on the well known TIMIT dataset BIBREF17 , which contains audio samples from 630 different speakers of the eight main American dialects and where each speaker reads a few phonetically rich sentences. We used the same pre-processing procedures as BIBREF16 . For example, the non-speech interval from the beginning and the end of the sentences were removed. Following the same protocol of BIBREF16 , we have used five utterances of each speaker for training the network and the remaining three for evaluation. Moreover, we also split the waveform of each audio sample into 200ms chunks with 10ms overlap, and then these chunks were used to feed the network.
For training, we configured the network to use the RMSprop as optimizer with mini-batches of size 128 along with a learning rate of INLINEFORM0 , INLINEFORM1 , and INLINEFORM2 . The AM-Softmax comes with two more parameters than the traditional Softmax, and the new parameters are the scaling factor INLINEFORM3 and the margin size INLINEFORM4 . As mentioned before, we set the scaling factor INLINEFORM5 to a fixed value of 30 in order to speed up the network training. On the other hand, for the margin parameter INLINEFORM6 we carefully did several experiments to evaluate the influence of it on the Frame Error Rate (FER).
We also have added an INLINEFORM0 constant of value INLINEFORM1 to the AM-Softmax equation in order to avoid a division by zero on the required places. For each one of the experiments, we trained the models for exactly 352 epochs as it appeared enough to exploit adequately the different training speed presented by both competing models. To run the experiments, we used an NVIDIA Titan XP GPU, and the training process lasts for about four days. The experiments performed by this paper may be reproduced by using the code that we made available online at the GitHub.
Results
Several experiments were made to evaluate the proposed method against the traditional SincNet approach. In every one of them, the proposed AM-SincNet has shown higher accurate results. The proposed AM-SincNet method requires two more parameters, the scaling parameter INLINEFORM0 and the margin parameter INLINEFORM1 . We have decided to use INLINEFORM2 , and we have done experiments to evaluate the influence of the margin parameter INLINEFORM3 on the Frame Error Rate.
The Table TABREF6 shows the Frame Error Rate (FER) in percentage for the original SincNet and our proposed method over 352 epochs on the test data. To verify the influence of the margin parameter on the proposed method, we performed several experiments using different values of INLINEFORM0 in the range INLINEFORM1 . The table shows the results from the first 96 and the last 32 epochs in steps of 16. The best result from each epoch is highlighted in bold.
It is possible to see that traditional SincNet only gets better results than the proposed AM-SincNet on the first epochs when none of them have given proper training time yet. After that, on epoch 48, the original SincNet starts to converge with an FER around INLINEFORM0 , while the proposed method keeps decreasing its error throughout training.
In the epoch 96, the proposed method has already an FER more than INLINEFORM0 better than the original SincNet for almost every value of INLINEFORM1 excluding INLINEFORM2 . The difference keeps increasing over the epochs, and at epoch 352 the proposed method has an FER of INLINEFORM3 ( INLINEFORM4 ) against INLINEFORM5 from SincNet, which means that at this epoch AM-SincNet has a Frame Error Rate approximately INLINEFORM6 better than traditional SincNet. The Figure FIGREF7 plots the Frame Error Rate on the test data for both methods along the training epochs. For the AM-SincNet, we used the margin parameter INLINEFORM7 .
From Table TABREF6 , we can also see the impact of the margin parameter INLINEFORM0 on our proposed method. It is possible to see that the FER calculated for INLINEFORM1 got the lowest (best) value at the epochs 32 and 320. In the same way, INLINEFORM2 and INLINEFORM3 got the lowest values at epochs 16 and 336, respectively. The value INLINEFORM4 scores the lowest result for epochs 64 and 96, while INLINEFORM5 got the lowest score at epoch 80, and INLINEFORM6 reached the lowest value of epochs 48 and 352.
The INLINEFORM0 , INLINEFORM1 , INLINEFORM2 , and INLINEFORM3 does not reach the lowest values of any epoch in this table. Although the results in Table TABREF6 may indicate that there is a golden value of INLINEFORM4 which brings the best Frame Error Rate for the experiments, in fact, the difference of the FER calculated among the epochs may not be so significant. Indeed, at the end of training, all of the experiments with the AM-SincNet seem to approximate the FER to a value around INLINEFORM5 . In any case, AM-SincNet overcomes the baseline approach.
Conclusion
This paper has proposed a new approach for directly processing waveform audio that is highly inspirited in the neural network architecture SincNet and the Additive Margin Softmax loss function. The proposed method, AM-SincNet, has shown a Frame Error Rate about 40% smaller than the traditional SincNet. It shows that the loss function we use on a model can have a significant impact on the expected result.
From Figure FIGREF7 , it is possible to notice that the FER ( INLINEFORM0 ) from the proposed method may not have converged yet on the last epochs. Thus, if the training had last more, we may have noticed an even more significant difference between both methods. The proposed method comes with two more parameters for setting when compared with the traditional SincNet, although the experiments made here show that these extra parameters can be fixed values without compromising the performance of the model.
For future work, we would like to test our method using different datasets such as VoxCeleb2 BIBREF21 , which has over a million samples from over 6k speakers. If we increase the amount of data, the model may show a more significant result. We also intend to use more metrics such as the Classification Error Rate ( INLINEFORM0 ) (CER) and the Equal Error Rate ( INLINEFORM1 ) (EER) to compare the models.
Acknowledgment
This work was supported in part by CNPq and CETENE (Brazilian research agencies). We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan XP GPU used for this research.

Question: Do they compare computational time of AM-softmax versus Softmax?
Answer: No

Question: Do they visualize the difference between AM-Softmax and regular softmax?
Answer: Yes