# Comparison of Modern Deep Learning Models for Speaker Verification
## 5 Experiment Setup  
In this study, a custom-created dataset of non-English short utterances was used to evaluate speaker verification models based on speaker embeddings. Comprising speech recordings from 50 Ukrainian politicians (50 unique voices, 33 male and 17 female) sourced from YouTube interviews, the dataset features 10 clips per individual, each up to 10 s long. This amounts to a total of 500 recordings, collectively spanning 1 h and 30 min (about 100 s per speaker). Every entry within the dataset consists of single-channel (mono) audio sampled at 16 kHz, recorded in the.wav format. Additionally, it is worth noting that the age range of the speakers spans from 20 to 70. The dataset is designed to be diverse, including speakers of different genders and ages, to provide a comprehensive evaluation of the speaker verification models. This dataset is an important resource for research in speaker verification and in the field of speaker verification with short-duration recordings. In order to evaluate the performance of the speaker verification models, a test dataset of 4500 pairs of recordings was created. Half of these pairs consist of recordings of the same speaker, while the other half consists of recordings of different speakers. This allows for a comprehensive evaluation of the models' ability to accurately identify a speaker and differentiate them from others. The dataset is organized into 50 distinct classes or speakers, each corresponding to a different speaker. Within the designated directory for each class, there are 10 audio files containing the respective speakers' voices. The structural layout of the dataset is illustrated in Figure 2.  
The test set, derived from the dataset depicted in Figure 2, is specifically tailored for evaluating speaker verification performance. In this restructured dataset, each speaker class yields 90 audio file pairs. These pairs are evenly divided into two subsets: one containing 45 intra-class pairs, where each audio file is matched with another from the same speaker, and another with 45 inter-class pairs, formed by pairing a random audio file from the target class with one from a different, randomly chosen class. The composition and structure of the test set are detailed in Figure 3.  
To evaluate and compare each of the speaker verification models' performance, the Equal Error Rate (EER), False Acceptance Rate (FAR) and False Rejection Rate (FRR) were used, which are popular performance metrics in biometric systems. The False Acceptance Rate (FAR) represents the proportion of incorrect acceptances, indicating the likelihood that an unauthorized speaker is mistakenly verified. The False Rejection Rate (FRR) indicates the proportion of incorrect rejections, reflecting the frequency with which an authorized speaker is erroneously denied. The Equal Error Rate (EER) metric is a measure of the error rate of the system where the threshold is modified in such a way that the False Acceptance Rate and False Rejection Rate are equal [44], providing a single value to summarize the overall trade-off between the FAR and FRR values. The experiment is conducted in the following steps for each of the pairs in the collected dataset. Both audio files of a pair are passed to the speaker verification model as an input, where the model outputs an embedding. Both output embeddings (which correspond to the input audio files) are compared using the cosine similarity function, which is a number between 0 and 1. Then, based on the retrieved similarity and threshold, a decision is made: if similarity is equal to or greater than the threshold, the decision is to accept (the speakers in the input audio files are the same person), but if it is lower than the threshold, the decision is to reject (the speakers in the input audio files are not the same person). Figure 2: Dataset structure. into a single cost metric, reflecting the overall effectiveness of the system.