# U-Vectors: Generating Clusterable Speaker Embedding from Unlabeled Data
## 4 Experiments and Evaluations
### Result Analysis  
Two methods are implemented, AutoEmbedder (pairwise architecture) and a triplet architecture, to analyze the speech embeddings based on the proposed strategy. However, apart from these two strategies, the currently famous speech vector methods do not hold to the training properties considered in the paper. They mostly follow a supervised learning or domain adaption strategy. Hence, they are disregarded in this experiment. DenseNet121 [41], is used as a baseline architecture for both of the DL frameworks. Further, both models are connected with a dense layer consisting of 12 nodes. Therefore, both pairwise and triplet networks produce 12 dimension embedding vectors. L2-normalization is added on the output layer of the triplet network, as it is suggested that it increases the accuracy of the framework [42]. For AutoEmbedder architecture, the default l2-loss is implemented, whereas the triplet architecture is trained using semi-hard triplet loss. The training pipeline is illustrated in Figure 3.  
The evaluation process guarantees that both architectures are trained using the same dataset/data-subset. As the training process is unsupervised, the architectures receive the same data for the training and testing process. However, for the training process, the labels are unknown and generated based on the paper's assumptions. We refer to such a dataset as a training dataset. By ground dataset, we refer to the same dataset that further considers the ground truth values. For training both frameworks, we used a batch size of 128. The training is conducted using Adam [43] optimizer with a learning rate of 0.0005. The learning rate and batch size were determined using a grid search. Although batch size 64 and 128 result in better evaluation, batch size 128 was considered due to faster computation. The training phase's data processing includes heavy computational complexity, including online noise augmentation and spectrogram conversion. Each dataset is randomly augmented with a threshold range of \([0,0.07]\) for the augmentation process. Further, computing ACC, NMI, and ARI metrics require quadratic time complexity. Hence, we limit the number of speakers to 150. Instead of training on the overall dataset, we train on a sub-set of data, where each speaker contains a speech of 10 s. For testing the ground truth data, a random selection of 2-s speech is selected for each speaker. To inquire the architectures properly, we scramble the training dataset's pseudo labels, that produces a can-link impurity in the dataset labels. Hence, we use the impurity ratio as a training data situation to illustrate the rate of impure cluster assignments on the training data pseudo-labels. The models were kept training until the performance on ground truth labels did not improve within the previous 1500 epochs. Figure 4 illustrates a benchmark of the triplet and pairwise architectures while training on three different datasets, with speakers = 25 and impurity = 0. The triplet architectures smoothly learn from the training data and greatly overfits on the augmented training data. The ground dataset benchmark is also as expected since the triplet architectures' correctness first increases and gradually decreases due to overfitting. Hence, from the visualization, it can be acknowledged that the triplet network only memorizes the speech features concerning the pseudo labels assigned to them. On the contrary, the pairwise architecture generates a satisfactory performance, with some irregularities. In general, deep learning architectures produce higher accuracy on  
Figure 3: The training of both pairwise and triplet networks is done through the same data processing pipeline. Half of the inputs are randomly augmented and processed using MFCC. The MFCC of each frame is further passed to the DL frameworks. the training dataset than validation dataset. However, in such a case, the ground dataset's performance is mostly more elevated than the training dataset. Yet, the performance on the ground datasets generally decreases after 400 epochs.