Learning Repeatable Speech Embeddings Using An Intra-class Correlation Regularizer
Authors: Jianwei Zhang
Suren Jayasuriya
Visar Berisha
Venue: NeurIPS 2023 poster (2023)
Keywords: repeatability
embeddings
metric learning
intra-class correlation
intra-class variance
TL;DR: We propose a novel regularizer based on the intra-class correlation coefficient to enforce deep architectures to learn repeatable embeddings.

Abstract: A good supervised embedding for a specific machine learning task is only sensitive to changes in the label of interest and is invariant to other confounding factors. We leverage the concept of repeatability from measurement theory to describe this property and propose to use the intra-class correlation coefficient (ICC) to evaluate the repeatability of embeddings. We then propose a novel regularizer, the ICC regularizer, as a complementary component for contrastive losses to guide deep neural networks to produce embeddings with higher repeatability. We use simulated data to explain why the ICC regularizer works better on minimizing the intra-class variance than the contrastive loss alone. We implement the ICC regularizer and apply it to three speech tasks: speaker verification, voice style conversion, and a clinical application for detecting dysphonic voice. The experimental results demonstrate that adding an ICC regularizer can improve the repeatability of learned embeddings compared to only using the contrastive loss; further, these embeddings lead to improved performance in these downstream tasks.

Learning Repeatable Speech Embeddings Using An Intra-class Correlation Regularizer
Jianwei Zhang
School of Electrical, Computer and Energy Engineering
Arizona State University
Tempe, AZ 85281
j janwei. wwwng@asu.naee...mo edu
Suren Jayasuriya
School of Arts, Media and Engineering
School Electrical, Computer and Energy Engineering
Arizona State University
Tempe, AZ 85281 sj sjayasur edu
Visar Berisha
College of Health Solutions
School of Electrical, Computer and Energy Engineering Arizona State University
Tempe, AZ 85281 visar@asu. edu
Abstract
A good supervised embedding for specific machine learning task is only sensitive to changes in the label of interest and is invariant to other confounding factors. We leverage the concept of repeatability from measurement theory to describe this property and propose to the intra-class correlation coefficient (ICC) to evaluate the repeatability of embeddings. We then propose novel regularizer, the ICC regularizer, complementary component for contrastive losses guide deep neural networks to produce embeddings with higher repeatability. We use simulated data explain why the ICC regularizer works better minimizing the intra-class variance than the contrastive loss alone. We implement the ICC regularizer and apply it three speech tasks: speaker verification, voice style conversion, and a clinical application for detecting dysphonic voice. The experimental results demonstrate that adding an ICC regularizer can improve the repeatability learned embeddings compared to only using the contrastive loss; further, these embeddings lead improved performance in these downstream tasks.
Introduction
Embeddings, which relatively low-dimensional latent representations of high-dimensional inputs, are widely used in deep learning applications and often trained through supervised learning techniques. In such cases, effective embeddings should be sensitive changes in the target class (e.g., speaker identity, clinical class) while remaining invariant to unrelated factors (e.g., noise, natural variations of the data). Embeddings that satisfy this desired property have high repeatability, term borrowed from measurement theory where it characterizes the consistency between the outcomes of consecutive
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

measurements of the same target when the underlying conditions remain unaltered 23 451, For instance, in a text-independent speaker verification (TI-SV) task, the speaker embedding extracted from recordings with varying content from the same speaker should remain consistent. Similarly, in dysphonic voice detection task, voice feature embeddings derived from the vowel phonation of a healthy person over consecutive days should remain consistent. By achieving high repeatability. embeddings effectively capture essential features while disregarding irrelevant factors.
A key to improving repeatability is reducing the intra-class variance, which is often increased by various confounding factors. Several studies have proposed different approaches handling intra- class variation including increasing the variability of training data [60, 47 7 13 61] and novel learning algorithms [2 42]. However, repeatability is rarely explicitly considered during training or evaluation of embeddings. In most cases, only the downstream applications' performance is used to indirectly evaluate the embeddings' effectiveness. We posit that directly assessing repeatability, regardless of downstream application, can help improve the quality of learned latent representations.
In this paper, we propose to use the intra-class correlation coefficient (ICC) to evaluate embeddings' repeatability. The ICC was designed to assess the consistency between two or more quantitative measurements [36], and often used to evaluate the repeatability of metries across different fields [2 We further propose novel regularizer based on the ICC as a complementary component to traditional contrastive losses to enforce deep architectures to learn repeatable embeddings. Some contrastive losses, such as GE2E [54], push embeddings towards the centroid of the true class to reduce their intra-class variance. Via analysis and intuition, we explain why the ICC regularizer better focuses on minimizing the intra-class variance than contrastive loss alone and provide new perspective for latent representation learning.
33 49].
Repeatability is an especially difficult property to enforce in complex, high-dimensional signals like speech. Speech characteristics depend on the speaker's neurological and physiological state, the degrees of freedom in the speaking task, the recording setup, the environment, etc. [45]. These sources of variation challenge the development of embeddings for a particular task (e.g.. learning speaker embeddings). We use the ICC regularizer to improve the embeddings' repeatability in three speech tasks: speaker embeddings for TI-SV, zero-shot voice style conversion, and voice feature embeddings for clinical application of dysphonic voice detection. Our experimental results demonstrate that the proposed ICC regularizer can significantly improve the repeatability of learned embeddings, and embeddings with higher repeatability perform better in the downstream tasks. In the TI-SV task, the speaker embeddings' repeatability is significantly enhanced, and the EER decreases by ~ 10% compared to the methods without ICC regularizer. AB preference test results for zero-shot voice style conversion show that embeddings with higher repeatability are preferred 62% to 38% over those with lower repeatabilityy. Objective evaluation metrics further confirm that more repeatable embeddings lead to improved performance in the voice style conversion task. In the clinical application, we demonstrate that highly repeatable voice feature embeddings further improve the in-corpus classification accuracy by ~ 3% and model generalizability across different corpora.
We summarize our contributions as follows:
We connect the concept of repeatability from measurement theory to deep-learned embed- dings and suggest the ICC for evaluating the quality of embeddings.
We propose a novel regularizer, the ICC regularizer, as a complementary component for contrastive loss to regularize the deep-learned embeddings such that they are repeatable.
We illustrate the reason why the ICC regularizer better minimizes the intra-class variance than the contrastive loss and provide new perspective for latent representation learning.
Our experimental results demonstrate that ICC regularizer can improve the repeatability of learned embeddings, and embeddings with higher repeatability exhibit better performance in downstream tasks.
For reproducibility of our work, the code for the ICC regularizer and experiments is available open-source in our GitHub repository[]
2

---

2
Related Work
Learning invariant embeddings: Previous literature has proposed to improve the robustness of learned latent representations by learning invariant embeddings. Most of these studies made the em- beddings invariant to one or two types of variation, e.g. pose-invariance for re-identification [= 35
noise-invariance for speaker recognition 7 38) personality-invariane for emotion recognition [5
There are two main approaches to promoting invariance to confounding factors: (1) increase the vari- ability of training data, e.g., combining datasets from different domains [33, data warping [60] 47].
data augmentation by GANs [6]]: and (2) novel learning paradigms. such as variance-invariang
invariant risk minimization [2], and simultaneously enforcing equiv- ariance and invariance 422. In this work, we follow the second approach by introducing new regularizer for training embeddings.
covariance regularization
Intra-class and inter-class variance: The variation between multiple observations of a class (intra-class variance) and the variation between classes (inter-class variance) define the performance of many machine learning criteria. Minimizing intra-class variance and maximizing inter-class
variance keys many deep learning tasks, including classification (39 63
representation learning [17 29 411. and few-shot learning [10 g 46]. However, these works typically rely on
visualization (c.g.. t-SNE, UMAP) to show how well their methods minimize intra-class variance and maximize inter-class variance (29 34]. The proposed metric for repeatability, the ICC, can directly evaluate the performance of given method using an ereead--sttbllssed and well-understood metric from measurement theory. Furthermore, we can construct a regularizer centered around this metric.
Contrastive loss: Researchers have proposed finding task-relevant embedding features by contrastive representation learning and leveraging labeled data [50]. There are several popular contrastive losses in deep learning, such the triplet loss 3 31, tuple-based end-to-end (TE2E) loss 20), generalized end-to-end (GE2E) Ioss [54. momentum contrast (MoCo) [19], and SimCLR LR 9 A contrastive loss encourages inputs of the same label class have more similar latent representations compared to inputs from different classes. We select GE2E loss as a representative contrastive loss for comparison as it widely adopted and used for supervised learning. The similarities and differences between the contrastive loss and the ICC regularizer are discussed in Section 3.2
Intra-class correlation coefficient: Repeatability is most frequently measured via intra-class correlation coefficient (ICC) [56]. Shrout and Fleiss elaborated several cases and corresponding formulas for ICC
4 The ICC is used widely in different fields estimate the reliability and
repeatability, including clinical applications 45 [4], psychology and behavioral science [3] and
medical imaging (6 49). To the best of our knowledge, the ICC has not been applied for embeddings
training and evaluation.
3 Method
In this section, we present the intra-class correlation coefficient (ICC) for evaluating the repeatability of deep-learned embeddings. and we present a novel ICC regularizer for enforcing repeatability in learned embeddings during training. Then, we illustrate the similarities and differences between contrastive loss and ICC regularizer by analyzing the intra-class and inter-class variance in relation to these two elements. Finally, we highlight why the ICC regularizer cannot function independently and its requirement for hyperparameter fine-tuning.
3.1 Intra-class correlation regularizer
Shrout and Fleiss elaborated several cases and corresponding formulas for the ICC 43 and we select the ICC 1-1 formulation, which is a measure of absolute agreement (i.c., the model generates the same embeddings to different samples from the same target) [66 271, for assessing the repeatability of deep-learned embeddings.
In our problem formulation, we assume there are a set of high- high-dimensional data x E IR each with label y E [1. N). The goal to learn lower-dimensional latent representations E B. that exhibit maximum separability: embeddings belonging to the same class should be closely clustered together, while embeddings from different classes should be distinctly separated from each other [32]. An encoder with weights w and characterized by f(x; w), takes input and outputs
3

---

diver-dimensional representation During training, the weights of the encoder are modified by optimizing a loss function that depends on and labels y. Considering a dataset with y [l..., N] and M input samples per class, the embedding vector eji is defined as the & normalization of the encoder output f(x w) و N, wi M, sis L)):
o5 = Sej G0 2) c.:.- = f(066 w)
where the Xj represents the i-th sample of j-th class and the eji represents the corresponding embedding vector (the notation is similar to other contrastive losses), and e/. represents the 1-th embedding dimension of eyy.
Then ICC(e'), the ICC score for I-th embedding dimension, can be calculated as follows [6
MSB(c') MSw(e)) ICC(e)) MSn(e') + (M M)
Here the (S' represents inter-class (between-class) variance and MSw (c') represents the intra-class (within-class) variance for I-th embedding dimension. MS calculated
M M-E, (c) マソ
MSB(e')
where of Ex S//M represents the mean of 1-th embedding dimension for the j-th class and M 2/ x M) represents the overall mean of (-th embedding dimension. Intuitively, (Sn(ee) measures the inter-class variance of 1-th embedding dimension.
= Σ Ex
MSw (e') is calculated as
Σ Mo? J.1
MSw(e') N(M -)
where 03,, = Σt (c), -2)3 represents the variance of 1-th embedding dimension for j-th class. Intuitively, MSw (e)) measures the overall intra-class variance of I-th embedding dimension.
After obtaining the ICC score for each embedding point, we use the mean value of all embedding dimensions' ICC scores as the repeatability metric:
TCC(e) = Σt ICC(ee)
(5)
ICC score interpretation: When the learned embeddings exhibit perfect repeatability (characterized by intra-class variance MSw = 0 and inter-class variance MSn 0), the ICC score is equal to A decrease in the ICC score signifies reduced repeatability, which implies a relative increase in intra-class variance M SSw compared to inter-class variance MS If the intra-class variance M Sw exceeds the inter-class variance ASS,, the ICC score may become negative
We propose a novel regularizer, the ICC regularizer, for regularizing learned representations to enforce high repeatability. The ICC regularizer operates on each batch: we assume a single batch contains samples from N classes, and M input samples from each class, and the dimension of the embedding is L. The ICC loss firstly uses Equation 5 to calculate the mean ICC score ICC(e) for the embeddings e of current batch, and then the ICC regularizer can be written
"Often noted as MR in ICC related literature 43 16
'For instance, consider simple numerical example where there is N 2 classes and M samples per class. Class has mean μ1 0 and variance of = 100, class 2 has mean μl = 0.1 and variance of = 100. In this situation, MSg- = 0.3, MS 120, then ICC 19.19 < 0. M has big effect in this situation: when M is large, the ICC score is still negative but very close to 0.

---

Mic=1-ICCCee
(6)
In this paper, the ICC and ICC regularizer require equal class size, i..., M is the same for all classes. However, to expand the ICC regularizer usage for scenarios where the number of samples per class may not be equal, we provide an extended version of the ICC formulation and code for the imbalanced classes the Appendix A and our GitHub repository
3.2 ICC regularizer vs. contrastive loss
(n)
GE2E Loss
(b)
ICC Regularizer
[c)
SVM Classification Error Rate
0.00
0.5
6:55
Inter dazs Variance
0.41
dass Variance
Intra Intra-dass dass Variance
Intra-class Variance
The contour figures for (a) GE2E loss and (b) ICC regularizer value of intra- and inter-class variance. To explore the gradient trend of GE2E loss and ICC regularizer, some starting points (red dots) are selected, then the maximum gradient descent path (red dashed lines) of value is calculated and plotted. (c) The contour figure for SVM classification error rate on simulation data per intra- and inter-class variance.
Figure I:
The ICC regularizer and contrastive loss have similarities their optimization criteria: both aim to minimize the intra-class variance and maximize the inter-class variance. However, they exhibit different tradeoffs between the two variances. We use a Monte Carlo simulation to study the similarities and differences of the ICC regularizer and contrastive loss the intra-class and inter- class variance. We use the GE2E loss (54 as a representative contrastive loss in simulation
Simulation setup: In simulation, we vary the intra- and inter-class variances to generate samples as follows: (1) the intra-class variance, i.e., the variance of simulated embeddings within one class, varies from 0.02 to 2.0 with step size of 0.02; (2) the inter-class variance, i.e., the variance of class centroids, varies from 0.01 to 0.60 with a step size of 0.01. For pair of configurations (e.g. inter-class, intra-class pair), draw 400 samples from an 8 8-dimensional, 4-class Gaussian mixture such that the class-com condissondiionnal mean and variance yield the desired inter-class, intra-class variance pair. We calculate the ICC regularizer value and GE2E loss value for these random samples. We repeat the Monte Carlo simulation 100 times and plot the loss values in Figure [ for the ICC regularizer and GE2E loss function of the inter-class and intra-class variance. To explore their landscapes, select several starting points, then trace the path of maximum gradient descent.
The ICC regularizer and GE2E loss have very different contours. They both have low value when the inter-class variance high and the intra-class variance is low. However, when embeddings are normalized (see Equation IT. the normalization places upper limit on the total variance of embeddings and limits how large the inter-class variance can be (the inter-class variance cannot go to infinity). So, regularizer that places greater emphasis minimizing the intra-class variance for a bounded inter-class variance will naturally lead to embeddings with higher repeatability compared to a loss that focuses on simultancously maximizing both variances.
Discussion:
The GE2E loss tries optimize both intra-class and inter-class variances simultaneously: the maximum gradient descent direction of the GE2E loss overall is from the lower-right corner to the upper-left corner. This loss continues to decrease as the inter-class variance increases, even after the intra-class variance is small enough (the descent path of point I in Figure [a)). We note that the
'natps//github..cmm wtaaa//igi--iaan/iccc--e-ee-eeee *We implement the GE2E loss with 'softmax" type, i.c., using Equation 6 in their paper. The GE2E loss use two learnalle parameters w, b for their similarity metric calculation S = cos(-) b, but we fixed 10, = -5 as initial values suggested in their paper for simulation.
5

---

embeddings of different classes are already clustered well under this scenario. Therefore, further increasing the inter-class variance does not improve the separability between the embeddings as they are already separated. It is difficult for the GE2E loss to further minimize the intra-class variance without increasing the inter-class variance based on the contours of the loss. Our simulation explains findings from several studies showing that contrastive losses, including the GE2E, do not perform well in reducing intra-class variance [30 55
In contrast, the ICC regularizer focuses on decreasing the intra-class variance and places less emphasis on the inter-class variance, thereby enhancing repeatability of the learned embeddings. The minimum for the ICC regularizer occurs when the embeddings intra-class variance is approximately equal to or the inter-class variance reaches a relatively large value compared the intra-class variance. The ICC regularizer pushes the embeddings towards lower intra-class variance and not towards larger inter-class variance once the inter-class variance exceeds the intra-class variance (ensuring the embeddings are clustered well) as shown in Figure (b). This naturally leads to embeddings with better repeatability compared to the GE2E loss.
A simple analysis of gradients explains this observation. The gradient of the ICC regularizer with respect the two variance terms
ORice 0MSB
M x MSW
(MSB + (M 1) x )S
0Rice
M x MSB
(8)
aMSw (MSB + (M - 1) x MSw)²
When the intra-class variance (MSw) is already small, the derivative of the ICC regularizer with respect to the inter-class variance (MSn) is small in absolute value; therefore, the gradient descent step along the inter-class variance dimension is small. When the inter-class variance is relatively large, the derivative of the ICC regularizer with respect to the intra-class variance is larger, which means the gradient descent step along the intra-class variance dimension is relatively large. This is clear from Figure (b) where the points that begin with a small intra-class variance (e.g. point 2 in Figure (b)) do not focus on further increasing the inter-class variance to improve the ICC as it is unnecessary. In contrast, the trajectory of point in (a) goes towards increasing the inter-class variance, even when the intra-class variance is small and the clusters are already well separated.
The ICC regularizer is also better aligned with classification error rate. We use the same simulation data and train a SVM for each intra- and inter-class variance pair to generate the contour figure for SVM classification error rate. As we show in Figure [ (c), the classification error rate has similar contour when compared with the ICC regularizer values: the lower ICC regularizer (higher the repeatability), the lower the classification error rate. This result supports our hypothesis that embeddings with improved repeatability will benefit downstream applications.
In summary, given the constraints on the total variance of the embeddings, it is beneficial to focus on the intra-class variance for increasing representation repeatability. Although the optimization objectives of contrastive loss and ICC regularizer are similar, the contrastive loss focuses optimiz- ing intra-class and inter-class variance simultaneously whereas the ICC regularizer places greater emphasis on minimizing intra-class variance.
The ICC is not a replacement for contrastive loss: The ICC is computed for each dimension of the embedding independently. while contrastive loss is calculated for the entire embedding vector. It's important to note that high separability in each embedding dimension doesn' necessarily translate into good separability in the overall embedding space and can lead to learned embedding dimensions that are highly correlated; this can have a negative impact on downstream model performance [51]. Relatedly, the cosine similarity score is commonly used to evaluated the fidelity of learned embeddings. A high ICC value per dimension does not necessarily imply a high cosine similarity. For these reasons, we propose to use the ICC regularizer rather than a stand-alone loss.
The ICC regularizer requires hyperparameter fine-tuning: As with other regularizers, linear combination of contrastive loss and the ICC regularizer, represented by the equation Leamtr + XRICe. requires selection of the hyperparameter X. our experiments, we used grid search to determine the optimal X. For visualizations of the loss contour (similar to those in Figure I of the combined

---

contrastive+ICC loss, we refer the reader to the Appendix B where we have included the combined loss contour with varying À values.
4 Experimental Results
In this section, we regularize deep-lcarning models with the ICC to improve the embeddings' repeatability in three different speech tasks: (1) speaker embeddings for text-independent speaker verification, where the ICC regularizer ensures the embeddings are repeatable for the same person and the contrastive loss aims to separate embeddings of different speakers; (2) speaker embeddings for zero-shot voice style conversion, where using embeddings with better quality results in higher-quality conversion; (3) voice feature embeddings for dysphonic voice detection, where the ICC regularizer ensures the embeddings do not change from day to day for the healthy group and a contrastive loss forces maximum separability between dysphonic and healthy speech.
4.1 Task 1: text-independent speaker verification
rext-independent speaker verification (TI-SV) systems verify the speaker's identification using speech signal input without any constraints on the speech content. Many previous methods use speaker embeddings for the TI-SV task [54 24 48 26]. For TI-SV, the speaker embeddings should only capture the difference in the speaker identities and be invariant to all other confounding factors, including utterance content and length. Herein we include our proposed ICC regularizer when learning speaker embeddings to improve their repeatability.
Experiment setup: We select three well-known contrastive losses as baselines: (1) GE2E 54
(2) Angular Prototypical (AngleProto) [12]. and (3) SupCon (25). Then we use these contrastive losses with and without the ICC regularizer 34 which is described in Chung et al. paper [12], and compare the performance and quality of these speaker embeddings. We use the VoxCeleb & 2 development dataset for training, and VoxCeleb 1 testing dataset for TI-SV performance evaluation [37 [I]. For training VGG-M-40, each batch contains N = speakers and M = 30 utterances per speaker, and the loss formula L(e) 1.0 X (contr-(e) + 0.06 x Ricc(e), where the e is the embeddings of speakers' in one batch. For training FastResNet-34,, each batch contains N = 100 speakers and = 2 utterances per speaker, and the loss formula L(e) 1.0 x Leontr (e) + 0.25 x (ecc(e). During training, we use the Adam optimizer, maintaining a static learning rate 0.001 without implementing any learning rate schedule. The dropout rate set to 0.2 for all dropout layers. As for data augmentation: (1) use variation in input audio length by randomly fixing the audio duration within range of 1.5 to 3.0 seconds, and (2) we add Gaussian noise with SNR randomly selected between 15 60 dB. No other augmentation methods are used. The
train two encoders, VGG-M-40 and FastResNet-
is tuned on the development dataset. The EER for subjects in the development dataset used to determine the optimized hyperparamter. We use the EER and minDCF to evaluate the performance of TI-SV, ICC (Equation (5) to evaluate the repeatability of embeddings.
Table 1: TI-SV task EER and ICC results for contrastive losses with and without ICC regularizer.
VGG-M-40 minDCF
FastResNet-34 minDCF
EER
ICC
EER
ICC
GE2E 54
4.39%
0.2925
0.4494
2.49%
0.2133
0.7215
GE2E + ICC
3.96%
0.2778
0.5487
2.39%
0.2012
0.7366
AngleProto 12
4.36%
0.2809
0.4399
2.28%
0.1960
0.7501
AngleProto ICC
4.02%
0,2790
0.5455
2.17%
0.1871
0.7627
SupCon 25
3.91%
0.2791
0.5693
2.30%
0.1956
0.7500
SupCon ICC
3.78%
0.2597
0.6661
2.16% %
0.1867
0.7615
Table 1 presents the results of the TI-SV evaluation for contrastive losses, both with and without the TCC regularizer. When using VGG-M-40 and FastResNet-34 models, the GE2E achieves EER of 4.39% and 2.49% on the VoxCeleb test respectively. Incorporating the
Results:
7

---

ICC regularizer improves the GE2E's performance, reducing EER to 3.96% for VGG-M-40 and 2.39% for FastResNet-34. This represents a approximately 10% enhancement in performance for the VGG-M-40 model. The repeatability of speaker embeddings also improves with the ICC regularizer as the ICC score increased from 0.4494 to 0.5487 and from 0.7215 0.7366 for VGG-M-40 and FastResNet-34, respectively. In addition, the benefits of the ICC regularizer are also observed with two other contrastive loss methods. For the AngleProto method, introducing the ICC regularization achieves a 7.8% and 4.8% improvement over baseline models without ICC regularizer. Meanwhile, the SupCon method, records a 3.3% and 6.08% improvement over SupCon without ICC regularizer, also resulting in more repeatable embeddings.
The experimental results of the TI-SV task demonstrate that the proposed ICC regularizer can improve the repeatability of embeddings learned to be sensitive to speaker identities. The speaker embeddings with higher repeatability achieve improved performance on the TI-SV task.
4.2 Task 2: zero-shot voice style conversion
One important application of speaker embeddings is voice style conversion, i..., modifying the speech of source speaker to sound like that produced by another target speaker without changing the linguistic information [40] 21 [8). In the previous section, we used the ICC regularizer together with contrastive loss during training obtain speaker embeddings with higher repeatability. The embeddings with high repeatability should benefit downstream applications. While in the previous section we demonstrated that they benefit the TI-SV task for several different contrastive losses, in this section, we use a zero-shot voice conversion model, Auto VC [40]. and evaluate it with two different speaker embeddings generated from the previous section: (1) GE2E loss trained speaker embeddings which are less repeatable, and (2) GE2E + ICC regularizer trained speaker embeddings which are more repeatable. We choose the most challenging voice style conversion task, the zero-shot voice style conversion (unseen speaker to unseen speaker), to compare the downstream performance of these two embeddings with different repeatability.
We use exactly same training procedure described in Qian et al. 40 to train the two models. For evaluating the conversion quality perspectively, we conduct AB preference test to compare the generated samples: we randomly select 10 unseen source and 10 unseen target speakers from the VCTK corpus [52], generate total of 100 source-target speaker pairs For each listening test, 15 pairs are randomly selected from the 100 pairs. The order of presentation is randomized. A total of 18 listeners participated in this AB preference test. They were instructed to select the sample that better matched the target speaker without knowing what method was used to generate the samples. We also evaluate two methods objectively by using objective scores based on the word error rate (WER) and character eтror rate (CER). We use an opensource speaker encoder"] to calculate the speaker similarity score between target speaker's 's audio and transformed output. And we then use Wav2Vec to do ASR on the transformed output, and calculate the WER and CER by using jiwer moduld"
Experiment setup:
Table 2: AB preference result for zero-shot voice style conversion.
GE2E Loss GE2E Loss + ICC Regularizer
Selection Ratio
38. 10%
61.90%
Table 3: Objective evaluation result for zero-shot voice style conversion.
Speaker Similarity Score
WER
CER 0.3817
GE2E Loss GE2E Loss + ICC
0.2231
0.5810
0.2309
0.5109
0.3324
Results: The AB preference result is shown in Table On average, 61.90% of samples generated by the model trained with the more repeatable speaker embeddings were preferred over those trained
"We provide several examples on our project page:
? hhugginnfacc.o//ppcchbrraiiaaa www.ppeebrriiggkkre-cc.aaoooooee $ 'huggingface coldoes/transsoomrr//mddd dowp:w/veeld////2eeee222 sypi.orgppooec/iiiee//....
8

---

using the original GE2E loss speaker embeddings. The objective evaluation results are shown in Table 3] The objective evaluation metries demonstrate that speaker embeddings with higher repeatability also result in better performance across all these metrics for voice style conversion. All these results demonstrate that speaker embeddings with higher repeatability also result in better performance for voice style conversion task.
4.3 Task 3: assessment of vocal quality for dysphonic voice detection
Dysphonia is a term that refers to difficulty producing clear voicing during speech production. Automatic dysphonic voice detection by deep learning has attracted academic and clinical interest. To develop reliable clinical models, it is important to use highly repeatable voice features and embeddings [5 Zhang et al. proposed voice feature embeddings sensitive to vocal quality and robust across different corpora [62] However, in their work there is no constraint on the voice feature embeddings' repeatability.
We rebuild the Zhang et al. network and implement the ICC regularizer to enhance the repeatability of embeddings. We follow the procedure described in [62 for training the voice feature embeddings. To enforce repeatability using the ICC regularizer, we use the recordings healthy subjects from the mPower corpus [5] to ensure the embeddings do not change daily for the healthy person.
Model structure: We use the same encoder and MLP classifier from the Zhang et al. paper. A two-branch structure is used for optimizing the dysphonic sensitivity and repeatability of voice feature embeddings simultaneously. All three embeddings have dimension of 256. For more information about our model structures, we refer the reader to the Appendix C.
Training loss: We use the following formula for training:
0.5 RIcc 1.0 Leontr + 1.0 x Ledass. where the Ricc is the ICC regularizer on repeat-constraied embeddings, Leontr is the contrastive loss the voice feature embeddings, and Ledass is the classification loss.
Training and evaluation datasets: We use the Saarbruecken Voice Database (SVD) [57] as the training and in-corpus validation dataset for dysphomic voice detection, and the mPower corpus [5] is used only for improving the repeatability of voice feature embeddings. The Massachusetts Eye and Ear Infirmary (MEEI) database and Hospital Príncipe de Asturias (HUPA) [ dataset are used for cross-corpus testing datasets for dysphonic voice detection task, and the ALS 44 dataset is used for repeatability evaluation. The MEEI, HUPA, and ALS are unseen during training for all methods. A summary of these datasets are provided in Appendix D.
We perform cross-validation six times to characterize the variability in performance. We fixed the random seed to 233. The SGD optimizer is used with a learning rate 0.001 and other default settings. We use one NVIDIA Titan Xp graphic card to train our models. We train the model for 20k steps, which takes approximately 16 hours under our configurations.
Training details:
We compare against four baselines: (1) Zhang et al. 62]; (2) P. Harar et al. 18] which is based on recurrent convolutional neural network model; (3) L. Verde et al. [53], which used a conventional features set with different classical machine learning classifiers; (4) M. Huckvale al. (22,, which uses the ComPare feature set from the OpenSMILE toolkit with the SVM and neural networks methods. All baseline methods are rebuilt and trained on our data using the same procedures as they outlined in the original papers.
Baseline methods:
Evaluation metrics: We use balanced accuracy to evaluate the dysphonic voice classification accuracy. We use the ICC (Equation 5) evaluate the repeatability of our trained voice feature embeddings and other baseline methods" features. For a fair comparison, we evaluate the repeatability by using ALS dataset 44] which unseen all methods during training.
Table 4: Dysphonic voice detection accuracy and features' repeatability of our and baseline methods.
Dysphonic Voice Classification/ / [mean accuracy] (95% CT)
Repeatability ICC ALS
SVD Train
SVD Validation
MEET Testing
HUPA Testing
Proposed Method
0.7353 (0.004)
0.7289 (0.009)
0.8214 (0.004)
0.6894 (0.011)
0.5708
Zhung et (2022)
0.8003 (0.018)
0.7077 (0.011)
(0.014)
(0.008)
0.4368
Harar et al. (2017)
0.7742 (0.017)
0.6914 (0.009)
0.6614 (0.024)
0.4918 (0.008)
0,4743
Verde et (2018)
0.8910 (0.006)
0.6274 (0.009)
0,7042 (0.018)
(0.015)
0.0182
Huckvale et al. (2021)
0.7290 (0.019)
0.6255 (0.012)
0.6978 (0.044)
0.3487 (0.014)
0.2914

---

Results: Implementing the ICC regularizer significantly improves the repeatability of the voice feature embeddings as shown in Table 4 Our proposed method achieves the highest ICC score of 0.5708, while the Zhang et al. model only achieves an ICC score of 0.4368, an improvement of 30.68%. Similarly, the repeatability of embeddings generated with the ICC regularizer regularizer significantly exceeds that of all baseline methods.
Our experimental results demonstrate that the voice feature embeddings with higher repeatability also achieve better classification accuracy and generalizability. Our proposed method achieves good classification accuracy on SVD in-corpus validation 0. 0.7289 (=00009), MEEI cross-corpus testing 0.8214 (+0.004), and HUPA cross-corplr testing 0.6894 (+0.011), For a comparison, the original HUPA publication achieved aceuracy of 0.6962 (=0.047) with MFCC when trained and tested in-corpus [ Our proposed method's 's classification accuracy across the three corpora is quite good, considering the differences between the corpora.
The experimental results of voice feature embeddings for dysphonic voice detection task demonstrate that the ICC regularizer can improve the repeatability of embeddings and embeddings with higher repeatability exhibit better accuracy and generalizability in dysphonic voice detection.
Conclusion
This paper ports the concept of repeatability from measurement theory representation learning. We propose to use the ICC as an evaluation metric in representation learning and use the ICC regularizer as a complementary component for contrastive loss to regularize deep-leamed embeddings to be more repeatable. We use an example and intuition to explain why the ICC regularizer has better performance on minimizing intra-class variance than contrastive loss. We evaluate the ICC regularizer on three speech tasks that use learned embeddings: speaker embeddings for TI-SV and zero-shot voice style conversion, and voice feature embeddings for a clinical application. The experimental results demonstrate that the ICC regularizer can improve the repeatability of learned embeddings, and embeddings with higher repeatability exhibit better performance in downstream tasks. There several directions for future works: (1) applying the ICC regularizer to other domains, including computer vision and natural language processing: (2) extension to self-supervised methods that use contrastive-style training; (3) more thorough theoretical analysis of the ICC and its properties.
Potential Negative Societal Impact: Methods for learning new feature representations that focus on separability between classes can amplify biases that exist in the data. This is a well-known problem and it can oceur when the data used to train the representation model biased. This especially problematic in high-stakes applications like healthcare, where biased predictions or decisions can lead to unequal treatment or access. Safe deployment of models based on the feature representations proposed herein will require thorough validation to detect potential biases and mitigation strategies for dealing with them.
Acknowledgments and Disclosure of Funding
This work funded in part by Office of Naval Research grants N00014-21-1-2615 and N00014-23- 1-2406. The authors acknowledge Research Computing at Arizona State University for providing GPU resources that have contributed to the research results reported within this paper.
References
[1] Julián David Arias-Londoño, Juan I Godinn--lorente, Maria Markaki, and Yannis Stylianou. On combining information from modulation spectra and mel-frequency cepstral coefficients for automatic detection of pathological voices. Logopedics Phoniatrics Vocology, 36(2):60-60 69,
2011.
[2] Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk mini- mization. arXiv preprint arXiv: /907002990 2019.
[3] Vassileios Balntas, Edgar Riba, Daniel Ponsa, and Krystian Mikolajezyk. Learning local feature descriptors with triplets and shallow convolutional neural networks. In 2016 The British Machine Vision Conference (BMVC), volume (2), page 3, 2016.
10

---

[4] Adrien Bardes, Jean Ponce, and Yann LeCun. Vicreg: serianeeiivvarince--vvaai regular- ization for self-supervised learning. arXiv preprint arXiv:2105.04906, 2021.
[5] Brian M Bot, Christine Suver, Elias Chaibub Neto, Michael Kellen, Arno Klein, Christopher Bare, Megan Doerr, Abhishek Pratap, John Wilbanks, E Dorsey, et al. The mpower study, parkinson disease mobile data collected using researchkit. Scientific Data, 3(1):1-9, 2016.
[6] Alejandro Caceres, Deanna L Hall, Fernando O Zelaya, Steven CR Williams, and Mitul A Mehta. Measuring fmri reliability with the intra-class correlation coefficient. Neurolmage, 45 (3):758-768, 2009.
[7] Danwei Cai, Weicheng Cai, and Ming Li. Within-sample iariability-invariant loss for robust speaker recognition under noisy environments. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6469-6473, IEEE, 2020.
[8] Tianshi Cao, Mare Law, and Sanja Fidler. A theoretical analysis of the number of shots in few-shot learning. arXiv preprini arXiv:/909.1 /etp: 1722,, 2019.
[9] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, pages 1597-1607. PMLR, 2020.
[10] Wei- Yu Chen, Yen-Cheng Liu, Zsolt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. A closer look at few-shot classification. arXiv preprint arXiv:1904.04232, 2019.
[11] Joon Son Chung, Arsha Nagrani, and Andrew Zisserman. Voxceleb2: Deep speaker recognition. arXiv preprint 1rXiv:180.005622 2018.
[12] Joon Son Chung, Jaesung Huh, Seongkyu Mun, Minjae Lee, Hee Soo Heo, Soyeon Choe, Chiheon Ham, Sunghwan Jung, Bong-Jin Lee, and lcksang Han. defence of metric learning for speaker recognition. arXiv preprini arXiv:2003. 1982, 2020.
[13] Jiarui Ding and Aviv Regev. Deep generative model embedding of single-cell ma-seq profiles on hyperspheres and hyperbolic spaces. Nature Communications, 12(1))1-17, 2021.
[14] SB Eickhoff, M Goni. Juergen Dukart, et al. Exploring test-retest reliability and longitudinal stability of digital biomarkers for parkinson disease in the m- power data set: Cohort study, Journal of Medical Internet Research. 23(9):e26608-e220
2021.
[15] Brian S Everitt and David C Howell. Encyclopedia of Statistics in Behavioral Science-Volume
2. John Wiley & Sons, Ltd, 2021.
[16] Andy P Field. Intraclass correlation. Encyclopedia of Statisties in Behavioral Science, 2005.
[17] Zhifu Gao, Yan Song, Ian McLoughlin, Pengcheng Li, Yiheng Jiang, and Li-Rong Dai. Im- proving aggregation and loss function for better embedding learning in end-to-end speaker verification system. In Proc. Interspeech 2019, pages 361-365, 2019.
[18] Pavol Harar, Jesus B Alonso-Hernandezyy Jiri Mekyska, Zoltan Galaz, Radim Burget, and Zdenek Smekal. Voice pathology detection using deep learning: preliminary study. In 2017 International Conference and Workshop on Bioinspired Intelligence (IWOBI), pages 1-4. IEEE, 2017.
[19] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9729-9738, 2020.
[20] Georg Heigold, Ignacio Moreno, Samy Bengio, and Noam Shazeer. End-to-end text-dependent speaker verification. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5115-5119. IEEE, 2016.
[21] Tzu-hsien Huang, Jheng-hao Lin, and Hung-yi Lee. How far are we from robust voice conver- sion: A survey. 202/ IEEE Spoken Language Technology Workshop (SLT), pages 514-521. IEEE, 2021.
11

---

[22] Mark Huckvale and Catinca Buciuleac. Automated detection of voice disorder in the saarbrücken voice database: Effects of pathology subset and audio materials. Proc. Interspeech 2021, pages 1399-1403, 2021.
[23] JCGM. Evaluation of measurement data- guide to the expression of uncertainty in measurement. JCGM, 100(2008):1-111 2008
[24] Jee-weon Jung, Hee-Soo Heo. Ju-ho Kim, Hye-jin Shim, and Ha-Jin Yu. Rawnet: Advanced end-to-end deep neural network using raw waveforms for text-independent speaker verification. arXiv preprint arXiv: 1194.9944404444 2019.
[25] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in Neural Information Processing Systems, 13:18661-18673, 2020.
[26] Seong- Hu Kim, Hyeonuk Nam, and Kong-Hwa Park. Temporal dynamic convolutional neural network for text-independent speaker verification and phonemic analysis. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6742-6746. IEFE, 2022.
[27] Terry K Koo and Mae Y Li. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2):155-16) 2016.
[28] Helena Chmura Kraemer. The reliability of clinical diagnoses: state of the art. Annual Review of Clinical Psychology, 10:111-130, 2014.
[29] Yoohwan Kwon, Soo- Soo-Whan Chung, and Hong-Goo Kang. Intra-class variation reduction of speaker representation in disentanglement framework. arXiv preprint 01X10 2020.
[30] Nam Le and Jean-Mare Odobez. Robust and discriminative speaker embedding via intra-class distance variance regularization. Proc. Interspeech 2018, pages 2257-2261, 2018
[31] Chao Li, Xiaokong Ma, Bing Jiang, Xiangang Li, Xuewei Zhang, Xiao Liu, Ying Cao. Ajay Kannan, and Zhenyao Zhu. Deep speaker: an end-to-end neural speaker embedding system. arXiv preprint arXiv: / 1705.02304 2017.
[32] Zhe Li and Man Wai Mak. Speaker representation learning via contrastive loss with maximal speaker separability. In 2022 Asia-Pacifte Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). pages 962-967. IEEE, 2022.
[33] David Liljequist, Britt Blfving, and Kirsti Skavberg Roaldsen. Intraclass correlation-a discussion and demonstration of basic features. PLOS ONE, 14(7):a221984, 2019.
[34] Qiang Meng, Chixiang Zhang. Xiaoqiang Xu, and Feng Zhou. Learning compatible embeddings. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9939- 9948, 2021.
[35] Olga Moskvyak, Frederic Maire, Feras Dayoub, Asia O Armstrong, and Mahsa Baktashmot- lagh. Robust re-identification of manta rays from natural markings by learning pose invariant embeddings. In 2021 Digital Image Computing: Techniques and Applications (DICTA), pages 1-8. IEEE, 2021.
[36] Reinhold Müller and Petra Büttner. A critical discussion of intraclass correlation coefficients. Statistics in Medicine, 13(23-24) 1994.
[37] Arsha Nagrani, Joon Son Chung, and Andrew Zisserman. Voxceleb: a large-scale speaker identification dataset. arXiv preprint arXiv: 08612,, 2017.
[38] Raghuveer Peri, Monisankha Pal, Arindam Jati, Krishna Somandepalli, and Shrikanth Narayanan. Robust speaker recognition using unsupervised adversarial invariance. In ICASSP 2020-2020 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pages 6614-6618. IEEE, 2020.
[39] Rafat Pilarezyk and Wladyslaw Skarbek. On intra-class variance for deep learning of classifiers. Foundations of Computing and Decision Sciences, 44(3):285-301, 2019.
12

---

[40] Kaizhi Qian, Yang Zhang, Shiyu Chang, Xuesong Yang, and Mark Hasegawa-Johnson. Autove: Zero-shot voice style transfer with only autoencoder loss. In International Conference on Machine Learning, pages 5210-5219. PMIR, 2019.
[41] Xiaoyi Qin, Na Li, Chao Weng, Dan Su, and Ming Li. Cross-age speaker verification: Learning age-invariant speaker embeddings. arXiv preprint urXiv:2207.05929 2022.
[42] Mamshad Nayeem Rizve, Salman Khan, Fahad Shahbaz Khan, and Mubarak Shah. Exploring complementary strengths of invariant and equivariant representations for few-shot learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 10836- 1064,, 2021.
[43] Patrick E Shrout and Joseph L Fleiss. Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin, 86(2):420, 1979.
[44] Gabriela M Stegmann, Shira Hahn, Julie Liss, Jeremy Shefner, Seward Rutkove, Kerisa Shelton, Cayla Jessica Duncan, and Visar Berisha. Early detection and tracking of bulbar changes in als via frequent and remote speech analysis. NPJ Digital Medicine, 3(1):1-5, 2020.
[45] Gabriela M Stegmann, Shira Hahn, Julie Liss, Jeremy Shefner, Seward B Rutkove, Kan Kawa- bata, Samarth Bhandari, Kerisa Shelton, Cayla Jessica Duncan, and Visar Berisha. Repeatability of commonly used speech and language features for clinical applications. Digital Biomarkers, 122, 2020.
[46] Bo Sun, Banghuai Li, Shengcai Cai, Ye Yuan, and Chi Zhang. Fsce: Few-shot object detection via contrastive proposal encoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2021.
[47] Jennifer J Sun, Jiaping Zhao, Liang-Chieh Chen, Florian Schroff, Hartwig Adam, and Ting Liu. View-invariant probabilistic embedding for human pose. In European Conference on Computer Vision, pages 53-70. Springer, 2020.
[48] Yun Tang, Guohong Ding, Jing Huang, Xiaodong He, and Bowen Zhou. Deep speaker embedding learning with multi-level pooling for text-independent speaker verification. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1416-6120 IEEE, 2019.
[49] Ye Tian and Andrew Zalesky. Machine learning prediction of cognition from functional connectivity: Are feature weights reliable? Neurolmage, 245:1 18648, 2021.
[50] Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. What makes for good views for contrastive learning? Advances in Neural Information Processing Systems, 33:6827-688 6839, 2020.
[51] Laura Toloşi and Thomas Lengauer. Classification with correlated features: unreliability of feature ranking and solutions. Bioinformatics, 17144)1186-1199 2011.
[52] Christophe Veaux, Junichi Yamagishi. Kirsten MacDonald, et al. Cstr vetk corpus: English multi-speaker corpus for estr voice cloning toolkit. University of Edinburgh. The Centre for Speech Technology Research (CSTR), 2017.
[53] Laura Verde, Giuseppe De Pietro, and Giovanna Sannino. Voice disorder identification by using machine learning techniques. IEEE Access, 6:16246-16255, 2018.
[54] Li Wan, Quan Wang, Alan Papir, and Ignacio Lopez Moreno. Generalized end-to-end loss for speaker verification. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8579.4883. IEEE, 2018
[55] Yuheng Wei, Junzhao Du, and Hui Liu. Angular margin centroid loss for text-independent speaker recognition. In Proc. Interspeech 2020, pages 3820-3824, 2020.
[56] Matthew Wolak, Daphne Fairbairn, and Yale Paulsen. Guidelines for estimating repeata- bility. Methods in Ecology and Evolution, 3(1):129-137, 2012.
13

---

[57] Bogdan Woldert-Jokisz. Saarbruecken voice database, 2007. URL pattt::/stimmdi coli uni saarland and..and... de/
[58] Ruitong Xiao, Haitong Zhang, and Yue Lin. Dge-vector: new speaker embedding for zero- shot voice conversion. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6547-6551. IEEE, 2022.
[59] Hao-Chun Yang and Chi-Chun Lee. An attribute-invarian variational learning for emotion recognition using physiology. In ICASSP 2019-2019 IEEE International Conference Acous- lics, Speech and Signal Processing (ICASSP), pages 118--1188. IEEE, 2019.
[60] Mang Ye, Jianbing Shen, Xu Zhang, Pong C Yuen, and Shih-Fu Chang, Augmentation invariant and instance spreading feature for softmax embedding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
[61] Zhengxu Yu, Yilun Zhao, Bin Hong, Zhongming Jin, Jianqiang Huang, Deng Cai, and Xian- Sheng Hua. Apparel-invariant feature learning for person re-identification. IEEE Transactions on Multimedia, 24:4482-4492, 2021.
[62] Jianwei Zhang, Julie Liss. Suren Jayasuriya, and Visar Berisha. Robust vocal quality feature embeddings for dysphonic voice detection. JEEE/ACM Transactions on Audio, Speech, and Language Processing, 31: 11:1348- 1359, 2023.
[63] Quanhua Zhao, Shuhan Jia, and Yu Li. Hyperspectral remote sensing image classification based on tighter random projection with minimal intra-class variance algorithm. Pattern Recognition, 111:107635, 2021.
[64] Liang Zheng, Yujia Huang, Huchuan Lu, and Yi Yang. Pose-invariant embedding for deep person re-identification. IEEE Transactions on Image Processing, 28(9):4500-4509 2019.
14

---

Appendix
A ICC Formulation for Imbalanced Classes
The proposed ICC and ICC regularizer in the main paper require equal class size, i.c., M is the same for all classes. However, this not typically the case in actual usage. In this section, propose an extended version of the ICC and ICC regularizer, which are capable of handling datasets with unbalanced class sizes.
Assume there are N classes in the dataset or batch, and k2 is the number samples for j-th class, the Xji represents the i-th samples of the j-th class, eja represents the corresponding embedding vector, and ey represents the I-th embedding dimension of the Cji- The between-class variance can be written as:
MS Ey=iky
ky (c) - e
where ej represents the mean of I-th embedding dimension for the j-th class:
J
2i=1eh
(10)
kj
and el represents the overall mean of 1-th embedding dimension,
(11)
kj
Then ICC(e'), the ICC score for 1-th embedding dimension, can be calculated as follows:
ICC(e)) MSp(e - Ee 3. by
(12)
MSn(c)) + N プソ= PM
where oJ., = (e) 27 represents the within-class variance of I-th embedding dimension for i-th sample of the j-th class. The Equation] 12 computes the intra-class variance for each class individually. taking into account the number of samples present in each class, denoted as ky. Thus, Equation 12 effectively addresses issues related to class size imbalance.
Then the ICC score and ICC regularizer still be calculated by using
TCC(e) Σt ICC(e))
(13)
L
Rice 1 - ICC(e),
(14)
respectively, for the datasets with unbalanced class sizes.
B Hyperparamter Ablation for GE2E Loss with ICC Regularization
As described in the Section 3.2 of the main paper, the ICC regularizer cannot be used alone and requires hyperparameter fine-tuning when combined with the contrastive loss. Therefore, based on the simulation data in Section 3.2 in the paper, the simulation contour figures of the (1 Lare2 ARIce function value of intra- and inter-class variance for different values hyperparameter are provided in Figure B.I
15

---

Werdee Valsoo
0.35
0.0
04
0:
02
62
0.0
0.5
000
Inn dles Varlance
Iern dase Variance
Inta doss Varfance
BuviGEa Low oud Begolutbvr
81- Low B.siCC Segdacter
BENGGEE Com tude 30C Begolsbee
Inwe cwn Werce
0.3
05
0.15
8:15
0.15
8.1
Begakuike
2.55
86
0.
0.5
Verooes Valson
044
08
2:25
2.
63
61
6.1
02
04
34
08
1.8
1.2
1.
32
06
0.8
1.4
1.0
0.4
0.4
088
5.
18
cbe- Varaacc
dase
180
Vartance
Figure B.I: The simulation contour figures of the 11-A) ARice function value of intra- and inter-class variance, where À (0.1, 0.2, 0.3, 0.4, 0.,,, 0.9).
Referring to Figure B.1 it's observed that the impact of the hyperparameter À on the shape of the (1 -) + XRee function's 's contour lines is linear under the given simulation conditions. As the value of this hyperparameter increases, the contour of (1 A)LGEZE XRIce leans more towards the ICC regularizer. This means it increasingly emphasizes on reducing the intra-class variance.
C Model and Training Details for Task 3: Dysphonic Voice Detection
We rebuild the Zhang al. network 622 and implement the ICC regularizer enhance the repeatabil- ity of voice feature embeddings. We follow the procedure described in their work [62 for training the voice feature embeddings. To enforce repeatability using the ICC regularizer, use the recordings of healthy subjects from the mPower corpus to ensure the embeddings do not change daily for the healthy person.
Model Structure:
We use the same encoder and MLP classifier from the Zhang et al. paper 62 A
two-branch structure is used for optimizing the dysphonic sensitivity and repeatability of voice feature embeddings simultaneously as shown Figure where the MLP networks is composed by three linear layers and two Leaky-ReL.U activation layers (negative slope is 0.4). All three embeddings have a dimension of 256.
Training Loss: We use the following loss:
16

---

# I
Healthy NxM
Data warping
Spectrogram
Dataset
Encoder
I+
Dysphonic NxM
Top Embedding
M1 I Sub
MLP Networks
MLP Networks
x MM
MLP Classifier
mPower Dataset Healthy Subjects
Repeat Embedding [mPower Healthy]
Voice Feature Embedding
Predicted Label (Healthy / Dysphonic)
[SVD Healthyj & [SVD Dysphonic]
11 1 Sub N
x M
Input Sampling
ICC Regulurizer
Contrastive Loss
Classification
Figure C.2: The diagram training repeatability enhanced voice feature embeddings for dysphonic voice detection.
L = 0.5 x Rice + 1.0 x Leontr + 1.0 x Lelass>
(15)
where the Ricc the ICC regularizer on repeat-constraieed embeddings, Leontr the contrastive loss on the voice feature embeddings, and Lelass is the classification loss.
Training and Evaluation Datasets: We use the Saarbruecken Voice Database (SVD) as the training and in-corpus validation dataset for dysphonic voice detection task, and the mPower corpus (5) used only in training for improving the repeatability voice feature embeddings. The Massachusetts Eye and Ear Infirmary (MEEI) database and Hospital Príncipe de Asturias (HUPA) [I] dataset are used for cross-corpus testing datasets for dysphonic voice detection task, and the ALS 44 dataset is used for repeatability evaluation. The MEEI, HUPA, and ALS are unseen during training for all methods.
We perform cross-validation six times to characterize the variability in performance. We fixed the random seed 233. For each training batch, randomly select 16 dysphonic and 16 healthy voice recordings of the same gender from the SVD dataset; 8 healthy subjects from the mPower dataset, and 2 consecutive days' voice recordings for each subject. The SGD optimizer is used with learning rate of 0.001 and other default settings. We use one NVIDIA Titan Xp graphic card to train our models. train the model for 20k steps, which takes approximately 16 hours under our configurations.
Training Details:
Baseline Methods:
We comapre against four baselines: (1) Zhang al. 622 ; (2) Harar al. (18).
which is based on recurrent convolutional neural network model: (3) Verde et al. [53] which used conventional features set with different classical machine learning classifiers; (4) Huckvale et al. (22) which uses the ComPare feature set from the OpenSMILE toolkit with the SVM and neural networks methods. All baseline methods are rebuilt and trained on our data using the same procedures as they outlined.
Evaluation Metrics: We use balanced accuracy evaluate the dysphonic voice classification accuracy:
TP
TN
Balanced accuracy = x
+
(16)
2
TP+ TN FP
We use the ICC to evaluate the repeatability of our trained voice feature embeddings and other baseline methods' features. For fair comparison, we evaluate the repeatability by using ALS dataset, which is unseen to all methods during training.
17

---

C.1 Why use a two-branch structure?
use a two-branch structure for optimizing dysphonic sensitivity and repeatability em- beddings simultamcously: the top embeddings generated by the encoder further transformed to repeat-constrained embeddings and voice feature embeddings instead of directly using top embed- dings for repeatability constraints and dysphonic voice detection.
We have tested three models with different structures in this experiment, including: (1) one-embedding output, i.e., Zhang et al. networks structure without modification and there is only one embeddings output can be optimized as shown in Figure to transform the top embeddings to the voice feature embeddings and there are two embeddings outputs can optimized as shown in Figure C.4 (3) two-branch structure (three-embedding output).
(2) two-embeddingg output, i.e., add one MLP network
- O
Healthy NxM
Data warping
Spectrogram
SVD Dataset
Encoder
| IIII-
Dysphonic NxMM
Voice Feature Embedding
(SVD Healthy] & [SVD Dysphonic]
[mPower Healthy]
1-1- Sub
MLP Classifier
mPower Dataset Healthy Subjects
Predicted Label (Healthy / Dysphonic)
11- N
M
Input Sampling:
ICC Regularizer
Contrastive Loss
Classification Loss
Figure C.3: The diagram of one-embedding output model structure. This is not used for any results in this paper.
Healthy NxM
Data warping
Spectrogram
Dataset
Encoder
1+
Dysphonic NxM
Top Embedding
[mPower Healthy]
-I|IM Sub
MLP Networks
x M
MLP Classifier
mPower Dataset Healthy Subjects
Voice Feature
Embedding [SVD Healthy] & [SVD Dysphonic]
Predicted Label (Healthy / Dysphonic)
1 Sub N
x XM
Input Sampling:
ICC Regularizer
Contrastive Loss
Classification Loss
Figure C.4: The diagram of two-embedding output model structure. This is not used for any results in this paper.
Convergence problems with the architecture from Figure C3
During experiments, we noticed
that if the model has only one output, there is conflict between the ICC regularizer for repeatability
18

---

and contrastive loss for dysphonic voice detection, i.e., the model has difficulties converging. We believe this problem is because the optimization objectives of the two components are so different. The ICC regularizer focuses on the repeatability within the subject; however, the contrastive loss focuses on the dysphonic sensitivity between the healthy and dysphonia groups. The inconsistent range of these two objectives requires careful fine-tuning of the ratio between the ICC regularizer and contrastive loss. We did not find a good loss ratio, so we abandon this structure.
Convergence problems with the architecture from Figure C.4: After the one-embedding output structure failed, we considered letting the ICC regularizer and contrastive loss optimize two different embeddings, and then we tested the two-embedding output structure (Figure C.4). The encoder's s output is the top embeddings. and the MLP network is used to learn dysphonic voice feature embeddings. Then the ICC regularizer regularizes the repeatability of top embeddings, and the contrastive loss regularizes the dysphonic sensitivity of voice feature embeddings. The intuition that since the dysphonic voice feature embeddings are function of the top embeddings, the voice feature embeddings will also have the property of high repeatability. However, this structure failed to converge. The two-embedding output structure did not solve the conflict between the two loss's s optimization objectives.
Two-branch structure solves the convergence problems: After the one- and two-embedding output structures failed, we designed the two-branch (three-ebeedding output) structure, solving the convergence problem. The top embeddings generated by the encoder are further converted to repeatable embeddings and dysphomic voice feature embeddings using two MLP networks. The ICC regularizer regularizes the repeatability of repeatable embeddings, and the contrastive loss regularizes the dysphonic sensitivity of voice feature embeddings. Due to the repeatable and dysphonic voice feature embeddings being independently transformed from the top embeddings, the top embeddings are endowed with both properties. These are the embeddings we used in the paper.
D Summary of Used Databases in Paper
VoxCeleb [37]: A large-scale speaker recognition dataset consisting of short video clips from YouTube. It includes over 100,000 utterances from more than 1,200 celebrities across various professions and demographies.
An extension of VoxCeleb 1, VoxCeleb 2 is an even larger dataset featuring approximately million utterances from over 6,000 speakers. Together, VoxCeleb and VoxCeleb 2 offer rich resources for training and evaluating speaker recognition models.
VoxCeleb I1:
VCTK (The Voice Cloning Toolkit) [52]: VCTK is a speech dataset that includes recordings of various English accents. With over 44 hours of speech from 109 speakers, each speaking in their accent, VCTK provides valuable resource for multi-accent speech synthesis and recognition research.
MEEI (Massachusetts Eye and Ear Infirmary): Full name is Kay Elemetrics Corp., Disordered Voice Database, Version 1.03 (CD-ROM), MEEI, Voice and Speech Lab, Boston, MA (October 1994). The MEEI Voice Disorders Database is a collection of speech samples from individuals with and without voice disorders. Participants research to study voice pathology and develop systems to detect and analyze voice disorders. The MEEI database contains more than 1400 recordings of sustained phonations. which are collected from 53 healthy speakers and 657 speakers diagnosed with different types of dysphonia.
English speakers. It is often used in medical and clinical
The Saarbrücken Voice Database is collection of voice recordings used for various phonetic and clinical studies. Participants are German speakers. provides comprehensive set of voice samples, including those from individuals with different voice disorders, aiding in the research of voice quality and characteristics. SVD database contains the voice recordings from more than 2000 speakers (428 healthy females, 259 healthy males, 727 dysphonic females, 629 dysphonic males).
SVD (Saarbrücken Voice Database) [57:
HUPA (Hospital Príncipe de Asturias) [I] Similar to MEEI and SVD, HUPA collection of speech samples from individuals with and without voice disorders. Participants are Spanish speakers. HUPA contains /al sustained phonation recordings of 366 adult Spanish speakers (169 dysphonic and 197 healthy).
19


Reviews:
[{"review_id": "oaESNcx7JF", "reviewer": "Reviewer_Qvpp", "summary": "This paper proposes to use a traditional intra-class correlation coefficient (ICC) to assess the repeatability of speech embeddings learnt by neural network. The proposed ICC regularizer has characteristics similar to well-known contrastive loss which aims to minimize the intra-class variance and maximize the inter-class variance of training data. The authors propose to use the ICC regularizer not in isolation but in combination with the contrastive loss. This paper performs several types of experiments based on speech processing tasks such as speaker verification, voice style conversion, and dysphonic voice detection to demonstrates the effectiveness of the proposed regularizer. These tasks focus on the importance of detecting the repeatability for the target data.", "questions": "- It looks like that the proposed ICC regularizer corresponds to the special case of contrastive loss focused only on positive sample pairs. Although there are many variants in contrastive loss, the basic concept is to increase a capability of category classification of data by leveraging both positive and negative pairs. I am wondering if the training only with conventional contrastive loss could provide some improvements similar to the proposed method, for example, by simply changing the ratio of positive sample pairs (reducing negative sample pairs) in the last half of the training stage. What are advantages that cannot be obtained by just changing the ratio of positive and negative pair data?\n- It is difficult to understand the effectiveness of the proposed method correctly because experimental setups including data are very briefly written. For example, in the experiment described in Section 4.1, what are the training strategy of the model such as an optimizer, a learning rate scheduling, other regularization techniques (specAugment, DropConnect, etc.), data augmentation, and so on?\n- In addition, a summary of data used for each experiment (a few more details) would be very much helpful.", "limitations": "The same comments as one of weaknesses. The proposed ICC regularizer shouldn't be used in isolation but be used in combination with a conventional contrastive loss as clearly mentioned in the paper.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The considered scenario is interesting and important to the community. The proposed method is simple yet effective in assessing the repeatability in the target data.\n- This paper assesses the effectiveness of the proposed method not only with a single speech task but also with several tasks based on voice quality evaluation. The improvements look sufficient.", "weaknesses": "- ICC is very traditional metric, which is widely used for various data classification algorithms.\n- The proposed ICC regularizer shouldn't be used in isolation.\n- Lack of some experiments and details of experimental setup (see questions below).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer Qvpp for the comments.\n\n**Response to \"ICC is a traditional metric and widely used for various classification tasks\":** The ICC is primarily a statistical measure that describes how strongly units in the same group resemble each other. It's typically used in various research fields like psychology, sociology, or medicine, where repeated measurements are taken on the same subjects to assess the reliability of ratings or measurements. The ICC is not a commonly used metric in machine learning classification tasks which typically rely on accuracy, F1, AUC-ROC, etc. Sometimes the ICC is used in machine learning classification tasks related to clinical applications (Ugga et al. 2021) for assessing the reliability of models but we are not aware of other classification papers which utilize ICC.\n\nBased on existing literature, employing the ICC as a measurement tool for assessing the quality of learned embeddings represents a novel contribution. Utilizing the ICC regularizer to ensure the model learns consistent and repeatable latent representations further distinguishes our approach as innovative. Within representation learning, our introduction of the ICC metric and ICC regularizer constitutes a significant contribution.\n\n(Ugga et al. 2021) - Ugga et al. \"Meningioma MRI radiomics and machine learning: Systematic review, quality score assessment, and meta-analysis.\" _Neuroradiology_ 63 (2021): 1293-1304.\n\n**Response to \"ICC regularizer shouldn't be used in isolation\":** As outlined in Lines 224-225 of the manuscript, we advocate for using the ICC as a regularizer rather than as a stand-alone loss function. The ICC regularizer is designed to act as a complementary component in learning repeatable latent representations. It's important to recognize that most regularizers do not function as stand-alone loss functions; instead, the ICC regularizer is intended to work synergistically with other components of the loss function to enhance the reliability and coherence of the learned embeddings. As additional evidence of the utility of the regularizer, we have conducted the following additional experiments:\n\n(1) Run speaker verification (SV) testing on audio samples with noise added. We evaluate three SNR levels, 30, 35 and 40 dB. Adding the ICC regularizers improves the original loss for all 3 SNR levels evaluated. (Details please see response to Reviewer wtbk.)\n\n|Method|SNR=40dB|SNR=35dB|SNR=30dB|\n|--|--|--|--|\n|GE2E|5.64%|7.00%|9.04%|\n|GE2E+ICC|5.34%|6.76%|8.66%|\n|AngleProto|4.77%|5.76%|7.66%|\n|AngleProto+ICC|4.52%|5.59%|7.34%|\n|SupCon|4.35%|5.21%|6.25%|\n|SupCon+ICC|4.21%|5.09%|6.10%|\n\n(2) Evaluated the two methods in Section 4.2, voice conversion (VC), objectively on three different metrics: speaker similarity score, word error rate (WER), and character error rate (CER). (Details please see response to Reviewer 89AX.)\n\n(VC model used here is AutoVC)\n||Speaker Similarity Score|WER|CER|\n|--|--|--|--|\n|GE2E Loss|0.2231|0.5810|0.3817|\n|GE2E Loss + ICC Regularizer|0.2309|0.5109|0.3324|\n\nThe higher speaker similarity score, the better performance of model. The lower WER & CER, the better performance of model.\n\n(3) Re-run VC task with a SOTA VC model named FreeVC with our speaker encoder (manuscript Line 258-287) to conduct the experiment outlined in Section 4.2. (For details please see response to Reviewer 89AX.)\n\n(VC model used here is FreeVC w/o SR.)\n||Speaker Similarity Score|WER|CER|\n|--|--|--|--|\n|GE2E Loss|0.2753|0.2556|0.0755|\n|GE2E Loss + ICC Regularizer|0.2899|0.2163|0.0718|\n\nWhile this model is still training, the results from an intermedia evaluation of the model tell a consistent story: when the regularizer is added to existing cost functions, performance improves on several metrics.\n\n**Response to \"ICC regularizer corresponds to the special case of contrastive loss focused only on positive sample pairs\":** The ICC regularizer cannot be posed as a special case of contrastive loss focused only on positive sample pairs as it requires both positive and negative sample pairs. The ICC regularizer and contrastive loss have similarities in their optimization criteria: both aim to minimize the intra-class variance by using positive sample pairs and maximize the inter-class variance using negative sample pairs. However, compared to contrastive loss, the ICC regularizer results in a better trade-off between minimizing intra-class variance and maximizing inter-class variance when inter-class variance is relatively larger than intra-class variance. This was shown in Fig. 1 in the manuscript.\n\n**Response to \"changing the ratio of positive sample pairs (reducing negative sample pairs)\":** Reducing negative sample pairs could help the loss focus on minimizing the intra-class variance; however this comes at the cost of the inter-class variance decreasing due to the imbalanced training strategy. We use our GE2E loss simulation to demonstrate this: all configurations and parameters are kept the same as those from the simulation in Fig. 1 in the manuscript, except we add a scalar to the negative pairs score in the GE2E loss to simulate the impact of reducing negative sample pairs during training. We set the scalar to 0.25, i.e., the ratio of negative sample pairs to positive sample pairs is 1:4. The results of our experiment are shown in Figure A (available in the global response PDF). The figure demonstrates that the trajectory of the optimal solution is in the direction of decreased inter-class variance. This is clearly in an undesired direction as a good regularizer promotes lower intra-class variance, but higher inter-class variance. Comparing this solution to the one from the ICC in Fig. 1 in the manuscript, we clearly see that the ICC does both; however simply changing the ratio of samples between the two classes only reduces the intra-class variance.\n\n**Response to \"experiments and details of experimental setup\":** We will add these in the revision. Please see the **Global Response**."}, {"review_id": "jYFf7rIWOc", "reviewer": "Reviewer_wtbk", "summary": "In this paper, the concept of repeatability from measurement theory was introduced for representation learning. Intraclass Correlation (ICC) was proposed as an evaluation metric and the ICC regularizer was used as an additional element to contrastive loss for training. This aims to enhance the repeatability of deep-learned embeddings.\nSome synthetic examples and intuitive reasoning were used to illustrate why the ICC regularizer is superior in minimizing intra-class variance compared to contrastive loss. The ICC regularizer was assessed across three speech tasks: speaker embeddings for Text-Independent Speaker Verification (TI-SV) and zero-shot voice style conversion, along with voice feature embeddings for a clinical application. The experiments indicate that the ICC regularizer can boost the repeatability of learned embeddings and perform better in downstream tasks.", "questions": "See the comments above in the weakness section.", "limitations": "No discussion on negative societal impact was included in the paper", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper proposed a novel metric of ICC based on measurement repeatability. Although the term may be a little confusing, the motivation \nand intuition were well explained. The experimental results on three speaker/speech tasks are convincing.", "weaknesses": "To show the repeatability of speaker embedding for TI-SV task, more controlled experiments may be necessary to prove the effectiveness of ICC loss in addition to the ICC scores: for example, adding noise to the testing sample, or switching languages of the same speaker as it was tested in the NIST Speaker Recognition Evaluation. \n\nFor results on dysphonic voice detection in Table 3, other baseline models were definitely over-trained compared to the proposed model.\nWhy do you train your own models instead of quoting the numbers from the original papers? (see line 334-335). The results in Table 3 may be unfair comparisons.\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer wtbk for the comments.\n\n**Response to \"more controlled experiments may be needed\":** We follow the reviewer's suggestion and run more testing on audio samples with noise added.    We evaluate three SNR levels, 30, 35 and 40 dB. For each input audio, we randomly generated Gaussian noise and scale the noise to the desired energy level. We add the audio and the noise to get the noisy audio at the desired SNR level. The EER results are provided in the following table. The results show that adding the ICC regularizers improves the original loss for all 3 SNR levels evaluated. Furthermore, the performance gain is greater for the lower SNR (30dB) than for the other two conditions, although the differences are small.\n\n|Method|SNR=40dB|SNR=35dB|SNR=30dB|\n|--|--|--|--|\n|GE2E|5.64%|7.00%|9.04%|\n|GE2E+ICC|5.34%|6.76%|8.66%|\n|AngleProto|4.77%|5.76%|7.66%|\n|AngleProto+ICC|4.52%|5.59%|7.34%|\n|SupCon|4.35%|5.21%|6.25%|\n|SupCon+ICC|4.21%|5.09%|6.10%|\n\n**Response to \"why not quoting the numbers from the original papers\"**: We can quote the reported numbers from the original paper for Zhang, Harar, and Huckvale, although the results from Verde are not comparable. Firstly, it's important to realize that (Verde et al., 2018), did not utilize the complete SVD dataset; the authors relied on a subset of well-separated samples, painting an overoptimistic performance of how well the method works. Authors of the three baseline methods (Harar et al., 2017; Huckvale et al., 2021; Zhang et al., 2022) all express this concern about the (Verde et al, 2018) paper.\n\nWe list the methods' SVD testing accuracy of the original papers and our rebuilt results as follows:\n\n|Methods|SVD Testing Acc - Original Paper|SVD Testing Acc - Our Rebuilt|\n|--|--|--|\n|Our Proposed Method|N/A|0.7289|\n|Zhang et al. (2022)|0.7077|0.7077|\n|Harar et al. (2017)|0.6808|0.6914|\n|Huckvale et al. (2021)|0.6974|0.6255|\n\nWe will add this information to Table 3 in the revision. The new data does not impact our conclusion in the manuscript as our method outperforms the three baseline methods, Harar et al. (2017), Huckvale et al. (2021), and Zhang et al. (2022). And the following additional context explains why we had to rebuild the models:\n\nTwo baseline methods (Verde et al., 2018 and Huckvale et al., 2021) both employ SVM-based algorithms. Yet, neither of these studies provides details on the training accuracy within their papers. To ensure a comprehensive and fair evaluation, we decided to report the training accuracies for our method as well as the baseline methods. This required us to reproduce their methods, thereby allowing for a more transparent and reliable comparison.\n\nExcept for Zhang et al. (2022), all other baseline methods do not provide enough information to reproduce their work, which may lead to degraded performance of our rebuilt model compared to their original works. For example, Hara et al. did not release the code publicly, and Huckvale et al. did not provide their feature selection criteria, how the features were selected, or what the final list of features used in the model was.\n\n(Harar et al, 2017) - Harar, Pavol, et al. \"Voice pathology detection using deep learning: a preliminary study.\" _2017 international conference and workshop on bioinspired intelligence (IWOBI)_. IEEE, 2017.\n\n(Verde et al., 2018) - Verde, Laura, Giuseppe De Pietro, and Giovanna Sannino. \"Voice disorder identification by using machine learning techniques.\" _IEEE access_ 6 (2018): 16246-16255.\n\n(Huckvale et al., 2021) - Huckvale, Mark, and Catinca Buciuleac. \"Automated detection of voice disorder in the Saarbrücken voice database: Effects of pathology subset and audio materials.\" _Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH_. Vol. 6. International Speech Communication Association (ISCA), 2021."}, {"review_id": "OGllR8FFNp", "reviewer": "Reviewer_efaw", "summary": "In this paper, authors propose to use an intra-class correlation coefficient (ICC) regularizer as a complimentary component for contrastive loss to increase the repeatability of the supervised speech embedding. Authors conduct simulation for ICC regularizer and contrastive loss (GE2E loss) to explain their similarities and differences in the optimization criteria. Based on the experimental results on the speaker verification, voice style conversion, and a clinical application for detecting dysphonic three tasks, the authors claim that adding an ICC regularizer helps increase the repeatability and further improve the performance on downstream tasks.", "questions": "1. As this paper mentioned, the ICC regularizer requires hyperparameter fine-tuning. Are the hyperparameter tuned on a validation set or a development set? \n\n2. As this paper mentioned in Section 3.2, though the ICC increased, do the learned embeddings are more compacted within-class and more separate between-class in the overall embedding space? Authors can use t-SNE to visualize them.", "limitations": "The authors do not analyze the potential negative societal impact.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper leverages the concept of repeatability and states that higher repeatability can contribute to improved performance in downstream tasks. Authors think about the importance of intra-class variance and the proposed ICC regularizer places greater emphasis on minimizing it to help increase repeatability. The simulation results in Section 3.2 explains the intuition clearly. \n \n2. As a complementary component to contrastive loss, adding the proposed ICC regularizer successfully increase the repeatability and shows good performance in the three different tasks.", "weaknesses": "1. In Task 1, the performance of SV system is not good though adding the ICC regularizer improves its performance. In this paper, VGG-M-40 is trained on VoxCeleb1 & 2. The best result (3.78% EER) reported in Table 1 is not as good as other commonly used models for SV. In Chung et al. paper, the VGG-M-40 shows much worse performance than Thin-ResNet34 and Fast ResNet34. To make the experimental solid and results and claims more convincible, authors should pick a good baseline instead of use the VGG-M-40.     \n\n2. For better replicable, more experiment details should be provided, such as the training details of Task 1.     \n\n3. To have comprehensive evaluation results for Task 1, the evaluation metric minDCF can be used as well.\n\n4. Visualization (such as t-SNE) can be added as a comparison to see if the extracted speech or speaker embeddings have smaller intra-class variance in the space, though their ICC is decreased.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer efaw for the comments.\n\n**Response to Weakness 1:** In this study, our focus is not on demonstrating that the proposed method can reach SOTA performance across various tasks. Rather, we aim to illustrate that the proposed ICC regularizer can improve the repeatability of learned embeddings, improving the performance of downstream applications. Additional experiments demonstrate that even with models better than the ones used in the paper, the proposed ICC regularizer can still improve performance. For example, we re-run VC task with a SOTA VC model named FreeVC with our speaker encoder (manuscript Line 258-287) to conduct the experiment outlined in Section 4.2. (For details please see response to Reviewer 89AX.)\n\n(VC model used here is FreeVC w/o SR.)\n||Speaker Similarity Score|WER|CER|\n|--|--|--|--|\n|GE2E Loss|0.2753|0.2556|0.0755|\n|GE2E Loss + ICC Regularizer|0.2899|0.2163|0.0718|\n\nWhile this model is still training (authors suggest 900k steps), the results above represent an intermedia evaluation of the model. The higher speaker simialrity score, the better performance of model. The lower WER & CER, the better performance of model. As the results show, even at this intermediate evaluation, the consistent pattern from the SOTA VC model serves as tangible evidence that including the ICC regularizer with existing cost functions demonstrates consistent improvements across several performance metrics.\n\nWe ran out of time for training the Fast ResNet34 model. When the FreeVC model training is complete, we will train the Fast ResNet34 model referenced in Chung et al.'s paper for the speaker verification task and refine and optimize our methodology.\n\n**Response to Weakness 2:** In the revision, we will add following contents to Section 4.1 Experiment Setup:\n\n*During training, we employ the Adam optimizer, maintaining a static learning rate of 0.001 without implementing any learning rate schedule. The dropout rate is set to 0.2 for all dropout layers. As for data augmentation: (1) we use variation in input audio length by randomly fixing the audio duration within a range of 1.5 to 3.0 seconds, and (2) we add Gaussian noise with an SNR randomly selected between 15 to 60 dB. No other augmentation methods are used.*\n\n**Response to Weakness 3:** We follow the reviewer's comment and add the minDCF metric to Table 1. For the minDCF, hyper-parameters of $P_{target}=0.05$, $C_{fa}=C_{miss}=1$.\n\n||minDCF|\n|--|--|\n|GE2E|0.2925|\n|GE2E+ICC|0.2778|\n|AngleProto|0.2809|\n|AngleProto+ICC|0.2790|\n|SuperCon|0.2791|\n|SuperCon+ICC|0.2597|\n\nThe results demonstrate that the ICC regularizer improves model performance relative to the minDCF metric on the TI-SV task. The improvement in the metric is clear for all three baseline models to which the ICC regularizer is added.\n\n**Response to Weakness 4 and Question 2:** We follow the reviewer's comment and generate the figure of speaker embeddings t-SNE projection. We use two trained speaker encoders from Section 4.1, i.e., GE2E and GE2E+ICC, to generate the speaker embeddings. For the audio data, we randomly select 12 speakers from the test dataset and randomly select 10 samples from each speaker. After using the two encoders to extract all embeddings for all samples, we run t-SNE on these embeddings, and the results are shown in Figure B (in global response PDF).   Please take a look at Speaker No. 4 & 7: in Figure B(a), the two speakers’ embeddings are mixed with other speakers, but in Figure B(b), the two speakers become separate from other speakers due to the fact that the ICC regularizer improves the repeatability of embeddings relative to speaker identities.\n\n**Response to Question 1:** The hyper-parameter is tuned on the development dataset. The EER for subjects in the development dataset are used to determine the optimized hyperparamter. We will be sure to highlight this in the revision."}, {"review_id": "TNPReREx4b", "reviewer": "Reviewer_Yotw", "summary": "The paper presents a simple but seemingly effective idea to regularize contrastive embedding extractors. The regularizer aims at the intra-class correlation to ensure the repeatability of the embedding.\nThe effectiveness is confirmed experimentally for the task of speaker verification, voice style conversion, and dysphonic voice detection.", "questions": "The contrastive loss has a direct relationship to the mutual information, one could think that ICC is automatically included in the mutual information when only 2nd order statistics are taken into account, what could be the reason that this doesn't seem to be the case.", "limitations": "The introduced regularizer uses 2nd order statistics only.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The presented idea is well introduced and the illustration of the ICC regularizer action compared to the GE2E loss is insightful. The evaluation is sufficient.", "weaknesses": "The introduced regularizer uses 2nd order statistics only which could be a limiting factor on the embedding extractors but on the other hand this doesn't need to be required.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer Yotw for the comments.\n\nIt is true that the ICC relies only on second-order statistics and it’s possible that it may not capture complex, nonlinear relationships within or between groups. That said, it does have several benefits over other more complex measures:\n\n1. This simplicity often means fewer assumptions and parameters compared to some nonlinear measures. With small sample sizes, complex models that require estimation of numerous parameters may suffer from overfitting or instability, whereas the ICC may provide more robust estimates. \n2. The ICC is a standard measure used in many fields, including psychometrics, medical research, and reliability engineering. Its widespread acceptance means that results based on the ICC can be easily compared with other studies or benchmarks. \n3. The reliance on second-order statistics makes the ICC computationally efficient. \n4. To the best of our knowledge there are no commonly-used measures of repeatability that are based on higher-order statistics.\n\nWhile the example contrastive loss in our manuscript, the GE2E loss shares some high-level goals with MI maximization (such as learning informative representations), as far as we know, it does not have a direct mathematical relationship to MI. The connection between contrastive learning and MI is more explicitly established in other contrastive methods like InfoNCE loss, which are specifically designed to approximate MI maximization. Those loss functions are less commonly used in speech, owing to the superior performance demonstrated by GE2E loss in speech tasks. While the Pearson correlation (different from the ICC measure we use here) does have a connection with the MI,  to the best of our knowledge, there is no standard or direct mathematical formula that connects ICC and MI. They are derived from different statistical principles. "}, {"review_id": "e8KPE32Mth", "reviewer": "Reviewer_89AX", "summary": "The paper introduces the ICC regularizer, a novel regularization technique that complements the contrastive loss to enhance the repeatability of embeddings. The authors illustrate the reason why the ICC regularizer better minimizes the intra-class variance than the contrastive loss, offering a fresh perspective on latent representation learning. The experimental results demonstrate that the ICC regularizer enhances the repeatability of learned embeddings. Consequently, embeddings with higher repeatability exhibit superior performance in downstream tasks.", "questions": "1. In Section 4.2, ''GE2E Loss + ICC Regularizer'' shows superior speech naturalness on subjective evaluations. However, the objective metrics like speaker similarity scores and word error rate (WER) are missing.\n2. In Section 4.2, the paper aims at zero-shot voice style conversion, which requires highly generalized speaker embedding. However, the authors train and evaluate the zero-shot voice style conversion on a small dataset called VCTK. I would like to ask if the generalization performance of the proposed method be further improved by using larger training datasets? Additional experiments on larger dataset would make the paper more convincing.\n3. Although the audio samples with ICC regularizer is better, after listening to the audio samples in the supplementary materials, I think the audio quality should be further improved for practical scenarios. I would like to ask is this due to the used AutoVC model? Could the authors use the latest SOTA voice conversion model to improve the speech quality?\n\nI would like to raise my scores if the authors have fully addressed my concerns.", "limitations": "Some limitations of the work are discussed. The potential negative societal impact of the work is not addressed.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper links the concept of repeatability, derived from measurement theory, to deep-learned embeddings, which is novel and interesting. The authors present an explanation of how the ICC regularizer effectively reduces intra-class variance compared to the contrastive loss, offering a novel perspective on latent representation learning.\n2. To enhance the repeatability of the learned embeddings, this paper introduces a novel regularizer called the ICC regularizer, which is used as a regularizer for the contrastive loss.\n3. The experimental results indicate that using the ICC regularizer leading to better performance in three downstream tasks .", "weaknesses": "My main concerns are mainly related to the experiments. Please refer to the following Questions section for details.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer 89AX for the comments.\n\n**Response to Question 1:** We evaluate two methods objectively by using a speaker similarity score and word error rate (WER). We use an opensource speaker encoder (huggingface.co/speechbrain/spkrec-ecapa-voxceleb) to calculate the speaker similarity score between target speaker's audio and transformed output. And we use Wav2Vec2 (huggingface.co/docs/transformers/model_doc/wav2vec2) to do ASR on the transformed output, and calculate the WER and character error rate (CER) by using *jiwer* module (pypi.org/project/jiwer/2.5.1/). The results are given as follows:\n\n(VC model used here is AutoVC.)\n||Speaker Similarity Score|WER|CER|\n|--|--|--|--|\n|GE2E Loss|0.2231|0.5810|0.3817|\n|GE2E Loss + ICC Regularizer|0.2309|0.5109|0.3324|\n\nThe results demonstrate that speaker embeddings with higher repeatability also result in better performance across all three metrics for voice style conversion.\n\n**Response to Question 2:** Using VCTK dataset solely for training is a widely adopted strategy in voice conversion researches, including in SOTA models (Tang et al., 2022; Casanova et al., 2022; Li et al., 2023; ). Other larger speech datasets such as LibriTTS and Common Voice are only used for testing in select papers (Li et al., 2023).\n\nBoth LibriTTS and Common Voice are indeed substantial datasets, but they have different characteristics compared to VCTK.\n\n**VCTK**: Specifically designed for studying various speech processing tasks, including voice conversion. The dataset contains diverse accents but with relatively clean recordings.\n\n**LibriTTS**: LibriTTS is designed more for TTS (Text-To-Speech) tasks. It's derived from audiobooks, so the speaking style is primarily a reading style.\n\n**Common Voice**: This dataset has a broader goal of collecting voice data for a variety of languages to support voice technologies. The quality and style of recordings vary greatly in this dataset. What's more, many speakers have very limited recordings in this dataset.\n\nVoice conversion benefits from consistent and clean recordings, as it aims to capture and transfer speaker characteristics without extraneous noise or variability. And voice conversion requires capturing the nuances of individual speaker characteristics. So, compared to VCTK, the LibriTTS and Common Voice are not suitable for voice conversion model training.\n\nThe above information gives some insights about why most VC research prefers VCTK over larger but noisier databases. However, as we found the reviewer's comment very interesting, we tried to train the AutoVC model on the Common Voice dataset and tested the model on the VCTK dataset under the unseen-to-unseen speaker scenario. The model is trained with \"GE2E loss\" (with and without the ICC) speaker encoder described in Section 4.1. We only report objective scores, including the speaker similarity score, WER and CER here, as there was not enough time to conduct AB preference experiments.\n\n(VC model used here is AutoVC.)\n|AutoVC|Speaker Similarity|WER|CER|\n|--|--|--|--|\n|GE2E Loss + train on VCTK|0.2231|0.5810|0.3817|\n|GE2E Loss + train on Common Voice|0.1934|0.5958|0.3942|\n|GE2E + ICC Regularizer + train on VCTK|0.2309|0.5109|0.3324|\n|GE2E + ICC Regularizer + train on Common Voice|0.2131|0.5308|0.3544|\n\nThe results reveal that employing a larger dataset for training the VC model does not necessarily lead to improved performance. However, the proposed ICC regularizer is still effective when using a large training dataset and leads to a better performance on several metrics.\n\n(Tang et al., 2022) - Tang, Huaizhen, et al. \"Avqvc: One-shot voice conversion by vector quantization with applying contrastive learning.\" _ICASSP 2022_. IEEE, 2022.\n\n(Casanova et al., 2022) - Casanova, Edresson, et al. \"Yourtts: Towards zero-shot multi-speaker tts and zero-shot voice conversion for everyone.\" _ICML 2022_. PMLR, 2022.\n\n(Li et al., 2023) - Li et al. \"Freevc: Towards High-Quality Text-Free One-Shot Voice Conversion.\" _ICASSP 2023_. IEEE, 2023.\n\n**Response to Question 3:** We use a SOTA voice conversion model to conduct the experiment outlined in Section 4.2. We selected a recently published work, FreeVC, for this purpose. Details of FreeVC can be found on GitHub at github.com/OlaWod/FreeVC and the paper is accessible at arxiv.org/abs/2210.15418.\n\nFor the experiment, we utilized the version of FreeVC without speaker regularization  (w/o SR), as it has the fastest training speed compare other two versions and we have limited time for training this model. We trained it using the speaker encoder mentioned in our paper (Line 258-287). Both the training and testing were performed exclusively on the VCTK dataset. However, the training is longer than our expectation. We use the code provided by the author of FreeVC to train the model on an Nvidia A100 card. The model is still training, however we report an intermediate result below (the paper recommends 900k steps, results below are at ~500k steps) (Li et al., 2023). We report the objective scores, including the speaker similarity score, WER and CER for the 500k-steps-trained FreeVC model here.\n\n(VC model used here is FreeVC w/o SR, trained 500k steps.)\n||Speaker Similarity Score|WER|CER|\n|--|--|--|--|\n|GE2E Loss|0.2753|0.2556|0.0755|\n|GE2E Loss + ICC Regularizer|0.2899|0.2163|0.0718|\n\nEven though we did not complete the FreeVC training, the results have already shown improved output performance over the results reported in the paper by using a SOTA voice conversion model. The reported speaker similarity scores, WER and CER, demonstrate that speaker embeddings with higher repeatability also result in better performance for voice style conversion tasks. It is important to note that the intermediate results reported herein cannot be directly compared to those in the paper as the model is still training.\n\n(Li et al., 2023) - Li et al. \"Freevc: Towards High-Quality Text-Free One-Shot Voice Conversion.\" _ICASSP 2023_. IEEE, 2023."}, {"review_id": "JZ7ZbJcyfN", "reviewer": "Reviewer_hWT5", "summary": "Starting with the observation that supervised embeddings should vary from one class to another but not be sensitive to variations within a given class, this paper proposes to add an intra-class correlation (ICC) regularization to the contrastive loss in representation learning. This new regularizer is derived from measurement theory and encourages the embeddings to have a low intra-class variance, whereas contrastive losses pushes more the embeddings towards high inter-class variances. After a simulation illustrating the \"behaviour\" of these losses with respect to inter- and intra-class variance, the authors report good improvements using the proposed ICC regularizer for three different tasks. ", "questions": "Could that method apply to self-supervised approaches, where the notion of classes is less well defined?\n\nIs it competitive with other methods mentioned at the beginning of the paper? Or could it be complementary?\n\nAre the improvements with that method bigger on smaller datasets or are they independent of the dataset size? Is the method especially suited for some tasks and less for other (e.g. could that be useful for classification, or ASR?)?\n\nminor remark:\n  - sec 3.1: \"represents the i-th samples\" -> sample", "limitations": "yes", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "  - The paper is overall well written, the idea well explained and easy to follow and to understand\n  - The ICC regularizer seems to really focus on intra-class variance, i.e. make all occurences of the same class have the same embeddings, where contrastive loss does both at the same time and never really focuses on minimizing the intra-class variance.\n  - The method is relatively simple, very well explained and easy to understand. The simulation, comparison with contrastive loss and classification error rate is quite interesting.\n  - When the embeddings are constrained, it is indeed useful to focus on minimizing the intra-class variance to separate classes. The explanation and results are convincing.", "weaknesses": "  - Missing comparison with other techniques mentioned in the related work (data augmentation, etc.). Is the proposed approach competitive with these methods? Could it be complementary? How does it compare to VICReg [4] for example\n  - In the result section, the text mainly lists the results in the tables. Maybe a reformulation would allow more space for example to study the effect of the regularization weight", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer hWT5 for the comments.\n\n**Response to \"Is ICC competitive with other techniques? Or could it be complementary?\":** In Lines 30-31, we identify two primary groups of approaches for managing the intra-class variance: (1) enhancing the diversity of the training data, and (2) utilizing innovative learning algorithms. Below we discuss how the ICC is related to both types of approaches.\n\n1. Our results show that the ICC regularizer is complementary to approaches belonging to the first group. For example, data augmentation was implemented in the speaker verification task (Section 4.1) and dysphonic voice detection task (Section 4.3) for both contrastive loss only and contrastive loss + ICC regularizer experiments. The ICC regularizer provides additional improvement on top of data augmentation. Implementing the ICC regularizer can indeed contribute to further minimizing intra-class variance, thus complementing those methods.\n2. Our results show that the ICC regularizer provides benefit over GE2E, AngleProto, and SupCon (see Section 4.1). It is unclear that if ICC regularizer is complementary over all innovative learning algorithms. This will depend on the optimization criteria for each loss. For example, the algorithm proposed in (Rizve et al., 2021; [42] in the manuscript) focuses on using cosine similarity to construct the cost function. This is considerably different than the ICC and does not directly overlap with our ICC regularizer's functionality, we believe that our ICC regularizer could be a compatible addition to this algorithm. Since this focus does not directly overlap with our ICC regularizer's functionality, we believe that our ICC regularizer could be a compatible addition to this algorithm. In contrast, VICReg (Bardes et al., 2021; [4] in the manuscript) focuses on the variance of embedding vectors, which may be redundant with our regularizer.\n\nWe will add a discussion about \"Comparison between the ICC regularizer and other intra-class variance minimizing techniques\" in Section 3.2 of revised manuscript.\n\n(Rizve et al., 2021) - Rizve, Mamshad Nayeem, et al. \"Exploring complementary strengths of invariant and equivariant representations for few-shot learning.\" _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_. 2021.\n\n(Bardes et al., 2021) - Bardes, Adrien, Jean Ponce, and Yann LeCun. \"Vicreg: Variance-invariance-covariance regularization for self-supervised learning.\" _arXiv preprint arXiv:2105.04906_ (2021).\n\n**Response to \"more details about the effect of the regularization weight\":** We will add more context to manuscript about regularization weight in revision based on the information provided Supplemental Material - Appendix Section 2.\n\n**Response to \"apply to self-supervised approaches\":** Any self-supervised method that can use contrastive-style training can also use the ICC. For example, consider the self-supervised deep learning methods proposed in (Falcon et al., 2020; Ciga et al., 2022): here, the anchor-positive pair could be treated as an intra-class data pair, and the anchor-negative pair treated as an inter-class data pair. This is an interesting avenue of future work, and we will mention this possibility in the revision at the end of the paper.\n\n(Falcon et al., 2020) - Falcon, W., & Cho, K. (2020). A framework for contrastive self-supervised learning and designing a new approach. arXiv preprint arXiv:2009.00104.\n\n(Ciga et al., 2022) - Ciga, O., Xu, T., & Martel, A. L. (2022). Self supervised contrastive learning for digital histopathology. Machine Learning with Applications, 7, 100198.\n\n**Response to \"ICC regularizer effectiveness dependency on dataset size\":** The improvements with the given method might be more pronounced on smaller datasets. Regularization techniques, in general, tend to be more useful for smaller datasets. However, it's hard to provide an unequivocal answer without a deeper analysis of the specific method and context. Depending on the method and the nature of the data, the impact might vary.\n    \n**Response to \"ICC regularizer suitability for specific tasks\":** The method appears to be more suited for tasks where repeatability is a crucial factor, such as speaker verification and classification. It might be particularly beneficial for these applications, as consistency and reliability might be key in achieving high performance. On the other hand, for tasks like automatic speech recognition or pure end-to-end translation systems, where repeatability might be less critical, the method may not be as effective. Further analysis and experimentation would be required to determine the full scope of its applicability across different tasks and domains."}]

Author rebuttal:
We thank all reviewers for the comments.

**Response to "discussion on potential negative societal impact":**

Methods for learning new feature representations that focus on separability between classes can amplify biases that exist in the data. This is a well-known problem and it can occur when the data used to train the representation model is biased. This is especially problematic in high-stakes applications like healthcare, where biased predictions or decisions can lead to unequal treatment or access. Safe deployment of models based on the feature representations proposed herein will require thorough validation to detect potential biases and mitigation strategies for dealing with them. We will add this discussion to the revised paper.

**Response to "experiments and details of experimental setup":**

In the revision, we will add the following information to Section 4.1 Experiment Setup:

*During training, we use the Adam optimizer, maintaining a static learning rate of 0.001 without implementing any learning rate schedule. The dropout rate is set to 0.2 for all dropout layers. As for data augmentation: (1) we use variation in input audio length by randomly fixing the audio duration within a range of 1.5 to 3.0 seconds, and (2) we add Gaussian noise with an SNR randomly selected between 15 to 60 dB. No other augmentation methods are used.*

If accepted, we will add the following information about datasets we used in the supplemental material:

**VoxCeleb 1**: A large-scale speaker recognition dataset consisting of short video clips from YouTube. It includes over 100,000 utterances from more than 1,200 celebrities across various professions and demographics.

**VoxCeleb 2**: An extension of VoxCeleb 1, VoxCeleb 2 is an even larger dataset featuring approximately 1 million utterances from over 6,000 speakers. Together, VoxCeleb 1 and VoxCeleb 2 offer rich resources for training and evaluating speaker recognition models.

**VCTK** (The Voice Cloning Toolkit): VCTK is a speech dataset that includes recordings of various English accents. With over 44 hours of speech from 109 speakers, each speaking in their accent, VCTK provides a valuable resource for multi-accent speech synthesis and recognition research.

**MEEI** (Massachusetts Eye and Ear Infirmary): The MEEI Voice Disorders Database is a collection of speech samples from individuals with and without voice disorders. Participants are English speakers. It is often used in medical and clinical research to study voice pathology and develop systems to detect and analyze voice disorders. The MEEI database contains more than 1400 recordings of sustained phonations, which are collected from 53 healthy speakers and 657 speakers diagnosed with different types of dysphonia.

**SVD** (Saarbrücken Voice Database): The Saarbrücken Voice Database is a collection of voice recordings used for various phonetic and clinical studies. Participants are German speakers.  It provides a comprehensive set of voice samples, including those from individuals with different voice disorders, aiding in the research of voice quality and characteristics. SVD database contains the voice recordings from more than 2000 speakers (428 healthy females, 259 healthy males, 727 dysphonic females, 629 dysphonic males).

**HUPA** (Hospital Príncipe de Asturias): Similar to MEEI and SVD, HUPA a collection of speech samples from individuals with and without voice disorders. Participants are Spanish speakers. HUPA contains /a/ sustained phonation recordings of 366 adult Spanish speakers (169 dysphonic and 197 healthy).

Meta-review: This paper introduces an intra-class correlation (ICC) regularizer to promote repeatability of speech embeddings and thus improve their invariance in the downstream tasks.  The proposed ICC regularizer works as a complementary to the commonly used contrastive loss.  It can be shown to give rise to a better trade-off between minimizing intra-class variance and maximizing inter-class variance when inter-class variance is relatively larger than intra-class variance.  Experiments are carried out on multiple tasks on a variety of downstream tasks such as speaker verification, zero-shot voice style conversion and dysphonic voice detection.  Globally the idea is interesting and the work is solid.  Most of the concerns on the methodology and experiments have been cleared in the rebuttal.  The reviewers have reached a consensus to accept.
Decision: Accept (poster)