## 4 Proposed Method
### Model Update  
The MT network parameter set \(\xi\) is updated via the EMA of the student network parameter set \(\theta\) as in Equation (1). We set the smoothing coefficient hyperparameter to 0.99. Therefore, the MT network linearly reduces the weight of the past student network, which is far from optimal, and aggregates the latest information. Consequently, we expect the MT network to provide stable and accurate speaker embeddings to train the student network to be a temporal ensemble teacher. In contrast to the MT network, the weights of the student network are updated using the backpropagation algorithm. There are two loss functions for the student network: consistency and classification costs. First, to discriminatively project the utterances of \begin{table}
\begin{tabular}{c c c} \hline \hline
**Block** & **Block Structure** & **Output** \\ \hline Encoder & RawNet2 & 512 \\ \hline \multirow{3}{*}{Converter} & FC(512) & \multirow{3}{*}{512} \\  & BN and LeakyReLU & \\  & FC(512) & \\ \hline \multirow{3}{*}{Projector} & FC(512) & \multirow{3}{*}{512} \\  & BN and LeakyReLU & \\  & FC(512) & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The student network architecture of the proposed method. RawNet2 used as the baseline in this study is exploited as the encoder. The mean teacher network has an identical structure as the student network except for the projector. Figure 1: Overall structure of the proposed method. The student and mean teacher (MT) networks comprise an encoder \(f\) and converter \(g\), and the projector \(q\) is added only to the student network. The two networks are fed by separate minibatches \(m\) and \(m^{\prime}\), and extract the embeddings \(Z\) and \(Y^{\prime}\), respectively. The student network is trained by cosine similarity-based metric learning using \(Y^{\prime}\), along with speaker identification. The parameter set of the MT network \(\xi\) is updated with the exponential moving average of the student network parameter set \(\theta\). unseen speakers to the representation space, it is necessary to learn to reduce intraclass variance and increase interclass variance. Thus, we designated the consistency cost as a cosine similarity-based metric learning loss function using embeddings from the two networks, rather than the MSE between predictions, which considers only positive pairs, as in a previous study [5]. It is expected that the embedding space can yield greater interclass variance using a negative pair as well as a positive pair between the two network embeddings. In addition, using the cosine similarity metric, which is a measure of similarity between the embeddings in SV evaluation, it is possible to directly learn the embeddings that have a smaller intraclass variance for SV. We used a centroid-based metric learning loss function, such as the generalized end-to-end (GE2E) [16] or angular prototypical (AP) network loss function [8]. In the case of GE2E, we modified it to fit the proposed method. The original GE2E loss operates using the cosine similarity between each query embedding and the centroids in the minibatch. Herein, the query refers to each utterance for all speakers, and the centroid \(c\) is derived by averaging embeddings that belong to the same speaker. If the speaker of a query and a centroid are equal, the centroid is calculated while excluding that query. In the proposed method, only the student network is backpropagated. Hence, only the embedding from the student network \(Z\) is used as a query during training and is referred to as half GE2E (GE2E-H). \[c_{j}=\frac{1}{U}\sum_{l=1}^{U/2}(Z_{jl}+Y_{jl}^{\prime}), \tag{3}\]  
\[c_{j}^{(-i)}=\frac{1}{U-1}(\sum_{ \begin{subarray}{c}l=1\\ l=i\end{subarray} }^{U/2}Z_{jl}+\sum_{l=1}^{U/2}Y_{jl}^{\prime}), \tag{4}\]  
where \(c_{j}\) and \(c_{j}^{(-i)}\) represent the centroid of the \(j\)th speaker and the centroid calculated excluding the \(i\)th utterance, respectively. In addition, \(Z_{jl}\) is the \(l\)th student network embedding of the \(j\)th speaker.