## 1 Introduction  
Speaker verification (SV) is the task of authenticating whether a speaker of an unknown input utterance matches the target speaker, and it is widely used in applications, such as voice assistant systems [1, 2]. Recent SV systems are primarily studied as an open-set scenario that tests using the utterances of speakers not seen in the training phase, requiring strong generalization [2, 3]. Considering these characteristics of SV, many researchers have aimed to extract discriminative speaker embeddings from utterances by exploiting deep neural networks (DNNs). A speaker embedding is a high-dimensional feature vector that represents the characteristics of a speaker. This paper focuses on studying an improved training method of a DNN-based SV system to explore a better speaker separable representation space.  
We noted from the results of a study that solely averaging DNN parameters after each step in the training phase can converge to better local minima [4]. This technique is called \"temporal averaging\"; the temporal averaging of weights can lead to more stable and accurate results than the final weights when the training has been completed. Leveraging this knowledge, in the field of semi-supervised learning, [PERSON] et al. [5] proposed a novel framework that uses a temporal averaging model, called mean teacher (MT), to use unlabeled data for training. The MT framework comprises a teacher-student [6] setting, where the teacher network (i.e., MT) is updated with an exponential moving average (EMA) of a set of student network parameters at each training step. In other words, the MT is the temporal averaging model of the student network and can generate a relatively accurate pseudo label for unlabeled data. Thus, the student network learns unlabeled data by reducing the Euclidean distance from the MT network predictions.  
The MT network can be regarded as a temporal ensemble model in terms of aggregating information about the student network at each training step. In addition, the MT model, as a teacher network, provides high-quality predictions for the student network to learn unlabeled data stably [5]. From these perspectives, we consider the MT network to be an ensemble teacher [7] and hypothesize that it can be a sufficient auxiliary model to assist student network training even in the supervised learning domain. Therefore, this study proposes a method that can enhance the generalization performance of the SV task by adapting the MT framework in a supervised learning condition.  
We modified the following two factors in the existing MT framework. First, SV is essentially a task of comparing the similarity between embeddings. Therefore, we let the student network learn directly from the speaker embedding output from the MT network, not the pseudo speaker labels. Second, the consistency loss function between the student and the MT network is changed to a cosine similarity-based metric learning that uses the negative pairs together, rather than the mean square error (MSE), which considers only positive pairs. Thus, the student network can explore the embedding space suitable for SV tasks with a small intraclass variance and large interclass variance by mimicking the evaluation scenario in the training process [8].  
The contributions of this study are:  
* We introduced a novel supervised training method utilizing the MT framework for the SV task of the open-set evaluation condition.
* We analyzed the effectiveness of the proposed framework's components through ablation and comparison experiments.  
To train and evaluate the models, we used the entire VoxCeleb2 [9] and the VoxCeleb1 [10] datasets, respectively. The experimental results revealed that the proposed method demonstrated a relative error reduction of 11.61% compared to the baseline system.