Question: What systems are tested?
Evidence: FLOAT SELECTED: Table 2. % EER performance of VoxCeleb-based systems on BULATS and Linguaskill test sets.
FLOAT SELECTED: Table 1. % EER performance of BULATS-trained baseline systems on BULATS and Linguaskill test sets.
Performance of the two baseline systems is shown in Table TABREF9 in terms of equal error rate (EER). The x-vector system yielded lower EERs on both BULATS and Linguaskill test sets.
In addition to the models trained on the BULATS data, it is also interesting to investigate the application of “out-of-the-box" models for standard speaker verification tasks to this non-native speaker verification task as there is limited amounts of non-native learner English data that is publicly available. In this paper, the Kaldi-released BIBREF19 VoxCeleb x-vector/PLDA system was used as imported models, which was trained on augmented VoxCeleb 1 BIBREF17 and VoxCeleb 2 BIBREF18. There are more than 7,000 speakers in the VoxCeleb dataset with more than 2,000 hours of audio data, making it the largest publicly available speaker recognition dataset. 30 dimensional mel-frequency cepstral coefficients (MFCCs) were used as input features and system configurations were the same as the BULATS x-vector/PLDA one. It can be seen from Table TABREF10 that these out-of-domain models gave worse performance than baseline systems trained on a far smaller amount of BULATS data due to domain mismatch. Thus, two kinds of in-domain adaptation strategies were explored to make use of the BULATS training set: PLDA adaptation and x-vector extractor fine-tuning. For PLDA adaptation, x-vectors of the BULATS training set were first extracted using the VoxCeleb-trained x-vector extractor, and then employed to adapt the VoxCeleb-trained PLDA model with their mean and variance. For x-vector extractor fine-tuning, with all other layers of the VoxCeleb-trained model kept still, the output layer was re-initialised using the BULATS training set with the number of targets adjusted accordingly, and then all layers were fine-tuned on the BULATS training set. Here the PLDA adaptation system is referred to as X1 and the extractor fine-tuning system is referred to as X2. Both adaptation approaches can yield good performance gains as can be seen from Table TABREF10. PLDA adaptation is a straightforward yet effective way, while the system with x-vector extractor fine-tuning gave slightly lower EERs on both BULATS and Linguaskill test sets by virtue of a relatively “in-domain" extractor prior to the PLDA back-end.
FLOAT SELECTED: Table 2. % EER performance of VoxCeleb-based systems on BULATS and Linguaskill test sets.
FLOAT SELECTED: Table 1. % EER performance of BULATS-trained baseline systems on BULATS and Linguaskill test sets.
Performance of the two baseline systems is shown in Table TABREF9 in terms of equal error rate (EER). The x-vector system yielded lower EERs on both BULATS and Linguaskill test sets.
Both adaptation approaches can yield good performance gains as can be seen from Table TABREF10. PLDA adaptation is a straightforward yet effective way, while the system with x-vector extractor fine-tuning gave slightly lower EERs on both BULATS and Linguaskill test sets by virtue of a relatively “in-domain" extractor prior to the PLDA back-end.
Answer: BULATS i-vector/PLDA
BULATS x-vector/PLDA
VoxCeleb x-vector/PLDA
PLDA adaptation (X1)
 Extractor fine-tuning (X2)