## 4 Results and Comparison
### PNR-TDNN Experiments
#### 4.2.3 Discussion of PNR-TDNN Ablation Experiments  
Table 7 presents the performance of the proposed method, which is discussed in detail. The experiments are numbered for ease of reference. Experiments 1-3 modify the residual module of the Hu + EC(80), with the various residual networks depicted in Figure 6a, Figure 6c and Figure 6b, respectively. Experiment 4 introduces the SE-attentive mean mechanism. Experiment 5 employs the multi-head attentive statistics pooling mechanism. In experiment 6, we combine the SE-attentive mean with the multi-head attentive statistics pooling mechanism, integrating these enhancements into the Hu + EC(80). Experiments 7-9 are the enhanced versions achieved after applying fusion Res2 net, SE-attentive mean, and multi-head attentive statistics pooling mechanism. In comparison to the joint training approach where the pre-trained model and back-end model are trained together, as described in references [32; 38], our findings indicate that \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Transformer Encoders** & **Feed Forward Net** \\ \hline base HuBERT & Layers: 12 heads: 12 & dim: 3072 \\ small HuBERT & Layers: 4 heads: 12 & dim: 1536 \\ tiny HuBERT & Layers: 2 heads: 12 & dim: 512 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Parameters of compressed HuBERT. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Methodology**} & \multicolumn{2}{c}{**Dataset A**} & \multicolumn{2}{c}{**Dataset B**} & \multicolumn{2}{c}{**Dataset C**} & \multicolumn{2}{c}{**Params**} & \multicolumn{2}{c}{**Computational Efficiency**} \\  & **EER (\%)** & **MinDCF** & **EER (\%)** & **MinDCF** & **EER (\%)** & **MinDCF** & **Model** & **Flops** & **RTF (1 \(\times\) 10\({}^{-3}\))** \\ \hline Hu + EC(60) & 6.41 & 0.189 & 8.51 & 0.196 & 7.56 & 0.161 & 405.1 & 16.11 & 23.50 \\ Hu + EC(70) & 6.39 & 0.190 & 8.10 & 0.195 & 6.84 & 0.161 & 4  
the entire model tends to converge more effectively when the pre-trained model is entirely frozen during the training process. For experiments under noise-only interference conditions: (1) Employing pseudo-labels based on MFCC clusters of clean speech significantly enhances the performance of the PNR-TDNN architecture, surpassing that of the established baselines. (2) Figure 11a reveals that the EER for each PNR-TDNN experiences a marked decline beyond 30 epochs. Conversely, the MinDCF continues to refine. Research indicates that the points of optimal performance for EER and MinDCF are not reached concurrently. For experiments conducted under noise and reverberation interference conditions, our findings were as follows: (1) The PNR-TDNN consistently outperforms the baselines only when MFCC clusters with reverberation are utilized as pseudo-labels. (2) When the extended version of dataset A, comprising 66,000 utterances with noise and reverberation interference, is utilized as the training set, an improvement in the EER of the test set is challenging to achieve. However, when dataset A is reduced to create dataset B (51,000 utterances) and dataset C (30,000 utterances), a notable reduction in EER is observed. The study suggests that the limitation of pre-trained models in capturing concrete information, which is less specific than MFCC features, is magnified by the presence of an excessive number of labels during PNR-TDNN training. (3) Figure 11b,c illustrate that the EERs for datasets B and C remain consistent. The slower convergence of the loss function indicates that the pre-trained model exhibits robust resistance to over-fitting. The resilience enables the PNR-TDNN to progressively enhance its efficacy in the latter phase of the training process, a trend that shares similarities with the performance characteristics observed in CAM++. In comparison to the best model (CAM++) discussed in Section 4.2.1, experiment 8 yields an EER improvement of 9.54% while almost maintaining a similar MinDCF on dataset A. On dataset B, an 11.13% EER improvement is observed, accompanied by a slight improvement in the MinDCF.