## 5 Results
### In-Domain Models (Retrained on Basque Parliament Data)
Counterintuitively, the amount of training data available when the PRR threshold is 0 does not correspond to _all_ the training data, because the collection procedure described in Section 3 discards segments smaller than 3 s (_orphan_ segments, usually with a low PRR). Since PRR figures are higher for the acoustic models trained after realignment (because they have adapted to Basque Parliament data), the number of discarded low-PRR segments decrease, a sizeable amount of those formerly low-PRR segments are pasted to the surrounding segments and the amount of training data available increases. We have tried the two options, obtaining practically the same performance in both cases (see Tables 7 and 8). This suggests that those additional 120 h of the second option, besides having questionable transcripts, do not improve the acoustic models and can thus be discarded. It seems that the criterion of choosing an equivalent amount of material (in this case, the top ranking segments amounting to 998 h) is good enough. Figure 4 shows WER results on the test set vs. WER results on the tuning set for 20 random partitions defined of the development set. These results reveal that: (1) hyperparameter tuning is working fine; (2) WER figures are highly and negatively correlated, which makes them somewhat _complementary_: all the partitions considered in the experiments comprise the same set of segments, so the global WER would be the same (around 4%) for all partitions, but segments would be distributed in different ways in the tuning and test sets, so for any given partition, WER figures would be higher on the most difficult set and correspondingly lower on the easiest one; and (3) despite the average WER is statistically identical in both cases, the models trained on the top ranking 998 h yield slightly better results than the models trained on the top ranking 1118 h, maybe because those 120 additional hours do not have reliable transcripts and introduce some noise. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Set** & **Metric** & **Basque** & **Spanish** & **Bilingual** & **All** \\ \hline \multirow{3}{*}{Tuning} & Avg & 5.43 & 3.93 & 4.38 & 4.29 \\  & StdDev & 0.16 & 0.17 & 0.55 & 0.14 \\  & 95\% CI & 0.07 & 0.07 & 0.24 & 0.06 \\ \hline \multirow{3}{*}{Test} & Avg & 5.51 & 4.04 & 4.35 & 4.41 \\  & StdDev & 0.20 & 0.17 & 0.65 & 0.14 \\ \cline{1-1}  & 95\% CI & 0.09 & 0.07 & 0.29 & 0.06 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: %WER performance of acoustic models obtained after one iteration of the semisupervised data collection method in cross-validation experiments. Figure 3: Amount of training data collected (in hours) depending on the PRR threshold, for the baseline (black) and first-iteration realigned models (blue). The red point signals the PRR threshold (80%) used to retrain the baseline models (998 h of top ranking segments); the green point signals the PRR threshold (98.28%) corresponding to using the top ranking 998 h in the second retraining stage; and the blue point signals an alternative PRR threshold (95%) for the second retraining stage (1118 h of top ranking segments). Figure 2: Experimental framework, showing the semisupervised data collection pipeline and cross-validation experiments using the baseline acoustic models and the acoustic models obtained after one iteration of the pipeline. Top ranking segments can be chosen using a PRR threshold or a target amount of data. In any case, the performance attained by the second-iteration realigned models is only sligtly better than that obtained with the first-iteration realigned models. Nothing compared to the huge improvement observed when going from the baseline to the first-iteration realigned models. This suggests that further iterations of the data collection pipeline are likely to yield even smaller improvements.