# An Overview of the IberSpeech-RTVE 2022 Challenges on Speech Technologies
## 5 IberSpeech-RTVE 2022 Results and Discussion
### Text and Speech Alignment Challenge  
#### 5.3.1 Alignment of Re-Speaking Generated Subtitles (TaSAC-ST)  
Results for the four systems submitted by GTTS on the development and test sets of TaSAC-ST are shown in Table 16. In all cases, the acoustic models trained on generic databases (Primary and Con-2) perform better than those trained on BP materials (Con-1 and Con-3), suggesting that, despite being three times larger, the BP training dataset does not suitably match the TV broadcasts used in this evaluation. Clearly, using in-domain (RTVE) data to train the acoustic models would lead to better results. On the other hand, the originally submitted systems (Primary and Con-1) show degraded performance on the test set when compared to that obtained on the development set: the APTEM increases by 33.8% and 22.6%, respectively, and the mean of the alignment errors gets multiplied by almost 4, meaning that large alignment errors are being made. The alignment algorithm could be matching some words at the wrong end of relatively long non-speech intervals, thus introducing large alignment errors. This issue (the appearance of relatively long non-speech intervals) would be happening more frequently in the test set than in the development set and would explain the difference in performance. This latter issue gets fixed when using the modified kernel (see details in [27]): the APTEM of Con-2 and Con-3 is almost the same on both (development and test) datasets, while the mean of the alignment errors on the test set gets drastically reduced, from 4.0923 for the Primary system to 0.6053 for Con-2 and from 4.1990 for Con-1 to 0.7186 for Con-3, meaning 85% and 83% relative reductions, respectively. Table 17 shows the average performance of the best GTTS system (Con-2) on the three TV programs used in the development and test sets of TaSAC-ST, with the aim to discover performance variability depending on the type of TV program. APTEM and the mean of alignment errors are quite similar for AG (Agrosfera) and CO (Corazon), but clearly worse for AT (Aqui la Tierra). Differences are not large in terms of APTEM but remarkable in terms of the mean error, which may indicate that the alignment task could be hard in the case of challenging audio recordings and/or poor subtitles. 3.2 Alignment and Validation of Speech Signals with Partial and Inaccurate Text Transcriptions (TaSAC-BP)  
The BUT team did provide development results only for the contrastive system and test results only for the primary system. Performance figures for the baseline and BUT systems are shown in Table 18 (development) and Table 19 (test). The optimal performance \begin{table}
\begin{tabular}{c c c c c} \hline  & **Dev** & & **Test** \\ \hline
**System** & **APTEM** & **Mean-Err** & **APTEM** & **Mean-Err** \\ \hline Primary & 0.2950 & 1.2665 & 0.3950 & 4.0923 \\ \hline Con-1 & 0.3250 & 1.1493 & 0.3986 & 4.1990 \\ \hline Con-2 & 0.2950 & 0.8233 & 0.2927 & 0.6053 \\ \hline Con-3 & 0.3250 & 0.7950 & 0.3277 & 0.7186 \\ \hline \end{tabular}
\end{table} 
Table 16: Performance of GTTS primary and contrastive systems on the development (Dev) and test sets of TaSAC-ST. The Average Program Time-Error Metric (APTEM, in seconds) and the global mean of subtitle alignment errors (Mean-Err, in seconds) are shown. \begin{table}
\begin{tabular}{c c c} \hline
**Program** & **APTEM** & **Mean-Err** \\ \hline AG & 0.2850 & 0.5250 \\ \hline AT & 0.3100 & 0.9177 \\ \hline CO & 0.2910 & 0.6112 \\ \hline \end{tabular}
\end{table} 
Table 17: GTTS Con-2 system performance disaggregated per programs: AG (Agrosfera, 10 programs), AT (Aqui la Tierra, 6 programs) and CO (Corazón, 10 programs). is shown too, along with the optimal word confidence threshold. The search for the optimal performance is done by sorting the accepted words by confidence and then leaving aside words one by one from the bottom of the list (the words with the lowest confidence), measuring the score time in each case and keeping the best one.