# An Overview of the IberSpeech-RTVE 2022 Challenges on Speech Technologies
## 3 IberSpeech-RTVE 2022 Evaluation Tasks
### Text and Speech Alignment Challenge
#### 3.3.1 Alignment of Re-Speaking Generated Subtitles (TaSAC-ST)
Task Description  
The task consisted of aligning each word of the text with a segment of the audio file so that the audio content corresponds to the pronunciation of the given word. Alignments were required to be monotonous, that is, the sequence of timestamps had to be non-decreasing. Obviously, it was guaranteed that there was an optimal monotonous alignment between the audio signal \(X\) and the paired text \(W\). Let \(W=\{w_{1},w_{2},\ldots,w_{N}\}\) be the sequence of \(N\) words to be aligned with an audio signal \(X\), and let \(S=\{s_{1},s_{2},\ldots,s_{N}\}\) be the corresponding sequence of aligned segments in \(X\). Then, if a word \(w_{i}\) is aligned to a segment \(s_{i}=(t_{bg}^{(i)},t_{end}^{(i)})\) and another word \(w_{j}\) is aligned to a segment \(s_{j}=(t_{bg}^{(j)},t_{end}^{(j)})\), with \(i<j\), then the timestamps defining those segments must be \(t_{bg}^{(i)}<t_{end}^{(i)}\leq t_{bg}^{(j)}<t_{end}^{(i)}\). Non-monotonic alignments were not allowed and non-monotonic submissions were not accepted. The output of an alignment system was required to be a text file containing a line for each word in the paired text, each line including 5 columns (separated by any amount of spaces or tabs) with the following information:  
* \(t_{bg}\): A real number with the time when the segment starts. * \(t_{end}\): A real number with the time when the segment ends. * word : The word paired with the audio segment. * score: A real number reflecting the confidence on the alignment, the more positive the score, the higher the confidence; the more negative the score, the lower the confidence. * decision: A binary value (0/1), 0 meaning Reject and 1 meaning Accept. Since only the accepted words would be evaluated, this decision should be made by applying a confidence score threshold. The participants could submit results for at most five (one primary + four contrastive) systems. Each system should automatically align the paired text with the audio, taking into account that some parts of the audio should not be aligned with any text and that the paired text did not reflect exactly the audio contents. It was not allowed to listen to the audio or use any kind of human intervention (e.g., crowdsourcing). Otherwise, any approach could be applied with no limit to the type or amount of resources that the participants could use to perform the task, as long as the employed methods and resources were described with enough detail and, if possible, links to papers, data and/or software repositories were provided to make it easier to reproduce their approach. For each system, two separate result files were required, for the development and test sets, respectively. Finally, participants would be ranked according to the performance obtained by their primary systems on the test set. Performance Metric  
The ground truth is preprocessed before using it to compute the performance metric. First, the missing segments are added to the ground truth and assigned an out-of-vocabulary label (\(\mathbf{\pi}\)). Then, the borders between segments are redefined by excluding from evaluation a collar time \(t_{collar}\) around them (in this evaluation, \(t_{collar}=20\) ms): the starting time \(t_{bgg}\) of each segment is added \(t_{collar}/2\) while the ending time \(t_{end}\) of each segment is subtracted \(t_{collar}/2\). Since the objective of the alignment is to recover as much correctly transcribed speech as possible to train acoustic models for the development of ASR systems, the performance metric should reflect this objective, but also the negative impact of wrongly aligned segments that could seriously compromise this semi-supervised training strategy. Thus, the performance metric will be just the difference between the correct and the wrongly aligned times. Let \(S=\{s_{1},s_{2},\ldots,s_{N}\}\) be the output of the alignment system for the paired text \(W=\{w_{1},w_{2},\ldots,w_{N}\}\).