## 3 IberSpeech-RTVE 2018 Evaluation Tasks
### Speech-to-Text Challenge  
#### 3.1.1 Challenge Description and Databases  
The speech-to-text transcription evaluation consisted of automatically transcribing different types of TV shows. The main objective was to evaluate the state-of-the-art in automatic speech recognition (ASR) for the Spanish language in the broadcast sector.  
Training and Development Data  
The training partition consisted of all the audio files without human revised transcriptions, which means that only subtitles were available. The training partition contained up to 460 h of audio, half of them corresponding to a live show (\"La Manana\"). Participants were free to use this audio as they considered appropriate.  
For development, two partitions were defined. Partition dev1 contains about 53 h of audio and their corresponding human revised transcriptions. Partition dev2 contains about 15 h of audio and the corresponding human revised transcriptions and speaker change timestamps. For this challenge, both partitions could be used for either development or training.  
Training Conditions  
The speech-to-text systems could be evaluated over a closed-set or open-set training condition.  
* Closed-set condition: The closed-set condition limited the system training to use the training and development datasets of the RTVE2018 database. The use of pretrained models on data other  
\begin{table}
\begin{tabular}{l l l l l l} \hline \hline
**dev1** & **Hours** & **Track** & **dev2** & **Hours** & **Track** \\ \hline
20H & 9:13:13 & S2T & & & \\ Asuntos Publicos & 8:11:00 & S2T & & & \\ Comando Actualidad & 7:53:13 & S2T & & & \\ La Manana & 1:30:00 & S2T & & & \\  & & & & Millennium & 7:42:44 & SD, S2T \\ La noche en 24H & 25:44:25 & S2T & La noche en 24H & 7:26:41 & SD, S2T, MD \\ \hline  & **52:31:51** & & & **15:09:25** & \\ \hline \hline \end{tabular}
\end{table}
Table 2: Development (dev) dataset partition with shows and duration. S2T, speech-to-text; SD, speaker diarization; MD, multimodal diarization.  
\begin{table}
\begin{tabular}{l l l l} \hline \hline
**Show** & **S2T** & **SD** & **MD** \\ \hline Al filo de lo Imposible & 4:10:03 & & \\ Arranca en Verde & 1:00:30 & & \\ Dicho y Hecho. & 1:48:00 & & \\ España en Comunidad & 8:09:32 & 8:09:32 & \\ La Manana & 8:05:00 & 1:36:31 & 1:36:31 \\ La Tarde en 24H (Tertulia) & 8:52:20 & 8:52:20 & 2:28:14 \\ Latinoamerica en 24H & 4:06:57 & 4:06:57 & \\ Saber y Ganar & 2:54:53 & & \\ \hline  & **39:07:15** & **22:45:20** & **4:04:45** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Test dataset partition with shows and duration.  
than RTVE2018 was not allowed in this condition. Participants could use any external phonetic transcription dictionary.
* Open-set condition: The open-set training condition removed the limitations of the closed-set condition. Participants were free to use the RTVE2018 training and development set or any other data to train their systems provided that these data were fully documented in the system's description paper. Each participant team should submit at least a primary system in one condition, open-set or closed-set, but they could also submit up to two contrastive systems.  
Evaluation Data The evaluation data contained a set of eight different TV shows covering a variety of scenarios with a total of 39:07 h of audio (see Table 3). The selected shows were different from those included in the development partition with human revised transcriptions. Table 4 shows the main characteristics of the selected shows.