## 4 Data Extraction  
This section summarizes the procedures employed to build the training and test datasets of our bilingual Basque Parliament database. A more in-depth description is provided in [39]. Our goal was to produce two sets of audio excerpts, each audio excerpt lasting between 3 and 10 s, along with the corresponding transcriptions (which are expected to match the audio contents). As a result, two index files for the training and test datasets were created with one line per segment, each line containing the speech segment filename (with information about the session from which it was extracted), language and speaker tags, text-audio similarity (a percentage reflecting how well the transcription reflects segment contents), length (duration, in seconds) and segment transcription (see Figure 2).  
As noted in Section 3.2, we were already producing subtitles for the videos of plenary sessions, where the draft minutes were reasonably aligned with the audio and segmented into readable chunks. However, those subtitles did not meet the requirements of an ASR corpus: first, the texts of the subtitles could be not suitably normalized; second, the aligned texts might not fully match the audio contents; third, the segments presented as subtitles might not meet the duration constraints.  
So, we designed an iterative segment extraction procedure from Basque Parliament plenary session videos. Each session could span two or more videos. Audios were extracted from each video in chunks of two or fewer hours to make the alignments computationally feasible; on the other hand, the draft minutes were manually cut in order to match the audio chunks. An alignment was performed between each audio chunk and the corresponding text.  
The process starts with text normalization of the draft minutes. Then, the audio and the normalized texts are aligned, which involves applying an ASR system to the audio to  
Figure 2: A section of an index file, with 10 training segments extracted from the Basque Parliament.  
obtain a recognized sequence of tokens/units and transforming the normalized texts into a reference sequence of tokens/units (in the same way as we do to obtain the subtitles in Section 3.2). Finally, starting from those alignments, segments are collected in a recursive fashion: the longest segment meeting the duration constraints and yielding the highest alignment similarity is retrieved and the audio chunks generated at both sides of the retrieved segment are searched until no segments are left. Note, that _not all_ of the audio signal is recovered by this procedure but just a part of it; because of the imposed duration constraints many short segments (shorter than 3 s) are left in the way.  
To build a training set, we may retain all segments obtained in this way regardless of the quality of the alignments; that is, regardless of how well the transcriptions matched the audio contents. Instead, segments may be ranked by alignment quality and only the top-ranking segments would be kept, that is, those showing a high correspondence between the audios and their transcriptions. No matter how strict the selection of segments is, new acoustic models can be trained on the selected segments and the alignment and selection procedure can be repeated until no further improvement is observed. To measure the goodness of a set of training segments, an ASR system can be trained on those segments and its performance measured on an independent, manually supervised test set, obtained from parliamentary sessions not included in the training set. This iterative procedure stops when ASR performance does not improve or the improvement is too small. In the following subsections, we give details about each step of the procedure.