## 2 Search on Speech Evaluation
### Databases  
Three databases that comprise different acoustic conditions and domains were employed for the evaluation: the workshop talks MAVIR and broadcast news RTVE databases, which were used in previous ALBAYZIN SoS evaluations, and the SPARL20 database, which was the new one added for this evaluation and which contains speech from Spanish parliament sessions held from 2016. For the MAVIR and RTVE databases, three independent datasets (i.e., training, development and testing) were provided to participants. For the SPARL20 database, only test data were provided. This allowed measuring the system generalization capability to an unseen domain. Tables 1-3 show some of the database features, such as the training/development/test division, the number of word occurrences, the duration, the number of speakers and the average mean opinion score (MOS) [139]. The latter is included to show the quality of each speech file in the databases. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**File ID** & **Data** & **\#occ.** & **dur. (min)** & **\#spk.** & **Ave. MOS** \\ \hline \hline \multirow{2}{*}{Mavir-02} & train & 13,432 & 74.51 & 7 (7 ma.) & 2.69 \\ \hline \multirow{2}{*}{Mavir-03} & dev & 6681 & 38.18 & 2 (1 ma. 1 fe.) & 2.83 \\ \hline \multirow{2}{*}{Mavir-06} & train & 4332 & 29.15 & 3 (2 ma. 1 fe.) & 2.89 \\ \hline \multirow{2}{*}{Mavir-07} & dev & 3831 & 21.78 & 2 (2 ma.) & 3.26 \\ \hline \multirow{2}{*}{Mavir-08} & train & 3356 & 18.90 & 1 (1 ma.) & 3.13 \\ \hline \multirow{2}{*}{Mavir-09} & train & 11,179 & 70.05 & 1 (1 ma.) & 2.39 \\ \hline \multirow{2}{*}{Mavir-12} & train & 11,168 & 67.66 & 1 (1 ma.) & 2.32 \\ \hline \multirow{2}{*}{Mavir-04} & test & 9310 & 57.36 & 4 (3 ma. 1 fe.) & 2.85 \\ \hline \multirow{2}{*}{Mavir-11} & test & 3130 & 20.33 & 1 (1 ma.) & 2.46 \\ \hline \multirow{2}{*}{Mavir-13} & test & 7837 & 43.61 & 1 (1 ma.) & 2.48 \\ \hline \multirow{2}{*}{ALL} & train & 43,467 & 260.27 & 13 (12 ma. 1 fe.) & 2.56 \\ \hline \multirow{2}{*}{ALL} & dev & 10,512 & 59.96 & 4 (3 ma. 1 fe.) & 2.64 \\ \hline \multirow{2}{*}{ALL} & test & 20,277 & 121.3 & 6 (5 ma. 1 fe.) & 2.65 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Characteristics of the MAVIR database: number of word occurrences (#occ. ), duration (dur.) in minutes (min), number of speakers (#spk.) and average MOS (Ave. MOS). These characteristics are displayed for training (train), development (dev) and testing (test) datasets. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**File ID** & **Data** & **\#occ.** & **dur. (min)** & **\#spk.** & **Ave. MOS** \\ \hline \hline \multirow{2}{*}{LN24H-20151125} & dev2 & 21,049 & 123.50 & 22 & 3.37 \\ \hline \multirow{2}{*}{LN24H-20151201} & dev2 & 19,727 & 112.43 & 16 & 3.27 \\ \hline \multirow{2}{*}{LN24H-20160112} & dev2 & 18,617 & 110.40 & 19 & 3.24 \\ \hline \multirow{2}{*}{LN24H-20160121} & dev2 & 18,215 & 120.33 & 18 & 2.93 \\ \hline \multirow{2}{*}{millennium-20170522} & dev2 & 8330 & 56.50 & 9 & 3.61 \\ \hline \multirow{2}{*}{millennium-20170529} & dev2 & 8812 & 57.95 & 10 & 3.24 \\ \hline \multirow{2}{*}{millennium-20170626} & dev2 & 7976 & 55.68 & 14 & 3.55 \\ \hline \multirow{2}{*}{millennium-20171009} & dev2 & 9863 & 58.78 & 12 & 3.60 \\ \hline \multirow{2}{*}{millennium-20171106} & dev2 & 8498 & 59.57 & 16 & 3.40 \\ \hline \multirow{2}{*}{millennium-20171204} & dev2 & 9280 & 60.25 & 10 & 3.29 \\ \hline \multirow{2}{*}{millennium-20171211} & dev2 & 9502 & 59.70 & 12 & 2.95 \\ \hline \multirow{2}{*}{millennium-20171218} & dev2 & 9386 & 55.55 & 15 & 2.70 \\ \hline \multirow{2}{*}{EC-20170513} & test & 3565 & 22.13 & N/A & 3.12 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Characteristics of the RTVE database: number of word occurrences (#occ. ), duration (dur.) in minutes (min), number of speakers (#spk.) and average MOS (Ave. MOS). These characteristics are displayed for training (train), development (dev), and testing (test) datasets. The results for train and dev1 are not reported per file due to the large number of files (about 400 for train and about 60 for dev1).