## 5 Results
### Multimodal Diarization Evaluation  
Table 14 shows the results of the multimodal diarization evaluation in terms of the overall diarization error and the diarization error for each modality (speaker and face). These results are presented for each submitted system and for each show contained in the test set. Note that the number of different characters to identify was 39. Three out of the four systems that were submitted for this task obtained very similar results in terms of the overall DER, ranging between 23% and 30%. The best result was obtained by the system presented by G9-PLUMCOT with 23.4% of overall DER. It is worth remarking that thanks to the optimization of speaker diarization using face information, the team was able to reduce the DER from 29.1%, obtained by independent optimization, to 23.4%. Note that G11-ODESSA and G9-PLUMCOT used the same face system for this evaluation. The results obtained by G2-GPS-UPC were significantly worse due to the poor adjustment of the speaker diarization system. Except for G2-GPS-UPC, the speech modality achieved significantly better results than the face modality. The G1-GTM-UVIGO speaker diarization system was the one that obtained the best results, 17.3%, followed by the G11-PLUMCOT system with 17.6%. For the face modality, the system used by G11-ODESSA and G9-PLUMCOT obtained the best results followed by the G2-GPS-UPC system. Regarding the results obtained by the participants for the different shows included in the evaluation, \"La Manana\" (LM-20170103) presented the highest difficulty for both face and speech modalities mainly due to the fact that it is a live show with a large amount of overlapping speech and outdoor scenes. On the other side, \"La tarde en 24 Horas, Tertulia\" (LT24 HTer-20180222 and LT24 HTer-20180223) presented TV studio set scenes mostly, less overlapping speech, and only five or less speakers and characters per show. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Team** & **G4** & **G8** & **G10** & **G11** & **G19** & **G22** \\ \hline
**System** & C2 & C2 & P & P & P & P \\
**TV Show** & & & & & & \\ EC & 34.8 & 27.8 & **17.1** & 37.6 & 30.3 & 47.0 \\ LA24H & 30.7 & 31.3 & **18.2** & 31.1 & 30.7 & 44.6 \\ LM & **29.1** & 36.0 & 41.2 & 33.7 & 35.3 & 52.4 \\ LT24 HTer & 14.9 & 27.6 & **13.3** & 14.6 & 20.2 & 27.9 \\ \hline
**Overall DER** & 25.4 & 28.7 & **17.3** & 26.6 & 26.6 & 39.1 \\ \hline Missed Speech & **0.4** & 4.1 & 1.1 & 0.7 & 1.1 & 2.4 \\ False Speech & 4.8 & 3.5 & **2.5** & 2.8 & 3 & 1.3 \\ Speaker Error & 20.2 & 21.1 & **13.7** & 23.1 & 22.5 & 35.4 \\ \hline \hline \end{tabular}
\end{table} 
Table 13: Closed-set condition speaker diarization: DER (%), missed speech (%), false speech (%), and speaker error (%) for the best systems per team. Team and system descriptions are in Section 4.2.2. Best results by TV show are in bold. \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline
**Team** & \multicolumn{2}{c}{**G1**} & \multicolumn{2}{c}{**G2**} & \multicolumn{2}{c}{**G9**} & \multicolumn{2}{c}{**G11**} \\ \hline
**Modality/System** & Face/P & SPK/P & Face/P & SPK/C1 & Face/P & SPK/P & Face/P & SPK/C1 \\ Missed Speech/Face & 28.6 & 3.0 & 22.6 & **1.1** & **12.1** & 1.2 & **12.1** & **1.1** \\ False Speech/Face & **5.2** & 9.3 & **4.7** & 14.1 & 12.2 & 12.4 & 12.2 & 12.6 \\ Error Speaker/Face & **0.9** & 5.0 & 2.3 & 47.4 & **4.9** & 4.0 & **4.9** & 15.2 \\
**Overall DER** & 34.7 & **17.3** & 29.6 & 62.6 & **29.2** & 17.6 & **29.2** & 28.9 \\
**Multimodal DER** & 26.0 & & 46.1 & & **23.4** & & 29.1 \\ \hline
**TV Show** & & & & & & & \\ LM-20170103 & 57.7 & **35.5** & 43.0 & 74.6 & **44.3** & 43.7 & **44.3** & 63.1 \\ LT24 HTer-20180222 & 19.8 & 9.0 & 19.4 & 46.4 & **17.3** & **3.2** & **17.3** & 22.8 \\ LT24 HTer-20180223 & **19.0** & 8.2 & 21.5 & 63.1 & 20.2 & **6.3** & 20.2 & 6.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 14: Multimodal diarization: DER (%), missed speech/face (%), false speaker/face (%) and speaker/face error (%) for the best systems. Team and system descriptions in Section 4.3.