## 5 Results
### Speaker Diarization Evaluation
#### 5.2.2 Closed-set Condition Results  
Six teams participated in the closed-set condition. Table 12 shows the results of all the submitted systems. The best result was obtained by the team G10-VIVOLAB with a DER of 17.3% for the primary system. The second best result was obtained by the team G4-VG with a DER of 25.4% with the second contrastive system. One of the most noticeable differences between the more accurate system and the rest was the speaker error term, which was substantially lower than the rest. This lower speaker error term was correlated with the more accurate estimation of the number of speakers, as can be seen in Figure 2, where the average of the estimated number of speakers per show is presented. It is interesting to compare the open-set and closed-set conditions presented by G11-ODESSA. Contrastive System 2 (C2) was the same for both conditions, the only difference was the training dataset, Voxceleb data for the open-set condition and the RTVE2018 dataset for the closed-set condition. The results were very similar with a slight improvement on the open-set condition. Regarding the results per show, Table 13 presents the diarization errors for the best system of each team, showing a similar trend as in the open-set condition.  
\begin{table}
\begin{tabular}{l c c c c c c c c c c c c} \hline \hline
**Team** & \multicolumn{3}{c}{**G1**} & \multicolumn{3}{c}{**G11**} & \multicolumn{3}{c}{**G20**} & \multicolumn{3}{c}{**G21**} & \multicolumn{3}{c}{**G22**} \\ \hline
**System** & P & C1 & C2 & P & C1 & C2 & P & C1 & C2 & P & P & C1 & C2 \\
**DER** & **11.4** & 11.7 & 12.7 & 25.9 & 20.3 & 36.7 & 30.8 & 31.8 & 33.3 & 30.96 & 28.6 & 28.2 & 31.4 \\ Missed Speech & 1.1 & 1.9 & 1.2 & 0.7 & 0.7 & 0.7 & 0.7 & 0.7 & **0.6** & 0.9 & 2.4 & 2.4 & 2.4 \\ False Speech & 3.7 & 3.2 & 3.7 & 2.8 & 2.8 & 2.8 & 3.1 & 3.7 & 4.5 & 4.8 & **1.3** & 1.3 & 1.3 \\ Speaker Error & **6.6** & 6.6 & 7.8 & 22.4 & 16.8 & 33.2 & 26.9 & 27.4 & 28.2 & 25.2 & 24.9 & 24.5 & 27.7 \\ \hline \hline \end{tabular}
\end{table}
Table 10: Open-set condition speaker diarization: diarization error rate (DER) (%), missed speech (%), false speech (%), and speaker error (%) per team. Team and system descriptions are in Section 4.2.1. Best results are in bold.  
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Team** & **G1** & **G11** & **G20** & **G21** & **G22** \\ \hline
**System** & P & C1 & P & P & C1 \\ \hline
**TV Show** & & & & & \\ EC & 13.1 & 27.4 & 37.7 & 40.9 & 38.6 \\ LA24H & 15.0 & 29.3 & 39.5 & 36.7 & 34.3 \\ LM & 16.9 & 24.1 & 48.7 & 45.3 & 35.9 \\ LT24 HTer & 7.8 & 9.9 & 18.4 & 18.2 & 15.6 \\ \hline
**DER** & 11.4 & 20.3 & 30.7 & 30.9 & 28.2 \\ \hline \hline \end{tabular}
\end{table}
Table 11: Open-set condition speaker diarization: DER (%) for the best systems per team and TV show. Team and system descriptions are in Section 4.2.1.  
\begin{table}
\begin{tabular}{l c c c c c c c c c c c} \hline \hline
**Team** & \multicolumn{3}{c}{**G4**} & \multicolumn{3}{c}{**G8**} & \multicolumn{3}{c}{**G10**} & \multicolumn{3}{c}{**G11**} & \multicolumn{3}{c}{**G19**} & \multicolumn{1}{c}{**G22**} \\ \hline
**System** & P & C1 & C2 & P & C1 & C2 & P & C1 & P & C1 & C2 & P & P \\
**DER** & 26.7 & 26.5 & 25.4 & 34.6 & 31.4 & 28.7 & **17.3** & 17.8 & 26.6 & 30.2 & 37.6 & 26.6 & 39.1 \\ Missed Speaker & **0.4** & 0.4 & 0.4 & 3.1 & 2.5 & 4.1 & 1.1 & 1.1 & 0.7 & 0.7 & 0.7 & 1.1 & 2.4 \\ False Speaker & 4.8 & 4.8 & 4.8 & 3.1 & 3.2 & 3.5 & 2.5 & 2.8 & 2.9 & 2.8 & 3 & **1.3** \\ Speaker Error & 21.5 & 21.3 & 20.2 & 28.4 & 25.7 & 21.1 & **13.7** & 14.2 & 23.1 & 26.6 & 34.1 & 22.5 & 35.4 \\ \hline \hline \end{tabular}
\end{table}
Table 12: Closed-set condition speaker diarization: DER (%), missed speech (%), and speaker error (%) per team. Team and system descriptions are in Section 4.2.2. Best results are in bold.