# Multimodal Diarization Systems by Training Enrollment Models as Identity Representations +
## 7 Results
### Summary of Face and Speaker Results  
In this section, we collect the best result for the face and speaker subsystems, and we also divide the results into development and test set to better observe the difference in behavior of both subsystems. Moreover, our reference results obtained for the challenge [46] are included to better reflect the improvement achieved. Thus, the results of the other two participating groups [47, 48] that are publicly available (available online: [[http://catedrartve.unizares./albayzin2020](http://catedrartve.unizares./albayzin2020) results.html]([http://catedrartve.unizares./albayzin2020](http://catedrartve.unizares./albayzin2020) results.html) (accessed on 23 December 2021)) are been introduced in reference to the difficulty of this multimodal diarization challenge.  
Table 4 shows the DER results obtained in the development and test set for the face and speaker modalities. In addition to the separate results, we show the average result of the face and speaker diarization errors (\(FACE+SPEAKER\)). These results indicate a great mismatch between development and test results. We analyzed what type of video files composed both subsets and the length of these files, and we found that the development files are shorter and more similar than test files. Thus, we can see that the face and speaker subsystems perform better in the development files which are shorter videos, so the tracking process is easier to follow. Nevertheless, in the face subsystem, we observe that this difference is smaller than in the speaker subsystem.  
To better analyze these results, Table 5 presents a decomposition of the DER metric into the three terms of error. Focusing on the face modality errors, in the case of the development subset, we observe that the main cause of error is the probability of misses, which indicates that a large number of segments of the target identities have not been detected. Therefore, this effect can be motivated by using a threshold value that is too  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Subset** & **Modality** & **DER\%** & **DER\% Ours [46]** & **DER\% [47]** & **DER\% [48]** \\ \hline \multirow{3}{*}{DEV} & FACE & 51.26\% & 51.66\% & \(-\) & \(-\) \\  & SPEAKER & 37.45\% & 47.90\% & \(-\) & \(-\) \\  & FACE+SPEAKER & 44.36\% & 49.78\% & \(-\) & \(-\) \\ \hline \multirow{3}{*}{TEST} & FACE & 54.07\% & 61.79\% & 44.55\% & 67.31\% \\  & SPEAKER & 60.34\% & 72.63\% & 61.61\% & 131.59\% \\ \cline{1-1}  & FACE+SPEAKER & 57.20\% & 67.21\% & 53.08\% & 99.45\% \\ \hline \hline \end{tabular}
\end{table}
Table 4: Experimental results on RTVE 2020 Multimodal Diarization development and test sets, showing DER%. These DER% values were the result of the improvements introduced in this work, and the reference results for both modalities are also presented.  
Figure 5: Evolution of the different types of errors (MISS, FA, ID) for each different \(\gamma\), \(\beta\) parameter configuration.  
restrictive, while in the test subset, the misses term decreases, and especially relevant is the increase in false alarm errors as this illustrates the problems in discarding segments of nontarget faces when the number of enrollment identities is large. On the other hand, the distribution of errors produced in the speaker subsystem is quite different, as false alarms are much larger than misses in the test subset of data. Note that it is also related to the chosen threshold; however, in this case, the threshold is lower, so the target segments are mostly detected, but as a result, a high number of enrollment identities are assigned to segments of unknown identity.