## 6 Conclusions  
This paper presented a multi-domain and international SoS ALBAYZIN 2020 evaluation, which comprises two different tasks: spoken term detection and query-by-example spoken term detection. The STD systems were based on the Kaldi toolkit with SGMM and DNN acoustic models. In both cases, the systems were evaluated with and without including a data augmentation strategy. The QBe STD task was addressed using neural end-to-end systems, which constitutes the first attempt regarding end-to-end system construction in the SoS ALBAYZIN evaluation series. The most important conclusion drawn from this evaluation is that multi-domain search on speech is still challenging since the results showed a large variability when there is a change in the domain. On the other hand, the challenging conditions of the highly spontaneous speech in MAVIR data decreased the system performance with respect to unseen domains (i.e., SPARL20 data) for both STD and QBe STD tasks. Regarding query properties, it was shown that INL query detection was _easier_ than OOL query detection for the STD task, since the query language matched the target language. However, for the QBe STD task, for which the systems were trained on a different language that sometimes matched that of the OOL queries, OOL query detection iswas comparable or even improved the system performance for INL queries. QBe STD systems also showed better performance for INV query detection than for OOV query detection, due to the more diverse properties of OOV queries. Regarding single-word and multi-word query detection for the STD task, the evaluation results showed that single-word query detection was, in general, _easier_ than multi-word query detection, since less words are needed to be detected. However, for the QBe STD task, the opposite occurred. This could be due to differences in the system architecture (standard ASR system vs. end-to-end system), along with the fact that longer queries typically produce less FAs on template-matching QBe STD scenarios and also in end-to-end neural systems, such as the one presented in this evaluation. Given the best performance for each speech domain, there is still plenty of room for improvement. Specifically, the performance of the STD systems degrades with the data augmentation technique. This means that more research is needed in that topic to boost the performance. On the other hand, end-to-end systems for the QBe STD task deserve more research to improve their performance, in particular with respect to the realistic setup without information about query or test word alignments. This encourages us to maintain the SoS ALBAYZIN evaluation in the next years, focusing on multi-domain SoS and the applicability of this technology to new unseen challenging domains. This evaluation suffered from a large decrease in the number of participants, likely due to the adverse global situation related to the COVID-19 pandemic. Specifically, in the next few months, we will be launching the ALBAYZIN 2022 STD evaluation to be held in November 2022 within the IberSPEECH conference. This new evaluation edition, for which we hope to substantially increase the number of participants, aims to provide new domains and more challenging data (i.e., more difficult search terms) and evaluation conditions (i.e., rank the submitted systems from weighting the system performance according to the most challenging domain). \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**SING**} & \multicolumn{2}{c}{**MULTI**} \\ \hline
**System ID** & **MTWV** & **ATWV** & **MTWV** & **ATWV** \\ \hline E2E + LA & 0.0214 & \(-\)116.7881 & 0.0000 & \(-\)77.9245 \\ \hline E2E & 0.0214 & \(-\)89.5284 & 0.0000 & \(-\)47.8828 \\ \hline E2E + ZNORM & 0.0000 & \(-\)16.9599 & 0.0000 & \(-\)3.3951 \\ \hline \hline \end{tabular}
\end{table} 
Table 30: System results of the QBe STD task for RTVE test data for single-word (SING) and multi-word (MULTI) queries. **Author Contributions:** Conceptualization, J.T. and D.T.T. ; methodology, J.T., D.T.T., J.M.R., A.R.M.