## 5 Post-Evaluation Analysis
### Spoken Term Detection
#### 5.1.2 System Analysis for Single and Multi-Word Queries  
An additional analysis based on single and multi-word queries was carried out for the STD task, and the results are presented in Tables 21 and 22 for MAVIR and RTVE data, respectively. The results show, in general, better performance for single than multi-word queries, regardless of threshold calibration issues. The only exception is the case of the _Kaldi DNN_ and the _Kaldi SGMM_ systems for RTVE data, where these systems perform better on multi-word queries than on single-word queries.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**INL**} & \multicolumn{2}{c}{**OOL**} \\ \hline
**System ID** & **MTWV** & **ATWV** & **MTWV** & **ATWV** \\ \hline Kaldi DNN + DA & 0.0025 & \(-\)0.0029 & 0.0000 & \(-\)0.0109 \\ \hline Kaldi SGMM + DA & 0.0062 & 0.0012 & 0.0000 & \(-\)0.0075 \\ \hline Kaldi DNN & 0.2633 & 0.2625 & 0.0727 & 0.0637 \\ \hline Kaldi SGMM & 0.2584 & 0.2572 & 0.0716 & 0.0708 \\ \hline \hline \end{tabular}
\end{table}
Table 19: System results of the STD task for RTVE test data for in-language (INL) and out-of-language (OOL) queries.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**SING**} & \multicolumn{2}{c}{**MULTI**} \\ \hline
**System ID** & **MTWV** & **ATWV** & **MTWV** & **ATWV** \\ \hline Kaldi DNN + DA & 0.0028 & \(-\)0.0186 & 0.0000 & 0.0000 \\ \hline Kaldi SGMM + DA & 0.0558 & 0.0445 & 0.0000 & 0.0000 \\ \hline Kaldi DNN & 0.4495 & 0.4509 & 0.1839 & 0.1601 \\ \hline Kaldi SGMM & 0.4705 & 0.4642 & 0.1667 & 0.1667 \\ \hline \hline \end{tabular}
\end{table}
Table 21: System results of the STD task for MAVIR test data for single-word (SING) and multi-word (MULTI) queries.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**SING**} & \multicolumn{2}{c}{**MULTI**} \\ \hline
**System ID** & **MTWV** & **ATWV** & **MTWV** & **ATWV** \\ \hline Kaldi DNN + DA & 0.0020 & \(-\)0.0052 & 0.0000 & 0.0000 \\ \hline Kaldi SGMM + DA & 0.0039 & \(-\)0.0011 & 0.0000 & 0.0000 \\ \hline Kaldi DNN & 0.2063 & 0.2063 & 0.3161 & 0.3161 \\ \hline Kaldi SGMM & 0.2048 & 0.2042 & 0.3161 & 0.3161 \\ \hline \hline \end{tabular}
\end{table}
Table 22: System results of the STD task for RTVE test data for single-word (SING) and multi-word (MULTI) queries.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**INL**} & \multicolumn{2}{c}{**OOL**} \\ \hline
**System ID** & **MTWV** & **ATWV** & **MTWV** & **ATWV** \\ \hline Kaldi DNN + DA & 0.0103 & \(-\)0.0024 & 0.0000 & \(-\)0.0087 \\ \hline Kaldi SGMM + DA & 0.0160 & 0.0100 & 0.0000 & 0.0000 \\ \hline Kaldi DNN & 0.0079 & \(-\)0.0036 & 0.0000 & 0.0000 \\ \hline Kaldi SGMM & 0.5424 & 0.5395 & 0.0625 & 0.0625 \\ \hline \hline \end{tabular}
\end{table}
Table 20: System results of the STD task for SPARL20 test data for in-language (INL) and out-of-language (OOL) queries.  
This discrepancy could be due to the fact that multi-word queries, on the one hand, may be easier to detect since they are typically longer than single-word queries; however, on the other hand, multi-word queries do convey more complex language modelling. Depending on the database domain, one factor can affect the system performance to a greater extent than another.