## 5 Experiments and Results  
Experiments were carried out with datasets, DNN methods (i.e., CNN, LSTM, and BiLSTM), and vectorization techniques (i.e., fastText and BERT) described in Section 3, Section 4.2, and Section 4.3, respectively. Moreover, DNN Keras models were optimized with the tpe.suggest and random.suggest algorithms presented in Section 4.4. Models were tuned to achieve as high an accuracy on the validation dataset as possible for each language, dataset, and classifier, with word embedding type tuned separately, and later evaluated in the testing phase. The performance of each model was evaluated with the accuracy metric, as presented in Equation (4). \[accuracy=\frac{N_{correct}}{N_{all}}, \tag{4}\]  
where \(N_{correct}\) and \(N_{all}\) stands for correctly predicted and all tested instances, respectively. The model is considered reasonable if its accuracy on the testing dataset is above random Equation (5) (assigning labels to instances according to their probabilities in the testing set) and majority Equation (6) (assigning all instances to a class having the largest probability in the training set) baselines (see Table 4). \[random\_baseline=\sum P^{2}\big{(}c_{j}\big{)}, \tag{5}\]  
where _P(c\({}_{j}\))_ is a probability of a class (intent). \[majority\_baseline=max\big{(}P\big{(}c_{j}\big{)}\big{)}, \tag{6}\]  
The testing results for English, Estonian, Latvian, Lithuanian, and Russian are summarized in Tables 5-9. \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline  & \multicolumn{8}{c}{**fastText embeddings**} \\ \hline  & \multicolumn{8}{c}{**chatbot**} & \multicolumn{8}{c}{**askUbuntu**} & \multicolumn{8}{c}{**webapps**} \\ \hline  & CNN & LSTM & BiLSTM & CNN & LSTM & BiLSTM & CNN & LSTM & BiLSTM \\ \hline tpe.suggest & 0.943 & 0.943 & 0.953 & 0.780 & 0.706 & 0.633 & 0.508 & 0.475 & 0.542 \\ \hline rand.suggest & 0.953 & 0.943 & 0.943 & 0.771 & 0.670 & 0.771 & 0.627 & 0.356 & 0.508 \\ \hline \multicolumn{8}{c}{**BERT embeddings**} \\ \hline  & \multicolumn{8}{c}{**chatbot**} & \multicolumn{8}{c}{**askUbuntu**} & \multicolumn{8}{c}{**webapps**} \\ \hline  & \multicolumn{8}{c}{**chatbot**} & \multicolumn{8}{c}{**askUbuntu**} & \multicolumn{8}{c}{**webapps**} \\ \hline  & CNN & LSTM & BiLSTM & CNN & LSTM & BiLSTM & CNN & LSTM & BiLSTM \\ \hline tpe.suggest & 0.943 & 0.943 & 0.953 & 0.780 & 0.706 & 0.633 & 0.508 & 0.475 & 0.542 \\ \hline rand.suggest & 0.953 & 0.943 & 0.943 & 0.771 & 0.670 & 0.771 & 0.627 & 0.356 & 0.508 \\ \hline \multicolumn{8}{c}{**BERT embeddings**} \\ \hline  & \multicolumn{8}{c}{**chatbot**} & \multicolumn{8}{c}{**askUbuntu**} & \multicolumn{8}{c}{**webapps**} \\ \hline  & CNN & LSTM & BiLSTM & CNN & LSTM & BiLSTM & CNN & LSTM & BiLSTM \\ \hline tpe.suggest & 0.953 & 0.943 & 0.943 & 0.872 & 0.752 & 0.761 & **0.644** & 0.407 & 0.508 \\ \hline rand.suggest & **0.972** & 0.934 & 0.943 & **0.890** & 0.706 & 0.780 & 0.254 & 0.525 & 0.492 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Evaluated accuracies on the Estonian testing datasets with optimized models. For the other notation, see Table 5. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset** & **Random Baseline** & **Majority Baseline** \\ \hline
**chatbot** & 0.558 & 0.670 \\
**askUbuntu** & 0.283 & 0.367 \\
**webapps** & 0.186 & 0.271 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Calculated random and majority baselines for different datasets.