## 3 Materials and Methods
### Model Training
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Labels**} & \multicolumn{2}{c}{**Split 0**} & \multicolumn{2}{c}{**Split 1**} & \multicolumn{2}{c}{**Split 2**} & \multicolumn{2}{c}{**Split 3**} & \multicolumn{2}{c}{**Split 4**} \\  & **Train0** & **Test0** & **Train1** & **Test1** & **Train2** & **Test2** & **Train3** & **Test3** & **Train4** & **Test4** \\ \hline \(y\) & 42 & 8 & 42 & 8 & 40 & 10 & 36 & 14 & 40 & 10 \\ \(y\)-\(d\) & 12 & 5 & 12 & 5 & 15 & 2 & 13 & 4 & 16 & 1 \\ \(n\) & 36 & 13 & 39 & 10 & 40 & 9 & 42 & 7 & 39 & 10 \\ \(n\)-\(d\) & 8 & 2 & 8 & 2 & 9 & 1 & 7 & 3 & 8 & 2 \\ \(sno\) & 298 & 83 & 300 & 81 & 308 & 73 & 306 & 75 & 312 & 69 \\ \(so\) & 25 & 5 & 24 & 6 & 26 & 4 & 23 & 7 & 22 & 8 \\ \(ack\) & 10 & 3 & 12 & 1 & 12 & 1 & 8 & 5 & 10 & 3 \\ \(dno\) & 39 & 5 & 38 & 6 & 30 & 14 & 35 & 9 & 34 & 10 \\ \(query\) & 30 & 4 & 29 & 5 & 27 & 7 & 24 & 10 & 26 & 8 \\ \(f\) & 31 & 10 & 35 & 6 & 33 & 8 & 33 & 8 & 32 & 9 \\ \(fa\) & 36 & 6 & 32 & 10 & 30 & 12 & 37 & 5 & 33 & 9 \\ \(fe\) & 40 & 13 & 42 & 11 & 42 & 11 & 45 & 8 & 43 & 10 \\ \(fp\) & 54 & 9 & 48 & 15 & 50 & 13 & 53 & 10 & 47 & 16 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Distribution of 5 folds (groups) of data (13 classes). Figure 4: Summary of model training. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Hyperparameter**} & \multicolumn{2}{c}{**SVM 1**} & \multicolumn{2}{c}{**RF 2**} & \multicolumn{2}{c}{**BERT 3**} \\  & **Range of Values** & **Hyperparameter** & **Range of Values** & **Hyperparameter** & **Range of Values** \\ \hline C & 10–100 & max\_depth & 10–110, None & batch\_size & 8, 16, 32 \\ kernel & radial basis function (rbf) [77], linear & max\_features & ‘auto’, ‘sqrt’ & learning\_rate & \(2\times 10^{-5},3\times 10^{-5},5\times 10^{-5}\) \\  & & min\_samples\_leaf & 1, 2, 4 & epochs & 20, 30, 40 \\  & & min\_samples\_split & 1–10 & n\_estimators & 10–190 \\  & & bootstrap & True, False & \\ \hline \hline \end{tabular}  
* Hyperparameter names as per skist-learn documentation.
* Hyperparameter names as per Hugging Face documentation.  
\end{table} 
Table 5: Hyperparameter grid for each model. We then moved on to fine-tuning the pretrained BERT model. The data for BERT were utterances that were preprocessed (not encoded), as discussed in Section 3.4, and the labels were encoded by the aforementioned label encoder. The same 5 folds of data that were used for the traditional ML classifiers were also used to cross-validate the BERT model to keep the datasets consistent. The BERT pretrained model and tokenizer were retrieved from the Hugging Face Transformers library [78] specifically, we chose the _bert-base-uncased_ version on Hugging Face, which consists of 12 transformer encoders stacked together, with a hidden state of 768, 12 attention heads, and 110 million parameters, and ignores case in words (e.g., \"Camel\" and \"camel\" are the same). For feature extraction, we used the Fast Tokenizer [79] from Hugging Face. BERT models have variations for several tasks, such as token classification, text classification, language modeling, and question answering. The BERT model for sequence classification has an additional layer for fine-tuning BERT for text classification tasks. All 12 layers of BERT were left unfrozen; hence, the 12 layers were fine-tuned. First, we conducted an in-house randomized search CV over a fixed grid of hyperparameters (see third column of Table 5 on the 5 folds of data. The weight decay was kept constant at 0.01, and the default _AdamW_[80] optimizer was used. On training, we obtain CV scores for each hyperparameter combination on each fold of testing data. The best parameter combination is the one that produced the highest average f1-score over the 5-folds of data. In Section 4, we report the average accuracies and the f1-scores across all 5 folds of data, and highlight the best fine-tuned classifier, as determined by 5-fold CV, with the best parameters found via randomized search CV.