## 5 Discussion  
From the results in Table 7, we observe that the SVM baseline classifier had an acceptable overall accuracy and f1-score; however, it failed to classify the classes _n-d_ (no, with explanation) and _y-d_ (yes, with explanation) and performed poorly on _so_ (opinionated state \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Dialogue Act**} & **SVM Baseline** & **RF Baseline** & **Fine-Tuned BERT** \\  & **Accuracy = 0.85** & **Accuracy = 0.84** & **Accuracy = 0.92** \\  & **F1-Score = 0.72** & **F1-Score = 0.74** & **F1-Score = 0.87** \\ \hline \(y\) & 0.75 & 0.75 & 0.84 \\ _y-d_ & 0.00 & 0.50 & 0.80 \\ \(n\) & 0.94 & 0.88 & 0.88 \\ _n-d_ & 0.00 & 0.00 & 1.00 \\ _sno_ & 0.87 & 0.86 & 0.94 \\ _so_ & 0.57 & 0.67 & 0.67 \\ _ack_ & 1.00 & 1.00 & 0.67 \\ _dno_ & 0.83 & 0.83 & 0.88 \\ _query_ & 0.86 & 0.80 & 0.86 \\ _f1_ & 0.94 & 0.88 & 0.94 \\ _fa_ & 0.91 & 0.91 & 1.00 \\ _fe_ & 0.84 & 0.84 & 0.95 \\ _fp_ & 0.81 & 0.73 & 0.88 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Classification report of best classifiers on Split 2. Figure 5: Confusion matrices of SVM (**top left**), RF (**top right**), and fine-tuned BERT (**bottom center**). The color gradient to the right of each matrix corresponds to the number of samples, i.e., the grid cell with \(\geq\)70 samples will be filled in yellow. ments) utterances. On observing the data, an utterance in the \(n\)-\(d\) class looks like \"No but I can try\", and an utterance in the \(n\) class looks like \"No ma'am\". The features for the SVM model did not take into account the order of words, since one-hot encoding was performed by the position of the word in the vocabulary, rather than by the order of occurrence of the words in the utterance (see Appendix A.1 for example). The presence of \"No\" in both utterances may be why the utterances in \(n\)-\(d\) were misclassified as the \(n\) label. However, on viewing the confusion matrix for SVM in Figure 5, we see that the \(n\)-\(d\) sample got misclassified as _sno_. This misclassification was also observed with the \(y\)-\(d\) utterances being misclassified as _sno_ class utterances, possibly due to the presence of words common to both classes and/or model bias towards the majority _sno_ class; however, there are not enough samples in the test set of the minority classes for an analysis. The RF model performs better for \(y\)-\(d\), where 1 sample is classified as _sno_ (see Figure 5) out of the 2; however, \(n\)-\(d\) is misclassified completely. The performance on _so_ improved in the RF classification report. The BERT model's features, on the other hand, involve integer-encoded utterances, where the order of the words in the utterance is preserved and each feature vector is padded to a maximum fixed length (see Appendix A.2, for example). Hence, it performed well on the two classes that the SVM model missed, but the accuracy on _so_ remained the same. Figure 6 shows two graphs where we plot the average accuracy and average f1-score of the three models (see Table 6) with standard error bars to present a visual representation of the variation in results from the 5-fold CV across the three classifiers. Independent one-tailed \(t\)-tests were conducted between the results of SVM and RF, SVM and fine-tuned BERT, and RF and fine-tuned BERT. The \(p\)-values are visualized on the dotted lines between the pairs. Results show that there was no statistical significance between the results of SVM and RF (\(t\) (4) = 1.25, \(p\) = 0.123 for mean accuracy and \(t\) (4) = 1.19, \(p\) = 0.135 for mean f1-score). However, the results between SVM and fine-tuned BERT (\(t\) (4) = \(-2.11\), \(p\) = 0.034) for mean accuracy and \(t\) (4) = \(-3.46\), \(p\) = 0.004 for f1-scores) and RF and fine-tuned BERT (\(t\) (4) = \(-3.56\), \(p\) = 0.004 for mean accuracy and \(t\) (4) = \(-3.93\), \(p\) = 0.002 for f1-scores) were statistically significant. The fine-tuned BERT performed significantly better than SVM and RF. In Section 1, we described how our project was motivated by [PERSON] et al.