## 5 Results
It is important to note that a query with an exact match or strict semantic match is also considered relevant. The accuracy metric is a combination of exact match and strict semantic match, while weak accuracy corresponds to relevance. in Table 1 clearly show a significant improvement in our implementation in the 'Overall Feel' and Repetition categories while showing similar performance to the original BB3 in the 'Hallucination' and 'Engagingness' categories. It is important to note that although the Turkers were instructed to be as objective as possible, the results may slightly deviate from the hypothetical rankings made by a different group of Turkers; Tables 5 and 6 provide insights into the differences of rankings among Turkers. Table 7 shows an overview of latencies of particular chatbot's pipelines and relative speed with respect to the original BB3 chatbot. Our enhanced chatbot has an estimated latency of 1.5 s and is more than three times faster than BB3. For enhanced lucidity, we also include the standard deviations of ratings from the Turkers (Tables 5 and 6). This not only illustrates the degree of consensus or disparity amongst the Turkers but also serves as an indicator of the impartiality or subjectivity embodied in the evaluation. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Category**} & \multicolumn{5}{c}{**Setting**} \\ \cline{2-6}  & **BB3 Original** & **Our BB3 Version** & **BB3 No Memory** & **BB3 No Entity** & **Vanilla BB3** \\ \hline Repetitions & 0.16 & 0.11 & 0.17 & 0.12 & 0.07 \\ Irrelevancy & 0.03 & 0.04 & 0.03 & 0.04 & 0.05 \\ Contradictions & 0.04 & 0.02 & 0.07 & 0.04 & 0.05 \\ Hallucinations & 0.04 & 0.03 & 0.04 & 0.03 & 0.02 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Standard deviations between Turker’s rating with respect to Table 2. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multicolumn{1}{c}{**Model**} & \multicolumn{2}{c}{**Accuracy [\%]**} & \multicolumn{2}{c}{**F1 [\%]**} & \multicolumn{2}{c}{**Speedup**} \\ \hline
**Task Name** & **Model Name** & **Eval** & **Test** & **Eval** & **Test** & **Test + Eval** \\ \hline search decision & BB3 & 84.0 & 63.0 & 86.2 & 71.3 & 1x \\ search decision & ELECTRA-small & 88.1 & 69.7 & 89.3 & 73.5 & 50x \\ search decision & BERT-tiny & 90.3 & 78.0 & 90.6 & 76.4 & 192x \\ search decision & DeBERTa-xsmall & 92.2 & 85.0 & 92.3 & 82.6 & 17x \\ \hline memory decision & BB3 & 76.1 & 69.7 & 36.9 & 81.6 & 1x \\ memory decision & ELECTRA-small & 87.3 & 59.8 & 42.5 & 59.8 & 50x \\ memory decision & DeBERTa-xsmall & 92.1 & 72.0 & 92.1 & 82.7 & 17x \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance of BB3 model and our fine-tuned models on two classification tasks: search and memory decision. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Category**} & \multicolumn{5}{c}{**Setting**} \\ \cline{2-6}  & **BB3 Original** & **Our BB3 Version** & **BB3 No Memory** & **BB3 No Entity** & **Vanilla BB3** \\ \hline Overall Feel \(\uparrow\) & 0.54 & 0.36 & 0.39 & 0.45 & 0.47 \\ Repetition \(\downarrow\) & 0.45 & 0.51 & 0.38 & 0.31 & 0.20 \\ Hallucination \(\downarrow\) & 0.65 & 0.67 & 0.58 & 0.60 & 0.23 \\ Engagingness \(\uparrow\) & 0.44 & 0.22 & 0.31 & 0.26 & 0.40 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Standard deviations between [PERSON]’s rating with respect to Table 1. Depending on the rating category, an upward arrow (\(\uparrow\)) indicates that higher score is better; a downward arrow (\(\downarrow\)) indicates that lower is better. Figure 4: Memory decision results on test set. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Category**} & \multicolumn{4}{c}{**Setting**} \\ \cline{2-5}  & **BB3 Original** & **Our BB3 Version** & **Vanilla BB3** & **BB3 with APIHUB** \\ \hline Speedup & 1x & 3.3x & 7.2x & 1.2x \\ Latency (mean) & 5.1 s & 1.5 s & 0.7 s & 4.6 s \\ \hline \hline \end{tabular}
\end{table}