## 5 Results and Evaluation  
In this paper, we used the recall R at position \(k\) in \(n\) number of candidates, denoted as R\({}_{n}\)@\(k\), as the evaluation metrics, as it is also used in DAM [1], SMN [2], and other novel retrieval chatbot models [20; 24; 26; 27; 28; 29] we selected to compare their performance with that of the mDNA model. Here, R\({}_{n}\)@\(k\) is defined as (the number of relevant retrieved response candidates at top-\(k\))/(total \(n\) number of retrieved response candidates). We represent this mathematically as below:  
\[\text{R}_{n}\text{@}k=\frac{\text{TP}}{\text{TP}+\text{FN}}, \tag{12}\]  
where TP and FN represent true positive and false negative of retrieved response candidates, respectively. This evaluation metrics formula is suitable for information retrieval tasks, and is popularly used to measure and score the performance of information retrieval systems in terms of the degree at which the selected output response candidates match with the context of the input utterances. In our task, the mDNA model needs to select a response from \(n\) number of candidate responses. In this process, if the true response is among the response candidates, it is referred to as the true positive. Figure 4 shows the overview representation plot of R\({}_{10}\)@1, 2, and 5, with respect to different values of epochs which shows that the model achieved a stable training (and learning). Moreover, to further analyze the performance of our model, we selected some representative models based on retrieval dialog systems (that are either based on Transformer or gated-RNN respective), DAM [1], SMN [2], DualEncoder [23], MV-LSTM [26], Match-LSTM [27], Multiview [28], DL2R [29], and BERT Bi-Encoder+CE [20]. We compared the performance of these models to that of the mDNA model; and to perform ablation analysis, we removed the DAM model combined with the mDNA, and referred to it as mDNA\({}_{self}\). In Table 3, the recall evaluation results show that the mDNA performs competitively better than most of the selected chatbot models trained on Ubuntu Dialogue Corpus that were compared to it. With respect to the baseline DAM model, the performance of the mDNA increased by 3% on \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Training** & **Validation** & **Testing** \\ \hline \# context response pairs & 1 M & 500 K & 500 K \\ \hline \# candidates per context & 2 & 10 & 10 \\ \hline Avg. \# turns per dialogue & 7.71 & 7.33 & 7.54 \\ \hline Avg. \# words per utterance & 10.34 & 10.22 & 10.33 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Statistics of Ubuntu Corpus dataset. R\({}_{10}\)@1, 1.7% on R\({}_{10}\)@2 and 0.6% on R\({}_{10}\)@5, respectively. While the mDNA outperforms all other selected models compared to it, we observe that the recall evaluation scores of the mDNA and BERT Bi-Encoder+CE model [20] are similar, and therefore the performance gap between them is very insignificant. The BERT Bi-Encoder+CE model exploits the benefits of Bi-GRU and Bi-encoder system of BERT-Transformer architecture which enables its performance to be comparable to that of the mDNA. However, the capability of augmenting Transformer-based retrieval chatbot models for performance enhancement is a unique attribute to which the mDNA claims superiority. Lastly, owing to the important role of memory in the mDNA, we investigated to see if adjustment of the maximum length of utterance embedding is fed into the memory would affect the performance and effectiveness in any way. In this task, we initially set the dimension of utterance memory embedding to 350 and, subsequently, during multiple retraining sessions of the network, we altered the value from 350 to 300, 250, 200, and 150 at different corresponding intervals of the following training epochs: 3, 6, 9, 12, and 15, respectively.