# A Conversation History-Based Q&A Cache Mechanism for Multi-Layered Chatbot Services
## 5 Experiments
### Comparison Experiment and Results
#### 5.3.1 Q&A Set Size and Cache Size Relationship  
Case 1 (C1) had 50 Q&A pairs; the cache size was equal to 10, 15, 20, 25, and 30, and the caching sequence was random, mixed, and related. Figure 7a-c shows the results of C1 in random, mixed, and related sequences, respectively. These results show how the cache size and order affect the hit ratio. The random sequence does not show good hit ratios, regardless of the cache size (Figure 7a). The C1 results for the mixed-question sequence show how the hit ratio became higher after related questions were asked after the first 4000 steps (Figure 7b). The related-question sequence in C1 produced the best results with cache size 30 (Figure 7c). However, this result cannot be considered a good result because the cache size is more than 50% of the entire dataset. The bar graph of all cache sizes in Figure 7d shows the amount of difference depending on the question sequence. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Case** & **Q&A Set** & **Cache Size** & **Questions Sequence** & **Steps** \\ \hline Case 1 & 50 & 10, 15, 20, 25, 30 & Random, mixed, related & 10,000 \\ Case 2 & 100 & 10, 15, 20, 25, 30 & Random, mixed, related & 10,000 \\ Case 3 & 150 & 10, 15, 20, 25, 30 & Random, mixed, related & 10,000 \\ Case 4 & 200 & 10, 15, 20, 25, 30 & Random, mixed, related & 10,000 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Information on experimental cases. Figure 7: Experimental results of C1. (**a**)—results of C1 with the random sequence; (**b**)—results of C1 with the mixed questions sequence; (**c**)—results of C1 with the related questions sequence; (**d**)—the overall results of C1. The results for C2 with all cache sizes and random-sequence questions show almost the same results as C1. The mixed-sequence results show improvement after related questions were asked. The sequence of related questions had a high hit ratio, particularly with cache sizes equal to 25 and 30. The random sequence had a hit ratio less than 30%. The hit ratio of the mixed-question sequence was less than 40%. The highest hit ratio had cache size 30, and the lowest hit ratio had cache size 10. In the case of C3, the experiment produced approximately the same results, with an insignificant difference between the hit ratio averages. As the number of questions grew, the hit ratio decreased. With the mixed-question sequence, the hit ratio increased after step 4000, and with the related-question sequence, the hit ratio also increased. In the related-sequence test of C3, all cache sizes produced a hit ratio greater than 80%. The random sequence in C4 produced the same low hit ratio as in the other cases (Figure 8a). The mixed-question sequence produced a low hit ratio before the related questions were asked, after which the hit ratio increased. The related-question sequence of C4 produced an increasing hit ratio for all cache sizes, as shown in Figure 8c. The comparison of all cache sizes in C4 shows a significant difference between the related and random sequences (Figure 8d). The first produced an average hit ratio slightly below 80%, but the other one produced a hit ratio less than 30%. The four experimental cases used the same parameters, with the only difference being the number of questions. The experimental results showed that the sequence of the questions affected the hit ratio. A logical conversation with a user would create a better frequency graph of Q&A pairs than random conversations would. The experiment demonstrated the difference between cache sizes and the number of Q&A sets. Table 4 shows the average hit ratio of all cases and all cache sizes in random and related sequence of questions. Figure 8: Experimental results of C4. (**a**)—results of C2 with the random sequence; (**b**)—results of C2 with the mixed questions sequence; (**c**)—results of C2 with the related questions sequence; (**d**)—the overall results of C2. The results in the table show how the sequence of questions produces the difference in hit ratio.