## 4 Experiment and Discussion
### Designing and Evaluating Experimental Models  
Firstly, we deployed a baseline model based on BERT2 BERT, which is widely chosen as a baseline model in text generation tasks [43]. We then designed different experimental models to demonstrate the effectiveness of using different contexts for response generation in the conversational model. To this end, we designed three models as follows:  
* **BERT LC (only left-context based on BERT)** This model addresses the shortcomings of traditional generative models that only generate responses based on the current sentence without considering information from previous conversation turns. It does so by reusing historical information to predict responses, exploiting contextual information from prior turns in the conversation. * **BERT RC (only right-context with BERT and reinforcement learning)** Encoder-decoder architectures have been proven to yield good results in text generation tasks. However, they still lack the ability to process information by considering its impact on the future. To address this limitation, this approach leverages the strengths of the BERT2 BERT method to capture the semantic information of a previous conversation and combines it with the power of reinforcement learning algorithms to manage the information flow for future conversations. * **BERT FC (full context with BERT and reinforcement learning)** This model combines the left-context and right-context to take advantage of them. The left-context allowsthe model to leverage the information already passed in the conversation. At the same time, the right-context helps the model capture the influence of the information flow on future conversations. We experimented with different context sizes (i.e., different numbers of utterances on the left and right sides of the context) to determine the most appropriate context length for these data. Our best model will then be compared to the latest relevant studies. Results are presented in Table 2. From these results, we have the following observations:  
* We presented the BLEU and ROUGE scores to measure conversation generation for correlation with human judgment. We showed that three of our models improved the BLEU score compared to the baseline model. However, the difference in BLEU score between the baseline and the proposed model (with only left-context) is slight. BERT is a state-of-the-art language model that uses a transformer-based architecture to learn contextual relationships between words in a text corpus. One of the main advantages of BERT is its ability to integrate contextual information into its understanding of text. Therefore, adding more contextual information to it may be difficult to further enhance the model's effectiveness. * When comparing the use of a single context, we found that the left-context-based model improved the average BLEU score by 5% and the average ROUGE score by 7% compared to the baseline. In contrast, the right-context-based model correspondingly achieved 8% BLEU score and 10% ROUGE score. However, using both the left-context and the right-context, we also find that the RL-based models generate more coherent outputs when compared to the baseline model. Specifically, the model improved 12% average BLEU score and 13% average ROUGE score. Figure 5 visually represents the scores for the four experimental models we trained. In general, using contextual information, either left-context or right-context can help the model better capture the mutual information among utterances in conversation. Among the four experiments, the proposed model with full context achieved the highest BLEU and ROUGE scores because they not only consider history information but also capture information in future utterances. In addition, we evaluated our proposed model based on the length of the simulated conversation with 1, 3, 5, and 7 turns (as shown in Table 3). These results are visually represented in Figures 6 and 7. When compared with the baseline, our best model still increased the average BLEU score by 24% and the average ROUGE score by 29%.