## 1 Introduction  
Conversational agents aim to enable computers to communicate with humans by automatically generating a response to each user input. Researchers have been working on developing robust, scalable, and context-aware systems for a long time. One of the key challenges they are focused on is generating meaningful and consistent dialogue responses that align with the conversation history, which this paper also addresses. While previous studies have indicated that incorporating additional information can enhance the accuracy of models, it is still uncertain how contextual information impacts a conversation as a whole, including its connection to future outcomes and how different contextual factors affect the conversation. Our goal is to describe and understand how contextual factors contribute to model performance improvement. In this study, we examine, explore, and exploit the contexts humans use in everyday decision making. Based on the principle that the more information the better, we have also presented different generation-based models that utilize context information in a conversation. This study proposes a Deep Reinforcement Learning model for analyzing the influence of different contextual information on responses in a conversation and how to combine them to improve a conversation's coherence and consistency. Existing chatbot systems can be classified into rule-based, information retrieval-based, and generation-based systems. Rule-based methods rely on a knowledge base of pre-defined question-answer pairs. When an input is received, the system matches it to the most similar question pattern in the knowledge base and retrieves the corresponding answer [1]. However, these predefined rules take time to expand and maintain. Additionally, these systems respond to user questions in a stereotyped manner without utilizing the conversation history. Another method based on the Information Retrieval (IR) approach is available [2; 3; 4]. This approach is suitable when a large set of conversations is available. The principle of IR-based models is to select the corresponding response based on a given input from a dataset (containing a set of conversation or question-answer pairs). However, a disadvantage of IR-based models is their limited ability to understand semantic differences between input contexts. Additionally, IR-based models can only respond within the dataset, making them suitable for query-type lookups rather than forming a conversation. Recently, generation-based approaches have attracted much attention in NLP, especially in machine translation and chatbots. These approaches treat a conversation as a source-to-target problem. Seq2 Seq models based on LSTM have shown excellent performance in generating responses, such as in [5; 6; 7]. The strength of this approach lies in its ability to generate answers in a generalized way that is not dependent on predetermined rules. The mechanism of this method mimics human thinking, including two stages: the first stage of encoding (i.e., understanding) the question and the second stage of generating the corresponding answer. Therefore, most current research continues this approach and focuses on developing deep neural models to improve the quality of conversations, which is also the approach in our work. However, most Seq2 Seq-based models are trained on single-turn conversations, making them incapable of handling long-context conversations. Additionally, due to numerous generic responses in the training dataset, Seq2 Seq models tend to generate generic responses regardless of the input [8; 9; 10]. In reality, humans use contextual reasoning to make daily decisions. Context refers to the collection of texts surrounding a word used in a sentence or phrase of interest. Some studies have used previous utterances to generate current ones or reinforcement learning methods to keep track of the conversation's history, such as [11; 12]. However, these studies typically only considered partially observed context when generating responses at the current position.