Abstract: Emotional language generation is one of the keys to human-like artificial intelligence. Humans use different type of emotions depending on the situation of the conversation. Emotions also play an important role in mediating the engagement level with conversational partners. However, current conversational agents do not effectively account for emotional content in the language generation process. To address this problem, we develop a language modeling approach that generates affective content when the dialogue is situated in a given context. We use the recently released Empathetic-Dialogues corpus to build our models. Through detailed experiments, we find that our approach outperforms the state-of-the-art method on the perplexity metric by about 5 points and achieves a higher BLEU metric score.

Introduction: Rapid advancement in the field of generative modeling through the use of neural networks has helped advance the creation of more intelligent conversational agents. Traditionally these conversational agents are built using seq2seq framework that is widely used in the field of machine translation BIBREF0. However, prior research has shown that engaging with these agents produces dull and generic responses whilst also being inconsistent with the emotional tone of conversation BIBREF0, BIBREF1. These issues also affect engagement with the conversational agent, that leads to short conversations BIBREF2. Apart from producing engaging responses, understanding the situation and producing the right emotional response to a that situation is another desirable trait BIBREF3. Emotions are intrinsic to humans and help in creation of a more engaging conversation BIBREF4. Recent work has focused on approaches towards incorporating emotion in conversational agents BIBREF5, BIBREF6, BIBREF7, BIBREF8, however these approaches are focused towards seq2seq task. We approach this problem of emotional generation as a form of transfer learning, using large pretrained language models. These language models, including BERT, GPT-2 and XL-Net, have helped achieve state of the art across several natural language understanding tasks BIBREF9, BIBREF10, BIBREF11. However, their success in language modeling tasks have been inconsistent BIBREF12. In our approach, we use these pretrained language models as the base model and perform transfer learning to fine-tune and condition these models on a given emotion. This helps towards producing more emotionally relevant responses for a given situation. In contrast, the work done by Rashkin et al. BIBREF3 also uses large pretrained models but their approach is from the perspective of seq2seq task. Our work advances the field of conversational agents by applying the transfer learning approach towards generating emotionally relevant responses that is grounded on emotion and situational context. We find that our fine-tuning based approach outperforms the current state of the art approach on the automated metrics of the BLEU and perplexity. We also show that transfer learning approach helps produce well crafted responses on smaller dialogue corpus.

Question: What is the state-of-the-art approach?
Answer: For particular Empathetic-Dialogues corpus released Raskin et al. is state of the art (as well as the baseline) approach. Two terms are used interchangeably in the paper.