Emotion recognition in conversations is crucial for the development of empathetic machines. Present methods mostly ignore the role of inter-speaker dependency relations while classifying emotions in conversations. In this paper, we address recognizing utterance-level emotions in dyadic conversational videos. We propose a deep neural framework, termed conversational memory network, which leverages contextual information from the conversation history. The framework takes a multimodal approach comprising audio, visual and textual features with gated recurrent units to model past utterances of each speaker into memories. Such memories are then merged using attention-based hops to capture inter-speaker dependencies. Experiments show an accuracy improvement of 3−4% over the state of the art.
Thus, REF used memory networks for emotion recognition in dyadic conversations, where two distinct memory networks enabled inter-speaker interaction, yielding state-of-the-art performance.
Conversational Memory Network for Emotion Recognition in Dyadic Dialogue Videos.
Proceedings of the conference. Association for Computational Linguistics. North American Chapter. Meeting
Proceedings of the conference. Association for Computational Linguistics. North American Chapter. Meeting
Medicine Computer Science