Sentiment Analysis and Emotion Detection in conversation is key in a number of real-world applications, with different applications leveraging different kinds of data to be able to achieve reasonably accurate predictions. Multimodal Emotion Detection and Sentiment Analysis can be particularly useful  as applications will be able to use specific subsets of the available modalities, as per their available data, to be able to produce relevant predictions . Current systems dealing with Multimodal functionality fail to leverage and capture  the context of the conversation through all modalities, the current speaker and listener(s) in the conversation , and the relevance and relationship between the available modalities through an adequate fusion mechanism . In this paper, we propose a recurrent neural network architecture that attempts to take into account all the mentioned drawbacks , and keeps track of the context of the conversation, interlocutor states, and the emotions conveyed by the speakers in the conversation . Our proposed model out performs the state of the art on two benchmark datasets on a variety of accuracy and regression metrics.
Sentiment Analysis and Emotion Detection in conversation is key in several real-world applications, with an increase in modalities available aiding a better understanding of the underlying emotions. Multi-modal Emotion Detection and Sentiment Analysis can be particularly useful , as applications will be able to use specific subsets of available modalities, as per the available data . Current systems dealing with Multi-modal functionality fail to leverage and capture - the context of the conversation through all modalities, the dependency between the listener(s) and speaker emotional states , and the relevance and relationship between the available modalities . In this paper, we propose an end to end RNN architecture that attempts to take into account all the mentioned drawbacks . Our proposed model , at the time of writing, out-performs the state of the art on a benchmark dataset on a variety of accuracy and regression metrics.
R
a number of
several
R
different applications leveraging different kinds of data to be able to achieve reasonably accurate predictions. Multimodal
an increase in modalities available aiding a better understanding of the underlying emotions. Multi-modal
A
,
D
the
R
their available data, to be able to produce relevant predictions
the available data
R
Multimodal
Multi-modal
A
-
R
current speaker and
dependency between the
R
in the conversation
and speaker emotional states
D
through an adequate fusion mechanism
R
a recurrent neural network
an end to end RNN
D
, and keeps track of the context of the conversation, interlocutor states, and the emotions conveyed by the speakers in the conversation
R
out performs the
, at the time of writing, out-performs the
R
two benchmark datasets
a benchmark dataset