  Speech Emotion Recognition (SER) task has known significant improvements over
the last years with the advent of Deep Neural Networks (DNNs). However, even
the most successful methods are still rather failing when adaptation to
specific speakers and scenarios is needed, inevitably leading to poorer
performances when compared to humans. In this paper, we present novel work
based on the idea of teaching the emotion recognition network about speaker
identity. Our system is a combination of two ACRNN classifiers respectively
dedicated to speaker and emotion recognition. The first informs the latter
through a Self Speaker Attention (SSA) mechanism that is shown to considerably
help to focus on emotional information of the speech signal. Experiments on
social attitudes database Att-HACK and IEMOCAP corpus demonstrate the
effectiveness of the proposed method and achieve the state-of-the-art
performance in terms of unweighted average recall.
