## 1 Introduction  
In what [PERSON] et al. [1] name the _\"common view\"_, certain emotion categories are reliably signaled or revealed by specific configurations of facial-muscle movements. The latter are referred to as emotional expressions and facial expressions. We will not argue here on the plausibility of this widespread assumption, which pervades both research and the development of commercial applications; such ongoing and passionate debate is out of the scope of this paper (see [1] for details). Yet, whatever the agreement with the pronouncements of this common view, it is surprising to notice how a straightforward fact has been hitherto overlooked to some extent: the facial apparatus, aside from being used in communicating affect, is much the same involved in articulation when subjects are speaking. This is allegedly occurring in spontaneous interactions and as a matter of fact, even a neutral expression of a speaking subject may be confused with emotional expressions [2]. This phenomenon was first described by [PERSON] in [3] who coined the term _\"speaking effect\"_. As an illustration of the argument, Figure 1 depicts an expression of _surprise_ from the CK+ [4] dataset and a frame from the RAVDESS [5] dataset of a speaking subject expressing the _neutralexpression and pronouncing the word _\"talking\"_. The action unit (AU) activation intensities are also shown, produced (by using the OpenFace toolbox [6]) according to the Facial Action Coding System (FACS) [7], a taxonomy of human facial movements by their appearance on the face. AUs 1, 2 and 5, belonging to the brow and eye areas of the face are strongly activated only in the true expression of _surprise_. However, AUs 25 and 26 which belong to the lower orofacial area are activated in both cases with similar intensities, corroborating that speech articulation may take the semblance of emotional expressions. Limited research efforts have been devoted to the problem, though the _speaking effect_ has gained attention by virtue of the growing efforts toward detecting affective states from a variety of modalities, cogently visual and auditory. Thus, work has been done on how to compensate for it and hence improve recognition accuracy. Indeed, such an effect is one of the hindrances in replicating accuracies of the acted datasets on spontaneous interactions. Our chief concern here is the following research question. How can we account for the _speaking effect_ in the design of an expression recognition system, rather than filtering it out as noise [8; 9; 10], in order to improve the system performance? Specifically, we attack the problem of whether and under what circumstances adding to a deep recognition model input features related to the speech articulation process will increase emotion recognition accuracy in speaking subjects. To the best of our knowledge, deep learning has not been explicitly applied to such effect, while generally it is widely applied to affective facial expression recognition. This question was addressed from an empirical standpoint. To said end we developed two deep learning neural network models and tested their performances first on facial features only; subsequently, both facial and speech articulation features were combined. The models were furthermore trained multiple times on varying numbers of consecutive frames to determine the dependence between the temporal context and the accuracy increase. Our main finding is that of a statistically significant increase in accuracy when features related to speech articulation are added in input, the increase being greater with more consecutive frames given in input. In the next section we provide the necessary background and review the literature on the subject. In Section 3 we then describe the experimental setup, models, data and parameters used. Finally, in Section 4 we present the results, and in Section 5 the conclusions, respectively. Figure 1: (**Left**) An expression of _surprise_ from the CK+ [4] dataset.