## 4 Methodology
### The Architecture  
The proposed architecture for classifying human emotions consists of the Convolutional Neural Network - Recurrent Neural Network (CNN-RNN) model. The CNN-RNN models were first proposed for image-captioning applications [50], where the CNN models are used to extract visual features from images followed by the recurrent units such as the gated recurrent unit (GRU) [51] or long short term memory (LSTM) [52] to generate captions. The model we proposed comprises two InceptionResNet [53] inspired CNN modules built on a bidirectional RNN [54], consisting of a bidirectional-GRU with 128 units (see Figure 3). The feature map created by the CNN module was concatenated with the raw signals through a connection of a \(1\times 1\) convolutional layer. To keep the spatial dimensions identical, we used \(1\times 1\) convolution operations. A total of 64, \(1\times 1\) kernels were applied to the raw sensor data. This concatenation was inspired by DenseNet [55], which is an image classification model. The benefit of InceptionNet [56] is that it has variable-sized kernels that enable it to detect features of different sizes. Similarly, the ResNet [57], or having a residual connection, lets the model learn the identity; thus, the model is capable of bypassing a module if it does not contribute to classification. For the sequential data, RNNs perform better; therefore, the feature map generated by the CNN modules is given to the bidirectional GRU. The motivation behind introducing the dense connection was that, if CNNs are of no benefit, the raw sensor data are still processed by RNNs resulting in a better classification accuracy. To the best of our knowledge, this type of dense connection, where a feature map generated by CNNs is concatenated with the raw signals, has never before been experimented with for any kind of classification problem. In our experimentation, we found that this significantly improves the performance of the entire model. Figure 3 shows the overall proposed architecture and presents the InceptionResNet modules that are incorporated into the model.  
The proposed model has five different kernels, i.e., \(1\times 1\), \(1\times 3\), \(1\times 5\), \(1\times 7\), and \(1\times 9\). The input is convolved with 64, \(1\times 1\) kernels before applying 10 kernels of sizes \(1\times 3\), \(1\times 5\), \(1\times 7\), and \(1\times 9\). This reduces the overall complexity of the model as pointed out in the existing literature [58]. We tested it by adding different-sized kernels sequentially, which improves the performance significantly. The selection of the number of kernels and their spatial dimensionswere based on empirical results (see Section 5.3). Moreover, the addition of a max-pool also improves performance. The feature map generated by the kernels and max-pool was then concatenated. The final output was then passed to 64 kernels of \(1\times 1\) convolutions to change the spatial dimensions. The input feature map was also passed through 64, \(1\times 1\) kernels, and feature maps generated from both were then added. The overall architecture has three such modules in it, which were selected by empirical analysis.  
The batch normalization and dropout were set to 0.25, and were applied after each module and to the feature map generated by the Bi-GRU units. This significantly reduces the training time. The feature vector generated by Bi-GRU was then reshaped and flattened before passing it to a fully connected layer of 512 neurons. The number of neurons in the dense layer was also selected empirically. A dropout of 0.25 was then applied for the sake of regularization.