## 3 Proposed Methodology
### Feature Representation for Expression Classification
It performs linear operations with \(\mathcal{X}_{inputs}\) parameters and generates \(\mathcal{X}_{output}\) parameters [56] that are also connected to the next layer as inputs. It utilizes dense connections between layers with matching feature map size \(\mathcal{X}^{\prime}_{l}=g^{\prime}\Big{(}\mathbf{W}^{T}\mathcal{X}^{\prime} _{l-1}\Big{)}\), where \(g^{\prime}\) is the activation function, e.g., ReLU defined as \(p(x)=max(0,x)\). * _Batch Normalization:_- Batch is used to normalize the inputs of the previous layers at each batch, maintaining the values in a comparable range with the mean equal to 0 and the standard deviation equal to 1. This helps the CNN model to prevent skews at any one particular point and increases the computation speed. We applied the batch normalization after every convolution layer and then passed these values to the ReLU activation function. Batch normalization acts as a regularizer and allows the model to use higher learning rates [57]. It is used in various image classification problems and achieves higher accuracy with fewer training steps. Batch normalization also has a beneficial effect on the gradient flow through the network by reducing the dependence of gradients on the scale of their parameters or initial values. It also regularizes the model and reduces the need for dropout layers. We calculated the batch normalization mathematically as follows: For a a mini-batch \(\chi\) of size m and with values of \(x^{(l)}\), i.e., activation and omit \(l\) for clarity, the mini-batch is expressed as \(\chi=(x_{1...m})\). \(\theta\) and \(\psi\) are the learning parameters, \((\hat{x}_{1...m})\) are normalized values, and \(y_{1...m}\) are their corresponding linear transformations denoted by batch normalizing transform, i.e., \(\mathcal{BN}_{\theta,\psi}:x_{1...m}\to y_{1...m}\). Thus, consider the following: mini-batch mean, \(\underline{\cdot}=\frac{1}{m}\sum_{i=1}^{m}x_{i}\); mini-batch variance, \(\underline{\cdot}\in\frac{1}{m}\sum_{i=1}^{m}(x_{i}-\mu_{\chi})^{2}\); normalization, \(\hat{x}_{i}=\frac{x_{i}-\mu_{\chi}}{\sqrt{\underline{\cdot}\in\mathcal{C}_{i}}+g}\); and scale and shift, \(y_{i}\leftarrow\theta\hat{x}_{i}+\psi=\mathcal{BN}_{\theta,\psi}(x_{i})\). * _Regularization:_- Regularization strategies are designed to reduce the test error of a machine learning algorithm, possibly at the expense of the training error [58]. The popular regularization methods that exist in the field of deep learning [59] are dropout, R1-regularization, and discriminative regularization, among others. We employed the dropout regularization technique on the penultimate layer \(\alpha=[\bar{a}_{1},\bar{a}_{2},...,\bar{a}_{\text{F}}]\) (F are the numbers of filters) for our proposed deep CNN model with constrain:norms of the weight vector [60]. The dropout regularization technique drops a unit during the training time with a specified probability. Dropout prevents co-adaptation of the network's hidden units by randomly dropping out a portion or setting the hidden units to zero during forward and backward propagation. The neural network becomes too reliant on particular connections. Instead of using \(\gamma=\omega\times\alpha+\delta\) for output hidden unit \(\gamma\) in forward propagation, here, dropout uses \(\gamma=\omega\times(\alpha\otimes\beta)+\delta\), where the operator '\(\otimes\)' performs element-wise matrix multiplication and \(\beta\in\mathbf{R}^{\text{F}}\) is the masking vector of the Bernoulli random variable. At test time, all units are present and the learned weight vectors are scaled by \(\mathcal{P}\) such that \(\vec{\omega}=\mathcal{P}\times\omega\), where \(\vec{\omega}\) represents the class score computed without dropout. The advantage of using dropout is that it prevents artificial neural networks from over-fitting. Intuitively, dropout can be thought of as creating an implicit ensemble of neural networks. This means that a selected subset of units for each training sample, including their incoming and outgoing connections, are temporarily removed from the network.