# Effective Class-Imbalance Learning Based on SMOTE and Convolutional Neural Networks
## 3 Methodology
### Models
#### 3.2.2 Proposed Convolutional Neural Network  
Convolutional operations are the main components in Convolutional Neural Network (CNN)-based models. These operations enable CNNs to extract and learn the salient features existent in the input data [61]. CNN comprises different layers that output feature maps, resulting in sliding different kernels on the input and applying activation functions [62]. Compared with DNNs, the major advantage of CNNs over DNNs is their capability to reduce the computational cost in each layer. The convoluted features extracted by these models are compact representations of the input data, which can be further used in downstream tasks such as classification [63].  
In this paper, the second method proposed for the binary classification of the input data is a CNN-based model. The architecture of this model is demonstrated in Figure 10.  
As is seen in Figure 10, our proposed model consists of 4 layers (two 1-dimensional convolutional layers and two fully connected layers). After each hidden layer, a non-linear activation function (ReLU) is applied to the output. To make our training process more efficient, we have experimented with several loss functions, among which Focal Loss (FL) [64] claimed better supervision of the network. In fact, it was invented to address the issue of class imbalance. FL belongs to the cost-sensitive methods which were originally introduced in the case of object detection, where the imbalance between background and salient object is often frequent. FL is a modification to the cross-entropy loss in that during the training procedure; the neural network receives more cost for wrongly predicting complex training samples.  
Figure 9: The architecture of the proposed DNN-based classifier.  
More precisely, the cross-entropy loss function is among the most common loss functions in deep learning that originates from information theory. It is seemingly identical to the negative log-likelihood loss function, and for the binary classification problems, the binary cross-entropy loss function, denoted by \(I_{BCE}\) is as follows:  
\[l_{BCE}(y,\widetilde{y})=-(y\log(\widetilde{y})+(1-y)\log(1-\widetilde{y})) \tag{2}\]  
Here \(y\), \(\widetilde{y}\in\{0,1\}^{N}\), where \(N\) is the number of samples, \(\widetilde{y}\) is the predicted value, and \(y\) denotes the ground truth table.  
The problem with the cross-entropy loss function is that in the case of imbalance classification, the larger class overwhelms the loss by dominating the gradient [65]. Hence, to obtain the Focal loss function, one can simplify and rewrite Equation (2) in the following way:  
\[\text{CE}(\text{p},\text{y})=\left\{\begin{aligned} -\log(\text{p}),& \text{if }\text{y}=1\\ -\log(1-\text{p}),&\text{if }\text{y}=0\end{aligned}\right\} \tag{3}\]  
Name the probability of predicting the ground truth class \(p_{t}\) and define \(p_{t}\) as:  
\[\text{CE}\text{p}_{t}=\left\{\begin{aligned} \text{p},& \text{if }\text{y}=1\\ 1-\text{p},&\text{if }\text{y}=0\end{aligned}\right\} \tag{4}\]  
Therefore, \(l_{BCE}\) can be rewritten and simplified as:  
\[l_{BCE}(p,y)=CE(p_{t})=-\log(p_{t}) \tag{5}\]  
Finally, FL augments a modulating factor \(\alpha(1-p_{t})^{\gamma}\) to the binary class entropy loss function, where \(\gamma\) > 0 is a tunable focusing parameter which yields the following equation:  
\[FL(p_{t})=-\alpha_{t}(1-p_{t})^{\gamma}\log(p_{t}) \tag{6}\]  
Figure 10: The architecture of the proposed CNN-based classifier.