# Hybrid Oversampling and Undersampling Method (HOUM) via Safe-Level SMOTE and Support Vector Machine
## 3 Preliminaries
### Classification Techniques Used in the Present Study
#### 3.1.4 Artificial Neural Networks (ANNs)  
Artificial neural networks were first proposed by [PERSON] and [PERSON] [36]. The methodology of the artificial neural network is the mathematical modeling of the learning process that imitates the working principle of the human brain. In this way, the algorithm can perform fundamental functions like learning, remembering, and generating new information from existing data.  
The backpropagation algorithm [37] can be used for training data. The flow of the procedure is inspired by [PERSON]'s book [38]. The steps of the backpropagation algorithm as well as its index and parameters are summarized as follows:  
\(p\): number of instances (\(p=1\ldots n\)).  
\(m\): iteration number.  
\(A\): a small number to update \(m\).  
\(\mu\): learning rate parameter, \(\mu=A/m\).  
\(i\): number of features (\(i\) is 2 in Figure 2).  
\(l\): number of nodes in the hidden layer (h = 1...H) (\(H\) is 3 in Figure 2).  
\(W\)(\(i\),\(h\)): weights between input and hidden layer.  
\(X\)(\(h\)): weights of output from hidden layer.  
\(u_{p}(i)\): input value of \(i\)th feature for \(p\)th instance.  
\(b\): bias.  
\(Y\): output value.  
1. Initialization of Weights and Biases: Random initial values are assigned between 0 and 0.5 for \(W(i,h)\), \(X(h)\), and \(b\). This step is crucial as the network needs proper starting values to begin the training process.  
2. Forward Propagation:  
* Calculating \(Vp^{\prime}\)(h): \(Vp^{\prime}\)(h) is calculated for each hidden node and instance using Equation (9), which involves the weighted sum of inputs to the hidden nodes.  
\[Vp^{\prime}(h)=\sum_{i=1}^{l}W(i,h)*u_{p}(i) \tag{9}\]
* Applying the Logistic Function: Using the logistic function in Equation (10), \(Vp(h)\) is calculated for each instance and hidden node. The logistic function transforms the weighted sum into an output between 0 and 1, which is commonly used in neural networks for binary classification tasks.  
\[Vp(h)=\frac{1}{1+e^{-Vp^{\prime}(h)}} \tag{10}\]  
3. Calculating the Predicted Output: the predicted output value \(Op\) is computed for each instance using Equation (11), which involves the bias term and the weighted sum of hidden node outputs.  
\[Op=b+\sum_{h=1}^{H}X(h)Vp(h) \tag{11}\]  
4. Backpropagation (Updating Weights and Biases):  
* Updating the Bias (\(b\)): the bias term is adjusted based on the error between the predicted output and the actual output using Equation (12).
* Updating \(W(i,h)\) Weights: the weights between the input and hidden layer \(W(i,h)\) are updated using Equation (13).
* Updating \(X(h)\) Weights: the weights between the hidden and output layer \(X(h)\) are updated using Equation (14).  
\[b=b+\mu\sum_{p=1}^{n}(Yp-Op) \tag{12}\]  
\[W(i,h)=W(i,h)+\mu\sum_{p=1}^{n}(Yp-Op)X(h)Vp(h)(1-Vp(h))\;u_{p}(i) \tag{13}\]  
\[X(h)=X(h)+\mu\sum_{p=1}^{n}(Yp-Op)Vp(h) \tag{14}\]  
5. Error Computation: calculating the sum squared error (SSE), which is computed using Equation (15) to evaluate the network's performance.  
\[SSE=\sum_{p=1}^{n}(Yp-Op)^{2} \tag{15}\]  
6. Iteration and Termination: Steps 2-5 are repeated until a termination criterion is met, for example, when a small SSE value is achieved or the maximum number of iterations is reached. This iterative process fine-tunes the weights and biases of the network to minimize the error between predicted and actual outputs.  
This sequence aligns with the iterative nature of the backpropagation algorithm. It trains neural networks by updating weights and biases based on the calculated errors until the network learns to make accurate predictions.