# LSTM and Bat-Based RUSBoost Approach for Electricity Theft Detection
## 4 Simulation Results and Discussion
### Performance of LSTM-RUSBoost Model for ETD  
In this section, we describe the problems of the conventional models described in Section 2; i.e., the class imbalance problem, overfitting issue, the handling of missing values in the dataset and lack of parameter optimization during classification. Firstly, we present the performance of our proposed model on raw data. Initially, the missing values in the dataset are filled with the interpolation method. Additionally, the data are normalized using the min-max scaling method. The visualization of the data in Figure 5 shows the imbalanced distribution of labels; i.e., the number of thieving users is represented by \"1\", while \"0\" shows honest customers. The consequences of using imbalanced data are that it trains the classifier on a majority dataset, which results in the classifier being biased towards identifying fraudulent electricity consumers. In this regard, the RUSBoost method is effective, as already described in Section 3.4. It achieves the benefits of RUS and adaptive boosting techniques. It is efficient in dealing with data imbalance problems. In order to show the effects of imbalanced data, we also implement SMOTE along with the SVM technique. The authors in [22; 23] used SMOTE to solve the data imbalance problem. In the SMOTE technique, the samples of the minority class are increased by taking the information from their nearest neighbor. This gives rise to the overfitting problem, and more execution time is required to run the process. The training complexity of SVM is highly dependent on the input data. Figure 6 shows that, due to training with imbalanced data, the performance of SVM is poor and it fails to classify the fraud users successfully. The model is trained on a balanced dataset after SMOTE is applied to generate synthetic data. Although the SMOTE-SVM method shows better results in comparison with SVM only, SMOTE adds synthetic data samples to the minority class, and the synthetic data do not represent real theft cases. Comparatively, our proposed method performs better for the imbalanced dataset in terms of all performance metrics. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Hyper-Parameters** & **Range of Values** & **Values** \\ \hline C & 0.001, 0.01, 0.1 & 0.1 \\ \hline R & 11 norm, 12 norm & 12 norm \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Hyper-parameter values of LR. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Hyper-Parameters** & **LSTM Values** & **CNN Values** \\ \hline Batch size & 50 & 130 \\ \hline Dropout & 0.2 & 0.01 \\ \hline Optimizer & Adam & Adam \\ \hline Epochs & 20 & 40 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Hyper-parameter values of CNN and LSTM. During model validation, the hyper-parameters of RUSBoost are tuned to find optimal values. The hyper-parameters are the learning rate, estimator and sampling strategy. We optimize the learning rate of RUSBoost. Setting the hyper-parameters manually is a time-consuming task. The bat algorithm is efficient for finding the optimal value of the learning rate. The results in Figure 7 show that parameter tuning enhanced the performance of our proposed model in terms of all performance metrics. Table 9 shows the confusion matrix of the true negative (TN), true positive (TP), false negative (FN) and false positive (FP) rates of our proposed technique. Our goal in ETD is to maximize the TPR and minimize the FPR. The confusion matrix shows good results for our proposed model. It achieves a high percentage of TPR with a low FPR. In order to perform a reliable evaluation, we also evaluate our model wit the ROC curve. The ROC curve for our proposed model is shown in Figure 8. It is obtained by plotting the TPR against the FPR. To train and test data, ROC covers a large area under the curve, which shows that the model has accurately predicted the theft cases. Moreover, as mentioned in Section 3, the existing models [19; 20; 21; 22; 23; 24; 25] tend to overfit when dealing with imbalanced data.