## 4 Results
### Evaluation Metrics  
The confusion matrix quantifies all classification outcomes of a model, providing an intuitive representation of classification accuracy. In this paper, electricity theft samples are considered as the positive class, while normal electricity usage samples are treated as the negative class. The definitions of true positive (TP), false positive (FP), false negative (FN), and true negative (TN) are summarized in Table 3.  
\begin{table}
\begin{tabular}{c c} \hline
**Attack Type** & **Modification** \\ \hline
1 & \(\hat{x}_{t}=\alpha_{t}x_{t},\alpha_{t}=random(0.2,0.8)\) \\ \hline
2 & \(\hat{x}_{t}=\max(x_{t}-\gamma,0),\gamma<\max(X)\) \\ \hline
3 & \(\hat{x}_{t}=\min(x_{t},\gamma),\gamma<\max(X)\) \\ \hline
4 & \(\hat{x}_{t}=\begin{cases}0,\quad t_{start}<t<t_{start}+\Delta\ t\\ x_{t},\quad otherwise\end{cases}\) \\ \hline
5 & \(\hat{x}_{t}=\tilde{\alpha}mean(X),\alpha=random(0.2,0.8)\) \\ \hline
6 & \(\hat{x}_{t}=x_{n-t}\) \\ \hline \end{tabular}
\end{table}
Table 2: Six types of FDI attacks.  
Figure 5: Example of daily electricity consumption and its tampered data corresponding to six types of FDI attacks.  
\begin{table}
\begin{tabular}{c c c} \hline
**Samples** & **Predicted Negative** & **Predicted Positive** \\ \hline Actually negative & TN & FP \\ Actually positive & FN & TP \\ \hline \end{tabular}
\end{table}
Table 3: Confusion matrix of electricity theft detection results.  
Based on the above confusion matrix, we use the \(F1\) score (\(F1\)), the area under the receiver operating characteristic (ROC) curve (AUC), recall, and false positive rate (\(FPR\)) to evaluate the performance of the proposed electricity theft detection framework.  
_Recall_, also known as true positive rate (\(TPR\)), evaluates a model's effectiveness in identifying all actual positive instances within a dataset, highlighting its sensitivity to true positives. It is calculated as follows.  
\[Recall=\frac{TP}{TP+FN} \tag{14}\]  
\(FPR\) quantifies the rate at which negative samples are misclassified as positive, reflecting the model's tendency for false alarms. It is computed as follows:  
\[FPR=\frac{FP}{FP+TN} \tag{15}\]  
In anomaly detection, we generally aim for a model that has a low \(FPR\) while maintaining a high TPR.  
_Precision_ quantifies the accuracy of positive predictions, calculated as follows:  
\[precision=\frac{TP}{TP+FP} \tag{16}\]  
\(F1\) captures the trade-off between _precision_ and _recall_, offering a single metric to evaluate performance, particularly in scenarios with imbalanced data. It is calculated as follows:  
\[F1=\frac{2\times precision\times recall}{precision+recall} \tag{17}\]  
AUC evaluates a model's capability to differentiate between classes over varying decision thresholds, with larger values reflecting stronger discriminative power.  
Model size reflects the storage space occupied by the model file. For devices with limited storage resources (such as smart meters), a smaller model size is important. In this article, model size refers to the size of the trained DNN model. The OC classifier (e.g., OCSVM/iForest/LOF) requires negligible additional storage.  
Training time refers to the time required for a model to complete a full training cycle on the training data. For models in electricity theft detection that require frequent updates or retraining, a shorter training time is important. In this article, the training time is the sum of the DNN training time and the OC classifier training time.  
Prediction speed reflects how many observations per second the model can process. For scenarios that require rapid response (such as electricity theft detection), a higher prediction speed is necessary.