## 4 Experiment and Analysis
### Matching Electricity Consumption Patterns for New Customers  
This experiment aims to verify the effectiveness of the DistilBERT-based model in the task of user electricity consumption classification. Firstly, we used the above clustered labeled dataset for DistilBERT fine-tuning to build a downstream classification task model. Then, 15,000 new sample data points were selected from measurements that were not included in the aforementioned cluster analysis. Following data preprocessing, a test dataset for the classification model was created. Given the need to assess the classification performance of DistilBERT, the sampling period and user type of the test sample were known in this instance. In this paper, the pre-trained DistillBERT encoder was used with a six-layer Transformer encoder with 66 million parameters and a model size of 768. The initial parameters of the DistillBERT model are presented in Table 3.  
The classification of user electricity consumption behaviors was derived from a labeled dataset. The variation in classification accuracy was observed as network experiments with different initial learning rates and iteration counts, as depicted in Figure 12. Specifically, when set at an initial learning rate of 0.001, and a cycle count of 50 iterations, the loss trajectories for both the training and testing datasets demonstrate a convergence towards zero, indicating the completion of model training, as illustrated in Figure 13. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**JLDBH** & **YHBH** & **Jan** & **Feb** & **Mar** & **Nov** & **Dec** & **Pat** \\ \hline
4013 XX & 4014 XX & 4529.73 & 5195.29 & 3183.92 &... & 3573.03 & 4211.42 & 0 \\
4013 XX & 4014 XX & 1468.26 & 1697.26 & 1115.15 &... & 1215.15 & 1390.69 & 1 \\
4013 XX & 4014 XX & 2553.65 & 2876.26 & 1794.82 &... & 2117.70 & 2414.48 & 2 \\
4013 XX & 4014 XX & 181.69 & 233.27 & 192.47 &... & 129.21 & 167.91 & 3 \\
4013 XX & 4014 XX & 1839.56 & 2149.62 & 1375.06 &... & 1554.40 & 1790.82 & 4 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Electricity consumption characteristic dataset. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter Name** & **Default Value** \\ \hline vocab\_size & 30,522 \\ max\_position\_embeddings & 512 \\ num\_attention\_heads & 12 \\ num\_hidden\_layers & 6 \\ hidden\_size & 768 \\ intermediate\_size & 3072 \\ hidden\_act & sigmoid \\ Batch Size & 32 \\ Initial Learning Rate & 0.001 \\ Epochs & 50 \\ Dropout & 0.1 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: DistillBERT parameters. To further validate the effectiveness of the DistillBERT algorithms, we compared DistillBERT with baselines, including classical classification models and deep learning models. The results from each algorithm were recorded and compared with the annotated labels, and the accuracy, precision, recall, and F1 score for each algorithm were calculated. \[\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}} \tag{30}\]  
\[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{31}\]  
\[\text{F1}=2.\frac{\text{precision}\times\text{recall}}{\text{precision}+\text{ recall}} \tag{32}\]  
where TP stands for the number of true positives (accurate matches), FP indicates the number of false positives (incorrect matches), and FN denotes the count of false negatives (the number of unmatched matches that actually should have been matched). The evaluation values of KNN, SVM, CNN, LSTM, Transformer, and other partial classification models were compared. The comparative results are presented in Table 4.  
Figure 12: Variability in accuracy across different learning rates. Figure 13: Training set loss function curve. From the above analysis, it can be concluded that DistilBERT and Improved-C clustering can not only assign cluster labels to complex and irregular original electricity data, but also automatically categorize the electricity characteristics of new users, enabling us to analyze users' electricity behavior according to the existing large-scale power big data.