## 3 Experimentation and Results

Table 5: Load recognition results obtained by the feed-forward ANN where it has been well trained and validated on the class-imbalanced training data. Figure 9: Training trajectory of the feed-forward, multilayer (1-15-12) ANN. In this experiment, more NIALM data instances, a total of 989 NIALM data instances, are sampled from the house environment and used to experimentally verify the performance of a feed-forward, multilayer ANN (Figure 5a) in terms of load recognition. A network configuration of 1-23-12 of a feed-forward, multilayer ANN was constructed, specified, and used. Also, the feed-forward, multilayer ANN was trained on 643 randomly sampled training data instances (~65% of the whole dataset) and tested on the remaining 346 test data instances. The training trajectory of the feed-forward, multilayer ANN is shown in Figure 10. The resulting MSE was 0.044 (the initial MSE was 1.110). The feed-forward, multilayer ANN can learn from the training data instances across all available CPU workers on the PC used. Table 8 tabulates the load recognition results obtained by the feed-forward, multilayer ANN where it has been well-trained and -validated. In Table 8, the \(F_{1}\) score is shown to each load class. Table 9 also tabulates the load recognition results obtained by the well-trained and -validated feed-forward, multilayer ANN applied on class-imbalanced test data. In Table 9, TPR vs. FPR is also shown for each load class. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Class** & **FPR** & **TPR** & **AUC** \\ \hline
1 & 0.29 & 0.77 & 0.74 \\ \hline
2 & 0.07 & 0.75 & 0.84 \\ \hline
3 & 0.20 & 0.74 & 0.77 \\ \hline
4 & 0.03 & 0.67 & 0.82 \\ \hline
5 & 0.02 & 1.00 & 0.99 \\ \hline
6 & 0.04 & 0.33 & 0.65 \\ \hline
7 & 0.08 & 0.92 & 0.92 \\ \hline
8 & 0.02 & 0.79 & 0.88 \\ \hline
9 & 0.23 & 0.94 & 0.86 \\ \hline
10 & 0.02 & 0.85 & 0.91 \\ \hline
11 & 0.03 & 0.67 & 0.82 \\ \hline
12 & 0.01 & 0.25 & 0.62 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Obtained TPRs vs. FPRs, for ROC curves, from the well trained and tested feed-forward, multilayer ANN applied on the class-imbalanced test data; for the 12 classes the AUCs obtained are also shown. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Electrical**} & \multicolumn{3}{c}{**CO [1]**} & \multicolumn{3}{c}{**The Presented GA-Based**} & \multicolumn{3}{c}{**The Presented ANN-Based**} \\  & \multicolumn{3}{c}{**NIALM**} & \multicolumn{3}{c}{**NIALM**} \\ \cline{2-9}  & **Precision** & **Recall** & \(F_{1}\) **Score** & **Precision** & **Recall** & \(F_{1}\) **Score** & **Precision** & **Recall** & \(F_{1}\) **Score** \\ \hline
**fridge** & 0.30 & 0.41 & 0.35 & 0.18 & 0.41 & 0.25 ** & 0.63 & 0.76 & 0.68 [FOOTNOTE:6]Footnote 6: Combinatorial optimization-based NIALM, which was implemented as a widget in the NILLMTK developed for the purpose of performing preliminary work for NIALM data preparation and providing a few load recognition approaches for load disaggregation. [FOOTNOTE:6]Footnote 6: Combinatorial optimization-based NIALM, which was implemented as a widget in the NILLMTK developed for the purpose of performing preliminary work for NIALM data preparation and providing a few load recognition approaches for load disaggregation. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Class** & **Precision** & **Recall** & \(\boldsymbol{F_{1}}\)** **Score** & **Number of Appliance Instances** \\ \hline
1 & 0.88 & 0.82 & 0.85 & 210 \\ \hline
2 & 0.88 & 0.98 & 0.93 & 45 \\ \hline
3 & 0.63 & 0.62 & 0.62 & 105 \\ \hline
4 & 0.67 & 0.91 & 0.77 & 11 \\ \hline
5 & 0.86 & 0.84 & 0.85 & 122 \\ \hline
6 & 1.00 & 0.90 & 0.95 & 89 \\ \hline
7 & 0.78 & 0.82 & 0.80 & 51 \\ \hline
8 & 0.33 & 0.60 & 0.43 & 5 \\ \hline
9 & 0.94 & 0.93 & 0.93 & 208 \\ \hline
10 & 0.67 & 0.71 & 0.69 & 17 \\ \hline
11 & 0.90 & 0.82 & 0.86 & 126 \\ \hline
12 & 0.00 & 0.00 & 0.00 & 0 \\ \hline Avg./total & 0.86 & 0.84 & 0.85 & 989 \\ \hline \hline \end{tabular}
\end{table}