# Selection of Features Based on Electric Power Quantities for Non-Intrusive Load Monitoring
## 7 Test and Results
### Load Classification Results  
Following the NCA suggestion, for each number of features, a bunch of training sessions was done varying the number of hidden neurons from 10 to 100 for an MLP, choosing as input the 8 top features. Results are reported in Table 4 for the network with 50 hidden neurons showing the best validation error. Figure 6 shows the confusion matrix. Since training, validation and test sets show similar performance, the confusion matrix is built for the entire dataset. In order to verify the goodness of the approach, the MLP classifier was trained varying the number of hidden neurons from 10 to 100. The network with 50 hidden neurons shows the minimum validation error, so it has been assumed as the best model. Figure 7 shows the performance in term of cross-entropy obtained by progressively entering the input features following the ranking provided by NCA algorithm. As it can be noticed, increasing the number of features initially improves model performance. The major improvement is obtained adding the fundamental active power. Then, the validation error fairly decreases, and best results are obtained when 8 features are selected. Further increasing the input dimension does not provide more accurate classifications. Thus, the test confirms the goodness of the choice suggested by the NCA algorithm. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Set** & **ce** & **TP** & **FP** & **FN** & **Precision** & **Recall** & **F-Score** \\ \hline Training & 0.05 & 1182 & 70 & 70 & 0.94 & 0.94 & 0.94 \\ Validation & 0.05 & 151 & 9 & 9 & 0.94 & 0.94 & 0.94 \\ Test & 0.08 & 157 & 16 & 16 & 0.91 & 0.91 & 0.91 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance of MLP trained with the top 8 features. Figure 6: Confusion matrix for the entire dataset. The model performance was compared with those shown when the MLP is fed with the fundamental active and reactive power \(\Delta\text{P}_{1}\) and \(\Delta\text{Q}_{1}\), widely applied in literature, also by the authors [30; 31], and adding to them the harmonic power that includes the information on the non-sinusoidal current. Table 5 reports the results for the described tests. Although the obtained performances are slightly worse than those achieved by the NMR top features, results confirm that relevant information is carried out by the fundamental and harmonic active power features in different time instants. Performance evaluation and comparison of different approaches to NILM remain open issues in literature because the authors test their techniques on different datasets, with different criteria and metrics. In this paper, in order to make a meaningful comparison, the techniques proposed in [22; 24] and [9] have been considered. In all three cases BLUED dataset is used and the performance is evaluated with the same indices considered in the present paper. In particular, in [22] the authors apply a complex multivariable hybrid energy disaggregation approach to a subset of BLUED database consisting of 24 h of monitoring. The study is limited to the following six devices: fridge, air compressor, hair dryer, backyards lights, bathroom upstairs light, bedroom lights for 107 events, obtaining a F-score equal to 91.8%. Table 6 reports the obtained performance for five individual appliances both for [22] and the present paper. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Input** & **Set** & **ce** & **TP** & **FP** & **FN** & **Precision** & **Recall** & **F-Score** \\ \hline \multirow{4}{*}{(3 rd and 4 th time instant)} & Training & 0.09 & 1138 & 114 & 114 & 0.91 & 0.91 & 0.91 \\  & Validation & 0.11 & 143 & 17 & 17 & 0.89 & 0.89 & 0.89 \\  & Test & 0.11 & 154 & 19 & 19 & 0.89 & 0.89 & 0.89 \\ \hline \multirow{4}{*}{\(\text{P}_{1},\text{Q}_{1},\text{P}_{\text{H}}\) (3 rd and 4 th time instant)} & Training & 0.06 & 1158 & 94 & 94 & 0.92 & 0.92 & 0.92 \\  & Validation & 0.06 & 148 & 12 & 12 & 0.925 & 0.93 & 0.93 \\ \cline{1-1}  & Test & 0.08 & 157 & 16 & 16 & 0.91 & 0.91 & 0.91 \\ \hline \hline \end{tabular}
\end{table}