## 4 Experiment and Result
### Performance Metrics  
The consistent evaluation of the result is mentioned as a difficult challenge in the literature on NILM [14; 15; 16]. Moreover, accurate estimates of appliances power consumption patterns are helpful in identifying peak hours, quantifying the contribution of each appliance during the peak and off-peak hours, and assess the efficiency levels of different appliances, which help to design demand management strategies that aim to improve energy efficiency and reduce costs through demand peak shifting and replacing low-efficiency devices.  
In our paper, a set of three metrics, F-score, power contribution error, and R2 score, is used to assess the quality of identifying and disaggregation result of our ILDACP method. The metrics are picked from others' literature, because it is a complete evaluation with necessary estimation characteristics involved [16].  
F-score (Fs) provides the capability of NILM algorithms to identify the appliances' operating states for each appliance \(i\), which is defined in the following Equation (10)  
\[Fs_{i}=\frac{2\times PC_{i}\times RC_{i}}{PC_{i}+RC_{i}} \tag{10}\]  
where \(PC_{i}\) and \(RC_{i}\) are the recall and precision for appliance \(i\). \(PC_{i}\) and \(RC_{i}\) are described in Equation (11).  
\[\left\{\begin{array}{l}RC_{i}=\frac{TP_{i}}{TP_{i}+FN_{i}}\\ PC_{i}=\frac{TP_{i}}{TP_{i}+FP_{i}}\end{array}\right. \tag{11}\]  
Table 1 shows the operating state prediction of appliances. According to Table 1, \(TP_{i}\) is the number of examples that are correctly labeled as ON state (true positive), \(FP_{i}\) corresponds the OFF state examples incorrectly labeled as ON state (false positive), and \(FN_{i}\) represents the ones classified as OFF being actually ON (false negative).  
Based on above description, F-score ranges from 0 (0% accuracy in state detection) to 1 (100% accuracy in state detection). It evaluates how good the algorithm is in identifying the operating states of each appliance. However, it does not give the information about the power consumption of each appliance \(i\).  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**State** & **Actual ON** & **Actual OFF** \\ \hline Prediction ON & TP & FP \\ Prediction OFF & FN & TN \\ \hline \hline \end{tabular}
\end{table}
Table 1: F-score parameters description.TP: true positive; FP: false positive; FN: false negative; TN: true negative.  
The power contribution error (PCE) shows the goodness of NILM models to estimate the power consumption assigned to each appliance. It gives the information on the accuracy in assigning the total power consumption to each appliance \(i\), according to the Equation (12). The closer the estimated contribution of appliance \(i\) is to the true contribution, the closer \(PCE_{i}\) is to 0.  
\[PCE_{i}=\frac{\left|\sum_{t=1}^{H}y_{t}^{i}-\sum_{t=1}^{H}\hat{y}_{t}^{i} \right|}{\sum_{t=1}^{H}P_{t}} \tag{12}\]  
where \(y_{t}^{i}\) and \(\hat{y}_{t}^{i}\) are the measured true and estimated power consumption for appliance \(i\) in time \(t\); \(P_{t}\) is the total power consumption of all the appliance in time \(t\). The more accurate NILM model is, the closer to 0 the PCE values are. The PCE provides more information than F-score to describe the estimation of power contribution of appliances. The R2 score evaluates the accuracy of power trace retrieve. For each appliance \(i\), it is defined as  
\[R2_{i}=1-\frac{\left(\sum_{t=1}^{H}y_{t}^{i}-\sum_{t=1}^{H}\hat{y}_{t}^{i} \right)^{2}}{\sum_{t=1}^{H}\left(y_{t}^{i}-\sum_{t=1}^{H}y_{t}^{i}/H\right)^{ 2}} \tag{13}\]  
The more similar the estimated consumption trajectory of the appliance \(i\) is to the measured consumption trajectory, the closer \(R2_{i}\) is to 1.  
This set of metrics assesses the performance of NILM algorithms in different viewpoints, and the values that were calculated by these metrics offer electric utilities and decision makers the reference to help designing energy demand management strategies.  
Figure 4: _Cont._