## 3 Experiments and Results
### Performance Analysis
#### 3.2.5 Test Scenario-5  
A more detailed performance analysis of the device classification model based on the current shapelets is carried out using confusion matrices calculated for the applied classifiers. From Tables 1-4, it can be implied that the best classification results are obtained for the setup: \(p=1,m=130\) & \(i_{max}=15\). Therefore, the confusion matrices for the classifiers are calculated for this setup. Tables 5-9 show the evaluated confusion matrix for ensemble, binary DT, discriminant analysis, naive Bayes and \(k\)-NN, respectively. From Table 5, it can be observed that ensemble classifies drill, fan, hair dryer, hedge trimmer, lamp, planer and router perfectly. However, ensemble partially misclassifies grinder with saw, paint stripper with drill, sander with vacuum cleaner, saw with grinder & hedge trimmer and vacuum cleaner with sander. The overall performance is significantly good. Table 6 shows that binary DT classifies drill, fan, hair dryer, lamp, paint stripper and planer perfectly. However, it partially misclassifies grinder with hedge trimmer & saw, hedge trimmer with saw, router with drill, sander with vacuum cleaner, saw with grinder, hedge trimmer & sander and vacuum cleaner with grinder, sander & saw. The overall performance is considerably good. Table 7 presents that discriminant analysis classifies fan, paint stripper and planer perfectly. However, to some extents it partially misclassifies drill with planer, router & saw, grinder with drill, hedge trimmer & saw, hair dryer with lamp, hedge trimmer with paint stripper, saw & vacuum cleaner, lamp with hedge trimmer, router with saw, sander with grinder, saw with drill, fan, hedge trimmer, paint stripper, router & vacuum cleaner and vacuum cleaner with grinder, sander & saw. The overall performance is poor and can not be considered as an efficient NILM solution. From Table 8, it can be noticed that naive Bayes classifies fan, hair dryer, hedge trimmer, planer and router perfectly. However, it does not recognize drill and saw to any extent; while it partially misclassifies grinder with hedge trimmer, lamp with hair dryer, paint stripper with grinder, sander with grinder and vacuum cleaner with drill, grinder & sander. The overall performance is very poor and can not be considered for an NILM application. From Table 9, it can be observed that \(k\)-NN classifies paint stripper, planer, sander and vacuum cleaner perfectly. However, it partially misclassifies drill with router & saw, fan with lamp, grinder with drill & saw, hair dryer with lamp, hedge trimmer with saw, lamp with hair dryer, router with drill, saw with grinder, hedge trimmer, router & sander. The overall performance is pretty good. \begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline
**Length** & \multicolumn{2}{c}{**Ensemble**} & \multicolumn{2}{c}{**DT**} & \multicolumn{2}{c}{**DA**} & \multicolumn{2}{c}{**NB**} & \multicolumn{2}{c}{\(k\)**-NN**} \\ \hline  & **Acc. (\%)** & **Loss** & **Acc. (\%)** & **Loss** & **Acc. (\%)** & **Loss** & **Acc. (\%)** & **Loss** & **Acc. (\%)** & **Loss** \\ \hline
80 & 88.09 & 0.12 & 83.81 & 0.14 & 56.65 & 0.30 & 43.77 & 0.55 & 81.85 & 0.11 \\ \hline
90 & 89.06 & 0.09 & 85.14 & 0.16 & 60.03 & 0.31 & 44.09 & 0.55 & 82.24 & 0.13 \\ \hline
100 & 89.05 & 0.08 & 87.97 & 0.12 & 63.00 & 0.31 & 44.22 & 0.54 & 83.81 & 0.10 \\ \hline