## 6 Testing Results  
### Testing of the Detection Techniques Using Simulated Signals  
Simulations of the six different PQDs were made using the MATLAB Simulink model shown in Figure 11. In total, 300 simulations of each PQD were generated to create the dataset, each one containing different start times and duration of the disturbance. This signals were stacked together in a cell array along with its corresponding labels. Data augmentation was applied as explained in Section 4.2. The cell array was shuffled and divided into 75% for training and 25% for testing. This architecture had a training time of 671 min or 11.18 h with an accuracy of 79.14% for the validation. Figure 8: Experimental setup for power quality disturbances generation. Figure 7: LSTM-CNN architecture block diagram. The confusion matrix of the LSTM for training and validation is shown in Figure 9a,b along with the testing in Figure 9c. This architecture had a training time of 443 min or 7.38 h with an accuracy of 84.58% for the validation. The confusion matrix of the CNN for training and validation is shown in Figure 10a,b along with the testing in Figure 10c. This architecture had a training time of 751 min or 12.52 h with an accuracy of 83.66% for the validation. The confusion matrix of the CNN-LSTM for training and validation is shown in Figure 11a,b along with the testing in Figure 11c. This architecture had a training time of 51 min or 0.85 h with an accuracy of 84.78% for the validation. The confusion matrix of the CNN-LSTM for training and validation is shown in Figure 12a,b along with the testing shown in Figure 12c. On each of the confusion matrix of the compared architectures, the precision and the recall was calculated. The precision of a classifier is defined as the number of retrieved relevant items as a proportion of the number of retrieved items for any given class [32]. In other words, it is the ratio between the positive identifications that are actually correct and the entire set of positive identifications of any given class. Recall, on the other hand, is defined as the number of retrieved relevant items as a proportion of all the relevant items, for any given retrieved set [32]. In other words, the proportion of the actual positives that where identified correctly. The comparison of precision and recall of different architectures are shown in Figure 13. The precision and recall of the LSTM-CNN with adjusted hyperparameters had superior results with respect to the other architectures due to the fact that it had better scores for classifying the transient in both training and testing. The LSTM training gave an accuracy of 79.14% where most of the problems were found in the Transient disturbance as shown in the precision and recall plots. The architecture was not able to detect the transient disturbance either in the training or the validation signals, that is, the LSTM classified the Transient signal as a No Fault in 100% of the cases. Concerning the other classes it resulted in 10.2%-14.6% misclassification. The CNN needs to be fed with an image which, in this case, represents the spectral components and amplitude of the signal of interest; hence a STFT was done on the augmented dataset. The training of the CNN gave an accuracy of 84.58% which is an improvement with respect to the LSTM. The problem with this architecture is that it classified 89% of the No Fault as a Transient from the training dataset and an 88.6% from the validation. Again, it showed confusion between the two classes as shown in the precision and recall plots. As regards to the the hybrid CNN-LSTM, two similar strategies were tested. The first strategy consisted in joining the LSTM and CNN that were used in the previous experiment and the second was using the combined architectures while adjusting the hyperparameters. The first strategy, exploiting the CNN-LSTM, resulted in an improvement of the classification performances of almost all of the the disturbances except for the Transient which resulted in 100% misclassification as with the LSTM.