## 4 Feature Visualization Analysis Based on RDCA
### Convolution Filter and Extracted Features Visualization  
It is particularly crucial to have a reliable interpretation of the extracted features from the black box in RDCA. Moreover, from the perspective of PQDs diagnosis structure optimization, the interpretation can also contribute to design model expectations. In this work, we sought to establish the connection between PQD diagnosis and extracted features. Concretely, we take two methods to visualize the features in different layers from convolution filter and feature output aspects, namely gradient ascent and de-convolutional network (Deconvnet) [25,29]. The first method is gradient ascent [30], which aims at perturbing the input with noise and observing the change of RDCA prediction in every layer. The basic principle is applying gradient ascent on the input of the ConvNet so as to maximize the activation value of filter. Thus, a three-layer RDCA (RDCA-3) is built. The size of convolution filters are set to C1: 9 \(\times\) 9 \(\times\) 8, \(P1\): 2 \(\times\) 2, C2: 9 \(\times\) 9 \(\times\) 16, \(P2\): 2 \(\times\) 2, C3: 9 \(\times\) 9 \(\times\) 24, \(P3\): 2 \(\times\) 2. The corresponding size of output features per max-pooling layer are 64 \(\times\) 128 \(\times\) 8, 32 \(\times\) 64 \(\times\) 16, and 16 \(\times\) 32 \(\times\) 24, respectively. The effect of the filter on a certain disturbance image with the size of 128 \(\times\) 256 \(\times\) 1 is illustrated in Figure 5. Here we set the L13 as input of the RDCA-3. Two pictures of each convolutional layer are picked by setting the threshold value that has the maximum loss value. The orange line at the top of the image represents the amplitude of the fundamental component. The second layer has some preliminary and uncertain shapes. It shows that the texture is clearer in the third layer, and it can be regarded as multiple complex combinations of the former layers. In the CNN model, the lower the convolutional layers' number is, the closer the signal is to the original signal, which means no useful information has been learned. The more the convolutional layers' number is, the higher the feature order is and more abstract, and many typical local features can be extracted. Therefore, the features in Layer3 are some typical local features in L13, indicating that information has been learned. \begin{table}
\begin{tabular}{l c c c} \hline
**Label** & **Categories** & **Label** & **Categories** \\ \hline L1 & Normal & L9 & Voltage Spike \\ L2 & Voltage Swell & L10 & Swell with Harmonics \\ L3 & Voltage Sag & L11 & Flicker with Harmonics \\ L4 & Voltage Interrupt & L12 & Interrupt with Harmonics \\ L5 & Harmonics & L13 & Sag with Harmonics \\ L6 & Transient & L14 & Swell with Transient \\ L7 & Flicker & L15 & Sag with Transient \\ L8 & Voltage Notch & L16 & Spike with Transient \\ \hline \end{tabular}
\end{table} 
Table 1: The sixteen types of PQDs. The details of the L13 disturbance image become increasingly abstract and high level when the layer of RDCA goes deeper. Particularly, the harmonic component becomes blurred while the characteristics of the fundamental wave are retained in the third layer. Figure 5 manifests that the convolution operation has the benefit to extract edge information and amplitude information of PQDs through a combination of features. To further explore the feature information, we focus on the output of the feature maps in the encoder. Taking the single PQDs L1, L4, L5, L6, and L9, and combined PQDs L13, L14, and L15 as examples, the output features of the third layer in RDCA-3 are illustrated in Figure 6. The size of feature maps in the third layer is 16 \(\times\) 32. According to the comparison of automatically extracted features in Figure 6, it can be seen that different PQD signals have unique characteristics. For instance, it clearly shows that there is a distinct recessed area and a small block in class L4 and L6, respectively, these corresponding high-level features indicate that the signal is voltage interrupt and transient disturbances.