## 2 Materials and Methods
### Feature Extraction
#### 2.2.2 Convolutional Autoencoder and Deep Features  
The convolutional autoencoder (CAE) presented in this section is designed to extract \"deep features\". These features are extracted automatically during the CAE training phase.  
Figure 1: Experimental setup.  
Deep features do not have physical meaning but are designed to minimize information loss of the input-output mapping of the CAE. Deep features (denoted as d1, d2,..., d6) were extracted from continuous wavelet transform (complex Morlet wavelets) scalograms using a convolutional autoencoder (CAE). Matrix-wise standardization was used in the pre-processing stage. Training data consisted of 44,598 AE signals, transformed into the aforementioned scalograms.  
The structure of the convolutional autoencoder for deep feature extraction is depicted in Figure 2. Model input is a 2-D matrix of size 48 \(\times\) 432, which is, before any computations, split into four stripes of 12 \(\times\) 432 matrices. The subsequent layer operations are performed separately and independently (without parameter sharing) on each of the stripes. Two convolutional layers (filter dimension 3 \(\times\) 5, ReLU activation and strides 1, 3) and 2 \(\times\) 2 max-pooling layers (C+P layers) with a stride of 2 were used. Then, all of the resulting output feature maps were concatenated back into one merged feature map. A single convolutional layer (filter dimension 3 \(\times\) 3, ReLU activation and strides 1, 1) and a 2 \(\times\) 2 max-pooling layer with a stride of 2 follow. The resulting 2-D feature map is flattened to 1-D and connected to three fully connected layers--FCL 1, FCL 2, and FCL 3. These layers complete the CAE's encoder used to extract deep features after the whole model (with decoder) has been trained. The number of neurons set in FCL 3 translates to the dimensionality of the feature vector, which encodes each scalogram as a specific series of numbers--deep features.  
Figure 2: Convolutional autoencoder (CAE) for deep feature extraction.  
The second part of the autoencoder (decoder) consists of layers that perform inverse operations and are also stacked in reverse order compared to those in the encoder. The carefully chosen input dimensions (scalograms) enable the layer hyperparameters of the encoder and decoder to be the same, while also fulfilling the model's input and output dimension matching requirement. The decoder consists of two fully connected layers (FCL 4 and FCL 5), a transposed convolution and upsampling layer (TC+U layers), a feature map splitting layer, two consecutive TC+U layers (in each of the resulting feature maps), and a final feature map concatenation layer.  
Various convolutional autoencoder architectures were designated by selecting the following configuration parameters:  
* Number of filters used in layers C+P (1) and TC+U (3),
* Number of filters used in layers C+P (2) and TC+U (2),
* Number of filters used in layers C+P (3) and TC+U (1),
* Number of neurons in fully connected layer FCL 2,
* Number of neurons in fully connected layer FCL 3,
* Number of training epochs,
* Batch size of training samples.  
Different CAE configurations are denoted as \"s1-s2-s3-s4-s5-s6-s7\" and the results for selected configurations are summarized in the Results section. Model training was performed on an NVIDIA GPU GeForce GTX 750 Ti, and lasted for each configuration up to 2 h. Configuration parameter s4 has, by far, the greatest impact on the model's complexity. The loss function used was squared error and for the optimization algorithm, ADAM [26] was chosen.