# Deep Dense and Convolutional Autoencoders for Machine Acoustic Anomaly Detection

Gabriel Coelho, Pedro Pereira, Luis Matos, Alexandrine Ribeiro, Eduardo C. Nunes, André Ferreira, Paulo Cortez, André Pilastri

<figure><img src="image_1.png" /></figure>

To cite this version:

Gabriel Coelho, Pedro Pereira, Luis Matos, Alexandrine Ribeiro, Eduardo C. Nunes, et al. Deep Dense and Convolutional Autoencoders for Machine Acoustic Anomaly Detection. 17th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2021, Crete, Greece. pp.337-348, 10.1007/978-3-030-79150-6_27. hal-03287664

HAL Id: hal-03287664

https://inria.hal.science/hal-03287664

Submitted on 15 Jul 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Deep Dense and Convolutional Autoencoders for Machine Acoustic Anomaly Detection

Gabriel Coelho<fnref n="1" />[0000-0001-8352-2637], Pedro Pereira<fnref n="1" />[0000-0002-6169-8778], Luis Matos<fnref n="1" />[0000-0001-5827-9129], Alexandrine Ribeiro<fnref n="3" />, Eduardo C. Nunes<fnref n="1" />, André Ferreira<fnref n="2" />, Paulo Cortez<fnref n="1" />[0000-0002-7991-2090], and André Pilastri<fnref n="3" />[0000-0002-4380-3220]

**Abstract.** Recently, there have been advances in using unsupervised learning methods for Acoustic Anomaly Detection (AAD). In this paper, we propose an improved version of two deep AutoEncoders (AE) for unsupervised AAD for six types of working machines, namely Dense and Convolutional AEs. A large set of computational experiments was held, showing that the two proposed deep autoencoders, when combined with a mel-spectrogram sound preprocessing, are quite competitive and outperform a recently proposed AE baseline. Overall, a high-quality class discrimination level was achieved, ranging from 72% to 92%.

**Keywords:** Acoustic anomaly detection · Unsupervised learning · Autoencoders · Convolutional neural network.

## 1 Introduction

With the advent of the Industry 4.0 phenomenon, the amount of digital data is growing exponentially. In effect, currently there is a widespread usage of interconnected sensors that can capture diverse physical aspects of the productive process (e.g., images, sound, temperatures, torque, energy consumption values). All this data can be used by Artificial Intelligence (AI) and Machine Learning (ML) to extract valuable productive analytics. A particularly relevant ML task is anomaly detection, which intends to distinguish abnormal events from normal ones [18,34]. In industrial processes, the early detection of operating machines with a defects by using ML can potentially [25, 31]: reduce maintenance time and costs; prevent or reduce production stops, and increase the safety of human operators that operate the machines. In this work, we focus on ML methods for Acoustic Anomaly Detection (AAD) [8], which aims to detect abnormal

<footnote n="1">¹ ALGORITMI Centre, Dep. Information Systems, University of Minho, Guimarães, Portugal
a82137,id6927,id6929@alunos.uminho.pt, pcortez@dsi.uminho.pt</footnote>

<footnote n="2">² Bosch Car Multimedia, Braga, Portugal,
andre.ferreira2@pt.bosch.com</footnote>

<footnote n="3">³ EPMQ - IT Engineering Maturity and Quality Lab, CCG ZGDV Institute, Guimarães, Portugal
andre.pilastri@ccg.pt</footnote>
behaviours using audio data. In particular, we aim to automatically detect, be-
forehand, if a given industrial machine is not working correctly, by using only the
sound produced by it. Several studies addressed this issue as an unsupervised
ML task, since data labeling is highly costly and time consuming, requiring great
manual human work subject to errors [4].

Over the years, several algorithms were applied to unsupervised AAD prob-
lems, including Isolation Forest (IF) [9, 10] and One-Class Support Vector Ma-
chines (OCSVM) [4,29]. Following the success of Deep Learning, there has been
a growing usage of neural network architectures for AAD. In particular, Au-
toEncoders (AE) are becoming popular for unsupervised AAD [15, 24]. When
compared with other ML approaches (e.g., IF and OCSVM), AE present the
advantage of requiring a lower computational effort [19].

AEs compressed the input features into a lower dimensional space, named
latent space, learning their most relevant relationships, and are composed by two
main components [5,22]: an encoder that maps the input vector (features) into
the latent space, via a nonlinear transformation; and a decoder that attempts to
reconstruct the reverse transformation to the original input signal. The difference
between the original input vector and the AE output is called reconstruction
error [3]. This error can be used to detect anomalies. AEs assume that normal
and anomalous events follow different distributions and it is trained to learn the
normal multi-dimensional space of the data, by using only normal event records,
aiming to minimize the reconstruction error on such data. When an AE tries
to reconstruct new unseen data containing anomalies, the reconstruction errors
are higher and by using a predefined threshold, the samples can be signaled as
anomalous [5,22].

Following on good results obtained in previous studies [14, 18, 26, 32], in this
work we address unsupervised AAD task in industrial machines using two dif-
ferent AE architectures: deep Dense and Convolutional. In order to use audio as
input, it is often necessary to preprocess the raw data by extracting features from
the signal. In this work, we use Mel Frequency Energy Coefficients (MFECs),
which are a popular sound preprocessing method [7,28]. Moreover, we use two
public datasets [17, 27] to test the proposed AEs that are fed with MFECs.
For benchmark purposes, we compare the Dense and Convolutional AEs with a
baseline AE architecture that was recently proposed [16].

The paper is organized as follows: Section 2 describes the used datasets, the
audio features used and its extraction process, the proposed AE architectures,
and the evaluation process. Section 3 presents the experimentation developed
and obtained results. Lastly, final conclusions are discussed in Section 4.

2 Materials And Methods

2.1 Dataset

The data used for this task comprises parts of the ToyADMOS [17] and the
MIMII [27] datasets, consisting of the normal and anomalous operating sounds of
six types of toy/real machines, as obtained from the DCASE 2020 challenge [16]. The data is divided into two datasets (development and evaluation) for 6 different machine types: ToyCar, ToyConveyor, Slider, Pump, Fan, and Valve.

The ToyCar and ToyConveyor data belong to ToyADMOS dataset. This dataset involved miniature machines (toys) that were damaged deliberately to record anomalous behavior. As for the MIMII Dataset, the sounds were recorded from different industrial machines, aiming to resemble a real-life scenario. In the development datasets, each machine type has 4 different specific machines, except for ToyConveyor, which has only 3. Each machine sound was recorded using only one microphone and sampled at 16 kHz.

The machine sound datasets include normal and anomaly labels that are available for the test data, allowing to estimate the AAD performance of the ML models. Regarding the evaluation data, it contains audio for new machines (new IDs) in each machine type, both for model training and testing. Table 1 summarizes the analyzed datasets. A different number of approximately 10 second Waveform Audio File (WAV) files is used for each machine.

<table><caption>**Table 1.** Summary of the machine AAD datasets.</caption><thead><tr><th rowspan="3"></th><th colspan="3">Development</th><th colspan="3">Evaluation</th></tr><tr><th colspan="3">Machine Audio Files</th><th colspan="3">Machine Audio Files</th></tr><tr><th>ID</th><th>Train</th><th>Test</th><th>ID</th><th>Train</th><th>Test</th></tr></thead><tbody><tr><td rowspan="4">Toy Car</td><td>01</td><td>1000</td><td>614</td><td>05</td><td>1000</td><td>515</td></tr><tr><td>02</td><td>1000</td><td>615</td><td>06</td><td>1000</td><td>515</td></tr><tr><td>03</td><td>1000</td><td>615</td><td rowspan="2">07</td><td rowspan="2">100</td><td rowspan="2">515</td></tr><tr><td>04</td><td>1000</td><td>615</td></tr><tr><td rowspan="3">Toy Conveyor</td><td>01</td><td>1000</td><td>1200</td><td>04</td><td>1000</td><td>555</td></tr><tr><td>02</td><td>1000</td><td>1155</td><td>05</td><td>1000</td><td>555</td></tr><tr><td>03</td><td>1000</td><td>1154</td><td>06</td><td>1000</td><td>555</td></tr><tr><td rowspan="4">Fan</td><td>00</td><td>911</td><td>507</td><td>01</td><td>911</td><td>426</td></tr><tr><td>02</td><td>916</td><td>549</td><td>03</td><td>916</td><td>458</td></tr><tr><td>04</td><td>933</td><td>448</td><td rowspan="2">05</td><td rowspan="2">1000</td><td rowspan="2">458</td></tr><tr><td>06</td><td>915</td><td>461</td></tr><tr><td rowspan="4">Pump</td><td>00</td><td>906</td><td>243</td><td>01</td><td>903</td><td>2016</td></tr><tr><td>02</td><td>905</td><td>211</td><td>03</td><td>606</td><td>213</td></tr><tr><td>04</td><td>602</td><td>200</td><td rowspan="2">05</td><td rowspan="2">908</td><td rowspan="2">348</td></tr><tr><td>06</td><td>936</td><td>202</td></tr><tr><td rowspan="4">Slider</td><td>00</td><td>968</td><td>456</td><td>01</td><td>968</td><td>278</td></tr><tr><td>02</td><td>968</td><td>367</td><td>03</td><td>968</td><td>278</td></tr><tr><td>04</td><td>434</td><td>278</td><td rowspan="2">05</td><td rowspan="2">434</td><td rowspan="2">278</td></tr><tr><td>06</td><td>434</td><td>189</td></tr><tr><td rowspan="4">Valve</td><td>00</td><td>891</td><td>219</td><td>01</td><td>679</td><td>220</td></tr><tr><td>02</td><td>608</td><td>220</td><td>03</td><td>863</td><td>220</td></tr><tr><td>04</td><td>900</td><td>220</td><td rowspan="2">05</td><td rowspan="2">899</td><td rowspan="2">500</td></tr><tr><td>06</td><td>892</td><td>220</td></tr></tbody></table>
## 2.2 Feature Extraction

MFCCs, which are derived from the mel-cepstrum representation of the audio, is one of the best known and most popular audio processing features [30]. However, when computing MFCCs, a Discrete Cosine Transform (DCT) is applied to the logarithm of the filter bank outputs, resulting in decorrelated MFCC features. Therefore, they have the drawback of having non-local features, which makes them unsuitable for Convolutional AE (CAE) processing.

In this paper, we address a feature for audio signal processing named MFECs, which are log-energies derived directly from the filter-banks energies. These are similar to MFCCs, yet they do not include the DCT operation. This feature provided good results in detecting different audio sounds and classification of sounds in previous studies [2, 13, 33].

To prepare the features for the first proposed deep learning architecture, the Dense AE, some operations were made. Audio data are buffered in fixed-length 1-second intervals with 50% overlap. For each audio buffer obtained, the segment is then divided into 64 ms analysis frames, with 50% overlap and 128 MFECs are extracted from the magnitude spectrum of each frame. In this way, 5 time-frames are concatenated to form a 640-dimensional input vector as shown in Figure 1.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Feature extraction procedure for the Dense AE.</figcaption></figure>

The second deep learning architecture, the Convolution Neural Network (CNN) AE, requires a different feature extraction method. For each audio, 128 log mel-band energy features were extracted from the magnitude spectrum, con-
sidering 64 ms analysis frames with 50% overlap. Then, each feature was nor-
malized to zero mean and unit standard deviation by using statistics from the
training data. Finally, the mel spectrogram was segmented every second into 32
column data, with approximately 100 ms of hop size. This procedure is shown
in Figure 2.

<figure><img src="image_4.png" /><figcaption>Fig. 2. Feature extraction procedure for the CNN AE.</figcaption></figure>

## 2.3 Autoencoder Architectures

The two proposed AEs contain a large number of hyperparameters. In order to select the best architectures, we have first conducted several preliminary experiments, in which we only used development data, selecting the best configuration (in terms of the reconstruction error) when varying element such as the number of hidden layers and units per layer. Once the neural architecture was selected, it was fixed and applied to all datasets. For both AEs, the training only uses normal machine sounds.

The first proposed architecture consists of a deep fully-connected AE (top
of Figure 3), which was adopted in the baseline AE proposed in [16]. The best
preliminary results were achieved by a Dense AE that includes encoder and decoder components with four fully-connected layers with 512 hidden units, followed by Batch Normalization, with all neural nodes using the popular ReLU as activation function. The Batch Normalization layer allows reduce the internal covariate shift, discarding the need of dropout, and normalizes the inputs for each batch of data [12]. As for ReLU, it presents the advantage of non-saturation of its gradient, which greatly accelerates the convergence of stochastic gradient descent compared to other activation functions, including logistic or hiperbolic tangent [20]. The bottleneck layer is set as one fully-connected layer with 8 hidden units, resulting in an 8-dimensional latent space. To train the AE only normal event audio was used, aiming to learn the data normal event distribution. Turning to the loss function, we adopted the popular Mean Squared Error (MSE), which is more sensitive to extreme errors and that is computed as:

$$MSE_i = \frac{\sum_{k=1}^{I} (x_{i,k} - \hat{x}_{i,k})^2}{I} \quad (1)$$

where i denotes a data instance, $x_{i,k}$ the k-th input value for instance i, $\hat{x}_{i,k}$ the AE predicted output response for the same input and I the total number of inputs of the AE.

Recently, CNNs have achieved promising results on many AAD benchmarks [6, 11, 21]. By integrating 2D convolutional operations in an AE structure, CNN AEs are capable of learning the spatial structure of the input features and reconstruct them while taking into account their spatial structural patterns. Based on this property, the second proposed deep learning architecture for the unsupervised AAD task consists of a deep CNN AE (shown in the bottom of Figure 3). With such an architecture, the AAD task is handled as a computer vision problem by exploring image-like time-frequency representations of audio. The encoder and decoder networks are comprised of convolutional blocks, each consisting of 2D Convolution and Batch Normalization layers, using ReLU as the activation function. The encoder network is composed by a stack of five convolutional layers with 32, 64, 128, 256, and 512 convolutional filters, kernel sizes of 5, 5, 5, 3, and 3 to capture local patterns, and strides of (1, 2), (1, 2), (2, 2), (2, 2), and (2, 2), respectively. The feature size map is reduced throughout the encoder by the convolution operation stride. The bottleneck consists of a layer with 40 convolutional filters, reducing the encoder feature maps to a 40-dimensional compressed input representation. Concerning the decoder network, it starts with a fully-connected layer that increases the latent space dimensionality, equalizing encoder last layer's shape, followed by five 2D transposed convolutional layers that mirror the encoder layers.

Regarding the training algorithm used to train both Dense AE and CNN AE architectures, we employed the Adam optimizer, which also was used in [16], using a learning rate of 0.001. Both AE were trained to minimize MSE between input and its reconstruction (the loss function). The training procedure was iterated up to a maximum of 100 epochs. In each epoch, 10% of training data is randomly divided for validation, which is used for evaluating training process
<figure><img src="image_5.png" /><figcaption>Fig. 3. Proposed AE Network architectures: Dense AE (top) and CNN AE (bottom).</figcaption></figure>

evolution, by computing the reconstruction error on such data. If MSE does not improve on validation data after 10 epochs, an early stopping callback is activated, ending the training process and storing the weights of the model that achieved a lower reconstruction error on validation data. The batch size for both Dense AE and CNN AE architectures was set as 512 and 64, respectively.

Once the AE is trained, the reconstruction error for an unseen sound sample $j$ is used as the decision score ($d_j$), where $d_j = MSE_j$. An anomaly class label is considered true if $d_j > Th$, where $Th$ is a decision threshold.

## 2.4 Evaluation

We evaluated our methods, we used two popular metrics on AAD that are based on the Receiver Operating Characteristic (ROC) analysis [17, 27]: Area Under the ROC Curve (AUC) and partial-AUC (pAUC). The ROC curve shows the False Positive Rate (FPR) versus the True Positive Rate (TPR) for different threshold values ($Th$). In this study, the positive class is the anomaly.
AUC represents the overall ML discrimination performance, while pAUC focuses on a particular range of interest from the ROC curve, defined in this work as the FPR values from 0 to 0.1, which reflects in a model with fewer false alarms. Both metrics are not influenced by unbalanced data, which in occurs in our datasets. The AUC and pAUC values can be interpreted as follows: 50% performance of a random classifier; 60% - reasonable; 70% - good; 80% - very good; 90% - excellent; and 100% - perfect.

As mentioned in Section 2.1, the datasets used in this work contain a total of 6 types of machines, most of them containing 4 specific machine data, except ToyConveyor that contains 3. We used development data to select two first fix AE architectures and then train the models for each specific machine. As for the predictive results, they are measured on the test sets from the evaluation datasets. A single model was created for each machine type and evaluated over each specific machine, using both AUC and pAUC.

## 3 Results

The proposed dense and CNN AE architectures were implemented in the Python programming language, using the TensorFlow-GPU library [1]. The computational experiments were conducted using two different GPUs (Titan Xp and 1080Ti). To evaluate the model performance, both AUC and pAUC metrics were used, as defined in Section 2.4. Table 2 presents the obtained predictive results for each specific machine, also showing the average value for each machine type. For comparison purposes, the Baseline system results from [16] are also provided in the table.

In terms of the average AUC and pAUC values for each machine type, the Dense AE outperforms the Baseline system in all machine types. Furthermore, the Baseline system only achieved better results in 5 of the 23 analyzed specific machines, namely ToyCar ID 3, pump (IDs of 0, 2, and 4) and slider ID 0. Regarding the CNN AE overall performance, it outperformed the Baseline system, although the latter achieved a higher average AUC values for 2 of 6 machine types (ToyCar and pump).

The two proposed AE architectures<fnref n="4" /> are quite competitive in terms of mean AUC and pAUC values, outperforming the Baseline system for all machine types. The Dense AE obtains the best average AUC and pAUC results for ToyCar, ToyConveyor and fan, while the CNN AE achieves the best AAD measures for the slider and valve tasks. Turning to the pump machine, the best AUC value is provided by the Dense AE and the best pAUC is returned by the CNN AE. In particular, when considering the AUC measure, a high quality anomaly class discrimination was achieved by the proposed AEs, since most AUC values are above 70%. Furthermore, the CNN AE architecture obtained excellent results for slider machine type, presenting the highest AUC value (91.77%).

<footnote n="4">⁴ https://github.com/APILASTRI/DCASE_Task2_UMINHO</footnote>
<table><caption>**Table 2.** Comparison of AUC and pAUC results for all AE architectures for each machine (best average values are denoted in **bold**)</caption><thead><tr><th rowspan="2">Machine Type</th><th rowspan="2">Machine ID</th><th colspan="2">Baseline</th><th colspan="2">Dense AE</th><th colspan="2">CNN AE</th></tr><tr><th>AUC (%)</th><th>pAUC (%)</th><th>AUC (%)</th><th>pAUC (%)</th><th>AUC (%)</th><th>pAUC (%)</th></tr></thead><tbody><tr><td rowspan="4">ToyCar</td><td>1</td><td>81.36</td><td>68.40</td><td>83.87</td><td>72.64</td><td>81.59</td><td>71.88</td></tr><tr><td>2</td><td>85.97</td><td>77.72</td><td>87.56</td><td>80.35</td><td>85.46</td><td>79.92</td></tr><tr><td>3</td><td>63.30</td><td>55.21</td><td>63.12</td><td>55.02</td><td>62.73</td><td>55.08</td></tr><tr><td>4</td><td>84.45</td><td>68.97</td><td>88.60</td><td>76.68</td><td>82.38</td><td>69.60</td></tr><tr><td colspan="2">Average</td><td>78.77</td><td>67.58</td><td><strong>80.79</strong></td><td><strong>71.17</strong></td><td>78.04</td><td>69.12</td></tr><tr><td rowspan="4">ToyConveyor</td><td>1</td><td>78.07</td><td>64.25</td><td>81.67</td><td>69.41</td><td>79.90</td><td>62.71</td></tr><tr><td>2</td><td>64.16</td><td>56.01</td><td>68.04</td><td>58.31</td><td>67.78</td><td>54.85</td></tr><tr><td>3</td><td>75.35</td><td>61.03</td><td>79.59</td><td>63.64</td><td>80.11</td><td>62.53</td></tr><tr><td>4</td><td>75.35</td><td>61.03</td><td>79.59</td><td>63.64</td><td>80.11</td><td>62.53</td></tr><tr><td colspan="2">Average</td><td>72.53</td><td>60.43</td><td><strong>76.43</strong></td><td><strong>63.79</strong></td><td>75.93</td><td>60.03</td></tr><tr><td rowspan="4">fan</td><td>0</td><td>54.41</td><td>49.37</td><td>56.73</td><td>49.72</td><td>51.77</td><td>49.05</td></tr><tr><td>2</td><td>73.40</td><td>54.81</td><td>79.60</td><td>54.00</td><td>72.71</td><td>55.51</td></tr><tr><td>4</td><td>61.61</td><td>53.26</td><td>70.11</td><td>54.11</td><td>62.60</td><td>52.80</td></tr><tr><td>6</td><td>73.92</td><td>52.35</td><td>81.69</td><td>55.15</td><td>80.05</td><td>53.19</td></tr><tr><td colspan="2">Average</td><td>65.83</td><td>52.45</td><td><strong>72.03</strong></td><td><strong>53.25</strong></td><td>66.78</td><td>52.63</td></tr><tr><td rowspan="4">pump</td><td>0</td><td>67.15</td><td>56.74</td><td>66.94</td><td>56.83</td><td>66.37</td><td>54.95</td></tr><tr><td>2</td><td>61.53</td><td>58.10</td><td>60.77</td><td>60.31</td><td>54.31</td><td>53.58</td></tr><tr><td>4</td><td>88.33</td><td>67.10</td><td>87.00</td><td>66.32</td><td>94.64</td><td>77.26</td></tr><tr><td>6</td><td>74.55</td><td>58.02</td><td>77.53</td><td>60.32</td><td>76.97</td><td>58.05</td></tr><tr><td colspan="2">Average</td><td>72.89</td><td>59.99</td><td><strong>73.06</strong></td><td><strong>60.94</strong></td><td>72.07</td><td><strong>60.96</strong></td></tr><tr><td rowspan="4">slider</td><td>0</td><td>96.19</td><td>81.44</td><td>96.12</td><td>82.30</td><td>98.86</td><td>94.47</td></tr><tr><td>2</td><td>78.97</td><td>63.68</td><td>79.55</td><td>64.42</td><td>84.06</td><td>69.33</td></tr><tr><td>4</td><td>94.30</td><td>71.98</td><td>95.44</td><td>76.14</td><td>97.69</td><td>87.82</td></tr><tr><td>6</td><td>69.59</td><td>49.02</td><td>77.22</td><td>49.56</td><td>86.46</td><td>53.16</td></tr><tr><td colspan="2">Average</td><td>84.76</td><td>66.53</td><td>87.08</td><td>68.10</td><td><strong>91.77</strong></td><td><strong>76.20</strong></td></tr><tr><td rowspan="4">valve</td><td>0</td><td>68.76</td><td>51.70</td><td>74.61</td><td>52.28</td><td>78.69</td><td>52.59</td></tr><tr><td>2</td><td>68.18</td><td>51.83</td><td>76.68</td><td>52.72</td><td>85.02</td><td>55.92</td></tr><tr><td>4</td><td>74.30</td><td>51.97</td><td>79.58</td><td>50.96</td><td>82.59</td><td>53.68</td></tr><tr><td>6</td><td>53.90</td><td>48.43</td><td>57.78</td><td>48.73</td><td>69.03</td><td>50.22</td></tr><tr><td colspan="2">Average</td><td>66.28</td><td>50.98</td><td>72.16</td><td>51.17</td><td><strong>78.83</strong></td><td><strong>53.10</strong></td></tr></tbody></table>

## 4 Conclusions

In this paper, we proposed two AutoEncoder (AE) deep learning architectures for an unsupervised Acoustic Anomaly Detection (AAD) task: a Dense AE and a Convolutional Neural Network (CNN) AE. The two AE architectures were applied to six different real-world industrial machine sound datasets. Using development records from the datasets and sound energy features from mel-spectrograms to preprocess the raw sounds, several preliminary experiments were conducted in order to tune the AE hyperparameters, namely in terms of hidden layers and nodes and activation functions. Then, the selected AE architectures were trained and tested using the evaluation instances from the public domain datasets.

Overall, competitive results were obtained by the Dense and CNN AEs when compared with a recently proposed baseline AE architecture [16]. For two machine types (slider and valve), the best results were achieved by the CNN AE, while the Dense AE provided the best results for the remaining machines (ToyCar, ToyConveyor, fan, and pump). In general, a high anomaly class discrimi-
nation was achieved by both proposed AEs, ranging from 72% (good) to 92% (excellent discrimination level).

As future work, we aim to explore different deep learning architectures for AAD, such as Variational AEs [23]. Furthermore, we intend to study the effect of using audio data augmentation techniques (e.g., pitching, time-shifting, Generative Adversarial Networks) or signal frequency filtering tools, aiming to further improve the AAD results.

## 5 Acknowledgments

This work is supported by the European Structural and Investment Funds in the FEDER component, through the Operational Competitiveness and Internationalization Programme (COMPETE 2020) - Project n 039334; Funding Reference: POCI-01-0247-FEDER-039334.

## References

1. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mane, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viegas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X.: Tensorflow: Large-scale machine learning on heterogeneous distributed systems (2016)

2. Afrillia, Y., Mawengkang, H., Ramli, M., Fadlisyah, Fhonna, R.P.: Performance measurement OfMel frequency cepstral coefficient(MFCC) method in learning system of al- qur’an based InNaghamPattern recognition. Journal of Physics: Conference Series **930**, 012036 (dec 2017). https://doi.org/10.1088/1742-6596/930/1/012036, https://doi.org/10.1088/1742-6596/930/1/012036

3. An, J., Cho, S.: Variational autoencoder based anomaly detection using reconstruction probability. Special Lecture on IE **2**(1), 1–18 (2015)

4. Aurino, F., Folla, M., Gargiulo, F., Moscato, V., Picariello, A., Sansone, C.: One-class svm based approach for detecting anomalous audio events. In: 2014 International Conference on Intelligent Networking and Collaborative Systems. pp. 145–151. IEEE (2014)

5. Charte, D., Charte, F., Garca, S., del Jesus, M.J., Herrera, F.: A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines. Information Fusion **44**, 7896 (Nov 2018). https://doi.org/10.1016/j.inffus.2017.12.007, http://dx.doi.org/10.1016/j.inffus.2017.12.007

6. Chen, C., Yuan, W., Xie, Y., Qu, Y., Tao, Y., Song, H., Ma, L.: Novelty detection via non-adversarial generative network. arXiv preprint arXiv:2002.00522 (2020)

7. Chu, S., Narayanan, S., Kuo, C.C.J.: Environmental sound recognition with time-frequency audio features. Audio, Speech, and Language Processing, IEEE Transactions on **17**, 1142 – 1158 (09 2009). https://doi.org/10.1109/TASL.2009.2017438
8. Duman, T.B., Bayram, B., İnce, G.: Acoustic anomaly detection using convolutional autoencoders in industrial processes. In: International Workshop on Soft Computing Models in Industrial and Environmental Applications. pp. 432–442. Springer (2019)

9. Farzad, A., Gulliver, T.A.: Unsupervised log message anomaly detection. ICT Express 6(3), 229–237 (2020)

10. Harar, P., Galaz, Z., Alonso-Hernandez, J.B., Mekyska, J., Burget, R., Smekal, Z.: Towards robust voice pathology detection. Neural Computing and Applications pp. 1–11 (2018)

11. Hershey, S., Chaudhuri, S., Ellis, D.P., Gemmeke, J.F., Jansen, A., Moore, R.C., Plakal, M., Platt, D., Saurous, R.A., Seybold, B., et al.: Cnn architectures for large-scale audio classification. In: 2017 ieee international conference on acoustics, speech and signal processing (icassp). pp. 131–135. IEEE (2017)

12. Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning. pp. 448–456. PMLR (2015)

13. Jam, M.M., Sadjedi, H.: Identification of hearing disorder by multi-band entropy cepstrum extraction from infant's cry. In: 2009 International Conference on Biomedical and Pharmaceutical Engineering. pp. 1–5 (2009)

14. Kawaguchi, Y., Endo, T.: How can we detect anomalies from subsampled audio signals? In: 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP). pp. 1–6. IEEE (2017)

15. Kohlsdorf, D., Herzing, D., Starner, T.: An auto encoder for audio dolphin communication. In: 2020 International Joint Conference on Neural Networks (IJCNN). pp. 1–7. IEEE (2020)

16. Koizumi, Y., Kawaguchi, Y., Imoto, K., Nakamura, T., Nikaido, Y., Tanabe, R., Purohit, H., Suefusa, K., Endo, T., Yasuda, M., Harada, N.: Description and discussion on DCASE2020 challenge task2: Unsupervised anomalous sound detection for machine condition monitoring. CoRR abs/2006.05822 (2020)

17. Koizumi, Y., Saito, S., Uematsu, H., Harada, N., Imoto, K.: Toyadmos: A dataset of miniature-machine operating sounds for anomalous sound detection. In: 2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WAS-PAA). pp. 313–317. IEEE (2019), https://ieeexplore.ieee.org/document/8937164

18. Koizumi, Y., Saito, S., Uematsu, H., Kawachi, Y., Harada, N.: Unsupervised detection of anomalous sound based on deep learning and the neyman-pearson lemma. IEEE/ACM Transactions on Audio, Speech, and Language Processing 27(1), 212–224 (2018)

19. Koizumi, Y., Saito, S., Yamaguchi, M., Murata, S., Harada, N.: Batch uniformization for minimizing maximum anomaly score of dnn-based anomaly detection in sounds (2019)

20. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25, 1097–1105 (2012)

21. Li, J., Dai, W., Metze, F., Qu, S., Das, S.: A comparison of deep learning methods for environmental sound detection. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 126–130. IEEE (2017)

22. Liu, Y., Zhuang, C., Lu, F.: Unsupervised two-stage anomaly detection (2021)

23. Ntalampiras, S., Potamitis, I.: Acoustic detection of unknown bird species and individuals. CAAI Transactions on Intelligence Technology (2021). https://doi.org/10.1049/cit2.12007
24. Oh, D.Y., Yun, I.D.: Residual error based anomaly detection using auto-encoder in smd machine sound. Sensors **18**(5), 1308 (2018)

25. Panfilenko, D., Poller, P., Sonntag, D., Zillner, S., Schneider, M.: Bpmn for knowledge acquisition and anomaly handling in cps for smart factories. In: 2016 IEEE 21st International Conference on Emerging Technologies and Factory Automation (ETFA). pp. 1-4. IEEE (2016)

26. Provotar, O.I., Linder, Y.M., Veres, M.M.: Unsupervised anomaly detection in time series using lstm-based autoencoders. In: 2019 IEEE International Conference on Advanced Trends in Information Theory (ATIT). pp. 513-517. IEEE (2019)

27. Purohit, H., Tanabe, R., Ichige, T., Endo, T., Nikaido, Y., Suefusa, K., Kawaguchi, Y.: MIMII Dataset: Sound dataset for malfunctioning industrial machine investigation and inspection. In: Proceedings of the Detection and Classification of Acoustic Scenes and Events 2019 Workshop (DCASE2019). pp. 209-213 (November 2019)

28. Purwins, H., Li, B., Virtanen, T., Schlüter, J., Chang, S., Sainath, T.N.: Deep learning for audio signal processing. IEEE J. Sel. Top. Signal Process. **13**(2), 206-219 (2019). https://doi.org/10.1109/JSTSP.2019.2908700, https://doi.org/10.1109/JSTSP.2019.2908700

29. Rovetta, S., Mnasri, Z., Masulli, F.: Detection of hazardous road events from audio streams: An ensemble outlier detection approach. In: 2020 IEEE Conference on Evolving and Adaptive Intelligent Systems (EAIS). pp. 1-6. IEEE (2020)

30. Sharma, G., Umapathy, K., Krishnan, S.: Trends in audio signal feature extraction methods. Applied Acoustics **158**, 107020 (2020)

31. Sonntag, D., Zillner, S., van der Smagt, P., Lörincz, A.: Overview of the cps for smart factories project: Deep learning, knowledge acquisition, anomaly detection and intelligent user interfaces. In: Industrial internet of things, pp. 487-504. Springer (2017)

32. Tagawa, T., Tadokoro, Y., Yairi, T.: Structured denoising autoencoder for fault detection and analysis. In: Asian Conference on Machine Learning. pp. 96-111 (2015)

33. Torfi, A., Iranmanesh, S.M., Nasrabadi, N.M., Dawson, J.M.: 3d convolutional neural networks for cross audio-visual matching recognition. IEEE Access **5**, 22081-22091 (2017)

34. Zhu, T., Wang, J., Cheng, S., Li, Y., Li, J.: Retrieving the relative kernel dataset from big sensory data for continuous queries in IoT systems. Eurasip Journal on Wireless Communications and Networking **2019**(1) (dec 2019). https://doi.org/10.1186/s13638-019-1467-4