# Anomaly Detection through Grouping of SMD Machine Sounds Using Hierarchical Clustering
## 2 Related Work
### Autoencoder-Based Anomaly Detection  
[PERSON] et al. [11] proposed the convolutional autoencoder (CAE) [12] anomaly detection method and verified it with the NSL-KDD dataset [13], which consists of data selected from the KDD 99 dataset, which is an intrusion simulation dataset on the network. The convolution layer of the CAE that reduces the dimension of input data serves as an encoder, and the deconvolution layer of the CAE that reconstructs the reduced dimension serves as a decoder. This process can be expressed by the following equations:  
\[\mathbf{h}^{k}=\sigma\Big{(}\mathbf{x}\varoccleft W_{x}^{k}+\mathbf{b}_{x}^{k}\Big{)}, \tag{1}\]\[\hat{x}=\sigma\left(\sum_{k\in H}h^{k}\otimes W^{k}_{\hat{x}}+b^{k}_{\hat{x}}\right), \tag{2}\]  
where \(\sigma\) refers to sigmoid, one of the activation functions, and \(\v@arrow\) refers to the 2D convolution operation. \(k\) refers to the number of feature maps of hidden representations, and \(W\) and \(b\) are the weights and biases of the convolution layer and the deconvolution layer. [PERSON] et al. converted 1D data into 2D data by applying triangle area maps (TAMs) to apply the convolution autoencoder to 1D data; TAM converts each 1D vector \(x=[x_{0},x_{1},\ldots,x_{d}]\) into a \(d\times d\) matrix \(Y\) according to the following equation: \(y_{ij}=x_{i}\times x_{j}\), where \(i,j\in[1,d]\). [PERSON] et al. [14] proposed a method for anomaly detection in multivariate time series data using a long short-term memory (LSTM) [15]-based autoencoder. LSTM is a neural network that improves the gradient vanishing problem, which is a disadvantage of RNN, and is calculated in the following equations:  
\[i_{t}=\sigma(W_{ix}x_{t}+W_{hf}h_{t-1}+b_{i}), \tag{3}\]  
\[g_{t}=\tanh\big{(}W_{xg}x_{t}+W_{hg}h_{t-1}+b_{g}\big{)}, \tag{4}\]  
\[f_{t}=\sigma(W_{xf}x_{t}+W_{hf}h_{t-1}+b_{f}), \tag{5}\]  
\[o_{t}=\sigma(W_{ix}x_{t}+W_{hg}h_{t-1}+b_{o}), \tag{6}\]  
\[c_{t}=f_{t}\circ c_{t-1}+i_{t}\circ g_{t}, \tag{7}\]  
\[h_{t}=o_{t}\circ\tanh\big{(}c_{t}\big{)}, \tag{8}\]  
where \(x_{t}\) denotes the input at the time step \(t\); \(i_{t}\) and \(g_{t}\) denote the input gates; \(f_{t}\) and \(o_{t}\) denote the forget and output gates; and \(h_{t}\) and \(c_{t}\) denote the hidden and cell states, respectively. Moreover, \(W\), \(b\), \(\sigma\), and \(\circ\) denote the weight matrices, bias vectors, the sigmoid function, and the Hadamard product (i.e., element-wise product), respectively. The encoder layer of the LSTM autoencoder compresses the multivariate time series input data \(\mathbf{x_{t}}=[x^{0}_{t},x^{1}_{t},\ldots,x^{L-1}_{t}]\) into the hidden states of the encoder \(\mathbf{h_{t}}=[h^{0}_{t},h^{1}_{t},\ldots,h^{k-1}_{t}]\) as follows:  
\[\mathbf{h_{t}}=\mathbf{LSTM}(\mathbf{x_{t}},\mathbf{h_{t-1}}),\mathbf{z}=f(\mathbf{h_{t}}), \tag{9}\]  
where \(t\) is the time step of input \(\mathbf{x}\), \(L\) is the number of variables of input \(\mathbf{x}\), \(k\) is smaller than \(L\), \(f\) is a non-linear function, and \(\mathbf{z}\) is the latent representation. The decoder layer reconstructs the latent representation \(\mathbf{z}\) to \(\hat{\mathbf{x}_{t}}=[\hat{x}^{0}_{t},\hat{x}^{1}_{t},\ldots,\hat{x}^{L-1}_{t}]\) as follows:  
\[\mathbf{h^{\prime}_{t}}=\mathbf{LSTM}\big{(}\mathbf{z},\mathbf{h^{\prime}_{t-1}}\big{)},\hat {\mathbf{x}_{t}}=g(\mathbf{h^{\prime}_{t}}), \tag{10}\]  
where \(\mathbf{h^{\prime}_{t}}\) is the hidden state of the decoder at time step \(t\) and \(g\) is the non-linear function. [PERSON] et al. used the sum of squared errors (SSEs) to calculate the reconstruction error.