# Uncertainty-Aware Time Series Anomaly Detection
## 3 Proposed Approach  
In this section, we describe the approach we developed, called LSTMAE-UQ (code is available at [[https://github.com/p199671/LSTM-AE-UQ.git](https://github.com/p199671/LSTM-AE-UQ.git)]([https://github.com/p199671/LSTM-AE-UQ.git](https://github.com/p199671/LSTM-AE-UQ.git)) (accessed on 1 September 2024) ) (Long Short-Term Memory Autoencoder with Aleatoric and Epistemic Uncertainty Quantification), to address the problem of detecting uncertain anomalies in univariate time series data. LSTMAE-UQ integrates LSTM cells in the autoencoder architecture and quantifies uncertainties in both data and model. We chose this combination for two main reasons: Firstly, by using LSTM, we can capture the temporal structure of time series data. Secondly, an autoencoder is an unsupervised learning approach that does not require labels for learning. Additionally, it allows for univariate and multivariate time series anomaly detection and sets the basis for epistemic uncertainty quantification by its multi-layer neural network structure. The combination of LSTM and the autoencoder can learn to capture relevant features of time series data and aims to reconstruct them properly. Moreover, unlike previous works which focus mainly on epistemic uncertainty, our approach considers both uncertainties separately. To this aim, we adapt the model inspired by [8] to output an estimate of data uncertainty as well as epistemic uncertainty using MC Dropout [11].  
The architecture of LSTMAE-UQ is illustrated (to ease readability, we do not show the averaging over the MC samples for reconstructions and uncertainty for the reconstruction error) in Figure 1. The task of the described architecture is to learn representative features of the input data given the window size and reconstruct it properly. The input data consists of one observation for each timestep. We reconstruct these observations which should be as close to the mean-normalized input as possible (see example in Table 1). Through a separate head, apart from the reconstruction output, the model also returns the variance, which corresponds to the respective data uncertainty.  
An autoencoder consists of two parts: an encoder and a decoder. In our work, each of the encoders and decoders is constructed of a two-layer LSTM network (Table 2). The first layer of the encoder takes the time series as input. The second layer, smaller in size than the first, encodes the output of the input layer to a lower dimensional representation. The dimensions of the encoding is 0.8 times the input window size. The first layer of the decoder uses the encoding as input. The output is passed to the second decoder layer which aims to reconstruct the input dimensions. Additionally, we split the head of the network and added a small fully connected network. This second output layer will be trained to calculate an estimate of aleatoric uncertainty.  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Encoder** & \multicolumn{3}{c}{LSTM:in \(\rightarrow\) Drop \(\rightarrow\) LSTM:encoding} \\
**Decoder** &
\begin{tabular}{c} LSTM:encoding \(\rightarrow\) Drop \(\rightarrow\) LSTM:reconstruction \(\rightarrow\) Drop \(\rightarrow\) FC:out \\ LSTM:encoding \(\rightarrow\) Drop \(\rightarrow\) LSTM:uncertainty \(\rightarrow\) Drop \(\rightarrow\) FC:out \\ \hline \hline \end{tabular}
\end{table}
Table 2: Network Layers.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Original Input Window** & \multicolumn{3}{c}{**Normalized Input Window**} & **Desired Reconstruction** \\ \hline \((1\) & \(4\) & \(4)\) & \((-2\) & \(1\) & \(1)\) & \((-2\) & \(1\) & \(1)\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Example of desired reconstruction.