## LSTMCaps Autoencoder Network
### Experimental Results
#### 2.6.4 SKAB Anomaly Benchmark
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Hyperparameter** & **LSTMCaps Optimised for Outlier Detection** & **LSTMCaps Optimised for Changepoint Detection** \\  & **(LSTMCaps Outlier Detector)** & **(LSTMCaps Changepoint Detector)** \\ \hline Optimiser & Amsgrad [43] & Adam [40] \\ MAE Threshold Multiplier & 0.925 & 0.99 \\ Epochs & & 100 \\ Learning rate & & 0.003 \\ Time steps & & 3 \\ Capsule activation & relu \\ LSTM activation & tanh \\ Validation split & & 0.2 \\ Batch size & & 128 \\ Branched layer width & & 32 \\ Full layer width & & 256 \\ Loss function & & huber \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Optimal hyperparameters used to maximise outlier detection (left) and changepoint detection (right). \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Algorithm** & \(F_{1}\) & **FAR \%** & **MAR \%** & **NAB (Standard)** & **NAB (LowFP)** & **NAB (LowFN)** & **Overall Accuracy** \\ \hline \hline \multicolumn{1}{c}{_Perfect score_} & 1 & 0 & 0 & 100 & 100 & 100 & 1 \\
**LSTMCaps Changepoint Detector (Proposed)** & **0.71** & **14.45** & **30.86** & **27.39** & **17.08** & **31.13** & **0.49195** \\ MSCRED [44] & 0.7 & 16.82 & 31.28 & 26.13 & 17.81 & 29.53 & 0.48065 \\ LSTMCaps Outlier Detector (Proposed) & 0.74 & 21.66 & 18.74 & 21.58 & 5.12 & 27.49 & 0.4779 \\ LSTM [45] & 0.65 & 14.89 & 39.4 & 26.61 & 11.78 & 32 & 0.45805 \\ LSTM-AE [46] & 0.64 & 14.81 & 39.5 & 22.97 & 20.95 & 23.93 & 0.43485 \\ MSET [47] & 0.73 & 20.82 & 20.08 & 12.71 & 11.04 & 13.6 & 0.42855 \\ Isolation forest [48] & 0.4 & 6.86 & 72.09 & 37.53 & 17.09 & 45.02 & 0.38765 \\ Conv-AE [49] & 0.66 & 5.57 & 46.16 & 11.12 & 10.35 & 11.77 & 0.3856 \\ LSTM-VAE [50] & 0.56 & 9.04 & 54.75 & 21.09 & 17.52 & 22.73 & 0.38545 \\ Autoencoder [51] & 0.45 & 7.52 & 66.59 & 15.65 & 0.48 & 21 & 0.30325 \\ _Null score_ & \(0\) & _100_ & _100_ & \(0\) & \(0\) & \(0\) \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Comparison of the average scores out of five runs for each metric tested. The results in Tables 11 and 12 show that the proposed method optimised for outlier detection outperforms all other methods tested, achieving the best \(F_{1}\) score and the lowest False Negative rate out of the models tested. It also achieves the second highest False Positive rate out of the models. With regards to changepoint detection, the proposed method optimised for outlier detection does not perform as well. However, the proposed method optimised for change \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Source** & **Degrees of Freedom (DF)** & **Sum of Squares (SS)** & **Mean Square (MS)** & **F-Ratio** & \(p\)**-Value** \\ \hline Between Groups & 9 & 0.1537 & 0.0171 & 238.2433 & 0 \\ \hline Within Groups & 40 & 0.0029 & 0.0001 & & \\ \hline Total: & 49 & 0.1565 & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 13: ANOVA source table for the overall accuracy scores of the SKAB benchmark. Figure 10: Visualisation of SKAB results: NAB score plotted against the \(F_{1}\) score. point detection was able to outperform all ML-based methods and the majority of other methods, but is outperformed by the Isolation Forest algorithm due to the low False Negative rate of the Isolation Forest on the test data. However, Isolation Forest performs poorly with regards to outlier detection, making the algorithm unsuitable as a balanced solution for both problems. Similar outcomes can be seen for the best performing test iteration, with no improvement in relation to the other NNs and ML methods. As a balanced solution to both outlier detection and changepoint detection, the proposed model outperforms all other tested methods when optimised for changepoint detection. The main change that is made to optimise the proposed model for each detection task is a change in the threshold multiplier. With a more sensitive threshold, the proposed model is able to display strong performance in outlier detection at a cost to the changepoint detection performance.