## LSTMCaps Autoencoder Network
### Experimental Results
#### 2.6.2 Drone Dataset Anomaly Detection
The proposed model, Design D, outperforms the other models tested with anomaly detection with an average \(F_{1}\) score of 0.53, and a best \(F_{1}\) score of 0.60, which supports the conclusion that both the inclusion Capsules and the multi-channel input architecture play an important role in improving the performance of the model. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Model** & **Score** & **MAE Threshold 1** & **MAE Threshold 2** & **MAE Threshold 3** & **Precision** & **Recall** & \(F_{1}\) \\ \hline \multirow{3}{*}{Design A} & Average over 5 runs & 6.13 & 6.06 & 10.81 & 0.37 & 0.59 & 0.45 \\  & Best over 5 runs & 5.60 & 6.70 & 11.90 & 0.40 & 0.63 & 0.49 \\  & Standard Deviation & 0.63 & 0.49 & 0.75 & 0.03 & 0.04 & 0.03 \\ \hline \multirow{3}{*}{Design B} & Average over 5 runs & 4.27 & 4.12 & 9.98 & 0.51 & 0.41 & 0.45 \\  & Best over 5 runs & 4.31 & 4.08 & 10.62 & 0.57 & 0.51 & 0.54 \\  & Standard Deviation & 0.15 & 0.06 & 0.59 & 0.06 & 0.11 & 0.08 \\ \hline \multirow{3}{*}{Design C} & Average over 5 runs & 7.96 & 6.87 & 9.88 & 0.54 & 0.50 & 0.52 \\  & Best over 5 runs & 7.70 & 6.92 & 11.22 & 0.58 & 0.50 & 0.53 \\  & Standard Deviation & 0.71 & 0.64 & 1.01 & 0.05 & 0.01 & 0.02 \\ \hline \multirow{3}{*}{
\begin{tabular}{c} **Design D** \\ **(proposed)** \\ \end{tabular} } & **Average over 5 runs** & **4.37** & **4.20** & **9.79** & **0.53** & **0.54** & **0.54** \\  & **Best over 5 runs** & **4.50** & **4.21** & **9.81** & **0.61** & **0.59** & **0.60** \\ \cline{1-1}  & **Standard Deviation** & **0.15** & **0.07** & **0.61** & **0.06** & **0.06** & **0.05** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Best and average test results out of five runs for anomaly detection using optimised hyperparameters in Table 3 for each NN Design, where MAE is Mean Absolute Error, \(F_{1}\) is \(F_{1}\) Score (Equation (15)). Figure 7: Visualisation of anomalies detected in test data by the best iteration of each design. The plots in Figure 7 show that each model was able to detect all the anomalies in the data. However, Figure 7b,d demonstrate that using by Capsules, less False Positives are flagged in the data in comparison to Figure 7a,c, the plots for the non-capsule model variants. Due to the method of calculation, the \(F_{1}\) scores achieved by each model may not be fully representative of the abilities of each model. This aspect can be further explored and improved in future works. The results attained give evidence to show that both the hybridisation of the Capsule and LSTM layers and the multi-channel input model structure are both effective methods for improving the performance of the neural network with multivariate data, especially when used in conjunction with each other. Furthermore, the results also show that the proposed model is superior during training in terms of convergence and resilience to overfitting. Additionally, the \(p\)-value of 0.0302 calculated from the ANOVA source table in Table 6 suggests that the means of the experiment are statistically different, and unlikely to be a result of randomness as is the case with some NN models due to their stochastic nature. The next section will explore the effect of \"unclean\" training data on the performance of the model variants.