## 1 Introduction
### Literature Review and Research Gaps
This approach has seen great success in motor fault detection [12; 13], where in these cases frequency domain transformations were used to enhance the representation of the data for use with LSTM networks. Additionally, the use of various types of CapsNets in numerous cases was found to improve training and classification performance over smaller datasets [14; 15]. As well as hybridising signal processing and ML, many literatures also propose the hybridisation of NN types to take advantage of their advantages with different types of data; one popular hybridisation for TS data is RNNs and CNNs [11]. For TS analysis, Recurrent Neural Network (RNN) based models are generally used due to their ability to identify dependencies in sequential data using their internal memory [9; 16]. The LSTM network [17] is an RNN variant that was proposed to overcome the vanishing gradient problem [18] encountered by the traditional RNN architecture, which causes learning long-term dependencies of a TS feature. In recent times, LSTM has been a highly popular research topic due to the overwhelming demand for time series analysis and forecasting in commercial environments, hence fuelling the demand for more research into the improvement of its performance. One recently proposed method in [19] explored the usage of the LSTM layer in an Autoencoder architecture. The authors correctly identified that a large number of the current machine learning methods that are used are unsuitable for use practically, as they usually require the use of labelled data, which is impractical with time series data due to the large volumes being constantly produced. Furthermore, the authors go on to evaluate classical anomaly detection methods such as Support Vector Machines and Isolation forests as being flawed since they fail to account for the temporal aspect of the time series data and only take the current data into account. The proposed approach shows promise with the accuracy of detection and the wide application of its usage, however, the authors selectively used data that was loud enough to be detected by the autoencoder and did not explore the sensitivity of detection in the study. Furthermore, the authors assumed that the training data acquired was \"clean\" of any anomalies, which could be a reasonable assumption to make since the data was taken from an established dataset, but in a real-life use case, this may not be the case. However, since this is an unsupervised approach, it is expected that all initial errors will not be identified unless extensive data analysis is carried out before training the system, or if previous knowledge about the operation of the system being analysed is acquired. One main issue that the LSTM faces is its overfitting when used with gradient descent learning optimisation algorithms. Although very careful tweaking of the hyperparameters can help to reduce this issue, this is often highly inefficient and time consuming and is sometimes unavoidable with complex datasets. Recent studies have utilised the Convolutional Neural Network (CNN), which has proved to be a powerful tool in image classification [20,21], in time series forecasting and outlier detection tasks [22,23]. Furthermore, the hybridisation of the CNN and LSTM layers has also been shown empirically to be a proven method of improving outlier detection performance [11,24]. However, it is well documented that the CNN has a fundamental flaw in understanding spatial context in data and suffers from a loss of information due to the pooling layer; this is most prominently demonstrated in image classification tasks. The Capsule Network (CapsNet) [25] was proposed to address this flaw by \"encapsulating\" the entity being described in a vector format, where the length describes the probability of existence and the orientation of the vector describes the entity's characteristics, such as orientation and the special context that other traditional neural networks cannot capture.