# Improving Deep Learning Anomaly Diagnostics with a Physics-Based Simulation Model
## 4 Validation with the Long Short-Term Memory Autoencoder
### Diagnostics Validation with RN and RA Data  
The Diagnostics validation is performed using the x1\(\ldots\)x5 inputs given in the DS2 output data set, where the x5 indicates the real torque excitation of the machine while in operation. This validation processes only the yn,1 variable and the related x1\(\ldots\)x5n,2 variables, excluding the x6 signal, to evaluate the labeling process of the condition-monitoring data obtained. The validation of the anomaly labeling is evaluated using an LSTM autoencoder-neural network's capability to differentiate between the labeled normal and anomalistic states. The overall algorithm structure used in this study is illustrated in Figure 8, where the variable x6 is also annotated as it is considered within the Simulation validation and the Comparative validation of the model. The input layer consists of the x-variables annotated in Table 2 and the \(y\)-variable comprising the supervised label states of the machine condition. The data-split size between the training and test data in each validation is 80/20. In the encoder layers, the dropout function (DO) rate of 20% is enabled to randomly drop neurons out of the neural network. This helps to reduce model overfitting, as the network can learn from different combinations of neurons being dropped out [25]. It essentially governs the magnitude of changes applied to the model parameters. The network epochs are set to 50, with a batch size of 64 indicating the number of samples before the model weights are updated. The learning \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Total Data Points** & **(\%)** & **Normal State (RN)** & **(\%)** & **Anomaly State (RA)** & **(\%)** \\ \hline
11,355 & 100 & 9338 & 82.2 & 2017 & 17.8 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Labeled machine x6 data normal and anomaly states of the DS2. rate used is 0.0001, which determines the extent of adjustments made to the model during each update of its weights in response to the estimated error. The latent vector of the autoencoder represents the input data in a compressed form, where only the high-level data features are retained and summarized from the input layer [26]. The decoder layers are responsible for reconstructing the original data from the latent vector. It takes the compressed representation of the input data and uses it to reconstruct the original data. This is achieved by passing the latent vector through decoder LSTM layers 1 and 2 to create a reconstructed output that is as close as possible to the original data. All the layers are accompanied by a rectified linear unit (ReLU) activation function, which helps to reduce the variable space used; therefore, increasing the training speed as the search space becomes significantly smaller in a neural network [27]. The output layer of the LSTM autoencoder represents the prediction of the encoded features, yet in this research, the machine state prediction \(\hat{y}\) is observed. The reconstruction error of an LSTM autoencoder measures the degree of similarity between the input and output data of the autoencoder, which is widely computed as the mean square error (MSE) of the difference between the original input and the output [28] in Equation (5). It is especially valuable for anomaly detection because the squared part of the equation can amplify errors [29]. \[K(y,\hat{y})=\mid y-\hat{y}\mid\mid^{2} \tag{5}\]  
The reconstructed and classified data points for \(y\) in DS2 output are shown in Figure 9. The threshold boundary in the reconstruction error refers to a point at which the autoencoder is distinguishing between normal and anomalistic data [30]. The threshold boundary for the reconstruction error is typically determined through the tuning of different constants for optimal fault-detection accuracy [31], as different values may yield different levels of accuracy in the reconstruction. The empirically found threshold level of 0.3 is selected to minimize false-negative values from the results.