# A Case Study on Virtual HPC Container Clusters and Machine Learning Applications
## 4 ML on Docker
### Anomaly Detection Application
#### 4.2.3 Implementation
```
#loadandtransformdata scaler=StandardScaler() id_column=\"replicaId\" feature_columns=[\"cpuUsage\",\"memoryUsage\"] data=pandas.read_csv(csv_file) data=data[feature_columns+[id_column]] num_features=data[feature_columns].shape[1] data[feature_columns]=scaler.fit_transform(data[feature_columns] ]) data_frame=data.groupby(id_column)[feature_columns] data={key:valueforkey,valueingroups_frame}
#createatrainingdatasetinDataset train_x=[] fortrain_seriesindata.values(): partial_dataset=keras.preprocessing. timeseries_dataset_from_array( data=train_series, targets=None, sequence_length=time_steps) unbatched_dataset=partial_dataset.unbatch() partial_train_x=numpy.array(list(unbatched_dataset)) train_x.extend(partial_train_x) train_dataset=tensorflow.data.Dataset.from_tensor_slices(( train_x, train_x)) train_dataset=train_dataset.batch(batch_size)
```  
The implementation of the LSTM autoencoder model is presented in Listing 4, and it was developed based on the article in [46]. The network consists of sequentially arranged layers. The first layer of the network is the input, with dimensions determined by the length of the time series fragment time_steps and the number of features num_features. By default, the input layer accepts a matrix with dimensions of four by two. The next two layers consist of LSTM cells and serve the purpose of encoding information. The first LSTM layer has one hundred and twenty-eight cells, and the second has sixty-four. In this arrangement, data reduction (compression) occurs. Additionally, each LSTM cell recursively develops as many times as the length of the time series time_steps (which is typically four steps). The first LSTM layer generates output signals for each individual sample taken from the time series (return_sequences set to True). As a result, for each of the one hundred and twenty-eight LSTM cells, four outputs are generated. In the second LSTM layer, cells have only one output each, and the signal is only generated after processing the last (fourth) sample from a given sequence. A RepeatVector layer, serving as a kind of 'bridge', separates the encoding and decoding parts. It replicates the signals received from the output of the encoding part (from the second LSTM layer) as many times as the length of the time series time_steps. This step is necessary because the second LSTM layer merges the input matrix and removes the dimensions determined by time_steps, meaning that it returns a vector of encoded information at the output. The third and fourth LSTM layers decode information and have sixty-four and one hundred and twenty-eight cells, respectively, each developing four times (which resembles the encoding part, but arranged in the opposite direction). Both generate output signals for each processed sample (return_sequences is true). The final layer is a densely connected neural network (Dense), with a number of neurons equal to the number of features specified by num_features. There is also an intermediate TimeDistributed layer before Dense. At each timestep, the fourth LSTM layer produces one hundred and twenty-eight output signals simultaneously. Simply put, TimeDistributed combines these signals into single outputs, consequently leading to four outputs for each Dense neuron. In this way, the autoencoder ultimately returns a matrix at its output with the same dimensions as the input matrix. Before training and using the neural network, its model must be compiled using the compile method (line 13). In the arguments of this function, the optimization algorithm and the loss function are set. The fit method (line 16) initiates the neural network training process. This requires the training set to be passed. Additionally, the number of epochs (epochs) and the list of callbacks (callbacks), which is normally empty, are specified. Callbacks allow for intervention in the training process and are triggered at various stages.