## 5 Methods
### Application of CNN-LSTM for Artifact Removal
#### 5.1.4 Proposed CNN-LSTM Structure and Training
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Layer** & **Parameters** & **Input Size** & **Output** & **Number of Parameters** \\ \hline Input & Input containing signals with artifacts. A matrix of 256 samples per 6 EEG/EMG channels is fed to the network. & (256, 6, 1) & (256, 6, 1) & - \\ \hline Conv2D & \begin{tabular}{c} 128 filters of size [1, 6], padding “valid”, and ReLU activation function. \\ \end{tabular} & (256, 6, 1) & (256, 1, 128) & 896 \\ \hline Conv1D & \begin{tabular}{c} 256 filters of size 1, padding “valid”, and ReLU activation function. \\ \end{tabular} & (256, 1, 128) & (256, 1, 256) & 33,024 \\ \hline Reshape & \begin{tabular}{c} Prepares data for sequential analysis in the LSTM layer. \\ \end{tabular} & (256, 1, 256) & (256, 256) & - \\ \hline LSTM & \begin{tabular}{c} 256 LSTM units, gate activation—sigmoid function, and state calculation—hyperbolic tangent. \\ \end{tabular} & (256, 256) & (256, 256) & 525,312 \\ \hline Flatten & \
\begin{tabular}{c} Converts data into a one-dimensional feature vector for the Dense layer. \\ \end{tabular} & (256, 256) & (65 536) & - \\ \hline Dense &
\begin{tabular}{c} Number of neurons 256, linear activation function. \\ \end{tabular} & (65 536) & (256, 1) & 16,777,472 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Structure of the proposed CNN-LSTM. information when processing subsequent time points. Since EEG signals are sequential, the LSTM layer enables the model to capture long-term temporal relationships, including phase relations in the signal. This makes LSTM particularly effective for examining the dynamic processes occurring in biological signals. The Flatten layer transforms the LSTM output into a one-dimensional vector. This operation is necessary so that the data can be processed by the fully connected Dense layer that follows. By flattening, all features are combined into a single vector suitable for predicting the cleaned EEG signal. Finally, the network includes a fully connected Dense layer acting as a regression layer to predict the artifact-free EEG signal. This layer uses a linear activation function, allowing for the prediction of continuous values. Its output represents the cleaned EEG signal. The output vector of the model with dimensions (256, 1) corresponds to the cleaned EEG signal from a single channel. It refers to a specific time window from the input signal and represents the signal after artifact removal by the model. The total number of network parameters is 17,336,704. For training the model, the ADAM (adaptive moment estimation) algorithm was used. ADAM dynamically adjusts the learning rates for each weight in the neural network and combines the advantages of two popular optimization methods--RMSProp and momentum. As a result, it enables rapid and stable convergence, even with large datasets and complex models. ADAM is widely employed in CNN and LSTM training tasks. In the machine learning process, several key parameters in the ADAM algorithm influence how the model weights are updated. The learning rate was set to 0.001, determining the step size for weight updates and thus the model's rate of adaptation. A decay of 0.0 means that there was no reduction in the learning rate during training. The beta_1 parameter was set to 0.9, smoothing the first moment by calculating the moving average of gradients, while beta_2 was set to 0.999, smoothing the second moment by considering the moving variance of gradients. A small epsilon value of \(1\times 10^{-7}\) was used for numerical stability. The AMSGrad algorithm was not utilized, so the default ADAM approach was retained. These configurations were used throughout the training process. The model's loss function was mean squared error (MSE), which measures the difference between the predicted (artifact-removed) signal and the true artifact-free EEG signal provided by the generator. Minimizing the MSE ensures that the model accurately reconstructs the target signal, thus effectively removing artifacts from the EEG recordings.