# Hybrid Deep Learning Approach for Automated Sleep Cycle Analysis
## 3 Methodology
### Model Training  
Machine learning develops algorithms that learn patterns from data to perform tasks such as classification, regression, and clustering [39]. In deep learning, neural networks are algorithms inspired by the structure of the brain [40; 41], designed to recognize these patterns and learn complex relationships. A neural network consists of layers of artificial neurons, where each neuron processes inputs through a weighted sum followed by an activation function, which can be expressed as  
\[a=f(z)=f\left(\sum_{i=1}^{n}w_{i}x_{i}+b\right), \tag{8}\]  
where \(x_{i}\) denotes the inputs, \(w_{i}\) denotes weight vectors, \(b\) is the bias, \(z\) is the weighted sum, \(f(\cdot)\) is the activation function, and \(a\) is the output of the neuron [42]. Figure 5 provides a visual representation of the internal components of an artificial neuron.  
Non-linearity is introduced into the model through activation functions. Among the principal activation functions are the sigmoid (\(\sigma\)), hyperbolic tangent (tanh), rectified linear unit (ReLU), and softmax functions. These functions are described in Table 2.  
The structure of a neural network includes an input layer, hidden layers, and an output layer [40], as shown in Figure 6. The raw input data are fed into the network through the input layer, with each neuron corresponding to an individual feature. Neurons within the hidden layers transform signals from the preceding layer through activation functions. Predictions are generated in the output layer, which typically employs activation functions that map the output into a probabilistic interpretation.  
\begin{table}
\begin{tabular}{l l l} \hline \hline
**Activation** & **Function** & **Description** \\ \hline \multirow{2}{*}{Sigmoid} & \multirow{2}{*}{\(\sigma(z)=\frac{1}{1+e^{-z}}\)} & In binary classification, it maps real values to the range [0, 1][42]. \\ \hline \multirow{2}{*}{Tanh} & \multirow{2}{*}{\(\tanh(z)=\frac{e^{z}-e^{-z}}{e^{z}+e^{-z}}\)} & Transforms inputs to outputs between \(-1\) and 1 [42]. \\ \hline \multirow{2}{*}{ReLU} & \multirow{2}{*}{ReLU(\(z\)) = max(0, \(z\))} & Outputs the input if it is greater than zero; otherwise, it is zero [43]. \\ \hline \multirow{4}{*}{Softmax} & \multirow{4}{*}{Softmax(\(z_{i}\))} & Converts a vector of values into probabilities, where \(z_{i}\) and \(z_{j}\) are the weighted sums for classes \(i\) and \(j\), respectively. This function is particularly useful for multiclass classification tasks [40]. \\ \hline \hline \end{tabular}
\end{table}
Table 2: Classification results of the best model on the validation set.  
Figure 5: Basic structure of an artificial neuron with a weighted sum and activation function.  
Training a neural network involves adjusting the weights \(w_{i}\) and biases \(b\) to minimize the error between the predicted and actual outputs [44]. This process is carried out iteratively through the following stages: forward propagation, evaluation via loss functions, backpropagation, and parameter adjustment using optimization algorithms. Regularization techniques are often applied to prevent overfitting and improve the generalization performance. Moreover, the design of a neural network is influenced by factors such as the number of layers, the arrangement of neurons, and the type of layer used. The following sections outline the stages involved in training a neural network, including forward propagation, loss functions, backpropagation, optimization, and regularization strategies. Subsequently, different types of neural network layers are reviewed.