## 2 Methods
### Classification and Training Method  
In this study, the primary focus was on developing and refining a deep learning model based on a simple feed-forward neural network. This neural network was trained using the standard backpropagation algorithm and consisted of two hidden layers. To implement the models, we utilized Keras [39] with a TensorFlow [40] backend, which provided a robust framework for building and training our neural networks. A variety of hyperparameters were meticulously adjusted to optimize the performance of each deep neural network (DNN). Key parameters included the number of hidden layers, the number of neurons within each layer, the choice of activation function, the optimization method, and the regularization technique. Through rigorous experimentation, we identified the best-performing DNN architecture with two hidden layers of 120 neurons each, after testing depths of 2 to 8 layers and 50 to 200 nodes using a trial-and-error approach (grid search method). The output layer consisted of a single neuron designed to produce a regression output, which was critical for our specific application.  
Figure 2: Two-dimensional plots of the first and second principal components (class 0 indicates non-severe apnea patients, and class 1 indicates severe sleep apnea patients).  
The activation function chosen for each layer was the Rectified Linear Unit (ReLU) [41], known for its ability to introduce non-linearity into the model while maintaining computational efficiency. We trained the model using dropout [42] values ranging from 0.1 to 0.5 and, based on testing, selected an optimal dropout value of 0.3, which was then applied to each layer to effectively prevent overfitting. This regularization technique randomly deactivates a fraction of neurons during training, thereby promoting generalization. For optimization, we selected the Adam optimizer [43], which combines the advantages of two other popular optimization methods: AdaGrad and RMSProp. The learning rate for Adam was set at 0.001, a value that balances the speed of convergence with the stability of the training process. Additionally, L2 regularization was incorporated to further prevent overfitting by penalizing large weights, thus encouraging the model to maintain smaller and more generalized weights.  
Batch normalization [44] was applied after the first two layers to stabilize and accelerate the training process. This technique normalizes the output of the previous activation layer by adjusting and scaling the activations. By doing so, batch normalization helps to mitigate internal covariate shifts, leading to faster convergence and improved performance. The loss function played a dual role in this context, not only guiding the optimization process by measuring the difference between predicted and actual values but also acting as a regularization term to denote model accuracy. This dual role ensured that the model remained focused on minimizing errors while maintaining generalizability.  
The comprehensive architecture of the proposed DNN, encompassing all these elements, is depicted in Figure 3. This architecture represents a carefully balanced network designed to deliver optimal performance for the given task. Each component, from the choice of activation function to the implementation of regularization techniques, was selected based on extensive testing and validation, ensuring that the final model was both robust and effective. Through this meticulous design and training process, the resulting DNN was well equipped to handle the complexities of the dataset, providing accurate and reliable predictions. The integration of advanced techniques such as dropout, batch normalization, and the Adam optimizer, coupled with a well-structured architecture, underscored the efficacy of our approach in leveraging deep learning for complex classification tasks.  
Figure 3: The architecture of the proposed DNN.