# A Domain-Based, Adaptive, Multi-Scale, Inter-Subject Sleep Stage Classification Network
## 2 Materials and Methods
### Pre-Training  
For pre-training, the proposed model is mainly built on a feature encoder and two classifiers. The feature encoder contains three multi-scale residual blocks and one bidirectional gated recurrent unit (GRU) layer whose cell size is set to 64 [23; 24]. The multi-scale residual block proposed in the paper contains four branches of convolutional processing, which includes convolution layers followed by a LeakyRelu [25] activation layer to transform weighted values non-linearly, as shown in Figure 2. The block uses different sizes of convolutional kernels to extract the spatial features of single-lead ECG whose kernel sizes are 1, 5, 11, and 21, respectively. Among them, the convolutional kernel with a size of 1 aims to increase or decrease the dimensions of the input signal to maintain the same number of channels as other convolutional branches. The multi-scale residual module can be expressed as  
\[H_{m}\ =\ h(H_{m\ -\ 1})\ +\ \sum_{ks}F^{\prime}_{ks@dim}(H_{m\ -\ 1},\ W_{m\ -\ 1}), \tag{1}\]  
\[\h(H_{m\ -\ 1})\ =\ W^{\prime}_{m\ -\ 1}H_{m\ -\ 1}. \tag{2}\]  
In the formula, \(H_{m\ -\ 1}\) and \(H_{m}\) are the input and output of the \(m\)-th multi-scale residual block, respectively; \(\h\) is the dimension transformation function of the input signal; and \(W^{\prime}_{m\ -\ 1}\) is the weight of the kernel with a size of 1. \(F^{\prime}_{ks@dim}\) is the residual mapping where ks represents the different scales of residual branches, and dim is the number of kernel channels. For a deeper multi-scale residual block, \(l\) can be expressed as  
\[H_{l}\ =\ W^{\prime}_{l\ -\ 1}H_{l\ -\ 1}\ +\ \sum_{ks}F^{\prime}_{ks@dim}(H_{l \ -\ 1},\ W_{l\ -\ 1}), \tag{3}\]  
\[H_{l}\ =\ \prod_{i\ =\ m}^{L\ -\ 1}W^{\prime}_{i}H_{m}\ +\ \sum_{j\ =\ m}^{L\ -\ 2}(W^{\prime}_{j\ +\ 1}\sum_{ks}F^{\prime}_{ks@dim}(H_{j\ },\ W_{j}))\ +\ \sum_{ks}F^{\prime}_{ks@dim}(H_{l\ -\ 1},\ W_{l\ -\ 1}). \tag{4}\]  
For the deeper layer \(l\) in the multi-scale residual network, the shallow information and the multi-scale residual information can be directly transferred to the deeper layers to provide more detailed features. Figure 2: The architecture of the pre-training model. \(X_{li}\) and \(X_{li\ -\ 1}\) represent the current ECG sequence and the previous 30 s ECG sequence, respectively. \(Y_{ti}\) is the prediction of the current 30 s sleep stage. “X3” means 3 multi-scale residual blocks. The block is described in the dashed box, where “5@dim” in the convolution layer indicates kernel size 5 and “dim” kernels where “dim” is different for different blocks. For the residual networks [26] that have different convolutional scales, taking MESA as an example, the feature map of the first convolutional layer whose channel is set to 12 in the first residual block is shown in Figure 3. The details extracted by the convolutional kernels with scales of 5, 11, and 21 are obviously different [27]. The kernel with a scale of 5 pays more attention to high-frequency information, while the kernel with a scale of 21 pays more attention to low-frequency information. The multi-scale residual block integrates the information at these different scales through the residual structure, which not only provides more spatial information for classification but also increases the robustness of the changes in ECG. Figure 3: \(Cont.\)  
After this, the features that are extracted by the feature encoder are flattened and sent to the first classifier, which contains a dense layer to output the probability of classes via the softmax function. An additional branch of the model seeks to use the previous 30 s ECG to predict the current sleep stage. The information from the previous stage is classified by the second classifier, which includes a reshape layer, an LSTM layer, and a dense layer. The units of the LSTM are set to 128. However, there is the problem of class imbalance, as the different sleep stages occupy different proportions in the structure of sleep throughout the night [28].