## 3 Experiment Setup and Network Training
### Network Training  
In the network training, since we needed to compare the teacher network (biGRU), the undistilled student network and the distilled student network (1D-CNN), we first trained the biGRU to obtain an optimal teacher network. After this, the 1D-CNN was trained using KD, and a 1D-CNN with the same network structure was trained simultaneously without KD. The performance of the three networks was compared after the training was completed. We used the signal after the channel as the input, and the initial signal (both 49,152 \(\times\) 1) as the label for training. The batch size was 256. We used 30% of the signal as the training set, and 70% as the test set.  
Teacher model: The teacher model is a biGRU, and its structure is shown in Figure 1. The purpose of training the teacher model is firstly to create teacher labels for the subsequent KD and secondly to prepare for the subsequent performance comparisons. The biGRU layer has 70 hidden units (n = 70), the loss function used in this model uses the mean square error (MSE), the optimizer is Adam and the learning rate is \(10^{-3}\). The input window has a size of 47 input symbols, which are used together to recover one symbol. The teacher model was trained with 50 epochs and the loss curve is shown in Figure 5.  
Student model: In order to better learn the performance of the teacher model biGRU, the student model 1D-CNN that we adopted requires some features. First of all, since the teacher model is bidirectional and can learn both forward and reverse time series information, we also adopted a two-branch 1D-CNN structure, with one branch processing forward information and the other processing reverse information. We also verified that this structure would have better learning performance in the experiment. Secondly, we adopt a  
Figure 5: The loss curve of the teacher model biGRU, 1D-CNN and 1D-CNN with KD.  
1D-CNN structure with a dilation rate of 1. This is because a model with a dilated CNN structure can better learn the processing power of biGRU for time series information [18]. The convolution kernel will have a larger perceptual field of view, which helps the 1D-CNN to learn longer time series information [19]. The various parameters of a 1D-CNN are obtained by traversing the parameters through Bayesian optimization. Two layers of a 1D-CNN structure are used in each direction; the structural parameters of the first layer (4, 22, 1) mean that there are 4 filters with a kernel size of 22 and a dilation rate of 1, while the second layer (4, 23, 1) means that there are 4 filters with a kernel size of 23 and a dilation rate of 1. The activation function of 1D-CNN is Relu [20]. The training takes an alpha value of 0.3, which is the alpha value with the best performance by traversing. The same student model was also trained with 50 epochs, and the loss curve is shown in Figure 5.  
Student model without KD: This model directly uses the student model structure under KD to better compare the performance improvement brought by KD. In order to be as consistent as possible with the previous network training, the loss function used in the model is the mean square error (MSE), the optimizer is Adam, and the input window has a size of 47 input symbols, which are collectively used to recover one symbol. The model was similarly trained with 50 epochs and the loss curve is shown in Figure 5.