# Adaptive Modem Based on LSTM-AutoEncoder with Vector Quantization
## 5 Results
### Effectiveness and Complexity  
#### 5.4.1 Effectiveness  
First, we repeated experiments to verify the BER performance. Ten experiments were conducted for each order, and significance level analysis was performed on the BER data under different \(E_{b}/N_{0}\) conditions. Under small sample conditions, T distribution was used for independent testing, and the null hypothesis (H0) was set, as there was no significant  
Figure 7: Comparison of the BER performance of VQ-LSTM-AE with other models. (**a**) Second-order BER performance; (**b**) fourth-order BER performance; (**c**) eighth-order BER performance; (**d**) sixteenth-order BER performance; (**e**) thirty-second-order BER performance. difference in BER between the old and new models. The alternative hypothesis (H1) is that there is a significant difference in BER between the old and new models. The significance level was set to \(\alpha=0.05\); that is, if the \(p\) value is less than 0.05, we reject the null hypothesis and believe that the performance difference between the VQ-LSTM-VAE and DAN-AE models under this \(E_{b}/N_{0}\) condition is statistically significant. The statistical results are shown in Table 2.  
Most of the significance levels of orders 2, 4, and 16 in the table are greater than 0.05, which verifies the conclusion that the model proposed in this article has the same performance as DAN-AE in these three cases; the values for orders 8 and 32 were mostly less than 0.05, which also shows that the BER performance of VQ-LSTM-AE and DAN-AE is significantly different. From Figure 7, we can know that the performance of VQ-LSTM-AE is better than DAN-AE. During the training process, perplexity \(P\) can be used as an indicator to monitor the training effect of the codebook in real time. Perplexity \(P\) uses the information entropy of the category distribution to detect the learning effect of the model, which can be considered as the inverse of the exponential function of information entropy. Information entropy is a measure of the uncertainty of a random variable. Perplexity is positively correlated with information entropy. The larger the perplexity, the more signals can be distinguished. The calculation of perplexity \(P\) is shown in Formula (14). \(\overline{p}_{j}\) is the jth element in the average probability distribution of all samples, \(\varepsilon\) is a very small positive number used to avoid mathematical problems in logarithmic operations, and \(N\) is the number of elements in the probability distribution. \[P=\mathrm{e}^{-\sum_{j=1}^{N}\overline{p}_{j}log(\overline{p}_{j}+\varepsilon)} \tag{14}\]  
The perplexity monitoring during the training process of each order is shown in Figure 8. Figure 8a-c show the situations of the second, fourth, and eighth orders, respectively. The model can find the optimal solution very quickly and converge near the extreme values of 1.9, 3.9, and 7.9, which means that the required corresponding signal types are successfully distinguished; Figure 8d shows the training situation in the 16 th order. Compared with the second to eighth orders, it took more epochs to complete convergence, but the goal of a perplexity of about 16 was still achieved within 30 rounds; Figure 8e shows that the perplexity result converges to around 30, which is slightly away from the target 32, which is one of the reasons why the BER performance did not reach 32 QAM.