## 4 Reinforcement Learning for Improving Performance  
In Section 3, the maximum achievable capacity of the SUTCH is analyzed. In Section 4, the reliability of the SUCCH is taken into consideration by the Bit Error Rate (BER). Suppose the signal waveform of the SUCCH is as follows:  
\[\left\{{ \begin{array}{*{20}{c}}{{s_{1}}(t)=\sqrt{{\varepsilon_{b}}}}\\ {{s_{2}}(t)=-\sqrt{{\varepsilon_{b}}}}\end{array} }\right. \tag{43}\]  
Suppose the two signal waveforms in Equation (45) are transmitted with the same probability. Since the SUCCH spreads its power spectrum density over an ultra-wide bandwidth to exploit the underused (gray) spectral regions, the interference process caused by the PUs and the SUCCH can be considered as a Gaussian approximation. If the SUCCH transmits \({s_{1}}(t)\), after the despread-demodulation algorithm at the receiver of the SUCCH, the received signal is as follows:  
\[r=\sqrt{{\varepsilon_{b}}}+\frac{1}{{G_{\text{SUCCH}}}}\left({n+\sum\limits_{ y=1}^{Y}{\sigma_{PU}}+\sum\limits_{j=1}^{M}{\sigma_{\text{SUCCH}}}}\right) \tag{44}\]  
where \(n\) is additive Gaussian white noise with mean zero, variance \({N_{0}}/{2}\) and \({\sigma_{PU}}\), \({\sigma_{\text{SUCCH}}}\) represent the interference caused by the PUs and the SUTCH. \({G_{\text{SUCCH}}}\) is the spreading gain of the SUCCH. The receiving signal of the SUCCH is compared with the threshold zero, which is as follows:  
\[r\underset{s_{2}}{\overset{S_{1}}{\underset{S_{2}}{\overset{S_{1}}{\underset {S_{2}}{\overset{S_{1}}{\underset{S_{2}}{\overset{S_{1}}{\underset{S_{2}}{ \overset{S_{1}}{\underset{S_{2}}{\overset{S_{1}}{\underset  
\[ \begin{array}{ll}P_{e}&=\frac{1}{2}\left[P\left(e|s_{1}\right)+P\left(e|s_{2} \right)\right]\\ &=\frac{1}{2}\left[\int\limits_{-\infty}^{0}p\left(r|s_{1}\right)dr+\int\limits _{0}^{+\infty}p\left(r|s_{2}\right)dr\right]\\ &=Q\left(\sqrt{\frac{\frac{G_{\text{SUCCH}}\epsilon_{B}}{\frac{\Lambda}{2}+ \sum\limits_{j=1}^{r}\sigma_{pl}^{2}+\sum\limits_{j=1}^{M}\sigma_{\text{SUCCH }}^{2}}}{\sum\limits_{j=1}^{r}\sigma_{\text{BUCCH}}^{2}+\sum\limits_{j=1}^{M} \sigma_{\text{SUCCH}}^{2}}}\right)\end{array}  \tag{47}\]  
Suppose the control information of the SUCCH consists of 8 bits. According to Figure 4, the transmitter and receiver of the SUs need to coordinate access to the spectrum three times. Therefore, the probability of successful establishment for the SUCCH can be concluded. Furthermore, the total interference caused by the SUs is divided into two parts: \(Q_{\text{SUTCH}}\) and \(Q_{\text{SUCCH}}\). \(Q_{\text{SUTCH}}\) represents the interference caused by the activity of the SUTCH, while \(Q_{\text{SUCCH}}\) represents the interference caused by the activity of the SUTCH. The loading factor \(\Gamma\) is defined as the radio of \(Q_{\text{SUTCH}}\) and \(Q_{\text{SUCCH}}\), which is as follows:  
\[\Gamma\overset{\Delta}{=}\frac{Q_{\text{SUCCH}}}{Q_{\text{SUTCH}}},\quad 0< \Gamma<1 \tag{48}\]  
In consideration of the link access protocol design described above and the probability of successful establishment for SUCCH, the lower PSD of SUCCH means it may take more time to complete the setup procedure for the SUs. In other words, accessible subchannels will remain idle for a long period of time, which will lead to spectrum resource waste. However, increasing the transmitting power of the SUCCH will decrease the transmitting power of the SUTCH, because of the total interference constraint caused by the SUs is certain at a time-slot. Lower transmitting power of the SUTCH will lead to reduce the capacity of data. Therefore, it's a trade-off, which is essential to choose the appropriate transmitting power of SUTCH according to the characteristic of the activity of the PU. For this purpose, a hybrid access method based on Reinforcement Learning model is proposed to solve this problem. The most prominent feature of Reinforcement Learning model is its autonomous learning and online learning ability. By trial and error, Reinforcement Learning model can get a better strategy based on the subchannel environment.