## 4 Masking Threshold Constrained Time-Varying Filter  
### The Pass Region Generation  
For the single carrier pulse shaping signal, the time frequency distribution of the I/Q separated baseband signal shows a regularity. Based on this regularity, the fixed pass region is generated to design the time-varying filter, where the pass region generation procedure is:  
* Step 1: Generate the \(1/-1\) alternate data sequence. If the processing window length of the time-varying filter is \(NT_{b}\), the number of the generated data is \(N\). * Step 2: The data sequence passes through the pulse shaping filter, and the pulse shaped baseband signal \(s_{o}(n)\) is generated. * Step 3: Calculate the time frequency distribution of \(s_{o}(n)\), and obtain \(S_{o}(n,v)\). * Step 4: Generate the pass region \(R\) with the pre-defined masking threshold \(\gamma\). \(R:\{n,v\}\in\{|S_{o}(n,v)|\geq\gamma S_{o}^{max}\}\), where \(S_{o}^{max}\) is the maximum value of \(S_{o}(n,v)\), i.e., \(S_{o}^{max}=max\{|S_{o}(n,v)|\}\). * Step 5: Construct the time frequency weighting function \(M(n,v)\) according to the pass region \(R\), which is shown in (11). Figure 9: Realizing time-frequency equivalent analysis of baseband and RF by adding Hamming window and zero padding. (**a**) baseband complex signal, with hamming analysis window and zero padding, STFT, \(L_{h}\) = 40; (**b**) RF signal, with hamming analysis window and zero padding, STFT, \(L_{h}\) = 40; (**c**) baseband complex signal, with hamming analysis window and zero padding, CW, \(L_{h}\) = 40; (**d**) RF signal, with hamming analysis window and zero padding, CW, \(L_{h}\) = 40. Figure 10 shows the pass region with different masking thresholds \(\gamma\). It illustrates that a smaller masking threshold allows a wider pass region, and more energies of the desired signal are collected. However, unfortunately, the interference and noise in the pass region will pass through the time-varying filter. As the masking threshold \(\gamma\) grows, the pass region shrinks, and the energies at the edge of each symbol period are filtered out gradually. If the energy center of the interference signal is just located at the edge of each symbol period, more interference could be suppressed, but the desired signal will be distorted severely. Therefore, there should be an optimal masking threshold \(\gamma\) with less desired signal distortion and more interference suppression. The optimal masking threshold \(\gamma_{opt}\) is determined by the minimum bit error rate criterion. \[\gamma_{opt}=\arg\min_{\gamma}\{\text{BER}\} \tag{22}\]  
Figure 11 shows how the BER of the desired signal varies with the masking threshold \(\gamma\). In the figure, LPF represents the low pass Butterworth filter with a bandwidth of 2.5 MHz, and the filter order is 4; LTV represents the GWS based linear time varying filter. In the simulation, the \(E_{b}/N_{0}\) is set to 8 dB, and the power ratios of the desired signal to the interference signal are 5 dB and 8 dB, respectively. The analysis window is the Hamming window with length \(T_{b}\). It is shown that the BER with the LTV filter is worse than that with the LPF filter when the masking threshold is too small or too big. Because when the threshold is too small, the pass region contains more time frequency areas on the time frequency plane. In this case, although most of the energies of the desired signal are collected, the interference suppression effect is not obvious. In contrast, when the threshold is too big,  
Figure 10: The pass regions with different masking thresholds. (**a**) The pass region with \(\gamma\) = 0.04; (**b**) The pass region with \(\gamma\) = 0.1; (**c**) The pass region with \(\gamma\) = 0.2; (**d**) The pass region with \(\gamma\) = 0.26.  
the pass region shrinks, the interference locating at the edge of each symbol period can be suppressed. However, the distortion of the desired signal overwhelms the interference suppression, hence the BER with the LTV filter is worse than that with the LPF filter.