## 4 Simulation and Analysis
### System Performance Analysis  
Figure 6 illustrates the amplitude-frequency characteristics of various signals. Upon observing Figure 6, it is evident that the interference cancellation algorithm significantly improves the amplitude-frequency characteristics of the signal. However, there is a slight deviation from those of the 4 PAM signal. This discrepancy arises because, although the MHSA-1D CNN-Bi-LSTM interference cancellation algorithm effectively eliminates interference values at signal sampling points, residual interference values still exist at other moments outside these sampling points. Therefore, it can be concluded that the proposed algorithm exclusively eliminates interference affecting the target value of signal amplitude during transmission without altering the overall duration of FTN signals. The sampled values of three different signals are presented in Figure 7. Figure 7a illustrates the sampled values of the 4 PAM signal (prior to FTN shaping filtering), while Figure 7b displays the sampled values of the signal after FTN shaping filtering. Additionally, Figure 7c presents the sampled values of the FTN signal following the application of our proposed interference cancellation algorithm. A comparison between Figures 7a and 7b clearly reveals the presence of ISI introduced by employing FTN technique for modulation purposes. In contrast, a point-by-point comparison between Figures 7a and 7c demonstrates consistent amplitudes for almost all corresponding points, indicating that our \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **RMSE** & **MAE** & **R\({}^{2}\)** \\ \hline LSTM & 0.68733 & 0.56313 & 0.62213 \\ Bi-LSTM & 0.64597 & 0.55962 & 0.66623 \\ CNN-Bi-LSTM & 0.36303 & 0.31812 & 0.89458 \\ MHSA-1D CNN–Bi-LSTM & 0.31499 & 0.24220 & 0.92060 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance comparison of different models. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Indicator Name** & **Indicator Meaning** & **Formula** \\ \hline RMSE & The square root of the ratio of the square of the deviation of the & \\ predicted value from the true value to the number of & \(RMSE=\sqrt{\frac{1}{\hbar}\sum\limits_{i=1}^{n}(y_{i}-\hat{y}_{i})^{2}}\) \\  & The average of the absolute values of the deviations of all & \\ MAE & individual observations from the arithmetic mean [26]. & \(MAE=\frac{1}{n}(\sum\limits_{i=1}^{n}|y_{i}-\hat{y}_{i  
interference cancellation algorithm effectively eliminates the ISI caused by the FTN modulation technique, thereby showcasing its potential as a robust solution for enhancing reliability in FTN communication systems. Figure 8 illustrates the BER plots for various transceiver antenna configurations, with and without the interference cancellation algorithm, under medium turbulence channel conditions with \(\tau=0.8\), \(\lambda=1550\) nm, \(ipce=0.5\). \(ipce\) denotes the photoelectric conversion efficiency and is set to 0.5 to represent typical photodetector performance across diverse environmental conditions. This selection effectively accounts for optical signal attenuation and noise effects, ensuring system reliability under varying channel conditions. Moreover, opting for a value of 0.5 strikes a balance between signal quality and system complexity, thereby enhancing overall performance. Consequently, this setting guarantees stable and efficient model operation in real-world applications. Firstly, our proposed algorithm demonstrates comparable performance to the orthogonal system across all antenna configurations, indicating its effectiveness in eliminating ISI. Experimental results on BER performance with \(1\times 1\), \(2\times 4\), and \(4\times 2\) transceiver antenna configurations confirm that both the algorithms exhibit nearly identical performances, thereby showcasing their equivalent diversity gain.