## 1 Introduction  
The explosive growth of wireless devices and streaming services, e.g., 3D video, cloud office, virtual reality (VR), internet of vehicles, and 6G services, has led to the demand for higher transmission capacity and more spectrum resources [1, 2]. The Terahertz band (THz band), occupying the spectrum range of 0.3 THz to 10 THz, is a promising candidate for providing large capacity; it can provide a data capacity of hundreds of Gbps or even Tbps due to its substantial available bandwidth [3, 4, 5]. Moreover, the devices at THz band have a small and compact size, and can be monolithically integrated with other front-end circuits in portable terminals. THz communication techniques can be divided into two main categories: pure electronic schemes and photonics-aided schemes. Photonics-aided THz-wave signals can be adopted in fiber-wireless communication systems, which overcomes the bandwidth limitation and electromagnetic interference resulting from electronic devices [6, 7, 8]. Moreover, the characteristics of photonic technology have effectively promoted the seamless integration of wireless and optical fiber networks. A photonics-assisted transmission system at the THz band integrates the large-capacity, long-distance advantages of fiber-optic transmission and a THz wireless transmission link, and therefore it has excellent potential for future 6G communication. In previous years, several seamless integration systems of THz wireless and optical fiber networks have been demonstrated, enabled by photonics [9, 10, 11, 12, 13, 14, 15]. However, the nonlinear impairments induced by photoelectric devices and wireless links will cause severe degradation and significantly restrict the transmission rate and distance. Therefore, advanced equalization algorithms are required to compensate for these nonlinear impairments. To mitigate the distortions of nonlinearity, several nonlinear equalization methods have been widely researched, i.e., Volterra [16], Kernel [17], and Maximum Likelihood Sequence Estimator (MLSE) [18]. However, with the increase in transmission speed, conventional digital signal processing (DSP) algorithms cannot effectively equalize nonlinear noise such as relative intensity and modal partition noise in the practical system [19]. In addition, blind equalization, which performs adaptive channel equalization without the aid of a training sequence, has become a favorite of practitioners, such as the constant modulus algorithm (CMA) equalizer and direct-detection least mean square (DD-LMS) equalizer. As we know, CMA is a typical blind adaptive algorithm for adaptive blind equalizers in Quadrature Amplitude Modulation (QAM) format [20]. Nevertheless, CMA has a significant residual error after convergence, which is unsuitable for the nonlinear channel balance. Furthermore, more advanced artificial intelligence (AI) algorithms are considered as promising solutions for signal equalization in optical communication systems. Table 1 summarizes the typical photonics-aided wireless communication systems using AI equalization algorithms. Different paradigms based on artificial neural networks (CNN) [21; 22] and deep neural networks (DNN) [23; 24; 25; 26] have been proven to obtain better performance than conventional DSP algorithms, showing an excellent capability of mitigating nonlinear distortions. However, a CNN fails to fully account for the relevant characteristic information of the input sample, leading to performance degradation in the optical communication system [27]. Moreover, a CNN is unable to meet the demand for complex-valued equalization in wireless transmission systems [24]. Furthermore, a DNN requires many training sequences in order to achieve the optimum result, reducing the effective channel capacity. Due to the ability to learn the relevant information and make the nonlinear decision in feature space, the Deep Belief Network (DBN) is used for nonlinear equalization, which is more helpful in handling the receiver sensitivity issue in the THz-band wireless communication system [28].