## 1 Introduction  
With the continuous growth of high-bandwidth access network service demands, communication networks based on passive optical network (PON) technology have gained increasing attention from researchers. The 50 G PON standard has now been launched and commercialized [1], and the current research focus is to achieve higher capacity, lower cost, and lower latency. In order to meet the requirements of low cost and simple implementation for PONs, advanced modulation formats based on intensity modulation and direct detection (IM/DD), including discrete multitone (DMT), carrier-less amplitude phase (CAP) and four-level pulse amplitude modulation (PAM4) have attracted much attention due to the high spectral efficiency [2, 3, 4]. Among these, the low-complexity PAM4 has been identified by the IEEE and the 802.3 bs 40 G standard [5]. IM-DD systems operating in the C-band are susceptible to the effects of chromatic dispersion (CD) leading to signal distortion. Fiber Bragg gratings (FBGs) are commonly used as CD compensation modules. Additionally, methods such as single-sideband or vestigial sideband (SSB/VSB) modulation [6] and CD pre-compensation are often employed. However, these approaches require complex system configurations and expensive costs, limiting their commercial viability. In contrast, digital signal processing (DSP)-based equalization techniques enhance dispersion tolerance while maintaining cost-effectiveness. Common equalization methods include feedforward equalizers (FFEs), decision feedback equalizers (DPEs), and maximum-likelihood sequence estimation (MLSE). Based on this, some modified equalization algorithms have been proposed. In [7], a DFE based on a nonlinear filtering structure is proposed for a 40 Gb/s PAM4-PON system. In [8], pre-equalizers based on Volterra and post-equalizers based on LMS were used to achieve 50 Gb/s transmission over 20 km with a power budget of 27 dB. However, due to the low sensitivity of IM-DD, further increasing the transmission rate would lead to a decrease in link budget, which limits the application of IM-DD in high-speed PON systems. In contrast to IM-DD, coherent detection technology offers superior resistance to interference and higher sensitivity. It serves as a powerful method to mitigate channel impairments and enhance receiver sensitivity. Therefore, applying coherent detection to PON-based access networks holds significant potential for development [9; 10]. In addition, considering the cost of coherent optical systems, researchers propose to combine a coherent receiver with an intensity-modulated transmitter [11; 12], which promotes the application of PAM4 in coherent systems. Signal distortions including CD, linear, and nonlinear impairments also exist in coherent systems. With the decrease in DSP costs, many equalization algorithms based on coherent detection have also been widely discussed [13; 14]. In [15], the researchers used a Volterra equalizer and MISE for equalizing PAM4 signals, but additional phase recovery operations were required, which significantly increased the computational complexity. The multimodulus algorithm (MMA) can balance PAM4 signals without the need for carrier phase recovery operation. However, once the carrier phase noise is too large, the equalization effect will deteriorate, and the MMA cannot perform nonlinear compensation [16]. Furthermore, the application prospects of equalizers based on neural network (NN) models are noteworthy. In [17], a multi-purpose complex-valued neural network was used to equalize the QAM signal for a 200 Gb/s/\(\lambda\) PON with 20 km of transmission. Nevertheless, the use of these neural networks increases system complexity, leading to higher power consumption. DPEs, compared to FFFs, can compensate for nonlinear distortions and have lower complexity than Volterra and NN equalizers, making them a promising choice for PON.