## 1 Introduction  
Driven by the rapidly growing demand for emerging applications, such as the internet of things (IoT), 4K/8K-HD video streaming services and virtual reality (VR), the need for higher access network capacity is involved in offering a better user experience. A bunch of broadband fiber access network technologies is evolving to support the accelerating capacity growth requirements. Among these access technologies, the passive optical network (PON) becomes the dominant one due to its cost-effective advantage [1]. In recent years, thanks to the efforts of the PON standardization groups, beyond 10 Gb/s line rate per wavelength of PON systems has successfully been conducted. A system based on 25 Gb/s nominal line rate has been defined by the IEEE 802.3 ca 50 G Ethernet Passive Optical Network (EPON) Task Force [2]. The ITU-T Q2/SG15 has also already committed to the standardization of a higher line rate of 50 Gb/s [3]. At C-band, more complex digital signal processing (DSP) is required to achieve high-speed transmission based on direct detection due to high fiber chromatic dispersion (CD) [4]. O-band is a better choice in both high-speed PON downstream and upstream links because its lower CD helps to overcome inter-symbol interference (ISI) [5], and it has been successfully selected as the downstream wavelength of 50G time division multiplexed PON (TDM-PON) [6]. However, O-band suffers higher fiber attenuation loss, which greatly decreases the power budget. Coherent technologies [7] are proposed for improving the loss budget, but for the PON applications, expensive optical components, high DSP complexity and power consumption required by coherent systems make it difficult to be cost-effective. Hard-decision forward error correction (HD-FEC) is one of the simplest and most commonly used FEC technology [8; 9]. Recently, more powerful FEC technologies, e.g., soft-decision FEC (SD-FEC), further improved the power budget for high-capacity PON systems [10; 11] with an overhead of 15-35% [12]. Semiconductor optical amplifier (SOA) is maturely fabricated and widely used in O-band as a booster and pre-amplifier to improve the power budget [13; 14]. The combination of SOA and photodiode (PD) as a receiver provides another solution for high sensitivity detection in PON systems in addition to an avalanche photodiode (APD). Compared with directly modulated laser (DML), electro-absorption modulated laser (EML) has the advantages of lower frequency chirp and slighter CD-induced distortions, which is expected to have a better transmission performance in PON systems. Furthermore, EML and PD are easily integrated with an SOA, which only adds a small cost to the system. SOA will induce the nonlinearity named pattern effect due to gain saturation [15]. For 50 Gbaud four-level pulse amplitude modulation (PAM-4) signals, although the pattern effect is weaker than lower speed signals [16], it still degrades the system performance. Compared with the pattern effect mitigation methods based on optical techniques [17; 18; 19], schemes using DSP in the electrical domain [16; 20] have the advantages of low cost and high flexibility. Look-up-table (LUT) has been proven as a simple and effective algorithm to mitigate the pattern-dependent distortions [21], especially the SOA-induced pattern effect [16]. After optimizing the parameters of LUT, it has good performance and almost no sensitivity penalty. Meanwhile, it is often used as pre-compensation placed in the optical line terminal (OLT) side to reduce the complexity of the optical network unit (ONU) side, which makes it more appropriate for cost-sensitive PON systems. However, most of the current research works only focus on the pattern effect induced by SOA as a pre-amplifier. As a booster that is often used to increase the launch power, SOA will also induce a pattern effect as the output power is high enough. When an SOA booster and an SOA pre-amplifier are used at the same time, the nonlinear impairments will become more serious.