# Advanced DSP for Coherent Optical Fiber Communication
## 2 Principle of Coherent Detection and Linear Impairments Equalization  
The coherent receiver based on an intradyne system is shown in Figure 1a [11]. The input signal interferes with the local oscillator (LO) laser, which usually has the same frequency as the transmitter laser in the 90-degree optical hybrid device. Balanced detectors (BDs) are often used to reject the common mode noise. In order to detect both real and imaginary parts, the input signal is mixed with the real part of the LO in one arm and the imaginary part in the other arm through the 90-degree phase delay between the signal and the LO introduced by the 90-degree hybrid. Polarization diversity is introduced to ensure the detection of both polarizations of the signal. It is noted that the LO does not require the phase and polarization locking for the input signal. Then the electrical signal is digitalized using the ADCs with two samples per symbol, and then the DSP is further applied. Adaptive filtering is a subject of extensive research and many results have been reported [12]. Essentially, an equalizer of butterfly structure is preferred since adaptive algorithms must be applied to recover the polarization-division multiplexed (PDM) signals. The butterfly structured equalizer is shown in Figure 1b. Here we will first focus on the physical background and discuss the principle of coherent DSP for recovering the transmitted information. The details of algorithms will be explored after that. Why can the electrical field of the detected signal be recovered, and what are the physical reasons behind this? The answer lies in the model of the optical fiber transmission system. In order to simplify the discussion, we firstly consider the SMF system and linear impairments, which can be solved based on linear fiber optics. The SMF supports the propagation of two polarized lightwaves. Optical waves usually do not remain in the principle orientation when they are propagating through the optical fiber. In the long-haul SMF system, the polarization modes are strongly coupled. Signal propagation in each section can be modeled as a \(2\times 2\) matrix. When the fiber is longer than the correlation length, it can be modeled as a concatenation of multiple sections with independent characteristics as shown in Figure 2. The overall transmission matrix \(H\) is the product of all independent matrices. No matter what the orientation of polarization beam splitters is, signals in two polarizations (\(EX_{out}\) and \(EY_{out}\)) cannot be distinguished due to the serious coupling between two degenerate modes. Therefore, the necessary condition for recovering the transmitted signals is that the transmission matrix has to be reversible (or unitary for the best case). The output electrical field can then be linked to the input electrical field by the matrix. The polarization state variation during transmission can be deduced through a unitary Jones matrix \(J\). Regardless of the polarization dependent loss or signal attenuation (fully compensated through amplification), the overall transmission matrix \(H\) is definitely unitary and of course reversible. Compared with the CD, which can be regarded as constant, the channel transfer matrix varies with time, due to the rapid and random polarization coupling, so the adaptive DSP scheme has to be utilized. The equalizer with butterfly structure has been widely utilized. This type of equalizer can demultiplex the PDM signals with significant crosstalk and can also equalize linear impairments well. Figure 1: (**a**) Schematic of a typical coherent receiver; (**b**) Equalizer with butterfly structure. The reverse transfer matrix \(H^{-1}\) is comprised of four elements. Each element includes a tap-weight vector based finite impulse response (FIR) filter. The length of adaptive digital filter should be equal to, or a bit longer than, the impulse response spread of the distorted signal.