# Compensation of Limited Bandwidth and Nonlinearity for Coherent Transponder
## 3 Experimental Results
### Compensation for Bandwidth and Nonlinearity  
To demonstrate this technique, we generate 300 Gb/s signal using 64-QAM format at approximately 30 GBd. This maximum baud rate is limited by our DSP ASIC. The DSP ASIC currently has no built-in Volterra filter. Thus, our proposed DSP architecture cannot be realized directly through the data path. So, the pseudo-random binary sequence (PRBS) is first generated. Then, the Gray coding is applied to map the QAM signal so that there is only a one-bit difference between the adjacent constellation points. Next, the memoryless Volterra filter and the '_arcsin_' function are applied to the signal. In the end, the signal is convoluted with an FIR filter. There are multiple functionalities of the FIR filter: to overcome the limited bandwidth \(\alpha\) coming from the components on the data path; to perform Nyquist pulse shaping using root raised cosine (RRC) filter with the roll-off factor of 0.1 [20]; to compensate the skew among the tributaries [21]. The data stream is loaded into the on-chip RAM and the output of the DAC is set according to the data in the RAM.  
The coherent transmitter then converts the electrical signal to an optical signal. The optical output from the coherent transmitter is further captured by a coherent receiver, which consists of 90-degree optical hybrid, balanced photodiodes, and linear trans-impedance amplifiers. An integrated tunable laser assembly (ITLA) serves as the local oscillator (LO). The electrical signal is then converted from the analog domain to the digital domain through a real-time oscilloscope (80 G samples per second, 33 GHz bandwidth).  
The offline DSP is used to generate a constellation diagram and count the bit error rate. The DSP processing steps are similar to those described in [22]. First, a static equalization with a matched RRC filter is implemented in the time domain. Second, the chromatic dispersion is compensated in the frequency domain. Next, the polarization de-multiplexing and the compensation of polarization mode dispersion are implemented through an adaptive equalizer using the radius directed equalization (RDE) [23]. Then in the following steps, the timing is recovered using [PERSON]'s method, the frequency offset is estimated, the carrier phase is recovered, and the symbol is determined. The detailed diagram for the experimental setup is shown in Figure 6.  
Figure 5: Proposed DSP block diagram. (**a**) The proposed structure of the FIR filter is shown in detail. \(a_{0}\), \(a_{1}\), \(a_{2}\), and the rest are the tap coefficients determined from \(\alpha\). (**b**) The proposed structure of the Volterra filter is shown in detail.  
The constellation diagram can only be recovered when the limited bandwidth is compensated by the FIR filter. We compare the constellation diagram before and after the compensation of the nonlinearity, as shown in Figure 7. The gain of RF amplifier is set to a high value so that MZM is driven into a nonlinear region. As seen, without the compensation of nonlinearity, the constellation points are crowded and blurry. After the nonlinearity is mitigated, the constellation points are linearly distributed and separated.  
The improvement in \(\mathcal{Q}^{2}\) factor is approximately 0.3 dB. The improvement is smaller than expected due to the hardware implementation penalty at 64-QAM. In our setup, the implementation penalty is higher than expected, mainly due to the limit effective number of bit (ENOB) in DAC and ADC. The implementation penalty acts as a noise floor for our system. The contribution from the nonlinearity is small compared with the implementation penalty. After removing the nonlinearity, the performance of the coherent transponder is still dominated by the implementation penalty. Thus, the improvement from the mitigation of the nonlinearity is smaller than expected.