## 2 Principle  
As an example of illustration, in our proposed scheme, the signal frame structure is shown in Figure 1, QPSK signals are inserted into 16 QAM signals in the time domain periodically. The reason for inserting QPSK signals lies in the benefits of faster convergence and better tracking performance. The combination of QPSK pilot symbols and higher-order modulation signals such as 16 QAM or 64 QAM can improve the overall system performance. Another critical parameter is power ratio (PR) between QPSK and 16 QAM, and its investigation was carried out [22]. The power ratio between QPSK and 16 QAM is set to be 6.5 dB. In this paper, we will conduct an investigation on 16 QAM, 64 QAM is not included in this paper. The reason is the limit of experimental conditions. We plan to investigate 64 QAM in the next step further as soon as the conditions are improved. Assuming that chromatic dispersion, carrier frequency offset, and timing error have been compensated, the signals are only impaired by carrier phase noise and crosstalk between two polarizations. Therefore, the received signal is represented mathematically as follows:  
\[\left[ \begin{array}{c}Z_{x}(k)\\ Z_{y}(k)\end{array} \right]=\left[ \begin{array}{c}T_{x}(k)\\ T_{y}(k)\end{array} \right]\mathbf{D}(k)e^{i\theta(k)}+\mathbf{\omega}_{k}. \tag{1}\]  
where \(\left[ \begin{array}{c}Z_{x}(k)\\ Z_{y}(k)\end{array} \right]=\left[ \begin{array}{c}T_{x}(k)\\ T_{y}(k)\end{array} \right]\), \(\mathbf{D}(k),\ \theta(k),\ \mathbf{\omega}_{k}\) represent the \(k^{th}\) received signals, the \(k^{th}\) transmitted symbol, time-varying unitary polarization transformation matrix, carrier phase noise, and additive  
Figure 1: The frame structure of the combination of the pilot QPSK symbols and 16 QAM symbols. white Gaussian noise, respectively. Setting a fixed angular polarization rotation rate is a traditional way to simulate time-varying polarization. To evaluate the performance when dealing with the dynamic signal, we propose a time-varying polarization rotation rate method. The polarization rotation frequency drift rate is introduced, which is similar to acceleration. The polarization rotation under drift has the following expression:  
\[\mathbf{D}(k)=\left[ \begin{array}{cc}cos(T(k)\cdot W(k)+\psi)&sin(T(k)\cdot W(k)+ \psi)\\ -sin(T(k)\cdot W(k)+\psi)&cos(T(k)\cdot W(k)+\psi)\end{array} \right], \tag{2}\]  
\[W(k)=W_{0}+f*T(k). \tag{3}\]  
\(W(k)\), \(W_{0}\), \(f\), \(T(k)\), \(\psi\) describe the polarization rotation speed of \(k^{th}\) symbol, initial polarization rotation speed, polarization rotation frequency drift rate, the corresponding time of \(k^{th}\) symbol, and azimuth angle. The resultant EKF model is described mathematically by the following set of equations:  
\[\left[ \begin{array}{c}U_{x}(k)\\ U_{y}(k)\end{array} \right]=\mathbf{M}(k)\left[ \begin{array}{c}Z_{x}(k)\\ Z_{y}(k)\end{array} \right]+\mathbf{v}(k), \tag{4}\]  
\[\mathbf{M}(k)=e^{-j_{\theta}(k)}\left[ \begin{array}{cc}a(k)+jb(k)&c(k)+jd(k)\\ -c(k)+jd(k)&a(k)-jb(k)\end{array} \right], \tag{5}\]  
\[\mathbf{S}(k)=\mathbf{S}(k-1)+\mathbf{\varepsilon}(k). \tag{6}\]  
Equations (4) and (6) are the measurement equation and state equation, respectively. \(\mathbf{v}(k)\), \(\mathbf{\varepsilon}(k)\) are measurement noise and process noise respectively. \(\mathbf{S}(k)=\left[ \begin{array}{cccc}a_{k}&b_{k}&c_{k}&d_{k}&\theta_{k}\end{array}  \right]^{T}\). There are five elements in this state vector. The first four elements are related to polarization, and the last one is the element related to phase noise. By using the information of state vector, the received signal is recovered. \(\mathbf{U}_{k}\) is the recovered signal, \(\mathbf{U}_{k}=\left[ \begin{array}{cccc}U_{x}(k)&U_{y}(k)\end{array} \right]\). We find the closest ideal constellation to \(\mathbf{U}_{k}\) as the desired output \(\mathbf{U}_{c}\) and \(\mathbf{d}_{\mathbf{U}}=\mathbf{U}_{C}-\mathbf{U}_{k}\) is the residual measurement.