## 2 Background and Problem Setting  
### Long-Haul WDM Optical Communication System Using DP-16 QAM Modulation Scheme  
The conceptual diagram of the chaotic optical communication with two wavelength multiplexed channels using DP-16 QAM for each channel is exhibited and described in Figure 1. In this diagram, one channel is for chaotic optical communication (COC), and another is for conventional fiber optic communication (CFOC). Each channel, defined by the wavelength of its carrier wave, is coupled into the same optical fiber in DP-16 QAM data format [45; 46]. The chaos cryptography technique encrypts some crucial information at a given WDM channel \(\lambda_{c}\). In this paper, the chaos sequence is created by a logistic map using a retrieval rule as follows [17]:  
\[z_{n+1}=4z_{n}(1-z_{n}), \tag{1}\]  
where \(n=1,2,3\ldots\) is a positive integer, and \(z_{0}=[0,1]\) is a starting real number between 0 and 1. Therefore, it is easy to see that \(z_{n}\) always satisfies \(0<z_{n}<1\). The chaotic function has a probability distribution density as follows:  
\[p(z)=\frac{1}{\pi\sqrt{z(1-z)}}\quad\text{for }0<z<1. \tag{2}\]  
In this proposed system, both the laser diode transmitter (LDT) and the laser diode receiver (LDR) are constructed from a single-mode semiconductor laser with an external reflector and the same configuration. Both the COC channel and the CFOC channel are (de)multiplexed by means of a wavelength (de)multiplexer in the C-band of the third telecom window. The transmitter laser (LDT) of the COC channel emits a chaotic carrier, and an optical isolator (ISO) is used to ensure unidirectional transmission. The original message is superimposed on a chaotic carrier by the chaos masking sequence (CMS). As seen in Figure 1, the chaotic signal is implemented by simply adding the CMS signal to the output of the conventional 16 QAM modulated signal. On the receiver side, the chaotic signal is decoded by simply subtracting the received COC signal and a CMS signal that is synchronized to the form of the transmitter side. Information propagating along the long-haul optical fiber link is greatly affected by fiber loss, dispersion, and nonlinear effects. We place an erbium-doped fiber amplifier (EDFA) for the fiber loss compensation and a dispersion compensating fiber (DCF) for the dispersion compensation. The EDFA amplifier has the gain coefficient \(G\), determined by the formula \(G=\alpha L\). Here, \(\alpha\) is the fiber loss coefficient, \(L\) is the total length of the transmission link, and \(L_{DCF}\) is the length of the DCF fiber. On the receiver, the optical signal is de-multiplexed by the wavelength de-multiplexer (DEMUX) and photodetector after propagating over a long-haul section. Figure 1: Conceptual Conceptual diagram of the COC and CFOC channels in the long-haul WDM optical communication system using the \(DP-16 QAM\) modulation scheme. The dynamic behaviors of a couple of transmitters and receivers that are set up in a COC system can be described by well-known Lang-Kobayashi rate equations, with optical feedback and injection terms [29; 45] as follows:  
\[ \begin{split}&\frac{dE_{T,R}(t)}{dt}=\frac{1}{2}(1+i\psi)\Bigg{[} \frac{G[N_{T,R}(t)-N_{0}]}{1+\varepsilon|E_{T,R}(t)|^{2}}\Bigg{]}E_{T,R}(t),\\ &+k_{T,R}E_{T,R}(t-\tau)\exp(-\mathrm{i}\omega\tau)+k_{\mathrm{irj }}E_{\mathrm{ext}}(t),\end{split}  \tag{3}\]  
where \(E\) and \(N\) correspond the slowly varying complex electric field amplitudes and the carrier density in the laser cavity, respectively; \(T\) and \(R\) stand for transmitter and receiver; \(\omega\) is the angle frequency of the free operation laser; \(\tau\) is the round-trip time; and \(E_{\mathrm{est}}\) is the transmission link. Then, the COC signal is decoded, and the DP-16 QAM signal is also demodulated to recover the baseband signals of the external electric field amplitude at the input of the receiver. For the proposed COC and CFOC parallel transmission system, we consider a two-channel WDM system (each subscript denotes the channel number).