## 1 Introduction
However, this kind of test method apparently has not met the requirements of the next generation of intelligent optical networks, which changes from traditional out-of-band OSNR monitoring to in-band OSNR monitoring with higher monitoring accuracy. With the rapid development of high-speed coherent optical communication systems, some in-band OSNR monitoring technologies for coherent optical communication systems have been proposed, including Stokes parameters [8], delay-line interferometer (DLI) [9], four-wave mixing (FWM) [10], statistical moments [11], normalized autocorrelation function [12], error vector magnitude (EVM) [13], asynchronous delay sampling [14], and Golay sequence [15] based methods. However, many of the OSNR monitoring technologies mentioned above require additional hardware and monitoring equipment, which increases the complexity and cost of the system. Some technologies perform well under weak input power and weak nonlinearity systems and have high requirements for linear impairment compensation of receiving channel. In addition, some techniques based on training sequence may reduce the spectral efficiency of the system and need to be applied to the optical fiber nonlinear scene in the state of high transmission power. In view of this, the various OSNR monitoring technologies mentioned above have been unable to meet the continuous development of optical communication systems toward high-speed and high-frequency spectrum utilization. Therefore, in modern optical communication system, with the change of signal rate, transmission mode, and networking mode, the development of OSNR monitoring technology is also developing in the direction of high-speed, intelligent, digital and low-cost systems. In recent years, deep learning (DL) is the mainstream research direction of machine learning. It is also a part of the framework of artificial intelligence (AI) [16]. Its essence is to build a multilayercomputing model and learn the manifestation of data through multilayer abstraction. This technology has made technological breakthroughs in most research fields, including data mining [17], visual target detection [18], speech recognition [19], target recognition [20], quality of transmission (QoT) prediction [21], and medical diagnostic [22], as well as greatly improved the learning performance of related systems. Compared with traditional machine learning, DL is constantly deepened on the basis of artificial neural network theory. By adopting universal learning process and layer-by-layer learning feature representation, the impact of the introduction of artificial features on the system is weakened. At present, DL technology has also been well used in optical communications to promote the development of intelligent systems [23]. The methods of OPM using machine learning include support vector machine (SVM) [24], artificial neural network (ANN) [25], generalized regression neural network (GRNN) [26], and convolutional neural network (CNN) [27]. Recently, the use of deep neural network (DNN) instead of shallow ANN for optical performance monitoring has been extensively studied. DNN is a network structure with multiple hidden layers between the input and output layers. The additional layers of DNN will automatically extract input features, so that complex data can be modeled by fewer units than shallow networks [28]. In this paper, we demonstrate a technique which employs the improved binary particle swarm optimization (IBPSO) algorithm and the particle swarm optimization optimized deep neural network (PSO-DNN) structure in optical coherent receivers for OSNR monitoring. Particle swarm optimization (PSO) is a kind of swarm intelligence optimization algorithm in the field of computational intelligence. It is a global optimization intelligent algorithm that uses the foraging behavior characteristics of biological populations to simulate and solve practical optimization problems [29].