A domain‐adaptive method with cycle perceptual consistency adversarial networks for vehicle target detection in foggy weather

IET Intelligent Transport SystemsVolume 16, Issue 7 p. 971-981 ORIGINAL RESEARCHOpen Access A domain-adaptive method with cycle perceptual consistency adversarial networks for vehicle target detection in foggy weather Ying Guo, Corresponding Author Ying Guo [email protected] The Joint Laboratory for Internet of Vehicles, Ministry of Education - China Mobile Communications Corporation, Chang'an University, Xi'an, China Correspondence Ying Guo, The Joint Laboratory for Internet of Vehicles, Ministry of Education - China Mobile Communications Corporation, Chang'an University, Xi'an, China. Email: [email protected]Search for more papers by this authorRui-lin Liang, Rui-lin Liang The Joint Laboratory for Internet of Vehicles, Ministry of Education - China Mobile Communications Corporation, Chang'an University, Xi'an, ChinaSearch for more papers by this authorYou-kai Cui, You-kai Cui Institute of Intelligent Transportation, Zhejiang Institute of Communications Co., Ltd., Hangzhou, ChinaSearch for more papers by this authorXiang-mo Zhao, Xiang-mo Zhao The Joint Laboratory for Internet of Vehicles, Ministry of Education - China Mobile Communications Corporation, Chang'an University, Xi'an, ChinaSearch for more papers by this authorQiang Meng, Qiang Meng The Joint Laboratory for Internet of Vehicles, Ministry of Education - China Mobile Communications Corporation, Chang'an University, Xi'an, China Institute of Intelligent Transportation, Shandong Provincial Communications Planning and Design Institute Group Co., Ltd., Jinan, ChinaSearch for more papers by this author Ying Guo, Corresponding Author Ying Guo [email protected] The Joint Laboratory for Internet of Vehicles, Ministry of Education - China Mobile Communications Corporation, Chang'an University, Xi'an, China Correspondence Ying Guo, The Joint Laboratory for Internet of Vehicles, Ministry of Education - China Mobile Communications Corporation, Chang'an University, Xi'an, China. Email: [email protected]Search for more papers by this authorRui-lin Liang, Rui-lin Liang The Joint Laboratory for Internet of Vehicles, Ministry of Education - China Mobile Communications Corporation, Chang'an University, Xi'an, ChinaSearch for more papers by this authorYou-kai Cui, You-kai Cui Institute of Intelligent Transportation, Zhejiang Institute of Communications Co., Ltd., Hangzhou, ChinaSearch for more papers by this authorXiang-mo Zhao, Xiang-mo Zhao The Joint Laboratory for Internet of Vehicles, Ministry of Education - China Mobile Communications Corporation, Chang'an University, Xi'an, ChinaSearch for more papers by this authorQiang Meng, Qiang Meng The Joint Laboratory for Internet of Vehicles, Ministry of Education - China Mobile Communications Corporation, Chang'an University, Xi'an, China Institute of Intelligent Transportation, Shandong Provincial Communications Planning and Design Institute Group Co., Ltd., Jinan, ChinaSearch for more papers by this author First published: 13 April 2022 https://doi.org/10.1049/itr2.12190AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Foggy weather can cause such problems as blurred image information and the loss of image details, which may pose great challenges to road traffic target detection based on images and videos. In this study, we propose a domain-adaptive road vehicle target detection method to implement domain adaptation for the real foggy scene. We firstly constructed a highway vehicle detection dataset with foggy images (HVFD), which contains normal weather images and foggy images and provides a complete data support for vehicle detection based on computer vision. Secondly, by improving CycleGAN we designed an improved generative confrontation network (CPGAN), which realised the style transfer between foggy images and normal weather images. Finally, we formulated a YOLOv4 target detection framework according to the domain adaptation based on the pre-trained YOLOv4 fog vehicle detection model. The experimental results show that the method we put forward can effectively improve vehicle detection performance and reduce the work of manually labelling a large number of foggy image tags, which has a strong generalisation ability for computer vision-based applications in low-visibility weather. 1 INTRODUCTION With the continuous development and universal application of deep learning [1] and computer vision technology [2-4], the vision-based vehicle target detection has become the main technical method for road traffic target detection [5]. Under the conventional traffic environment, the current relevant research results can achieve good detection effects [6-9]. However, in the foggy environment, atmospheric light is scattered by water vapour and particles in the air, resulting in the overall whiteness of the image collected by the camera, the reduction of image contrast and the lack of image details, which significantly increase the difficulty of traffic target detection [10], especially on the highway when foggy weather is very common due to the influence of geographical characteristics and local microclimate environment. In addition, the high position of the camera and the fast speed of traffic flow may also make the traffic target detection more challenging. Many experts have studied the visual detection method of traffic targets in fog. One research idea is to execute image dehaze algorithm first. At present, the mainstream research direction focuses on the haze removal algorithms based on the prior knowledge or deep learning. The representative studies include a bright algorithm based on dark channels proposed by He Kaiming et al. [11], a colour attention priority algorithm by Zhu et al. [12], the DehazeNet method by Cai et al. [13], the AOD-NET method by Li et al. [14], the MSCNN approach by Ranftl et al. [15], and the cycle-dehaze's single image dehaze network by Denizen et al. [16]. Although the above algorithm can achieve good defogging effect, the defogging method based on image restoration focuses on the target texture and surface information in the image and cannot make full use of higher-level information, such as shape, contour, which usually loses effective information related to target detection and leads to poor detection performance. Therefore, the idea of giving priority to defogging treatment has strong limitations. Another common technical method for fog vehicle target detection is the use of deep learning. In the detection process based on deep learning, a significant number of manual vehicle bounding boxes are required to construct a dataset to train a convolutional neural network (CNN) [17-19]. To reduce the workload of manual marking, many researchers have enhanced foggy photos by inverting, cropping, zooming, and other processes [20], but there are still a limited number [21] of vehicle detection datasets that can be employed in foggy conditions. This is related to the difficulties of acquiring foggy image data as well as the labelling process. In response to this issue, some relevant researches presented domain adaption (DA) methods [22] in recent years. The idea is to map the data features of various domains in the same feature space, usually with two different datasets, and then use the annotated data from the source domain to solve the target domain's learning problem, or utilise the source domain data to improve the target domain's training effect. Target detection can adapt to different scenarios more easily using DA method, overcoming the challenge of getting labelled data in new target fields in practical applications, and therefore boosting the universality of deep learning models and the capacity to transfer and reuse information [23]. Recently, the methods based on generative adversarial networks (GAN) [24] have achieved great success [25-28]. Among the several GAN, the cycle-consistent adversarial network (CycleGAN) [29] has received a lot of attention. CycleGAN enables style transfer from one domain to another [30], and the confrontation training makes the generated image very similar to the real target domain image, which enables to implement domain confusion. However, the DA method based on CycleGAN only analyses pixel-level alignment of cross-domain and ignores image depth, which is easy to lose graphic details in adversarial training. After processing DA method, the next work is to detect the object of interest in the image. Domain adaptive faster R-CNN [31] is the first algorithm model that applies adversarial training to DA based target detection, since then, Faster R-CNN has become the most widely studied domain adaptive baseline network [32-35]. However, as a typical TWO-STAGE target detector, Faster R-CNN [36] has a long inferring time because of the network structure, which obtains the region proposal first and then classifies it. The ONE-STAGE target detector YOLOv4 [37, 38] can ensure high detection speed and provide similar or even better accuracy than faster R-CNN. Some research results show that it is feasible to construct Da target detection framework based on YOLOv4 [39, 40]. Some researchers use other networks such as RetinaNet as the baseline network of domain adaptation and have achieved good achievements [34]. Although scholars have achieved many success in domain adaptation, However, the existing studies mostly use public datasets for training and testing. There are still many changes from public datasets to practical applications, such as image style, background, brightness, image quality etc. And there is still a considerable domain transfer in the training set and test set, and the adaptability to the real traffic environment needs to be verified. According to the consideration mentioned above, this paper studies the use of deep learning and DA methods to implement vehicle target detection in real highway foggy environment. Our main purpose is to build an end-to-end object detection model that enables rapid real-time detection, and no matter what the weather is, we can detect without defogging or other preprocessing step We changed the CycleGAN network structure to achieve better image transfer effects from normal weather to foggy days without the need for paired training. After conducted DA algorithm, we got a target detection model in foggy environments by training YOLOv4 network. In the experimental part, we first verified the image transfer performance of CPGAN, and then verified the effectiveness of the proposed method on our data set through ablation experiment. Our contributions are summarised into the following four aspects: By improve the architecture of cycleGAN, a new countermeasure generation network CPGAN was advanced to improve image style transfer effect (image dehaze). We propose an end-to-end domain-adaptive object detection framework based on YOLOv4, during the training and testing phases, it requires neither paired samples of foggy and ground truth images nor any parameters of atmospheric scattering model, which Addresses an issue with the lack of a foggy dataset The proposed method can meet the demand of real-time detection, the average time cost of our method is only 0.041 s. Based on the real China highway monitoring data, the highway vehicle detection dataset with foggy images (HVFD) was constructed. The organisation of the paper is as follows. In Section 2, we introduce the domain adaptive method, and we propose an adversarial generation network CPGAN. In Section 3, we present the domain-adaptive YOLOv4 target detection framework based on pre-trained CPGAN with its components. In Section 4, we introduce the dataset we built and the other datasets we used in our experiments. Section 5 presents experimental results and Section 6 concludes the paper. 2 DOMAIN ADAPTIVE METHOD BASED ON IMPROVED CYCLEGAN Aiming at the problem that CycleGAN cannot complete the domain adaptation of image features, this paper investigates and improves CycleGAN's network structure, further reduces the difference in domain distribution, and implements the usage of annotated regular weather traffic images (source domain) to aid complete vehicle detection in foggy images (target domain) that are not annotated. The source domain X is an annotated normal weather traffic image set (each vehicle in each image is manually annotated with a vehicle bounding box), while the target domain Y is an unannotated foggy traffic image set. The research idea of this paper is to complete the mapping between the source domain X and the target domain by training two generators and two adversary discriminators. CycleGAN defines two generators G X → Y ${G_{X \to Y}}$ 、 F Y → X ${F_{Y \to X}}$ and two discriminators D Y ${D_Y}$ , D X ${D_X}$ for conversion between X and Y. G X → Y ${G_{X \to Y}}$ is the learning domain X to Y transfer function, F Y → X ${F_{Y \to X}}$ is the learning domain Y to X transfer function; G X → Y ${G_{X \to Y}}$ and F Y → X ${F_{Y \to X}}$ are the two discriminators, respectively; D Y ${D_Y}$ is used to judge whether the image is from the real image of Y or the image synthesised by G X → Y ${G_{X \to Y}}$ which approximates the target domain. D X ${D_X}$ is used to determine whether the picture is a real image from X or an image synthesised by F Y → X ${F_{Y \to X}}$ that approximates the source domain. The total loss function of the CycleGAN network is as follows: L CycleGAN G , F , D X , D Y = L GAN G , D Y , X , Y + L GAN F , D X , Y , X + λ L cycle ( G , F ) \begin{equation} \def\eqcellsep{&}\begin{array}{rcl} {{\cal L}_{{\rm{CycleGAN}}}}\left( {G,F,{D_X},{D_Y}} \right) &=& {{\cal L}_{{\rm{GAN}}}}\left( {G,{D_Y},X,Y} \right)\\[8pt] && +\; {{\cal L}_{{\rm{GAN}}}}\left( {F,{D_X},Y,X} \right)\\[8pt] && +\; \lambda {{\cal L}_{{\rm{cycle}}}}(G,F) \end{array} \end{equation} (1) Where λ represents the balance weight of the cycle consistency loss, L cycle ${{\cal L}_{\rm{cycle}}}$ is the cycle consistency loss in the cycle architecture, and L GAN ${{\cal L}_{{\rm{GAN}}}}$ is the counter-training loss. Assuming that after the image conversion cycle, each image in the domain X can be converted back to the original image, that is x → G ( x ) → F ( G ( x ) ) ≈ x $x \to G(x) \to F(G(x)) \approx x$ , this procedure is known as forwarding cycle consistency, as shown in Figure 1(a). Similarly, for each image y in the domain Y, G, and F should meet the reverse cycle consistent: y → F ( y ) → G ( F ( y ) ) ≈ y $y \to F(y) \to G(F(y)) \approx y$ as shown in Figure 1(b). Cycle consistency loss can be used to incentivise this behaviour: L cycle ( G , F ) = E x ∼ p data ( x ) F ( G ( x ) ) − x 1 + E y ∼ p data ( y ) G ( F ( y ) ) − y 1 \begin{eqnarray} {\mathcal{L}}_{\rm cycle}(G,F)&=&{{\mathbb{E}}_{x\sim {{p}_{{\rm data }}}(x)}}\left[ F(G(x))-{{x}_{1}} \right] \nonumber\\ && +\,{{\mathbb{E}}_{y\sim {{p}_{{\rm data }}}(y)}}\left[ G(F(y))-{{y}_{1}} \right] \end{eqnarray} (2) FIGURE 1Open in figure viewerPowerPoint Network structure of CPGAN. (a) Forward cycle, (b) Backward cycle FIGURE 2Open in figure viewerPowerPoint Domain-adaptive YOLOv4 vehicle detection model framework The adversarial training loss L GAN ${{\cal L}_{{\rm{GAN}}}}$ is expressed as: L GAN G , D Y , X , Y = E y ∼ p data ( y ) log D Y ( y ) + E x ∼ p data ( x ) [ log 1 − D Y ( G ( x ) ) ] \begin{eqnarray} {{\mathcal{L}}_{{\rm GAN}}}\left( G,{{D}_{Y}},X,Y \right)&=&{{\mathbb{E}}_{y\sim {{p}_{{\rm data}}}(y)}}\left[ \log {{D}_{Y}}(y) \right] \nonumber\\ && +\,{{\mathbb{E}}_{x\sim {{p}_{{\rm data}}}(x)}}[ \log \left( 1-{{D}_{Y}}(G(x))] \right.\qquad \end{eqnarray} (3) To train the generator and discriminator, the following requirements must be met: G ∗ , F ∗ = arg min G , F , D x , D y max L CycleGAN G , F , D x , D y \begin{equation}{G^ * },{F^ * } = \mathop {\arg \min }\limits_{G,F,{D_x},{D_y}} \max {{\cal L}_{{\rm{CycleGAN}}}}\left( {G,F,{D_x},{D_y}} \right)\end{equation} (4) Throughout the cycleGAN network confrontation training procedure, the conversion from foggy to normal weather is shown to be less successful, which further lowers the entire cycle training effect. Because a lot of colour and detail information is lost in the foggy image and after a forward or backward cycle, the cycle consistency loss is insufficient to restore all of the images. So this paper enhances cycleGAN, adds perceptual consistency loss based on traditional cycleGAN's cycle consistency loss, and proposes cycle perceptual consistency adversarial networks to achieve image pixel-level and feature-level cross domain adaptation. The CPGAN network is shown in Figure 1 and the above-mentioned loss of perceived consistency is stated as follows: L Perceptual = ϕ ( x ) − ϕ ( F ( G ( x ) ) ) 2 2 + ϕ ( y ) − ϕ ( G ( F ( y ) ) ) 2 2 \begin{equation} \def\eqcellsep{&}\begin{array}{rcl} {{\cal L}_{{\rm{Perceptual}}}} &=& \left\| {\phi (x) - \phi (F(G(x)))} \right\|_2^2\\[8pt] && +\; \left\| {\phi (y) - \phi (G(F(y)))} \right\|_2^2 \end{array} \end{equation} (5) Where ϕ is the feature extractor of the second and fifth pooling layer of the VGG16 [17] network. Unlike the cycle consistency loss that only aligns the domain differences between two different domains at the pixel-level, the cycle perceptual consistency loss aims to perform cross domain adaptation at the feature level. Therefore, the total network loss of CPGAN is expressed as: L CPGAN G , F , D x , D y = L GAN G , D Y , X , Y + L GAN F , D X , Y , X + λ ∗ L cycle ( G , F ) + γ ∗ L Perceptual ( G , F ) \begin{equation} \def\eqcellsep{&}\begin{array}{rcl} {{\cal L}_{{\mathop{\rm CPGAN}\nolimits} }}\left( {G,F,{D_x},{D_y}} \right) &=& {{\cal L}_{{\rm{GAN}}}}\left( {G,{D_Y},X,Y} \right)\\[8pt] && +\; {{\cal L}_{{\rm{GAN}}}}\left( {F,{D_X},Y,X} \right)\\[8pt] && +\; \lambda *{\cal L}_{{\rm{cycle}}}(G,F)\\[8pt] && +\; \gamma * {{\cal L}_{{\rm{Perceptual\;}}}}(G,F) \end{array} \end{equation} (6) G ∗ , F ∗ = arg min G , F , D x , D y max L CPGAN G , F , D x , D y \begin{equation}{G^ * },{F^ * } = \mathop {\arg \min }\limits_{G,F,{D_x},{D_y}} \max {{\cal L}_{{\rm{CPGAN}}}}\left( {G,F,{D_x},{D_y}} \right)\end{equation} (7) Where λ denotes the cycle consistency loss balance weight, while γ denotes the perceptual consistency loss balance weight. Finally, using Equations (6) and (7), the CPGAN design is optimised and generators G X → Y ${G_{X \to Y}}$ and F Y → X ${F_{Y \to X}}$ are generated. Since L cycle ${{\cal L}_{{\rm{cycle}}}}$ and L Perceptual ${{\cal L}_{{\rm{Perceptual\;}}}}$ extract pixel-level and feature-level losses, weight γ cannot be too large; otherwise, colour information will be lost following domain translation, and L cycle ${{\cal L}_{{\rm{cycle}}}}$ requires a greater weight than L Perceptual ${{\cal L}_{{\rm{Perceptual\;}}}}$ . 3 DA-BASED YOLOv4 VEHICLE TARGET DETECTION METHOD In this article, we define the set of labelled normal weather traffic images (with vehicle bounding boxes labelled in each image) is the Source domain as X, and an unlabelled set of foggy weather traffic images is the Target domain as Y. In this study, we have two tasks to solve: 1. Detect the vehicles in normal weather by YOLOv4 network; 2. Detect the vehicles in foggy weather by YOLOv4 with DA method. For task 1, we perform detection by training YOLOv4 network. Vehicle detection of traffic surveillance images in normal weather is a common supervised learning problem. YOLOv4 is an end-to-end learning system whose network parameters can be learned by backward propagation based on gradient descent using only input and output. The structure of YOLOv4 is divided into four major sections. They are Input, Backbone, Neck, and Prediction. YOLOv4 retains the network structure of YOLOv3 [20] using multi-scale features for object detection, integrates three scales (13×13, 26×26, and 52×52), and independently detects on the fusion feature map of multiple scales, in particular, the detection of small objects achieves better results. For Task 2, we need to train two networks, the CPGAN network and the YOLO V4 network, which use different data sets. Firstly, we train the CPGAN network, with normal weather images as the source domain X, and foggy images as the target domain Y. These images do not need to be paired, and do not contain any labels. Through adversarial training, the discriminators D X ${D_X}$ and D Y ${D_Y}$ are gradually optimised so that the images generated by generators G and F cannot be distinguished from the real images. After training, we are able to generate fake images of normal weather or foggy weather via generators F and G. Next, we use another dataset, HVFS, which contains labelled highway vehicle images in normal weather. With the weights of normal-foggy obtained in the previous step, we convert these images from normal weather to foggy weather. While the style transfer is completed, the pseudo-foggy images generated by the real marked image of normal weather are very similar to the real foggy images, so they have the same vehicle targets and can share target labels. In the process of style transfer, there is a certain probability of failure, so we need to remove this failure images manually. Finally, we send these images and labels to YOLOv4 network for training, and we get a vehicle detection weight of foggy weather, which can be directly used for vehicle target detection in foggy days without pre-processing such as image defogging. 4 DATASET After years of accumulation and development, research institutions have introduced different artificially labelled road scene public datasets, as shown in Figure 3. Although vehicle target detection has achieved good performance in public datasets, it still faces many changes in practical applications, such as image style, background, brightness, image quality changes. On the other hand, there are few vehicle detection datasets available of foggy days, due to the rarity of foggy images and the difficulty of marking images. In complex weather scenarios of different categories, such as fog, rain, and snow, it is not realistic to retrain a large number of images that mark bounding boxes. FIGURE 3Open in figure viewerPowerPoint Different datasets for vehicle detection. From top to bottom, left to right are (a) KITTI (b) BDD100K (c) Cityscapes (d) Foggy Cityscapes (e) HVFD (normal weather) created in this article (f) HVFD (foggy weather) In this study, we used two different datasets when training CPGAN network and YOLOv4 fog detection model: (1) Outdoor image dataset. In this article, a total of 4000 images from the public dataset RESIDE (realistic single image dehazing dataset) [41] which include RTTS (real-world task-driven testing set) were selected. These images including 2000 images of normal weather and 2000 images of foggy weather. The image pixels are 480×270. The image background includes a variety of outdoor scenes such as scenery, roads and buildings. 500 normal weather images and 500 fog images were randomly selected from the dataset as the test set, and the other 3000 normal weather images and fog images were used as the training set. (Figure 3 ) From top to bottom, left to right are (a) KITTI (b) BDD100K (c) cityscapes (d) foggy cityscapes (e) HVFD (normal weather) created in this article (f) HVFD (foggy weather) (2) HVFD (highway vehicle detection dataset with foggy images, HVFD). In view of the research needs, we created a highway scene image dataset based on the surveillance video data of a highway in China call HVFD. The composition of HVFD show in Table 1, and the dataset has been published in: https://drive.google.com/file/d/1ru9kUWOtcyRM3NKX4497DoCPHhBcZLNy/view?usp = sharing. HVFD contains 2400 traffic images of a highway scene in China, including traffic scenes such as bends, mountain roads, ramps, high traffic, and the dataset image size is 1280×720. HVFD contains three parts: normal weather training set, normal weather test set, and fog test set. The training set of normal weather contains 1931 manually marked normal weather traffic images, and 379 images were randomly selected from the normal weather training set as the normal weather test set; the foggy day test set contains 814 artificially marked foggy highway traffic images. The foggy day test set contains three scenarios, called Foggy_1, Foggy_2, Foggy_3. Among them, the visibility of images in Foggy_1 is between 100–200 m, that of Foggy_2 image are between 150–300 m, and that of Foggy_3 image is between 40–150 m. The visibility of Foggy 3 image is lower than that of Foggy_1 and Foggy_2, and the number of vehicles in Foggy_1 is larger than the other two scenes. TABLE 1. Objects of HVFD ImageNum Car Bus Van Truck SUV Train Normal weather 1931 10894 201 165 3680 306 Test Normal weather 379 2136 39 29 717 65 Foggy_1 432 492 37 18 1150 13 Foggy_2 244 1023 65 29 805 464 Foggy_3 138 518 47 33 1398 18 For vehicles running on the road, when the visibility is greater than 500 m, the impact on road traffic is very small [42], while when the visibility is less than 50 m, it is difficult for cameras to capture moving vehicles. Therefore, the visibility of fog images involved in this paper is between 50–500 m. 5 EXPERIMENTS AND ANALYSIS CPGAN foggy day adaptive training In this study, model training was carried out on a workstation equipped with 2.9 GHz CPU, 16 GB memory and NVIDIA Quadro P2200 TI GPU. In the CPGAN network training process, the balance weight λ was set to 10, the weight of cycle perception consistency loss γ was set to 0.0001, the training picture size was set to 256 × 256, the batch size was set to 1, the Adam optimiser learning rate was initialized to 0.0002, and the image buffer size was set to 50. Figure 4 shows the conversion effect of CPGAN network training iterations of 433, 17,976, 56,590, and 70,797,wherein column (1) is a real fog image, column (2) is a normal weather image generated by a real fog day image, column (3) is a fog day image generated by a synthetic normal weather, column (4) represents a true normal weather image, column (5) represents a fog day image generated by real normal weather, and column (6) represents a normal weather image generated by a composite fog day image. It can be seen that with the increase of iterations, the image conversion effect is becoming clearer, shown as follows: the background is continuous, the target is protruding and has not deformed, and no new interference is generated during the conversion. Finally, we saved the model for 80,000 iterations. FIGURE 4Open in figure viewerPowerPoint Comparison of CPGAN network training results. (a) Images trained 433 steps, (b) images trained 17,976 steps, (c) images trained 56,590 steps, (d) images trained 70,796 steps Figure 5 shows the loss curves of CPGAN, it can be seen that this method has fast convergence speed and the training process is stable. However, after about 40,000 iterations, the loss curve of F and G shows slightly overfitting, and other loss curves continue to descend slowly. FIGURE 5Open in figure viewerPowerPoint Train loss of GPGAN In order to verify the performance of the proposed CPGAN network, 1000 generated images from foggy image fields were randomly selected as the test data. We evaluated CPGAN and other methods by calculating the kernel inception distance (KID) [43] and the Frechet inception distance (FID) [44]. KID calculates the maximum average difference between two sets of features as the distribution distance between the two sets of data, and KID has an unbiased estimate, close to human perception. FID evaluates the similarity between the feature vector of a real image and the feature vector of the generated image to evaluate the similarity of the two sets of images, the mean and variance obtained by the inception network [45]. It means the lower the score of the two indicators, the closer the distribution distance between the real image and the generated image. We adopted three popular image style transfer methods as a reference: CycleGAN, UNIT [46], and MUNIT [47]. UNIT assumes that the data space of the source and target domains shares the same hidden space and develops an unsupervised image translation network in conjunction with VAE-GAN and cycle consistency loss; MUNIT assumes that the data of the source domain and the target domain have a shared hidden space, and then generates the results by exchanging these decoupled content and style hidden spaces. The test results are shown in Table 2. It can be found that CPGAN obtains the lowest kid and FID scores in all test methods, indicating that the converted image is closer to the real result, that is, the conversion effect is better. TABLE 2. Test results of image style-transfer on sunny and foggy weather FID KID CPGAN 101 0.019 CycleGAN 106 0.026 UNIT 105 0.029 MUNIT 108 0.036 Figu