## 3 Proposed Diverse and Realistic Marine Image Generation Framework
### Diverse Realistic Marine Image Generator  
We customized the MUNIT [22] by adding horizon-targeted loss to enable the generator to transform a virtual image into a corresponding realistic image. When training the generator, each feature is extracted using the style encoders \(E_{style}^{virtual}\)and \(E_{style}^{real}\), and the content encoders, \(E_{cont}^{virtual}\)and \(E_{cont}^{real}\), in the virtual and real-world domains, respectively. The extracted features intersect with \(G_{virtual}\Big{(}E_{style}^{real}(x_{real})\), \(E_{content}^{virtual}(x_{virtual})\Big{)}\) and \(G_{real}\Big{(}E_{style}^{virtual}(x_{virtual})\), \(E_{content}^{real}(x_{real})\Big{)}\) through the decoder to synthesize each content and style aspect, wherein the content is shared regardless of the domain, and the style is domain-specific. To generate the stylized image output, the intermediate result is once again provided as input to the encoder and decoder and subsequently synthesized to return to the style of the input image. The network is trained by calculating the loss based on the differences between the output and input images, as illustrated in Figure 4.  
Figure 3: Architecture of the proposed diverse realistic marine image generation framework. There are five main types of loss functions: image reconstruction loss, latent content loss, latent style loss, adversarial GAN loss, and horizon-targeted loss. In image reconstruction loss, \(L_{image}^{virtual}\) and \(L_{image}^{real}\) are loss functions that calculate and reflect the difference between images converted to other domains by the encoder and decoder. In latent content and style loss, \(L_{content}^{virtual}\), \(L_{content}^{real}\), \(L_{style}^{virtual}\), and \(L_{style}^{real}\) are loss functions for the latent space. In horizon-targeted loss, \(L_{horizon}^{virtual}\) and \(L_{horizon}^{real}\) are loss functions that calculate and reflect the difference between images converted to other domains based on the RoI (region of interest) containing the horizon. In adversarial GAN loss, \(L_{adv}^{virtual}\) and \(L_{adv}^{real}\) are loss functions that adjust the feature distribution of the input image to match that of the target domain. 1. Image reconstruction loss: Given an image sampled from the data distribution, we reconstruct the entire image after encoding and decoding. \(L_{image}^{real}\) is defined in a similar manner. \[L_{image}^{virtual}=\mathbb{E}_{x_{1}\sim p(x_{1})}\Big{[}\parallel G_{virtual }(E_{content}^{virtual}(x_{1}),\ E_{style}^{virtual}(x_{1}))-x_{1}\parallel\Big{]}\] (1)  
Figure 4: Architecture of training process of the generator. 2. Latent content loss: Given a latent content code sampled from the latent distribution, we calculate the latent content code required to reconstruct it after decoding and encoding. \(L_{content}^{real}\) is defined in a similar manner. \[L_{content}^{virtual}=\mathbb{E}_{c_{1}\sim p(c_{1}),s_{2}\sim p(s_{2})}\Big{[} \parallel E_{content}^{real}(G_{real}(c_{1},s_{2})-c_{1}\parallel\Big{]}_{1}\] (2)
3. Latent style loss: Given a latent style code sampled from the latent distribution, we calculate the latent style code required to reconstruct it after decoding and encoding. \(L_{style}^{virtual}\) is defined in a similar manner. \[L_{style}^{real}=\mathbb{E}_{c_{1}\sim p(c_{1}),s_{1}\sim p(s_{1})}\Big{[} \parallel E_{style}^{real}(G_{real}(c_{1},s_{2})-s_{1}\parallel\Big{]}\] (3)
4. Adversarial GAN loss: We employ GANs to match the distribution of virtual-domain images to that of the real-domain data. \(L_{adv}^{real}\) is defined in a similar manner. \[L_{adv}^{real}=\mathbb{E}_{c_{1}\sim p(c_{1}),s_{2}\sim p(s_{2})}[log(1-D_{ real}(G_{real}(c_{1},s_{2})))]+\mathbb{E}_{x_{2}\sim p(x_{2})}[log\ D_{real}(x_{2})]\] (4)
5. Horizon-targeted loss: We calculate the loss by comparing the RoI in the input data to that in the output data. \(L_{horizon}^{real}\) is defined in a similar manner.