# Translation-classification loss for SAR image understanding with deep learning

Antoine Bralet, Abdourrahmane M Atto, Jocelyn Chanussot, Emmanuel

Trouvé

► To cite this version:

Antoine Bralet, Abdourrahmane M Atto, Jocelyn Chanussot, Emmanuel Trouvé. Translation-classification loss for SAR image understanding with deep learning. Computer Vision and Image Understanding, 2025, 257, pp.104374. 10.1016/j.cviu.2025.104374. hal-05058869

HAL Id: hal-05058869

https://hal.science/hal-05058869v1

Submitted on 7 May 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# Translation-classification loss for SAR image understanding with deep learning

<figure><img src="image_3.png" /></figure>

Check for
updates

Antoine Bralet<sup>a,*</sup>, Abdourrahmane M. Atto<sup>a</sup>, Jocelyn Chanussot<sup>b</sup>, Emmanuel Trouvé<sup>a</sup>

## ARTICLE INFO

Communicated by David Jacobs

Dataset link: BigEarthNet-MM (Reference data)

MSC:

68T45

68U10

68T01

Keywords:

SAR-to-optical translation

Remote sensing

Deep learning

Translation-Classification Loss

## ABSTRACT

SAR-to-optical translator networks are especially used to overcome the lack of optical images under cloudy conditions. Those translations being used for downstream tasks, they require the reconstruction of reliable patterns with respect to the underlying objects. In this paper, we propose a novel training strategy to account for land-cover complexity through a conjoint Translation-Classification Loss (TCL). The proposed loss evaluates the classifiability of translated images with a pre-trained land-cover classifier by assessing the reliability of its predictions and the relevance of its extracted hidden features. This new loss is applied to nine translators from the literature and to a tenth architecture introduced in the paper. Experiments show that applying the TCL not only improves the credibility of structures, patterns and textures but it also allows for better class discrimination and transitions while avoiding unreliable hallucinated artifacts produced by standard losses in adversarial approaches.

## 1. Introduction

Due to their wide spatial coverage and high revisit frequency (up to 5 days for Sentinel-2), optical remote sensing images are widely exploited for land-cover segmentation (Interdonato et al., 2018; Chatterjee and Poullis, 2021), object recognition (Lee and Park, 2023), disaster damage assessment (Adriano et al., 2020), change detection (Ghorbanzadeh et al., 2022) or disaster monitoring and early warning systems (Desrues et al., 2019). These methods – and especially multi-temporal ones (Chelali et al., 2021) – require a regular availability of optical images whereas optical sensors are sensitive to meteorology and cloud cover. Therefore, multiple modality translation deep networks are developed to compute missing acquisitions from other remote sensing sensors. In particular, Synthetic Aperture Radar (SAR) to optical image translation ensures the generation of optical-like images independently from weather conditions and solar exposure. But this task presents three challenges. First, it is an ill-posed problem because SAR images provide spectral information in a single band (C-band for Sentinel-1) with strong structural properties while we aim at reconstructing multispectral images in three bands (RGB). Second and third, SAR image acquisition is sensitive to landscape topography and to land-cover complexity — causing geometrical distortions (Kaushik

et al., 2021; Bralet et al., 2022) and radiometrical variations (Wang et al., 2022b), respectively.

In this paper, we propose a Translation-Classification Loss (TCL) to take land-complexities into account when optimizing translator weights. The TCL first computes a translation error based on structural and radiometrical similarity criteria with respect to a true optical image. Then, it leverages a pre-trained land-cover multi-label classifier to compute a classification error based on the accuracy of the predicted labels and on the relevance of the extracted hidden features. Classification is favored over more sophisticated approaches such as segmentation or change detection tasks in this work to lighten the computational burden of the proposed approach. The objective is to demonstrate that even though the TCL is using a global evaluation of the translation through the classification task, it is still relevant to generate more complex and detailed patterns than the vanilla translator losses.

The TCL is applied to optimize nine standard approaches from the literature: Pix2pix (Isola et al., 2017) and CycleGAN (Zhu et al., 2017) which are shown to be the most promising in Zhao et al. (2022), Geometry-Consistent GAN (GCGAN) which demonstrates better robustness to image transformations, Style-based Recalibration Module generator (SRM) (Chen et al., 2022) which separates style and

<footnote><sup>a</sup> Université Savoie Mont Blanc, LISTIC, Annecy, 74940, France</footnote>

<footnote><sup>b</sup> University of Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, Grenoble, 38000, France</footnote>

<footnote>* Corresponding author.
E-mail address: antoine.bralet89@gmail.com (A. Bralet).</footnote>
content for the translation, IcGAN (Yang et al., 2022b) which includes a chromatic aberration loss to improve color reconstructions, CFRWD-GAN (Wei et al., 2023) which leverages wavelet transforms to reconstruct high resolution patterns, FG-GAN (Yang et al., 2022a) which contains a dense module within the latent space, StegoGAN (Wu et al., 2024) which is implemented to limit reconstruction artifacts due to non bijective patterns and finally SAR Distorted Image translator Network (SARDINet) (Bralet et al., 2022) which tackles SAR geometrical distortions in urban regions on high resolution images. We also propose a new version of SARDINet called SARDINetv2 to improve detail recovery, overall image contrast and make SARDINet performance comparable to other state-of-the-art methods on medium resolution images. Adversarial networks are implemented by disabling their discriminator, i.e by training only the generator to the translation task in a non-adversarial way, to avoid quantitative performance degradation experienced in Liu et al. (2021), Bralet et al. (2022).

Therefore, our contribution is twofold.

*   **A Translation-Classification Loss (TCL)** is introduced to evaluate the relevance of translated images from both regression and classification points of view. The novel loss is applied to six different non-adversarial architectures to prove its efficiency both quantitatively and qualitatively.
*   **An improved version of SARDINet** (Bralet et al., 2022) is proposed to enhance its detail recovery. The proposed version shows significant improvements over the original version.

The remaining of the paper is organized as follows. Section 2 details related works in modality translation and auxiliary classification. The proposed TCL is motivated and introduced in Section 3. SARDINetv2 and the different translator backbones used for experiments are described in Section 4. Finally, experiments and discussions are presented in Section 5 before concluding with perspectives in Section 6.

## 2. Related works

**Modality Translation:** Image modality translation is often used in the literature as a preprocessing step to enable cloud removal (Bermudez et al., 2018), change detection (Luppino et al., 2021; Fang et al., 2023), classification (Ley et al., 2018; Piva and Dubbelman, 2023), super-resolution (Feng et al., 2022) or image registration (Merkle et al., 2018; Du et al., 2022). Due to their performance in computer vision (Chen et al., 2022; Zhang et al., 2023; Jung et al., 2024), a wide variety of modality translators are based on Generative Adversarial Network (GAN) architectures (Goodfellow et al., 2014). Indeed, GAN derivatives – Pix2Pix (Isola et al., 2017) and CycleGAN (Zhu et al., 2017) – have shown the best SAR-optical translation capabilities among seven translators in Zhao et al. (2022) which encourages to explore and improve them.

Supervised networks are mostly inspired by Pix2pix (Isola et al., 2017) as in Li et al. (2020) where the architecture remains unchanged but is trained with the Structural SIMilarity Loss (SSIM) to improve detail consistency. Loss modifications are also explored in Yang et al. (2022b) where the authors propose the so-called chromatic aberration loss which compares a blurred version of the translation and the optical image to ensure more relevant translation, at the cost of contour sharpness. A simpler Nested U-Net based generator without batch normalization is proposed in Lee and Park (2023) to reconstruct images of military vehicles based on a new virtual dataset. Authors in Wang et al. (2022b) optimize their SAR-to-optical translator to mimic the decoder features of an aside optical image autoencoder. CFRWD-GAN is introduced in Wei et al. (2023) and includes a multi-scale latent processing and an additional wavelet decomposition branch to extract more relevant high resolution features. Wavelets are also exploited in Li et al. (2022) where the network first translates a SAR image into the wavelet decomposition of the grayscale optical image and then

colorizes it by style transfer. Authors of Zhang et al. (2022) propose to separate content and style features with Style-based Recalibration Modules (SRM) to keep relevant structures unchanged while transferring the style from one modality to another. Pix2pix is also extended to multi-temporal approaches as in He and Yokoya (2018), Liu et al. (2021) where past SAR-optical pairs are used as input in order to better associate SAR features to their corresponding optical ones and improve their translation capabilities — reducing seasonal variations errors.

On the other hand, to avoid SAR-optical mismatches due to changes occurring in between their acquisitions, CycleGAN is exploited for unsupervised translations. In particular, authors of Du et al. (2022) merge both Pix2pix and CycleGAN to train the network despite the corruption of SAR-optical pairs. Novel cascaded residual connections are introduced in CycleGAN generators in Fu et al. (2021) to enhance detail recovery in the translated image. The authors of Luppino et al. (2021) target modality confusion by introducing a loss which compares generators latent spaces and enforces their alignment. The contributions proposed in Guo et al. (2024) focus on multiscale architectures – both in the generator and the discriminator – and on a despeckling module to generate more reliable translations with the CycleGAN framework. A multiscale discriminator is also implemented in Yang et al. (2022a) in combination to a generator with dense convolutional blocks in the latent space. Multiple normalization strategies are also employed depending on the depth of the layers to improve the quality of the translation. The authors of StegoGAN (Wu et al., 2024) propose to modify the CycleGAN framework by separating matchable and unmatchable features from a modality to another. In this way, they limit the steganography effect which impacts the quality of the back translation in the CycleGAN framework.

Nevertheless, since all of the aforementioned architectures are based on adversarial training, they suffer from unstable performance and are sensitive to hyperparametrization. In addition, they mainly target visually credible results while remote sensing translations require more meaningful reconstructed patterns to be used for downstream tasks. Therefore, novel non-adversarial architectures are emerging as diffusion models in Bai et al. (2024) where the noisy input is conditioned by the SAR image to generate its corresponding optical translation. ADFNet is proposed in Xia et al. (2024) to merge both Sentinel-1 and MODIS acquisitions and reconstruct Landsat -8 optical images with a three stage network trained with the MAE loss only. We have also recently introduced a non-adversarial SAR Distorted Image translator Network (SARDINet) (Bralet et al., 2022) to account for strong geometrical distortions in SAR images and avoid reconstructing optical artifacts.

**Classification assistance:** Guiding a neural network with an auxiliary or a pre-trained classifier has shown significant improvements in standard computer vision tasks. Authors of Dhariwal and Nichol (2021) show that classification-guided diffusion models outperform GAN approaches for ImageNet-like image generation. In Odena et al. (2017), Auxiliary Classifier GAN (ACGAN) also shows more relevant and variable generations thanks to the classification task. A grayscale to RGB color translation is implemented in Iizuka et al. (2016) by including the features extracted by a classifier as input to the decoder of the translator to account for labels in the reconstruction. Authors of Kandala et al. (2022) exploit a latent space classifier to improve the performance of their transformers in image captioning.

Nevertheless, even though land-complexities have been shown to affect remote sensing translation capabilities (Wang et al., 2022b), to the best of our knowledge only few approaches leverage land-cover information to strengthen translation performance. In Ley et al. (2018), authors used a SAR-to-optical translator to use the encoder as SAR feature extractor. Their goal being to extract relevant features for SAR image classification but not to reconstruct reliable optical images that could be used a posteriori for a downstream task. The authors of Wang et al. (2022a) leverage both CNN and vision transformers for modality translation. Two branches communicate with each other in the encoder
of the generator in order to extract local and global SAR features. The latter are used by a CNN decoder to reconstruct the final optical image. Classification is introduced as an evaluator of the quality of the features extracted by the vision transformer, thus being applied only to SAR images without evaluating the final optical reconstruction. The most related work lies in Du et al. (2024) where the authors suggest a translation based on the same concept as ACGAN. Indeed, they exploit a discriminator designed under a U-Net++ architecture (Zhou et al., 2018) to perform both the real/fake classification and a change detection task. Therefore, the change detection is used as an evaluator of the relevance of the reconstructed patterns, while the classification only focuses on the visual aspect of the translation. Finally, authors in Fang et al. (2023) implement an optical style transfer network and evaluate its transferring performance at inference time with a pre-trained classifier — using prediction accuracy as a metric, but not for the translator training. Similarly, the translation performance in Zhang et al. (2024) are evaluated based on their classification and segmentation performed by a pre-trained network at inference time. It highlights that translation and classification performance is expected to behave similarly, showing their complementarity.

# 3. Method

## 3.1. Motivations

Based on the aforementioned literature, combining classification to image generation is not new. But a wide majority of methods apply an auxiliary classifier on latent features (Iizuka et al., 2016; Ley et al., 2018; Kandala et al., 2022; Wang et al., 2022a). Nevertheless, in the current SAR-to-optical translation context, applying classification to SAR extracted features is known to be less efficient than to optical features (Nava et al., 2021). In addition, this strategy may not be beneficial to the decoder since the classification loss is only backpropagated through the encoder. Finally, classifiers are often trained together with the generators (Iizuka et al., 2016; Odena et al., 2017; Ley et al., 2018; Wang et al., 2022a), thus optimizing the former on inaccurately generated images or features. Therefore, in order to leverage translation and classification complementarity (Wang et al., 2022b; Fang et al., 2023; Zhang et al., 2024), we propose to pre-train a classifier on real optical images. Its pattern recognition knowledge is then transferred to the translator by optimizing it with a new Translation-Classification Loss (TCL). The TCL not only combines radiometrical and structural translation criteria, but it also leverages the classifier to judge the quality of the generated optical image. This process consists of evaluating the accuracy of the predicted land-cover classes and comparing the features extracted by the classifier from the translated image and from its corresponding true optical image.

## 3.2. Translation-classification loss

In order to ensure the reliability and the consistency of the classification term of the TCL, a ResNet50 (He et al., 2016) multi-label classifier is trained offline, prior to translator optimization. This network is selected following the conclusions in Sumbul et al. (2021) stating that it is the most relevant among the five implemented approaches. The training resulted in more than 95% of accuracy on the evaluation dataset, indicating that the network has a strong knowledge of what to expect from an optical image. Its weights are then frozen to perform the Translation-Classification training of the translators as symbolized by the locks in Fig. 1.

Following the translation of the SAR input image by the backbone translator, the resulting optical image is fed into the frozen ResNet50 classifier to evaluate its relevance. In order to maximize the use of the downstream classification, a Translation-Classification Loss (TCL) not only evaluates ResNet50 label predictions against the actual land-cover but it also compares the hidden features of four layers with those

extracted from the true optical image. The selected layers are located before each downsampling stage. This strategy is similar to Wang et al. (2022b) where an aside optical autoencoder is trained to compare its decoder features with those of the translator. The major difference is that the proposed approach compares features extracted from the same modality – optical – while in Wang et al. (2022b), extracted features come from two different modalities which may be the source of domain adaptation issues. The TCL is thus evaluating the translated image for its similarity to the true optical image with the Translation Loss, but also for the relevance of the reconstructed patterns from a classification point of view with the Classification Loss.

**Translation Loss:** Following the works in Isola et al. (2017) and Li et al. (2020), the translation loss is composed of both the Mean Absolute Error ($L_{MAE}$) and the Structural SIMilarity error ($L_{SSIM}$). The former aims at evaluating the general colors and brightness of the generated image and has been shown more relevant than the Mean Squared Error in Isola et al. (2017). On the other hand, the SSIM is exploited to ensure the structural consistency of the reconstructed image. The objective is to transfer the structural properties from the SAR images to the reconstructed optical images.

**Classification Loss:** ResNet50 predictions from the translated image are first evaluated by the Weighted Cross-Entropy loss ($L_{WCE}$). To reduce class imbalance in the training, each class error is weighted proportionally to the number of samples belonging to the given class in the entire training dataset. Such an evaluation can only occur in a fully supervised context where land-cover labels are available (green box in Fig. 1). Thus, to ensure the applicability of the TCL despite the lack of ground truth labels, the classification loss is also composed of $L_{Feats}$ which compares hidden features extracted by the classifier as depicted by the blue arrow in Fig. 1. The features extracted from the translated image are compared to those extracted from its corresponding true optical image with the MAE according to Eq. (1).

$$L_{\text{Feats}}(\hat{I}, I) = \frac{1}{N} \sum_{i \in T} L_{\text{MAE}}(F_i(\hat{I}), F_i(I)) \quad (1)$$

considering $\hat{I}$ and $I$ as the translated and true optical images, respectively. $F_i$ denotes the extraction of the features at layer $i$. $T$ corresponds to the ensemble of the $N$ target layers.

**Specific Backbone Loss:** The main objective of this paper is to compare the performance of a TCL enriched backbone translator network with that obtained with their respective vanilla losses. Since the selected networks may have more sophisticated losses than the standard MAE loss, we train those network with their original configuration, i.e. by maintaining the use of their specific loss terms $L_{Spe}$. The calculation of each $L_{Spe}$ is detailed in Section 4.1 for each backbone translator.

**Translation-Classification Loss (TCL):** The final TCL is a linear combination of the four previous terms:

$$TCL(L(\hat{I}, I, y) = \lambda_0 L_{\text{MAE}}(\hat{I}, I) + \lambda_1 L_{\text{SSIM}}(\hat{I}, I) \\ + \lambda_2 L_{\text{Feats}}(\hat{I}, I) + \lambda_3 L_{\text{WCE}}(\hat{y}, y) \\ + L_{\text{Spe}} \quad (2)$$

considering $\hat{y}$ and $y$ the ResNet50 predictions and ground truth labels, respectively. The four $\lambda_x$ are hyperparameter weights used to balance the multiple loss terms.

# 4. Backbone translators

## 4.1. Standard translators

The first two translators to be considered are the widely exploited Pix2pix (Isola et al., 2017) and CycleGAN (Zhu et al., 2017). Indeed, they have shown the highest performance among the seven translators compared in Zhao et al. (2022). Pix2pix is based on an eight-stage U-Net (Ronneberger et al., 2015) and trained with the MAE loss — thus $L_{Spe}^{P2P}$ is set to 0 in Eq. (2). On the other hand, CycleGAN consists of
<figure><img src="image_4.png" /><figcaption>Fig. 1. Scheme of the strategy based on the Translation-Classification Loss. The SAR input is first processed by a translator and compared to the true optical image. Both images are then evaluated by the same frozen pre-trained classifier for feature comparisons. An additional fully supervised loss can be applied by comparing classifier results with the ground truth when land-cover labels are available ($L_{WCE}$ in the figure).</figcaption></figure>

two generators based on a nine-block ResNet (He et al., 2016) for SAR-to-optical and optical-to-SAR translations. Since it is an unsupervised approach, the MAE is not used in the vanilla loss. Instead, it is based on the cycle consistency loss and on the identity loss so that $L_{Spe}^{CG} = L_{Cycle} + L_{Idt}$. Cycle consistency evaluates the network's ability to reconstruct the input when it is translated twice — first to the target modality and then back to the source modality. The twice-translated image is then compared to the original input with the MAE loss as specified in Eq. (3). The identity loss given by Eq. (4) only ensures that an optical image remains unchanged when translated to the optical modality.

$$L_{Cycle} = L_{MAE}(I_S, G_{OS}(G_{SO}(I_S))) + L_{MAE}(I_O, G_{SO}(G_{OS}(I_O))) \quad (3)$$

$$L_{Idt} = L_{MAE}(I_S, G_{OS}(I_S)) + L_{MAE}(I_O, G_{SO}(I_O)) \quad (4)$$

where S and O denote the SAR and optical modalities, respectively, $I_Y$ the input image in modality X and $G_{AB}$ the generator translating from modality A to modality B.

The third selected backbone is based on CycleGAN to perform style transfer (Zhang et al., 2022). The main difference with CycleGAN is the introduction of Style Recalibration Modules (SRM) in each residual block. SRMs aim to extract input style characteristics through standard deviation and average pooling, convert them to output style characteristics and integrate them with the original features to compute style transfer. Similar to CycleGAN, the specific loss is calculated as $L_{Spe}^{SRM} = L_{Cycle} + L_{Idt}$.

The fourth method tested is the Geometry-Consistent GAN (GCGAN) introduced in Fu et al. (2019). The single SAR-to-optical generator has the same architecture as those of CycleGAN and its training is based on the identity loss and a geometry consistency constraint in $L_{Spe}^{GCG} = L_{Idt} + L_{Geom}$. The latter forces the network to match the translation of a rotated input image to the rotated version of the translated image as given by Eq. (5) by considering rot as the rotation transformation.

$$L_{Geom} = L_{MAE}(rot(G_{SO}(I_S)), G_{SO}(rot(I_S))) \quad (5)$$

The fifth state-of-the-art neural network is an improved cGAN from Yang et al. (2022b) which is referred to as IcGAN hereinafter. This network proposes to upgrade the architecture of Pix2pix by adding a skip connection from the input to the output of the decoder. The latter aims at bringing more structural consistency in the final reconstruction since the resolution of the extracted features is kept unchanged when applying a shallow convolutional block. Both the output of the standard Pix2pix generator and the skip connection are merged with

a succession of convolutional layers. To bring even more consistency when reconstructing the optical patterns, the authors also propose a chromatic aberration loss as defined in Eq. (6).

$$L_{CA} = L_{MSE}(Gauss(G(\hat{I})), Gauss(I)) \quad (6)$$

with $L_{MSE}$ the Mean Square Error Loss, Gauss a 2D Gaussian blur of mean 0 and standard deviation 3, I and $\hat{I}$ the true and translated optical images, respectively. Since the network is originally trained in combination with the MAE Loss which is already included in the TCL, we deduce that $L_{Spe}^{IcGAN} = L_{CA}$.

The sixth approach leverages Discrete Wavelet Transform (DWT) during the training. The so-called CFRWD-GAN proposed in Wei et al. (2023) mostly consists of an improved Pix2pix network. Two architectural modifications are applied: a latent cross-fusion reasoning module and a wavelet decomposition branch. The former substitutes the latent processing module in the original Pix2pix generator. Then to enhance contrast in the reconstructions, the DWT is applied to the input image. The latter is processed independently from the generator by a high-frequency feature encoder composed of a ResNet18 and a ResNet101 branch (He et al., 2016). Since the contribution lies in modifying the network's architecture, no specific loss is applied to constrain the network so that $L_{Spe}^{CFRWDGAN} = 0$.

The seventh neural network named FG-GAN in Yang et al. (2022a) is based on the same framework as CycleGAN. Instead of the original generator, the authors propose to use the so-called unbalanced generator. The peculiarity of this generator lies in the latent space where a dense convolutional block is included before a succession of residual blocks. The dense convolutional block consists in feeding each convolutional layer with the concatenation of all the features that are output from the previous convolutional layers. Thus, considering latent the features extracted by the encoder and out_i the output of the i-th layer of the dense block, the input of the l-th convolutional layer is $[latent, out_1, out_2, ..., out_{l-1}]$. In addition, the authors propose to use different types of normalization layers depending on the depth of the features. They alternate between standard batch normalization, instance normalization and layer normalization. Since the network is originally trained similarly to CycleGAN, the specific loss is calculated as $L_{Spe}^{FGGAN} = L_{Cycle} + L_{Idt}$.

The eighth model tested in this work is the StegoGAN model from Wu et al. (2024). StegoGAN is based on the assumption that the translation of an image modality to another might not be fully bijective. For instance, some SAR patterns can refer to different optical ones due to seasonal variations. These unmatchable features can limit
<figure><img src="image_5.png" /><figcaption>Fig. 2. Architecture of SARDINetv2 extended from SARDINet (Bralet et al., 2022) for more detailed reconstructions. Red dotted arrows represent skip connections.</figcaption></figure>

the capabilities of CycleGAN to translate unmatchable features back into the input modality for computing the cycle consistency loss. Such a limitation is referred to as steganography. To mitigate this effect, the authors propose to separate the translation of matchable and unmatchable features. Thus, the latent space obtained by translating an optical image into a SAR one is used to compute an “unmatchability” map $M$. By accounting for the mask during the translation process and separating the latent space between matchable and unmatchable features, the authors are able to limit the steganographic effect. To better guide the network, a constraint of sparsity is added on $M$ with the non-convex $L_{0,5}$ loss. Finally, a matchable consistency loss $L_{\text{Match}}$ is proposed to enforce the consistency of the patterns between the different reconstructions in the matchable regions. For the sake of readability, this loss is not described further is this paper. The specific loss of StegoGAN can be written as $L_{\text{StegoGAN}}^{\text{Spe}} = L_{\text{Idt}} + L_{\text{Cycle}} + L_{\text{Match}} + L_{0.5}$.

The last selected approach is the SAR Distorted Image translator Network (SARDINet) (Bralet et al., 2022) originally implemented to target the translation of SAR images suffering from geometrical distortions. Its architecture is composed of a three-branch encoder to deeply extract features at different resolutions. As for Pix2pix, $L_{\text{Spe}}^{\text{SARDINet}}$ is set to 0 since the network is originally trained with the MSE loss which is replaced here with the MAE loss in Eq. (2).

Each GAN-like network is trained without their discriminator in order to focus only on the performance of the generator. Indeed, in order to avoid using heavy discriminator regularization strategies (Zhang et al., 2024) and avoid the drops in metrics experienced in Liu et al. (2021), Bralet et al. (2022), we choose to focus on the quality of vanilla translations and avoid unwanted and untruthful artifact generated by an adversarial training. Thus, the latter is only considered as an increment for better looking reconstructions (see Section 5.4). In addition, for fair comparisons between the backbones, the translation tasks are always supervised in the experiments — introducing an additional MAE term in CycleGAN, GCGAN, SRM, FG-GAN and StegoGAN vanilla losses when they are trained without the TCL. The training of these networks is performed with Pytorch implementations available on Github except for IcGAN, CFRWD-GAN and FG-GAN which are implemented from scratch.

## 4.2. SARDINetv2

As stated in Bralet et al. (2022), SARDINet architecture can overcome SAR geometrical distortions but suffers from a lack of contrast and structure recovery. This lack of contrast is even more impacting the translation of medium resolution images such as that of Sentinel-1 and Sentinel-2 used for the experiments in Section 5. Therefore, to

<figure><img src="image_6.png" /><figcaption>Fig. 3. Zoom into the feature maps obtained after each upsampling stage of SARDINetv2 decoder and those transferred by the skip connections. To easy the visualization, the displayed feature maps are averaged. The symbols are the same as in Fig. 2.</figcaption></figure>

make SARDINet performance comparable to the standard state-of-the-art methods and avoid irrelevant translations (see SARDINet results in the supplementary materials), a new SARDINet version depicted in Fig. 2 is proposed. Four modifications are implemented compared to the original architecture in order to mitigate the loss of structure:

* **Encoder-decoder convolutional kernels:** In order to increase the variety of reconstructable optical patterns, the number of convolutional kernels between the encoder and the decoder is better balanced. The balancing consists to symmetrize the number of convolutional kernels at each resolution. Thus the convolutional layer in charge of reconstructing features with a resolution $r$ in the decoder is assigned the same number of kernels as that of the convolutional layer applied to extract features at resolution $r$ in the encoder.

* **Deeper latent features:** Two residual blocks are introduced in the latent space to facilitate the projection of SAR extracted features into the optical feature space.

* **Skip connections:** Similar to U-Net, skip connections are introduced between the encoder and the decoder, denoted as red dotted arrows in Fig. 2. Their role is to transfer structural properties of the SAR image to the translated optical image. Fig. 3 illustrates the features that are transferred from the encoder to the decoder thanks to the skip connections. As expected, the structural SAR properties cannot be maintained by the original encoder since they are high resolution features. Yet, with the use of the red dotted skip connections in Fig. 3, they can be transferred to the
<caption>**Table 1**</caption>

Quantitative results comparing the impact of the TCL on standard translator results. Best and second best results are displayed in bold and underlined, respectively. Accuracy is displayed in %.

<table><thead><tr><th rowspan="2"></th><th colspan="4">Without TCL</th><th colspan="4">With TCL</th></tr><tr><th>PSNR ↑</th><th>MSE ↓</th><th>FID ↓</th><th>Acc ↑</th><th>PSNR ↑</th><th>MSE ↓</th><th>FID ↓</th><th>Acc ↑</th></tr></thead><tbody><tr><td>Pix2Pix (Isola et al., 2017)</td><td>25.39</td><td>10.16e-3</td><td>4.34</td><td>15.47</td><td>25.55</td><td>9.89e-3</td><td><strong>3.81</strong></td><td><strong>34.68</strong></td></tr><tr><td>CycleGAN (Zhu et al., 2017)</td><td>25.17</td><td>13.26e-3</td><td>4.81</td><td>4.00</td><td><strong>25.67</strong></td><td>12.72e-3</td><td><strong>3.97</strong></td><td>31.82</td></tr><tr><td>GcGAN (Fu et al., 2019)</td><td>25.03</td><td>14.31e-3</td><td>6.66</td><td>6.94</td><td><u>25.59</u></td><td>12.19e-3</td><td>5.63</td><td>27.14</td></tr><tr><td>SRM (Zhang et al., 2022)</td><td>25.36</td><td>10.72e-3</td><td>4.36</td><td>11.54</td><td><u>25.59</u></td><td>10.06e-3</td><td>4.17</td><td>29.24</td></tr><tr><td>IcGAN (Yang et al., 2022b)</td><td>25.27</td><td>10.13e-3</td><td>4.45</td><td>16.63</td><td>25.42</td><td><u>9.37e-3</u></td><td>4.16</td><td>33.21</td></tr><tr><td>CFRWD-GAN (Wei et al., 2023)</td><td>25.49</td><td>9.36e-3</td><td>4.37</td><td>13.57</td><td>25.48</td><td>9.68e-3</td><td>4.21</td><td>32.28</td></tr><tr><td>FG-GAN (Yang et al., 2022a)</td><td>25.56</td><td>16.11e-3</td><td>6.46</td><td>9.52</td><td>25.48</td><td><strong>9.02e-3</strong></td><td>4.90</td><td>26.96</td></tr><tr><td>StegoGAN (Wu et al., 2024)</td><td>25.18</td><td>16.05e-3</td><td>5.79</td><td>4.22</td><td>25.22</td><td>10.45e-3</td><td>4.07</td><td>28.21</td></tr><tr><td>SARDINet (Bralet et al., 2022)</td><td>24.82</td><td>12.51e-3</td><td>4.64</td><td>17.22</td><td>25.25</td><td>11.85e-3</td><td>5.60</td><td>29.62</td></tr><tr><td>SARDINetv2</td><td>25.26</td><td>10.49e-3</td><td>5.07</td><td>13.34</td><td>25.32</td><td>10.32e-3</td><td>4.31</td><td><strong>35.88</strong></td></tr></tbody></table>

decoder in order to ensure the reconstruction of patterns that are more consistent with the input SAR image.

* Linear branch fusion: As symbolized by the circled *F* in Fig. 2, branches are merged with a learnable linear combination in order to maximize the relevance of the features transferred from the encoder to the decoder.

In the following, the original architecture is referred as SARDINet and the modified architecture as SARDINetv2.

# 5. Experimental results

## 5.1. Experimental setup

All experiments are computed with a random subset of images from the BigEarthNet-MM dataset (Sumbul et al., 2021). This dataset is composed of pairs of 120 × 120px coregistered Sentinel-1 and Sentinel-2 images acquired from June 2017 to May 2018 in 10 different European countries. Among the twelve available optical channels, the red, green and blue bands are retained to compose the true optical image. SAR images consist of Ground Range Detected products acquired in Interferometric Wide swath mode with dual polarization VV and VH in logarithmic scale. They were selected as temporally close as possible to the optical acquisitions to ensure relevant correspondence. Each pair was multi-labeled based on the CORINE 2018 land-cover map (Feranec et al., 2016) resulting in 19 labels. For the experiments, pairs with cloudy optical images and with seasonal snow are excluded to avoid unreliable optical images.

The pre-training of the ResNet50 classifier is applied to a subset of 156,087 images and trained until convergence — achieving 95% accuracy on the 39,022 evaluation images. Starting from the pre-trained ImageNet weights, the fine-tuning lasts for 38 epochs to minimize the WCE loss using an Adam optimizer and a learning rate of 1.10⁻⁴.

In order to train backbone translators, the training and evaluation datasets are limited to 34,412 and 7,520 images respectively, due to memory constraints. In order to fit image size requirements, the inputs of non-SARDINet networks are resized to 256 × 256 and their translations are downsampled to 120 × 120 for ResNet50 evaluation and metric computations. The networks are all trained for 150 epochs with a learning rate and batch-size set to 5.10⁻⁵ and 16, respectively. Their weights are updated by an Adam optimizer with the gradient moments weighted by β₁ and β₂ set to 0.5 and 0.999, respectively. The Translation-Classification Loss as described in Eq. (2) is balanced to give the same importance to the MAE and SSIM — thus λ₀ and λ₁ are identical in Eq. (2). The weights of classification terms are reduced by 3 orders of magnitude — resulting in λ₂ = λ₃ = λ₀·10⁻³. This choice forces the network to first ensure coherent translation with respect to the true optical image before leveraging the classification task to strengthen the relevance of the patterns. In order to follow the original implementations of each backbone, λ₀ and λ₁ are set to 100 for Pix2pix (Isola et al., 2017) and IcGAN (Yang et al., 2022b), 10 for

CycleGAN (Zhu et al., 2017), GCGAN (Fu et al., 2019), SRM (Zhang et al., 2022), CFRWD-GAN (Wei et al., 2023), FG-GAN (Yang et al., 2022a) and StegoGAN (Wu et al., 2024), and 1 for SARDINet (Bralet et al., 2022) and SARDINetv2. Specific non-adversarial loss terms L<sub>Spe</sub> remain unchanged from the original papers.

The performance of each translator is summarized in Table 1 by averaging the results over the 7,520 evaluation images. Translated images are evaluated based on four metrics, namely Peak Signal-to-Noise Ratio (PSNR), Mean Squared Error (MSE), Fréchet Inception Distance (FID) and the overall accuracy of ResNet50 predictions (Acc). The two former translation metrics aim at evaluating the similarity to the true optical image while the latter two focus on the exploitability of the translation for downstream tasks and therefore the relevance of reconstructed patterns. It is worth mentioning that the MSE and FID scores are computed on the output of the translator while the PSNR is applied after thresholding the output image between 0 and 1, and formatting the data in unsigned integer encoded on 8 bits. These metrics are used to assess the performance of the actual output of the network with the MSE and FID scores, and the further usability of the translations in a practical case where the images are encoded on 8 bits.

## 5.2. Translation-classification results

The resulting quantitative and qualitative translators performance trained with and without the TCL are given in Table 1 and in Fig. 4(a), respectively. For better visualization purpose, only the four approaches which obtained the best quantitative metrics are displayed in Fig. 4(a), i.e. CycleGAN, FG-GAN, Pix2pix and SARDINetv2. More visual results can be found in the supplementary materials comparing the reconstructions of each generator trained with and without the TCL. Among the conclusions that can be drawn, the most important one is that the TCL significantly outperforms all vanilla networks not only in terms of metrics but also in terms of qualitative results.

Indeed, the TCL helps to reveal the presence of new classes — in particular, in CycleGAN results of Fig. 4(a), the village appears when the TCL is used during the training. The forest at the bottom of the image also becomes more credible and textured than with the vanilla losses. It also helps to avoid mis-reconstructions by avoiding forest-like patterns in the middle of the village as in the FG-GAN vanilla reconstructions. Such forest-to-building corrections are even more visible on the additional visual results provided in the supplementary materials associated with the paper, e.g. in CycleGAN, GCGAN and StegoGAN reconstructions. In addition, colors, brightness and variations look more reliable with respect to the true optical image which explains the great benefits in terms of accuracy (> 20% of average increase for each network) but also in terms of similarity as depicted by the PSNR (>0.2 points of average increase) and the MSE (up to 2e-3 points of decrease).

Based on qualitative results, the use of the TCL has a significant impact on deblurring the translations and reconstructing high resolution details. Indeed, as reflected by the urban area, buildings are better distinguishable and even countable while on the first row of Fig. 4(a),
<figure><img src="image_7.png" /><figcaption>**Fig. 4.** Qualitative comparison of the impact of the TCL on standard translator networks (a) and of the ablation of each TCL term on SARDINetv2 reconstructions (b). The details of each ablation configuration can be found in Table 2. SAR images are colored with RG channels being VV and VH bands.</figcaption></figure>

the structure of the village cannot be properly estimated. This strong deblurring property not only helps to obtain better translation metrics but it also makes the results closer to actual optical patterns as reflected by the FID score with an average decrease of 0.7 points. The only network which FID gets increased is the original version of SARDINet. Indeed, the network cannot reconstruct high resolution details due to its architecture. First, this defect illustrates that this network is better adapted to high resolution images (Bralet et al., 2022) and that the modifications suggested in Section 4.2 are necessary in order to preserve the details of coarse resolution images and make the performance of the SARDINet architecture comparable to that of other models. Indeed, the vanilla SARDINetv2 improves SARDINet's translation performance by 0.44 PSNR points and 2.02e-3 MSE points and when combined to the TCL, every metrics show strong benefits thanks to the details enhancement brought by the architectural modifications. Second, it demonstrates that the TCL strategy cannot hallucinate artifacts for credibility purpose. Therefore, it strengthens the reliability of the proposed strategy in comparison with adversarial-based approaches. This aspect is discussed with more details in Section 5.4.

The aforementioned structural and deblurring impact coupled with the class pattern enhancement help to discriminate the different classes more easily. Indeed, the transition between the forest and the fields is enhanced for all backbone networks, highlighting the class discrimination ability of the proposed strategy. The TCL does not impact each network architecture in the same way. On the one hand, when the architecture allows detail transfers, the results are greatly improved as depicted by the reconstructions of CycleGAN (Zhu et al., 2017) and FG-GAN (Yang et al., 2022a). The blurry vanilla reconstructions are much more relevant in terms of structures (e.g. urban area) and in terms of color variations (e.g. the forest patterns differ from fields). On the other hand, Pix2pix (Isola et al., 2017) initial results already contained details. Therefore the TCL especially focuses on improving class transitions and color variations in order to accurately distinguish and recognize the land-cover. Such performance lead Pix2pix generator to achieve the best results among the six implemented methods when combined with the TCL. Finally, as illustrated by the metrics, the major impact of the TCL on SARDINetv2 lies in the improvement of the relevance of the patterns. Indeed, PSNR and MSE are slightly better while significant improvements in terms of FID and accuracy are obtained in Table 1. Thus, SARDINetv2 succeeds to reach stable

translation results while leveraging the TCL to improve the perceptual quality of its reconstructions.

Apart from visual and quantitative performance, it is worth devising on the computational cost of the TCL. The proposed experiments are computed with a NVIDIA RTX A6000 GPU with a memory capacity of 48 GB. Based on such a configuration, the computational cost of applying the TCL during the training is increased by 1 min 49 sec per epoch which means a total delay of less than 5 h for the entire training. Since the TCL allows significant quantitative and qualitative improvements, the trade off with the computational cost is highly acceptable.

Therefore, this study shows that the TCL is able to leverage appropriately available high-resolution features to reconstruct better visual results while improving the relevance of patterns for downstream tasks. Due to the lack of adversarial training, hallucinations and unreliable artifacts are avoided and the structural properties of the SAR image are successfully transferred to optical-like reliable patterns.

### 5.3. Ablation study

In this Section, the relevance of each term of the Translation-Classification Loss is evaluated. Since the performance of SARDINetv2 demonstrates the best use of the TCL for the downstream classification task (35.88% of accuracy in Table 1), this architecture is selected in this Section to conduct the ablation study. Thus, the results obtained with SARDINetv2 under eight different loss configurations are compared as detailed in Table 2 to study which terms of the loss are responsible for major improvements. The overall hyperparameter configuration is kept unchanged from Section 5.1. Based on Table 2 and Fig. 4(b) showing quantitative and qualitative results respectively, each loss term has a different and complementary positive impact on the final translated image. Only four configurations are displayed in Fig. 4(b) for the sake of readability.

Based on these results, the benefit of $L_{SSIM}$ lies in the structural enhancement as visible in the *Config. SSIM* in comparison with *Config. Init.* composed only of the MAE loss. Indeed, thin forests appear, urban regions are more distinguishable and the image is actually deblurred. It has a direct impact on translation metrics in Table 2 as it achieves the best performance (PSNR and MSE are 0.06 and 0.2e-3 points better, respectively) and all metrics of each combination are strengthened when the $L_{SSIM}$ is part of the loss. However when used alone,
<caption>**Table 2**</caption>

Quantitative comparison of the ablation study using SARDINetv2 as backbone translator. Best and second best results are displayed in bold and underlined, respectively. Accuracy is displayed in %.

<table><thead><tr><th rowspan="2">Name</th><th colspan="2">Translation</th><th colspan="2">Classification</th><th colspan="4">Metrics</th></tr><tr><th>L<sub>MAE</sub></th><th>L<sub>SSIM</sub></th><th>L<sub>Feats</sub></th><th>L<sub>WCE</sub></th><th>PSNR ↑</th><th>MSE ↓</th><th>FID ↓</th><th>Acc ↑</th></tr></thead><tbody><tr><td>Config. Init.</td><td>✓</td><td>✗</td><td>✗</td><td>✗</td><td>25.26</td><td>10.49e-3</td><td>5.07</td><td>13.34</td></tr><tr><td>x</td><td>✓</td><td>✗</td><td>✓</td><td>✗</td><td>25.21</td><td>10.38e-3</td><td>4.51</td><td>32.21</td></tr><tr><td>x</td><td>✓</td><td>✗</td><td>✗</td><td>✓</td><td>25.23</td><td>10.52e-3</td><td>4.94</td><td>27.82</td></tr><tr><td>x</td><td>✓</td><td>✗</td><td>✓</td><td>✓</td><td>25.25</td><td>10.73e-3</td><td>4.57</td><td><strong>39.96</strong></td></tr><tr><td>Config. SSIM</td><td>✓</td><td>✓</td><td>✗</td><td>✗</td><td><strong>25.46</strong></td><td><strong>10.06e-3</strong></td><td>4.37</td><td>15.80</td></tr><tr><td>Config. Feats</td><td>✓</td><td>✓</td><td>✓</td><td>✗</td><td><u>25.43</u></td><td><u>10.30e-3</u></td><td><u>4.22</u></td><td>28.29</td></tr><tr><td>x</td><td>✓</td><td>✓</td><td>✗</td><td>✓</td><td>25.41</td><td>10.47e-3</td><td>4.34</td><td>30.23</td></tr><tr><td>Config. TCL</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>25.32</td><td>10.32e-3</td><td><u>4.31</u></td><td><u>35.88</u></td></tr></tbody></table>

relevance metrics suffer from the uncertain transitions between classes (e.g. between the forest and the fields). This is greatly mitigated by using the $L_{Feats}$ classification term. Indeed, when combined with $L_{SSIM}$ in *Config. Feats*, it reaches the best balance in quantitative metrics. It shows that despite the lack of classification labels for the training, the TCL still has powerful capabilities to reconstruct relevant patterns. The last classification term $L_{WCE}$ for fully supervised training plays a major role in the relevance of the patterns without significantly affecting details enhancement. This behavior results from the fact that the WCE only evaluates the images from a global classification point of view and not from a local one. Therefore, when combined with the other terms in *Config. TCL*, it helps to significantly improve color transitions between classes and the overall brightness of the reconstructed images while slightly trading the structural benefits of the other terms as visible in the last row of Table 2. Thus, as reflected in this study, each term of the TCL provides different but complementary benefits to the final image to capture not only the structural quality and relevance brought by $L_{SSIM}$ and $L_{Feats}$, but also the overall colors, brightness and transitions by $L_{WCE}$.

## 5.4. TCL and adversarial training

In order to complete and highlight the benefits of the TCL, a brief qualitative study on the translation capabilities of Pix2pix when trained adversarially is proposed in this Section. Fig. 5 illustrates the results obtained with the configuration described in Section 5.1 but with an additional adversarial term (upper right image). In addition, in order to evaluate the applicability of the TCL in an adversarial context, the same adversarial training is also applied targeting the minimization of the TCL (bottom right image). Their respective non-adversarial counterparts are displayed on the left hand side for comparison purpose.

Based on these qualitative results, the vanilla adversarial training shows strongly distorted reconstructions compared to the actual urban area. However, this behavior is completely avoided when trained to minimize adversarially the TCL. Indeed, the overall structures of the image are reconstructed, showing the strong regularization capability of the TCL. Nevertheless, the drawbacks of adversarial training are still visible in the bottom right image. Indeed, the network hallucinates some building patterns in the red circled area and erases the actual forest and field patterns. In addition, the transition between the forest and the fields in the corner of the image is much less distinguishable than in the results of the non-adversarial Pix2pix (left images).

Therefore, these brief experiments show that the TCL is an effective tool for enhancing translation performance. Combined with an adversarial training, reconstructions are much more credible but the generation of artifacts still disturbs the final image which can cause critical issues for downstream tasks such as change detection or disaster monitoring. Thus, it confirms the hypothesis made in Section 4.1 stating that adversarial training can only be considered as an incremental contribution for better visual appreciations.

<figure><img src="image_8.png" /><figcaption>Fig. 5. Comparison of the results obtained by Pix2pix (Isola et al., 2017) when trained adversarially and in combination with the TCL.</figcaption></figure>

## 6. Conclusions and perspectives

In this paper, we proposed a Translation-Classification Loss (TCL) to leverage an auxiliary classification task in order to improve the relevance of SAR-to-optical translations. Unlike previous approaches from the literature, the classifier is applied to evaluate the translated image itself. In addition, it is pre-trained on real optical images and frozen during the translator training. These two aspects force the translator to integrate the knowledge acquired by the classifier in order to reconstruct truthful optical images. Experiments on ten different translator architectures and training strategies show that the TCL brings structural consistency, color variation credibility and class discrimination ability to the translated images while avoiding the reconstruction of unreliable hallucinated artifacts. A further ablation study shows that even without land-cover labels, the TCL significantly improves the similarity with the true optical images and the perceptual relevance of the generated images.

This study paves the way for better SAR image understanding through accurate optical translations and for further downstream task-driven neural networks. Land-cover classification is chosen here as it is a common remote sensing application. But a TCL-like strategy can be extended to change detection, segmentation or disaster monitoring to specialize the training for specific applications. Indeed, even though the computational burden would be heavier by using these auxiliary tasks, they would focus on the reconstruction of local patterns which could help to better localize and recognize specific classes. Based on the increase in performance shown by the TCL, it would be interesting to explore the trade-off between computational cost and performance induced by more sophisticated auxiliary tasks. Another perspective consists in providing more data as input to the network. Since the SAR-to-optical translation process is an ill-posed problem, supplementary
information such as the date of acquisition, the Digital Elevation Model or past optical images could help to further understand the style of the expected output image. In addition, a two-step methodology could be employed by separating the SAR-to-optical translation and the optical style transfer processes to adapt the output radiometry to the actual season. It is worth mentioning that the TCL can be easily applicable to all these methodologies with a least computational cost which makes it a powerful auxiliary tool to reconstruct more meaningful patterns.

### CRediT authorship contribution statement

**Antoine Bralet:** Writing – review & editing, Writing – original draft, Visualization, Validation, Methodology, Investigation, Conceptualization. **Abdourrahmane M. Atto:** Writing – review & editing, Validation, Supervision, Conceptualization. **Jocelyn Chanussot:** Writing – review & editing, Validation, Supervision, Conceptualization. **Emmanuel Trouvé:** Writing – review & editing, Validation, Supervision, Conceptualization.

### Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Antoine Bralet reports financial support was provided by Auvergne-Rhone-Alpes Region. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

### Acknowledgments

This work is supported by the region Auvergne-Rhône-Alpes (AURA, France) through the project IATOAURA.

## Appendix A. Supplementary data

Supplementary material related to this article can be found online at https://doi.org/10.1016/j.cviu.2025.104374.

### Data availability

Data are already available online on the BigEarthNet website. Code can be provided upon request.

### BigEarthNet-MM (Reference data) (BigEarthNet website)

### References

Adriano, B., Yokoya, N., Xia, J., Miura, H., Liu, W., Matsuoka, M., Koshimura, S., 2020. Learning from multimodal and multitemporal earth observation data for building damage mapping. ISPRS J. Photogramm. Remote Sens. 175, 132-143. http://dx.doi.org/10.1016/j.isprsjprs.2021.02.016.

Bai, X., Pu, X., Xu, F., 2024. Conditional diffusion for SAR to optical image translation. IEEE Geosci. Remote. Sens. Lett. 21, 1-5. http://dx.doi.org/10.1109/LGRS.2023.3337143.

Bermudez, J.D., Happ, P.N., Oliveira, D.a.B., Feitosa, R.Q., 2018. SAR to optical image synthesis for cloud removal with generative adversarial networks. ISPRS Ann. Photogramm. Remote. Sens. (ISSN: 2194-9042) IV-1, 5-11. http://dx.doi.org/10.5194/isprs-annals-IV-1-5-2018.

Bralet, A., Atto, A.M., Chanussot, J., Trouvé, E., 2022. Deep learning of radiometrical and geometrical sar distortions for image modality translations. In: 2022 IEEE International Conference on Image Processing. ICIP, Bordeaux, France, pp. 1766-1770. http://dx.doi.org/10.1109/ICIP46576.2022.9897713.

Chatterjee, B., Poullis, C., 2021. Semantic segmentation from remote sensor data and the exploitation of latent learning for classification of auxiliary tasks. Comput. Vis. Image Underst. 210, 103251. http://dx.doi.org/10.1016/j.cviu.2021.103251.

Chelali, M., Kurtz, C., Puissant, A., Vincent, N., 2021. Deep-star: Classification of image time series based on spatio-temporal representations. Comput. Vis. Image Underst. 208-209, 103221. http://dx.doi.org/10.1016/j.cviu.2021.103221.

Chen, H., Dong, L., Yang, H., He, X., Zhu, C., 2022. Unsupervised real-world image super-resolution via dual synthetic-to-realistic and realistic-to-synthetic translations. IEEE Signal Process. Lett. 29, 1282-1286. http://dx.doi.org/10.1109/LSP.2022.3175411.

Desrues, M., Lacroix, P., Brenguier, O., 2019. Satellite pre-failure detection and in situ monitoring of the landslide of the Tunnel du Chambon, French Alps. Geosci. 9 (7), 313. http://dx.doi.org/10.3390/geosciences9070313.

Dhariwal, P., Nichol, A., 2021. Diffusion models beat GANs on image synthesis. In: Advances in Neural Information Processing Systems, vol. 34, Curran Associates, Inc., pp. 8780-8794.

Du, Z., Li, X., Miao, J., Huang, Y., Shen, H., Zhang, L., 2024. Concatenated deep-learning framework for multitask change detection of optical and SAR images. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens. 17, 719-731. http://dx.doi.org/10.1109/JSTARS.2023.3333959.

Du, W.-L., Zhou, Y., Zhu, H., Zhao, J., Shao, Z., Tian, X., 2022. A semi-supervised image-to-image translation framework for SAR-optical image matching. IEEE Geosci. Remote. Sens. Lett. 19, 1-5. http://dx.doi.org/10.1109/LGRS.2022.3223353.

Fang, B., Chen, G., Kou, R., Paoletti, M.E., Haut, J.M., Plaza, A., 2023. CIT: Content-invariant translation with hybrid attention mechanism for unsupervised change detection. ISPRS J. Photogramm. Remote Sens. 204, 321-339. http://dx.doi.org/10.1016/j.isprsjprs.2023.09.012.

Feng, J., Jiang, Q., Tseng, C.-H., Jin, X., Liu, L., Zhou, W., Yao, S., 2022. A deep multitask convolutional neural network for remote sensing image super-resolution and colorization. IEEE Trans. Geosci. Remote Sens. 60, 1-15. http://dx.doi.org/10.1109/TGRS.2022.3154435.

Feranec, J., Soukup, T., Hazeu, G., Jaffrain, G., 2016. European Landscape Dynamics: CORINE Land Cover Data. CRC Press.

Fu, H., Gong, M., Wang, C., Batmanghelich, K., Zhang, K., Tao, D., 2019. Geometry-consistent generative adversarial networks for one-sided unsupervised domain mapping. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVPR, IEEE, Long Beach, CA, USA, pp. 2422-2431. http://dx.doi.org/10.1109/CVPR.2019.00253.

Fu, S., Xu, F., Jin, Y.-Q., 2021. Reciprocal translation between SAR and optical remote sensing images with cascaded-residual adversarial networks. Sci. China Inf. Sci. 64 (2), 1-15. http://dx.doi.org/10.1007/s11432-020-3077-5.

Ghorbanzadeh, O., Xu, Y., Zhao, H., Wang, J., Zhong, Y., Zhao, D., Zang, Q., Wang, S., Zhang, F., Shi, Y., Zhu, X.X., Bai, L., Li, W., Peng, W., Ghamisi, P., 2022. The outcome of the 2022 Landslide4Sense competition: Advanced landslide detection from multi-source satellite imagery. IEEE J. Sel. Top. Appl. Earth Obs. 15, 9927-9942.

Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y., 2014. Generative adversarial nets. In: Adv. Neural Inf. Process. Syst., vol. 27, pp. 1-9.

Guo, Z., Zhang, Z., Cai, Q., Liu, J., Fan, Y., Mei, S., 2024. MS-GAN: Learn to memorize scene for unpaired SAR-to-optical image translation. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens. 17, 11467-11484. http://dx.doi.org/10.1109/JSTARS.2024.3411691.

He, W., Yokoya, N., 2018. Multi-temporal sentinel-1 and -2 data fusion for optical image simulation. ISPRS Int. J. Geo- Inf. 7 (10), 389. http://dx.doi.org/10.3390/ijgi7100389.

He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition. CVPR, Las Vegas, NV, USA, pp. 770-778. http://dx.doi.org/10.1109/CVPR.2016.90.

Lizuka, S., Simo-Serra, E., Ishikawa, H., 2016. Let there be color! joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. ACM Trans. Graph. 35 (4), 110:1-110:11. http://dx.doi.org/10.1145/2897824.2925974.

Interdonato, R., Ienco, D., Gaetano, R., Ose, K., 2018. DUPLO: A dual view point deep learning architecture for time series classification. ISPRS J. Photogramm. Remote Sens. 149, 91-104.

Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A., 2017. Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA, pp. 1125-1134.

Jung, C., Kwon, G., Ye, J.C., 2024. Patch-wise graph contrastive learning for image translation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, Vancouver, Canada, pp. 13013-13021. http://dx.doi.org/10.1609/aaai.v38i12.29199.

Kandala, H., Saha, S., Banerjee, B., Zhu, X.X., 2022. Exploring transformer and multilabel classification for remote sensing image captioning. IEEE Geosci. Remote. Sens. Lett. 19, 1-5. http://dx.doi.org/10.1109/LGRS.2022.3198234.

Kaushik, S., Yan, Y., Ravanel, L., Magnin, F., Trouvé, E., 2021. Visibility analysis of glaciers on steep slopes in the European ALPS using terrasar-X/PAZ data. In: 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS. Brussels, Belgium, pp. 5505-5508. http://dx.doi.org/10.1109/IGARSS47720.2021.9554966.

Lee, I.H., Park, C.G., 2023. SAR-to-virtual optical image translation for improving SAR automatic target recognition. IEEE Geosci. Remote. Sens. Lett. 20, 1-5. http://dx.doi.org/10.1109/LGRS.2023.3312140.
Ley, A., Dhondt, O., Valade, S., Haensch, R., Hellwich, O., 2018. Exploiting GAN-based SAR to optical image transcoding for improved classification via deep learning. In: EUSAR 2018; 12th European Conference on Synthetic Aperture Radar. Aachen, Germany, pp. 1–6.

Li, Y., Fu, R., Meng, X., Jin, W., Shao, F., 2020. A SAR-to-optical image translation method based on conditional Generation Adversarial Network (cGAN). IEEE Access 8, 60338–60343. http://dx.doi.org/10.1109/ACCESS.2020.2977103.

Li, H., Gu, C., Wu, D., Cheng, G., Guo, L., Liu, H., 2022. Multiscale generative adversarial network based on wavelet feature learning for SAR-to-optical image translation. IEEE Trans. Geosci. Remote Sens. 60, 1–15. http://dx.doi.org/10.1109/TGRS.2022.3211415.

Liu, X., Hong, D., Chanussot, J., Zhao, B., Ghamisi, P., 2021. Modality translation in remote sensing time series. IEEE Trans. Geosci. Remote Sens. 60, 1–14. http://dx.doi.org/10.1109/TGRS.2021.3079294.

Luppino, L.T., Kampffmeyer, M., Bianchi, F.M., Moser, G., Serpico, S.B., Jenssen, R., Anfinsen, S.N., 2021. Deep image translation with an affinity-based change prior for unsupervised multimodal change detection. IEEE Trans. Geosci. Remote Sens. 60, 1–22. http://dx.doi.org/10.1109/TGRS.2021.3056196.

Merkle, N., Auer, S., Muller, R., Reinartz, P., 2018. Exploring the potential of conditional adversarial networks for optical and SAR image matching. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens. 11 (6), 1811–1820. http://dx.doi.org/10.1109/JSTARS.2018.2803212.

Nava, L., Monserrat, O., Catani, F., 2021. Improving landslide detection on SAR data through deep learning. IEEE Geosci. Remote. Sens. Lett. 19, 1–5.

Odena, A., Olah, C., Shlens, J., 2017. Conditional image synthesis with auxiliary classifier GANs. In: Proceedings of the 34th International Conference on Machine Learning. PMLR, Sydney, Australia, pp. 2642–2651.

Piva, F.J., Dubbelman, G., 2023. Exploiting image translations via ensemble self-supervised learning for unsupervised domain adaptation. Comput. Vis. Image Underst. 234, 103745. http://dx.doi.org/10.1016/j.cviu.2023.103745.

Ronneberger, O., Fischer, P., Brox, T., 2015. U-Net: Convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (Eds.), Medical Image Computing and Computer-Assisted Intervention. MICCAI 2015, In: Lecture Notes in Computer Science, Springer International Publishing, Munich, Germany, pp. 234–241.

Sumbul, G., de Wall, A., Kreuziger, T., Marcelino, F., Costa, H., Benevides, P., Caetano, M., Demir, B., Markl, V., 2021. BigEarthNet-MM: A large-scale, multimodal, multilabel benchmark archive for remote sensing image classification and retrieval [software and data sets]. IEEE Geosci. Remote. Sens. Mag. 9 (3), 174–180. http://dx.doi.org/10.1109/MGRS.2021.3089174.

Wang, Z., Ma, Y., Zhang, Y., 2022a. Hybrid cGAN: Coupling global and local features for SAR-to-optical image translation. IEEE Trans. Geosci. Remote Sens. 60, 1–16. http://dx.doi.org/10.1109/TGRS.2022.3212208.

Wang, H., Zhang, Z., Hu, Z., Dong, Q., 2022b. SAR-to-optical image translation with hierarchical latent features. IEEE Trans. Geosci. Remote Sens. 60, 1–12. http://dx.doi.org/10.1109/TGRS.2022.3200996.

Wei, J., Zou, H., Sun, L., Cao, X., He, S., Liu, S., Zhang, Y., 2023. CFRWD-GAN for SAR-to-optical image translation. Remote. Sens. 15 (10), 2547. http://dx.doi.org/10.3390/rs15102547.

Wu, S., Chen, Y., Mermet, S., Hurni, L., Schindler, K., Gonthier, N., Landrieu, L., 2024. StegoGAN: Leveraging steganography for non-bijective image-to-image translation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, WA, USA, pp. 7922–7931.

Xia, Y., He, W., Huang, Q., Chen, H., Huang, H., Zhang, H., 2024. SOSSF: Landsat-8 image synthesis on the blending of sentinel-1 and MODIS data. IEEE Trans. Geosci. Remote Sens. 62, 1–19. http://dx.doi.org/10.1109/TGRS.2024.3352662.

Yang, X., Wang, Z., Zhao, J., Yang, D., 2022a. FG-GAN: A fine-grained generative adversarial network for unsupervised SAR-to-optical image translation. IEEE Trans. Geosci. Remote Sens. 60, 1–11. http://dx.doi.org/10.1109/TGRS.2022.3165371.

Yang, X., Zhao, J., Wei, Z., Wang, N., Gao, X., 2022b. SAR-to-optical image translation based on improved cGAN. Pattern Recognit. 121, 108208. http://dx.doi.org/10.1016/j.patcog.2021.108208.

Zhang, T., Gao, F., Dong, J., Du, Q., 2022. Remote sensing image translation via style-based recalibration module and improved style discriminator. IEEE Geosci. Remote. Sens. Lett. 19, 1–5. http://dx.doi.org/10.1109/LGRS.2021.3068558.

Zhang, J., Liu, Z., Jiang, W., Liu, Y., Zhou, X., Li, X., 2023. Application of deep generative networks for SAR/ISAR: A review. Artif. Intell. Rev. 56 (10), 11905–11983. http://dx.doi.org/10.1007/s10462-023-10469-5.

Zhang, H., Yang, Y.-J., Zeng, W., 2024. Self-supervised multi-scale semantic consistency regularization for unsupervised image-to-image translation. Comput. Vis. Image Underst. 241, 103950. http://dx.doi.org/10.1016/j.cviu.2024.103950.

Zhao, Y., Celik, T., Liu, N., Li, H.-C., 2022. A comparative analysis of GAN-based methods for SAR-to-optical image translation. IEEE Geosci. Remote. Sens. Lett. 19, 1–5. http://dx.doi.org/10.1109/LGRS.2022.3177001.

Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J., 2018. UNet++: A nested U-Net architecture for medical image segmentation. In: Stoyanov, D., Taylor, Z., Carneiro, G., Syeda-Mahmood, T., Martel, A., Maier-Hein, L., Tavares, J.M.R.S., Bradley, A., Papa, J.P., Belagiannis, V., Nascimento, J.C., Lu, Z., Conjeti, S., Moradi, M., Greenspan, H., Madabhushi, A. (Eds.), Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Springer International Publishing, Granada, Spain, pp. 3–11. http://dx.doi.org/10.1007/978-3-030-00889-5_1.

Zhu, J.-Y., Park, T., Isola, P., Efros, A.A., 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision. In: Proc. IEEE Int. Conf. Comput. Vis., Venice, Italy, pp. 2223–2232.