In the case of up-sampling (e.g. CLC translated to OSO), this very wide resolution gap
necessarily implies using highly resolved additional data to obtain both a semantically
and geometrically more refined segmentation of the territory, e.g. to predict 2500 pixels
with potentially different target classes from a 25 pixels segment with a single source class.
However, Section 2.4 underlined that very few works study how to merge additional data
and maps in a context-aware fashion. Thus research must be conducted on this particular
topic. Examples of usable additional data are provided with the dataset and presented in
Section 3.4.

In the case of down-sampling (e.g. OSO translated to CLC), most current resampling
techniques use a grid-based approach (e.g. a cell of 100 OSO pixels is resumed into one
CLC pixel) to resample the land-cover map. Section 2.2.1 underlined that the technique
used for resuming the information, either nearest neighbour or majority voting, neglects
information for heterogeneous cells holding multiple classes. We argue that context and
class proportion should be used to obtain specific classes, e.g. a cell mixing OSO *Water*
and *Pastures* should be translated a CLC *Inland wetlands*.

From a machine-learning point of view, we underline that learning the MMU on the dataset
is a difficult task that requires a spatial context-aware translation framework that learns
rules such as: 'If two adjacent areas of discontinuous and continuous urban fabric occur,
each of them <25 ha, but in total >25 ha, they should be mapped as one single polygon,
and discontinuous urban fabric is privileged'. The main difficulty in learning the minimum
mapping unit is that errors in the source or target maps can be perceived as minimum
mapping rules when they exhibit systematic patterns. For instance, if small discontinuous
urban fabrics are often confused with industrial and commercial units, the method could
transform the previous rule in 'If two adjacent areas of industrial and commercial unit and
continuous urban fabric occur, each of them <25 ha, but in total >25 ha, they should be
mapped as one single polygon, and discontinuous urban fabric is privileged'. The impact
of errors on the dataset is addressed in the next Section.

3.3.3 Errors

Appendix E presents the confusion matrices of the France wide-land-cover maps (CLC, CGLS-LC100, OSO) computed on the ground-truth provided with the MLULC dataset (see Section 3.5.1) (official confusion matrices are available but produced using heterogeneous methods and does not cover the same spatial extent). From these confusion matrices we derive Figure 3.6, which presents the dispersion of per-class accuracies.

We observe that most of the land-cover maps exhibit a high precision variation depending
on the considered class. For instance, almost half of CGLS-LC100 level 2 classes exhibit
an accuracy of less than 70% and two classes of OSO exhibit a precision below 30%. From
a translation point of view, this is challenging, as errors in the source map have a high
risk of being mistranslated into the targeted nomenclature, *i.e.* translating from a map
with 70% accuracy is unlikely to give a target map with higher accuracy.
Finally, unlike semantic-based translation methods that are very sensitive to errors in source data, machine learnt methods are more sensitive to errors in target data. For instance, a per-class analysis of the OSO product reveals that the OSO *Road surfaces* are more than 50% of the time misclassified as *Industrial and Commercial units*. Thus, the translation from MOS *Transports* to OSO is likely to learn to translate into *Industrial and Commercial units*.

<figure><img src="image_20.png" /><figcaption>Figure 3.6: Box plot illustration of per-class accuracy dispersion for the three France-wide land-cover maps of the dataset. Per class accuracy are computed on the ground truth presented in Section 3.5.1.</figcaption></figure>

### 3.3.4 Spatial generalization

Since several maps do not cover the whole French territory, the number of available maps varies depending on the considered location, as shown in Figure 3.4. Using this particularity, one can study spatial generalisation to unseen areas on a real operational setup, e.g. translate the OSO map into MOS (less than 2% of the territory) to obtain a France wide MOS. However, this raises two main concerns. First, some OSO classes such as *glaciers*, *natural grassland*, or *bare soils* are never seen in the MOS original extent (see Figure 3.4). The translation from those OSO classes to a MOS class is never learnt. In this configuration, a first solution is to rely on rule-based non-contextual translation for those specific classes. Developing a machine learning strategy to address this issue is far more challenging and is discussed in Section 5.1. Consequently, they are hard to fit in the MOS nomenclature. Secondly, from a machine learning point of view, this difference in spatial extent creates a significant imbalance between the number of patch of the different LULC, making approaches performing multiple translations at the same time or multi-data fusion more difficult.

### 3.3.5 Conclusion

As we aimed to study translation in complex cases, the six selected land-cover maps exhibit complicated nomenclature and resolution translation problems and include a significant number of errors.
From the observation that the spatial co-occurrence between source and target classes observed between two maps of the dataset directly depends on the nomenclature, the resolution and the errors, we propose to summarise the difficulty by comparing the semantic-based nomenclature translation matrices provided in Appendix B with the spatial co-occurrence. As an illustration of this idea, we provide Table 3.2 in which we compare the observed percentage of spatial co-occurrence between OSO 2018 pixels at 10 m and CLC 2018 (at level 2 for readability) with their possible semantic association, e.g. one pixel labelled as *Discontinuous urban fabric* in OSO should semantically be translated into *Urban fabric* in CLC. In contrast, this association is only observed 45% of the time in the dataset.

<table><caption>Table 3.2: Semantic association between OSO and CLC level 2. The third column gives the observed percentage of spatial co-occurrence between one of the source class and all the proposed semantic translation(s) i.e. only 62% of OSO orchards pixels are translated into one of the semantically corresponding CLC class *Permanent crops* or *Heterogeneous agricultural areas*.</caption><thead><tr><th>OSO Classes</th><th>CLC Classes</th><th>Observed</th></tr></thead><tbody><tr><td>Dense urban</td><td>Urban fabric</td><td>87%</td></tr><tr><td>Sparse urban</td><td>Urban fabric</td><td>45%</td></tr><tr><td rowspan="2">Industrial and commercial Roads</td><td>Industrial, commercial and transport units</td><td>14%</td></tr><tr><td>Industrial, commercial and transport units</td><td>16%</td></tr><tr><td>Rapeseeds</td><td>Arable land / Heterogeneous agricultural areas</td><td>91%</td></tr><tr><td>Cereals</td><td>Arable land / Heterogeneous agricultural areas</td><td>90%</td></tr><tr><td>Protein Crops</td><td>Arable land / Heterogeneous agricultural areas</td><td>91%</td></tr><tr><td>Soy</td><td>Arable land / Heterogeneous agricultural areas</td><td>91%</td></tr><tr><td>Sunflower</td><td>Arable land / Heterogeneous agricultural areas</td><td>89%</td></tr><tr><td>Maize</td><td>Arable land / Heterogeneous agricultural areas</td><td>83%</td></tr><tr><td>Rice</td><td>Arable land / Heterogeneous agricultural areas</td><td>96%</td></tr><tr><td>Tubers</td><td>Arable land / Heterogeneous agricultural areas</td><td>96%</td></tr><tr><td>Orchards</td><td>Permanent crops / Heterogeneous agricultural areas</td><td>62%</td></tr><tr><td>Vineyards</td><td>Permanent crops / Heterogeneous agricultural areas</td><td>82%</td></tr><tr><td>Pastures</td><td>Pastures / Heterogeneous agricultural areas / Artificial non-agricultural vegetated areas</td><td>69%</td></tr><tr><td>Lawn</td><td>Shrub and herbaceous associations</td><td>39%</td></tr><tr><td>Shrub</td><td>Shrub and herbaceous associations</td><td>41%</td></tr><tr><td>Broad leaved</td><td>Forest / Artificial non-agricultural vegetated areas / Heterogeneous agricultural areas</td><td>82%</td></tr><tr><td>Coniferous</td><td>Forest / Artificial non-agricultural vegetated areas / Heterogeneous agricultural areas</td><td>79%</td></tr><tr><td>Mineral surfaces</td><td>Open space with little or no vegetation / Mine, dump and construction sites</td><td>86%</td></tr><tr><td>Sand</td><td>Open space with little or no vegetation</td><td>65%</td></tr><tr><td>Glaciers and snow</td><td>Open space with little or no vegetation</td><td>100%</td></tr><tr><td>Water</td><td>Inland water / Marine Water</td><td>84%</td></tr></tbody></table>

## 3.4 Ancillary data for enhancing the translation process

In the previous section, we explained that land-cover map translation is a difficult task that can even be impossible to achieve using just one source/target pair of maps due to a lack of semantic correspondence, a gap in resolution, and errors. In this section, we briefly present additional data that can be used to improve the translation quality when facing one of those cases. More specifically, we introduce data rich in semantic information to help with the lack of semantic information and errors and geometric information to help with the resolution gap. We investigate three sorts of data: Optical imagery, Synthetic aperture radar imagery, and Digital Elevation model. Mono-temporal data acquisitions are used to
increase the translation quality in those challenging cases. Multi-temporal acquisitions are deliberately ignored as we focus on the complementarity between additional data and land-cover maps for translation rather than producing the best quality maps. The choice of the data provider is made according to three main factors, the open access rights, which enable to share the dataset, the resolution of the data, which should be close to or higher than the most resolved map (10 × 10m), and the worldwide availability to enable comparable results on different spatial extents.

### 3.4.1 Optical imagery: Sentinel-2

Optical imagery, also termed optoelectronics, acquires information based on the reflection of the sunlight on the Earth's surface. The data is collected using mainly the wavelength between 400 and 1400 nanometers. Amongst the multiple optical data providers, we selected Sentinel-2 according to three criteria:

* A 10-meter spatial resolution, on par with highest resolved maps, to test the coarse to fine resolved translation in a non-ill-defined setup. As we do not seek to test the image potential for a given target land-cover, but it's complementarity with the source, more resolved images are irrelevant as the additional gain in translation accuracy would only be due to the image.

* The availability of cloud-free synthesis. As we proceed to a mono-temporal analysis at a France-wide scale, we require cloudless images to proceed with relevant data fusion.

* Free open access right enabling redistribution. This enables the distribution of the MLULC dataset.

The twin satellites Sentinel-2A and 2B are carrying multi-spectral optical sensors acquiring thirteen spectral bands at different spatial resolutions [72]. As we are mainly interested in gaining geometric information, we only keep the four 10-meter resolved bands (Red, Green, Blue and Near-infrared) in our analysis. The two satellites exhibit the same orbit but are phased at 180° to enable a temporal resolution of 5 days at the equator (slightly better at higher latitudes).

Two main levels of pre-processing Sentinel-2 images are distributed. Level 1 provides geometric correction to account for the main image distortions factor (satellite's motion, Earth's rotation, acquisition angle, orbit and terrain topography) to provide an accurate location for each pixel. Level 2 adds to level 1 the atmospheric properties, which are not constant in time and thus require corrections for applications performing a temporal analysis of an object, i.e. it removes exogenous factors such as atmospheric absorption or scattering.

As we do not plan to use multi-temporal acquisitions of Sentinel-2 but only a single one across all of France, those two processing levels are insufficient. Indeed, they do not
consider the cloud cover, which is more or less critical depending on the geographical areas and strongly degrades the data availability [289]. We relied on the Weighted Average Synthesis Processor (WASP) algorithm [109] implemented by the Theia French Land Data Center to produce a cloudless synthesis image. The core idea of the WASP processing chain is to perform a weighted average of all the Sentinel 2 images available during 45 days. The weight of each date is determined by taking into account both the time gap between this date and the mid of the 45-day date (to ensure a spectral coherence between adjacent pixels) and the likeliness of the pixel being cloud occulted. An almost cloudless single France wide-image with only a slight spectral shift between two tiles of different orbits is obtained. However, this processing chain still exhibits artefacts when all acquisitions during the 45-day periods (8 to 9 acquisitions) are cloudy on the same pixel. Artefacts are also observed on surfaces, such as snow and water, that change quickly over time.

Adding geometric information to the land-cover translation, such as the target segmen-
tation, using an image, implies using an image from which target segmentation can be
inferred. We observe that the potential of target segmentation inference from an image
widely depends on the considered date, especially for agricultural land-scapes (60% of
French territory) in which the visible segmentation varies across the year. The choice of
the 45-day period is essential as land-cover map mapping accuracy widely depends on the
date used due to seasonal variation [185]. A first constraint on selecting the date is that
the usage of 45 days (9 images) does not ensure a cloudless synthesis, especially in the
very cloudy months between October and March (the kept image must be outside of this
period). To our knowledge, no studies focused on the best dates to perform land-cover map
mapping over France. However, some thematic-specific studies on urban areas [248] have
demonstrated to produce more accurate land-cover maps during the Spring or Autumn
season. Since we previously mentioned that France's land-cover map mainly includes
croplands (nearly 60%) and that most of them are already harvested in the autumn season,
we selected a spring season cloudless synthesis centred on mid-April 2019. Ideally, we
would have processed the 2018 image to be as close as possible to land-cover maps, but
the synthesis of 2018 was too cloudy.

Last but not least, the image is aligned with the different land-cover maps in an alignment
step. It mainly consists of a re-projection from EPSG:4326 to EPSG:2154 on the same grid
as the one used for the land-cover map. This involves resampling, for which we relied on a
bi-cubic interpolation. As such, the effective resolution of the obtained image is slightly
under the original one.

3.4.2 Radar image: Sentinel-1

Radar imagery acquires information based on the reflection of a radar signal emitted by
the satellite on the Earth's surface. The data is collected using a wavelength between a cm
and a few meters. Since the reader might not be familiar with the SAR imaging system,
this section provide critical relevant information for understanding the dataset. However,
for a real in-depth explanation of the radar imaging system, we refer the reader to [14, 62, 219]. We selected the Sentinel-1 data following the same criterion list as the one used to
select the data provider for optical images.

The twin satellites Sentinel-1A and 1B carry a C-SAR (synthetic aperture radar) instrument acquiring in dual polarisation (HH+HV, VV+VH) with an incidence acquisition angle between 20-46°. This right-sided acquisition angle has many consequences, such as producing images with a pixel resolution varying across the satellite track. The Sentinel-1 data are either distributed with all the possible information (complex signal with phase and amplitude) and with the unequal pixel size in a mode called Single-Look Complex (SLC) or in a simpler form including only the modulus of the signal (with no phase information) in a resampled 10m × 10m pixel-sized image in a mode called Ground Range Detect (GRD). As our work does not involve phase exploitation, traditionally used for SAR interferometry, we only discuss the characteristics of the GRD product.

Radar images' main characteristics stem from the used wavelength in the case of Sentinel 1, the C-band, which corresponds to 5.5 cm. As a diffraction-limited system [209], a radar imaging system cannot acquire information on objects too small compared to the wavelength used. Clouds mainly consisting of water droplets of a millimetre width are seen through, enabling image acquisition even in cloudy weather. More generally, the use of a significantly higher wavelength than optical imagery brings widely different information. For instance, the signal tends to penetrate the tree and soil cover a bit before being reflected. It provides additional geometric information that is not seen in an optical image.

However, because the radar signal is coherently emitted in space and time, the multiple backscattered signal of a single pixel content creates some constructive and destructive interference. This sum of signals inside each pixel results in a noise-like pattern referred to as speckle, which can be modelled as a multiplicative noise (assuming a set of realistic assumptions referred to as the Goodman hypothesis [102]). As speckle makes the radar images usage considerably harder, various denoising (despeckling) methods have been proposed. Local spatial filtering methods have been proposed and perform local weighted means to reduce the noise at the cost of down-resolving the image. Temporal filtering methods have also been proposed. They use the random aspect of the noise by averaging multiple noise-independent acquisitions, with the drawback of being inconsistent on surfaces changing over time. Lastly, mixes between those strategies have also been proposed to alleviate the problems of both methods.

Since we mainly want to preserve the geometric information rather than the radiometric
one, we propose performing a temporal average of 3 years of sentinel-1 acquisition across
France (2017-2019). As the SAR acquisition is realized with an angle, each object can be
seen from two points of view depending on the orbit direction (ascending or descending).
We randomly selected the descending. Furthermore, we only included images from the S1B
satellite to avoid averaging with the S1A, which has very slightly yet different acquisition
parameters. This results in averaging up to 120 GRD images on some locations producing
a nearly noiseless SAR image.
Preprocessing steps include: calibration to consider the incidence angle and remove thermal noise, orthorectification and a temporal average. As the last step, we reprojected all the images in EPSG:4326 along the grid used for land-cover maps, which involves a bi-cubic resampling. All the processing steps except the temporal average are proceeded using the S1Tiling software<fnref n="6" />.

### 3.4.3 DEM: Alos-World3D

Elevation and land-cover are highly intertwined, making elevation a good additional data to increase the quality of land-cover translation. For instance, France vineyards tend to be cultivated on lands with a slope (that ensures excellent drainage properties) south oriented (to ensure high solar exposition).

Ideally, the digital elevation model (DEM) should be around 10 meters resolved to exhibit the exact resolution as the most resolved maps. However, even though some corresponding digital elevation models are available in France, they do not provide the worldwide extent mentioned. We choose to rely on one of the two publicly available worldwide DEM at 30×30m resolution: Alos-World3D [2] (AW3D30) was chosen over the SRTM due to its higher height accuracy [264].

The AW3D30 provides a single image, with each pixel representing the height of the object inside with a 5-meter precision. The data is reprojected in EPSG:2154 using the same grid as the one used for land-cover maps and a bi-cubic resampling algorithm.

Commonly computed DEM-derived features (slope, aspect, topographic position and roughness index) are also provided to help to increase the quality of the translation. In particular, we underline that high slope areas are often mineral areas, that some crops, such as vineyards, exhibit a specific slope orientation towards the sun (aspect), and that wetlands are often observed downhill (topographic position index).

## 3.5 Ground truth and quality measurement

### 3.5.1 Ground-truth datasets

Comparison between translated and target maps is the simplest way to assess the quality of the translation. Comparison can be performed pixel-wise all over the test set, offering many samples per class to evaluate absolute and per-class metrics. However, those measurements are maximised only when the translation exhibits the same errors as the target data, i.e. a translation corresponding at 100% to CGLS is a translation replicating the 30% error rate of CGLS. We refer to this comparison as an *agreement measure* rather than an accuracy measure. Moreover, it is worth noting that the agreement measure can only be computed

<footnote n="6">⁶https://gitlab.orfeo-toolbox.org/s1-tiling/s1tiling</footnote>
on the intersection between the source and target maps. For instance, the agreement measurement between the translation from MOS to OSO and OSO can only be computed on the Paris area, while the result of the CLC-to-OSO translation can be computed over entire France. Consequently, those two agreement measurements are not comparable.

Conversely, the comparison with an independent ground truth gives a better estimate of the accuracy. However, creating such a ground truth on each specific map spatial extent for all of the six maps with a large enough sample to compute significant per-class accuracies [85] is unrealistic for both time and lack of expertise reasons. This ground truth should be country-wide (to study generalisation to broader spatial extents) and with classes compliant with the specifications of each map. During this PhD, we released two ground truths datasets; one including ground truth annotation for the six land-cover maps of the MLULC dataset, including 2,300 points and one specifically focusing on CLC at level 2 but with a higher number of annotations (5,000) to ensure enough per-class sample to estimate per-class metrics. As the creation procedure of those two ground truths are identical, we only focus on the broad one (including the six land-cover maps) in the remaining to simplify the reading.

The choice of the ground truth sample size and sampling strategy directly influences the precision of the computed metrics. To enable a fair comparison between different land-cover maps, the sampling strategy consist in annotating the same sample for all land-cover maps, i.e. one ground truth point is annotated six times. As a direct consequence, stratified random sampling strategies aiming to provide enough samples per class to compute accurate per-class metrics are difficult to conduct. We choose to rely on pure random sampling, which provides accurate overall accuracy across the territory at the cost of poor per-class metrics estimation. For a suitable ground truth sample size *n* ensuring reasonable overall accuracy evaluation, we rely on Equation 3.1 [52, 230]:

$$n = \frac{z^2\alpha(1 - \alpha)}{m^2} \qquad (3.1)$$

where *z* is a percentile from the standard normal distribution, *α* is the real overall accuracy and *m* is the authorised margin of error. We arbitrarily choose *z* = 1.96 and *m* = 2%, corresponding to a 95% confidence interval with a 2% margin on the overall accuracy prediction as those values are the most commonly found in land cover literature. We choose to set *α* = 50% (the worst-case scenario leading to the largest available sample size), as we can not predict the real overall accuracy of the different translations. This results in an expected ground truth sample size *n* = 2,300.

These 2,300 points are randomly sampled from the test set; thus, they can not be used to compute per-class accuracy due to the low (or null) number of samples for rare classes. We also provide 400 additional points (non-randomly sampled), focusing on rare classes to ensure an arbitrary minimum number of 15 points per class to enable rough per-class metric estimation. The points in the ground truth are sampled with a minimum distance of 2.5 km to reduce spatial correlation. The MMU of CLC on linear elements does not
guarantee independence below this distance.

Ground truth labelling relies on photo-interpretation of Sentinel-2 (multiple dates available) and Spot 6 and 7 (High resolution) imagery, and two independent sources of information: (i) the French authoritative cartographic database (BD Topo), yearly updated at 2 m with more than a hundred classes, and (ii) the national Land Parcel Information System (RPG, Registre Parcellaire Graphique), a 10 m farmers declarative database for European Common Agricultural Policy (CAP) [36]. We consider the target data valid unless it disagrees with those sets, in which case photo interpretation is performed. Such ground truth exhibits multiple limitations:

* First, the BD Topo and RPG only cover about 75% of France since some structures are excluded (e.g., sidewalks), and information is lacking (missing farmer declarations, especially for crops not included in CAP subsidies). Thus exhaustive ground truth representation of some classes is not guaranteed.

* The generated ground truth is a partially corrected version of the original data instead of a completely independent ground truth (i.e., favourably biased toward the original data).

* The 400 additional points are mostly added on the rarest of the 44 CLC classes to increase their sample size. Thus they abide by the 25 ha MMU of CLC, which significantly affects statistics for other maps. For instance, most CLC *Sport and leisure facilities* additional points added are *golfs* since they cover large surfaces and subsequently artificially enriches the MOS *Artificial green urban areas* with numerous golfs.

* A France-wide ground truth can only be used to assess France-wide quality: evaluation of translation quality on a smaller extent (MOS, OCSGE) can not be performed.

The obtained ground truth enables fairly computing the original land-cover accuracy. The obtained accuracy for OSO (86%) and CGLS-LC100 (73%) (is on par with their official nomenclature (respectively 87% and 72% at the European level for CGLS-LC100) while the one of CLC strongly differs (89% on the ground truth compared to 94.2% in [218]). This stems from multiple variations with CLC's quality assessment protocol:

* Two different operators double-checked the CLC official validation dataset, while ours includes only one interpretation for each point.

* The official validation is achieved with respect to the CLC initial segmentation (to avoid taking into account geometric errors, separately evaluated), while ours correct CLC segmentation when needed. Therefore, our interpretation of the same point might differ, especially on the edges.

* The official validation is performed on the vector data, while ours is performed on a rasterised version, which tends to amplify CLC segmentation errors.
<table><caption>Table 3.3: Summary of the characteristics of the three data sets used for translation evaluation.</caption><thead><tr><th>Evaluation data</th><th>Target map</th><th>Random ground truth</th><th>Enriched ground truth</th></tr></thead><tbody><tr><td>Sample size</td><td>&gt;100,000 for all LULC</td><td>2,300</td><td>2,300+400</td></tr><tr><td>Minimum sample per class</td><td>&gt;1,000</td><td>0</td><td>10</td></tr><tr><td>Pros</td><td>Big sample size = per-class metrics are computable</td><td>France wide coverage for all maps</td><td>France wide coverage for all maps</td></tr><tr><td>Cons</td><td>Same errors as target data<br>Only covers the target extent</td><td>Small minimum sample per class</td><td>Partially biased to increase sample size of rare classes</td></tr><tr><td>Usage</td><td>Overall accuracy<br>Per class accuracy</td><td>Overall accuracy<br>Generalisation</td><td>Per class accuracy<br>Generalisation</td></tr></tbody></table>

### 3.5.2 Quantitative indices

Quantitative evaluation is performed using traditional land-cover metrics to compare with other works. In particular, we provide the confusion matrix to discriminate the nature of errors. The Overall Accuracy computation is used to account for global quality. LULC data sets are highly class-imbalanced: high accuracy can be achieved by correctly predicting the most frequent classes (often not the most difficult to discriminate). Therefore macro f1-score and Kappa are computed to assess the quality of the translated classes more accurately. Standard per-class metrics (precision, recall, f1-score) are also computed. We underline that other metrics used in semantic segmentation could have been used but exhibit either a perfect correlation with one of those indicators (IoU and f1-score), or are not commonly found in the land-cover field (mean Average Precision) preventing comparison with other works. Formulas for per-class metrics and overall metrics are given in Equations 3.2 and 3.3, respectively.

$$p_i = \sum_{j=1}^{c} \frac{m_{ii}}{m_{ji}}, \qquad r_i = \sum_{j=1}^{c} \frac{m_{ii}}{m_{ij}}, \qquad F1_i = \sum_{j=1}^{c} 2 \frac{p_i r_i}{p_i + r_i}. \qquad (3.2)$$

$$OA = \frac{\sum_{i=1}^{c} m_{ii}}{\sum_{i,j=1}^{c} m_{ij}}, \qquad mF1 = \frac{1}{n} \sum_{j=1}^{c} F1_i, \qquad K = \frac{OA - p_e}{1 - p_e}. \qquad (3.3)$$

$p_i$, $r_i$, and $F1_i$ are the precision, recall, and f-score for a given class $i$, respectively. $OA$ is the overall accuracy, $mF1$ is the macro f1-score, $c$ is the number of classes, and $m_{ij}$ is the element in the $j^{th}$ row of the $i^{th}$ column of the confusion matrix, i.e., the number of pixels of class $j$ classified as $i$. $p_e$ is the hypothetical overall accuracy obtained with a classifier replicating the class proportions randomly. These statistics are computed separately by comparing the translation with (a) the target LULC and (b) the ground truth. For clarity, we denote $OA_{ag}$ and $mF1_{ag}$ the metrics computed by comparing the translation to the
target and OA<sub>gt</sub> and mF1<sub>gt</sub> the metrics computed by comparing the translation to the ground truth. We underline that all the metrics displayed in this manuscript are rounded to the closest per cent, as we believe variations below a per cent are not interesting from an operational point of view. Moreover, to be statistically significant, it would require dozens of independent training for each experiment as the results between two different training of the models of this manuscript often vary up to 0.25%.

Evaluating the quality of the geometric preservation of a map is rarely done in land-cover map classification studies while being extensively studied in image reconstruction or denoising [262]. Thus, we lack predefined quantitative measures to assess it. We propose to use the Edge Preservation Index (EPI),also termed Edge correlation [199], traditionally used in [351]. It computes the correlation between edges in a pair of images. EPI is defined as [265]:

$$EPI = \frac{\Gamma(\Delta l - \bar{\Delta}l, \Delta t - \bar{\Delta}t)}{\sqrt{\Gamma(\Delta l - \bar{\Delta}l, \Delta l - \bar{\Delta}l)\Gamma(\Delta t - \bar{\Delta}t, \Delta t - \bar{\Delta}t)}} \quad (3.4)$$

where *l* is the map predicted by the network, *t* is the ground truth target, Δ denotes a high pass filter (we use a simple Laplacian kernel), Δl denotes the mean of the high pass filtered image, and Γ is defined as:

$$\Gamma(x, y) = \sum_{i,j \in (nr, nc)} x(i, j) \times y(i, j), \quad (3.5)$$

where *nr* and *nc* represent the number of pixels in rows and columns, respectively, of the given image.

Since we work on categorical data, edges are discretized (0 = non edge, 1 = edge). Ideally, the EPI should be computed between the prediction and perfectly segmented ground truth. Since creating an object-oriented ground truth would be overly time-consuming, we compute the EPI between the prediction and the target. It assesses the agreement between the prediction and the target edges rather than the true correlation between the prediction and a perfect segmentation. The larger the EPI value is, the more edges are maintained. The EPI value is highly dependent on the expected proportion of edge in the target, which makes it quite comparable to the dependence of the proportion of each class of the *kappa* value. Unlike Kappa, for which interpretation key enabling distinction between low and high agreement have been provided [173, 281], no such key exists for EPI. We propose to build an interpretation key for each land-cover maps (see Table H.1) using a protocol described in Appendix H. We propose to use the average of each threshold resulting in the following key: poor (EPI<0), slight (0 < EPI < 0.16), fair (0.16 < EPI < 0.30), moderate (0.30 < EPI < 0.45), substantial (0.45 < EPI < 0.60), almost perfect (EPI > 0.60).
# Mono land-cover map translation

This chapter tries to answer the question: How to translate from a source to a target leveraging various context information? In this direction, we decompose the chapter in the following sections.

First, Section 4.1 introduces baseline land-cover map translation methods directly inspired by the literature. As current literature relies principally on a resampling followed by nomenclature-level rule-based translations, these non-contextual methods are assimilated as a reference group to evaluate the performance of the context-aware methods presented in other sections.

Secondly, Section 4.2 investigates the potential of spatial context for translation. First, a method based on standard manually defined shape indicators is used to perform context-wise translation using the shape of the object a pixel belongs to. Secondly, a convolutional neural network is used to replace manually defined features with learnt ones, hoping to extract spatial context information better.

Section 4.3 focuses on the geographical context. First, we identify and compare two data sources of geographical context, i.e. ecoclimatic areas and geographical coordinates. Two different strategies are proposed to leverage these data based on independent model training per area or direct incorporation as learnable features to train a single model. We then focus on incorporating geographical coordinates in the Convolutional neural network proposed in the previous section.

Section 4.4 addresses the temporal context. A first investigation of the potential of multi-temporal analysis is conducted. In particular, a distinction between improvements due to higher temporal information and those due to a higher ability to identify source errors is conducted. Secondly, the effect of a temporal gap between the source and target map used for learning the translation is assessed and linked with real operational situations.

Section 4.5 addresses the cartographic context. We propose a loss-based method to learn translation when the target data is heavily noisy (30% when the target is CGLS) using the noise confusion matrix provided by the land-cover map producers.

All experiments are conducted on the MLULC dataset introduced in Section 3, leading to 26 possible translations with various resolutions and nomenclatures. Therefore we
expect the results to be generalizable to other land-cover maps. As no additional data
is considered in this section, we underline that the presented translations results from
coarsely resolved sources (CGLS and CLC) to highly resolved target maps (OSO, OCSGE
use, OCSGE cover, MOS) are ill-defined but still presented for reference.

4.1 Map translation without contextual information

4.1.1 Motivation

This section proposes a set of translation baselines, inspired by the state-of-the-art, that we used as references to estimate the potential of the context-wise methods introduced later on. The six proposed baselines, summarised in Table 4.1, reflects the diversity of translation approaches proposed in the current literature. We identify three key differences between translation methods: soft or hard association, semantic or statistics translation, and learnt or manually defined resampling. We review them briefly below.

Section 2.1.2 distinguished the hard association, *i.e.* one source class is translated into a single target class, from soft association, *i.e.* one source class have varying translations depending on the considered pixel. We underlined that current soft association techniques either require downsampling or merging multiple source. As this thesis mainly focuses on the translation framework rather than on merging hypothetically available maps, soft association baselines proposed in this section relies on source down-sampling.

We also distinguished baselines according to the nature of the translation performed: Semantic or Statistic. Semantic translation (**HardSem**, **SoftSem**) of source relies on a set of manually defined rules linking the source and target nomenclature with the main advantage of not requiring an existing target sample for training. Statistic based methods (**SoftStat**, **SoftMaxProba**, **SoftLearntFreq**, **SoftLearntGridPattern**) evaluate the correspondence between source and target classes by computing spatial co-occurence between source and target classes on pairs of spatially overlapping source and target.

Finally we distinguish machine-learnt statistic downsampling strategies (*SoftLearntFreq*, *SoftLearntGridPattern*) from manually defined ones (*SoftSem*, *SoftMaxProba*). Manually defined downsampling techniques combine translation conducted independently for multiple source pixels to obtain a single target class. This approach inherently ignores the potential translation synergies of the different source pixels. For instance, *Water* and *Grasslands* are not very close semantically and statistically from *Rice crops* when analysed separately. Conversely, when 2 pixels, one of *Water* one of *Grassland*, are to be translated together into a single target class, *Rice crops* is a semantically and statistically valid translation. The machine-learnt statistic downsampling strategies enable an analysis of those source class compatibilities instead of considering each translation independently.

In the next subsection, we present the implementation details of each corresponding
baseline used in this manuscript.
<table><caption>Table 4.1: Presentation of the six baseline methods used for comparison. We represent positive/negative characteristics in the forms of +/-. The number of signs denotes the degree of positiveness/negativeness of the method on an arbitrary scale ranging from -- (the method is the worst possible for this characteristic) + + + the method is the best.</caption><thead><tr><th rowspan="2"></th><th rowspan="2">Method</th><th rowspan="2">Nomenclature translation</th><th rowspan="2">Resolution translation</th><th colspan="4">Characteristics</th><th rowspan="2">Related works</th></tr><tr><th>Translated class diversity</th><th>Sensitivity to noise in target</th><th>Sensitivity to noise in source</th><th>Apply MMU rules</th></tr></thead><tbody><tr><td>HardSem</td><td>Semantic-based: Highest semantic affinity association</td><td>- After nomenclature translation<br>- Majority voting</td><td>-</td><td>+</td><td>++</td><td>--</td><td>--</td><td>[243, 305, 317]</td></tr><tr><td>HardStat</td><td>Statistic-based: Highest probability association</td><td>- After nomenclature translation<br>- Majority voting</td><td>-</td><td>--</td><td>--</td><td>--</td><td>--</td><td>[225]</td></tr><tr><td>SoftSem</td><td>Semantic-based: Highest combined semantic affinity association</td><td>- After nomenclature translation<br>- Majority voting</td><td>+</td><td>++</td><td>--</td><td>--</td><td>--</td><td></td></tr><tr><td>SoftMaxProba</td><td>Statistic-based: Highest combined probability association</td><td>- After nomenclature translation<br>- Sum of probability</td><td>+</td><td>--</td><td>--</td><td>--</td><td>--</td><td>[60, 92]</td></tr><tr><td>SoftLearntFreq</td><td>Statistic-based: Learnt class frequency association</td><td>- Simulatenous with nomenclature<br>- Frequency association</td><td>++</td><td>--</td><td>--</td><td>--</td><td>+</td><td>[56, 180]</td></tr><tr><td>SoftLearntGridPattern</td><td>Statistic-based: Learnt class spatial pattern association</td><td>- Simulatenous with nomenclature<br>- Spatial pattern association</td><td>++</td><td>--</td><td>--</td><td>--</td><td>+</td><td></td></tr></tbody></table>
4.1.2 Baselines implementation

Three of the six baselines mentioned below (HardSem, HardStat, SoftLearntFreq) are found in the literature. The three others are natural adaptations of close-by methods and are part of our contribution to the translation problem.

**HardSem** (Figure 4.1) is the most commonly found approach in literature. It relies on a separate nomenclature and resolution translation. Most works perform the resolution translation before the nomenclature translation. However, we argue that the resolution translation should always be performed on the target nomenclature to keep the most relevant information and thus proceed in this order: **HardSem** first applies a semantic translation, then the resolution one. The semantic translation is performed in a hard association manner, *i.e.* one source class is associated with one unique target class. Since finding the main correspondent of each source class into the target nomenclature is straightforward for a human operator, the association is performed manually instead of using a semantic harmonisation tool such as LCCS. The results of this one source/one target class association is provided in Appendix A with the nomenclature of each of the 6 maps. For instance, all the pixels of class *Individual housing* of the MOS map are translated into *Built up* in the CGLS-LC100 map. Most current papers perform the resampling using the nearest neighbour method. This is not detrimental when the target resolution is equal to or finer than the source. However, this resampling method is the worst possible when the target resolution is coarser than the source. *n* source pixels are resumed into a single target one using only the content of one of them, thereby ignoring all the others. Instead, we propose to perform, in this case, the resampling using a majority voting rule, *i.e.* the class of the target pixel is the most frequently observed one amongst the *n* source pixels. This resampling system is rarely used because it is more computationally intensive and rarely implemented in the standard image processing software.

<figure><img src="image_21.png" /><figcaption>Figure 4.1: The **HardSem** translation: Source $S_i$ and $S_j$ are translated into their closest semantic correspondent, target class $T_k$ and $T_l$, respectively. A down-resolution translation of factor 2 is then performed using majority voting.</figcaption></figure>

**HardStat** relies on the exact same strategy: nomenclature translation followed by a resolution translation (see Figure 4.2). The only difference is that the nomenclature translation is based on statistical matching. An association matrix, giving the spatial
co-occurrence proportion of each source with each target class, is computed. Each source class is translated into its most frequently spatially co-occurring target class. For instance, if a source class $S_i$ is observed at the same location than a target class $T_k$ in 10% of cases, class $T_l$ in 60% and class $T_m$ in 30%, $S_i$ is translated into class $T_l$. As some of the soft baselines presented use a machine-learning algorithm such as the Random forest, we underline that this method gives strictly equivalent results to training a random forest to perform the translation from source to target using only the source class as a feature.

<figure><img src="image_22.png" /><figcaption>Figure 4.2: **HardStat**: Source $S_i$ and $S_j$ are translated into the most frequent corresponding target, $T_m$ and $T_l$, respectively. A down-sampling of factor 2 is then performed by majority voting.</figcaption></figure>

**SoftSem** presented in Figure 4.3 can be used when the target resolution is coarser than the source one. It relies on a separate nomenclature and resolution translation. The nomenclature translation is achieved by using the soft association between the source and the target maps described in Appendix B. Each source class might be semantically linked with multiple target ones. Since we do not assume prior knowledge of the target segmentation, the resolution translation is performed with a grid-based approach. The possible target translations of the two source pixels are summed, and the target class with the most votes is assigned. Thus two pixels with the same source class can be translated differently depending on their neighbourhood.

<figure><img src="image_23.png" /><figcaption>Figure 4.3: **SoftSem**: Source $S_i$ and $S_j$ are associated to all their possible semantic correspondent ($T_k/T_l$ and $T_l/T_m$ respectively). A down-sampling of a factor 2 is then performed by majority voting.</figcaption></figure>
**SoftMaxProba** (Figure 4.4), based on [60], can be used when target's resolution is coarser than source's. It relies on a separate nomenclature and resolution translation. The nomenclature translation is achieved using the same association matrix as the **HardStat**. The average of all the observed proportions of co-occurrence for each of the *n* source pixels is performed, and the most probable one is then assigned to the target. This acts differently than the **HardStat** method, which only considers one possible translation per source pixel (the most frequently observed one). However, the statistical links between the source and target nomenclature do not consider multiple source class compatibility.

<figure><img src="image_24.png" /><figcaption>Figure 4.4: **SoftMaxProba**: the translated class is obtained by averaging the probability of each source class to correspond to each target class and choosing the most probable one.</figcaption></figure>

**SoftLearntFreq** directly inspired from [56], can be used when the target resolution is coarser than the source one (Figure 4.5). It relies on a simultaneous nomenclature and resolution translation and tries to alleviate the **SoftMaxProba** main limitation by considering multiple source class compatibilities. The core idea is that compatibility can be inferred from data through machine learning algorithms. For instance, one can learn to translate differently depending on the proportions of each source class amongst n-pixels to translate into one. Additionally, it enables us to learn thresholds rules such as *"it is forest only if trees cover more than 70% of the surface"*. Unlike [56], we do not assume prior knowledge of the target segmentation, relying instead on a grid-based segmentation at the target resolution. This approach appears more realistic in most land-cover translation problems, as the target segmentation is usually unknown. Moreover, [56] analysed the proportions of classes using a discriminant analysis while we replaced it with a random forest, showing better results in our case. **Implementation details:** A cross-validation experiment is conducted to estimate the best random forest parameters according to the *mF1<sub>ag</sub>* using a grid search approach on the number of trees (50 to 500, with a 50 step), maximum tree depth (from 5 to 40, with a 5 step) and the minimum samples per leaves (from 10 to 1 to 101, with a step 10). Even though slight variations are observed between different map 100 trees, with a 25 maximum depth and 10 minimum samples per leaf
appears sufficient to achieve the best mF1 score for all maps at ±1%.

<figure><img src="image_25.png" /><figcaption>Figure 4.5: **SoftLearntFreq**: The frequency of each source class is used to train a random forest. Translates differently depending on the proportion of each source class.</figcaption></figure>

**SoftLearntGridPattern** (Figure 4.6) is very close to **SoftLearntFreq**. However, instead of learning how to translate source class proportions, the method directly processes the *n* pixels preserving their local spatial arrangement. We replace the random forest with a convolutional neural network, the ResNet50, as a random forest would be unable to learn all the possible patterns due to the very high number of configurations (16 in the Figure 4.6 with only two source classes and a small downsampling factor of 4). **Implementation details:** We used the classical ResNet50 implementation [114] in its pytorch implementation<fnref n="1" />.

<figure><img src="image_26.png" /><figcaption>Figure 4.6: **SoftLearntGridPattern**: The translation of each source class is performed by a ResNet50 trained to translate differently based on the local class spatial arrangement.</figcaption></figure>

<footnote n="1">¹https://pytorch.org/hub/pytorch_vision_resnet/</footnote>
4.1.3 Results

4.1.3.1 Qualitative analysis

<figure><img src="image_27.png" /><figcaption>Figure 4.7: Sample of results for six different areas with each of the six baseline methods. The class/label correspondence tables of each LULC are available in Appendix A.</figcaption></figure>
We categorize those results into three qualitatively homogeneous groups: the semantic group (**HardSem**, **SoftSem**), the simple statistic group (**HardStat**, **SoftMaxProba**), and the machine-leaning based downsampling group (**SoftLearntFreq**, **SoftLearnt-GridPattern**). The distinction between those three groups is especially visible in the second column for which OSO 19: *Woody moorlands* is translated as CGLS 20: *Shrubland* by semantic-based methods, as CGLS 12: *Open forest* by simple statistic methods or as a mix between CGLS 20: *Herbaceous vegetation* and *Shrubland* by more machine learning based methods. The difference between the semantic group and the other is mainly observed for classes for which the main semantic correspondent differs from the statistically most observed one. Such differences are often observed on erroneous source classes, as semantic translations assume that the description of a class and its real content is identical, *i.e.* no errors in the source map. For instance, in the second column (OSO to CGLS-LC100), the source OSO maps widely overestimates *Bare rock* areas (in red) which are wrongly translated into CGLS-LC100 *Bare / sparse vegetation* (in grey). Conversely, statistic methods determine the translation based on the real noisy content of the source and target class resulting in different translations, *e.g.* most OSO *Bare rock* corresponds to *Herbaceous vegetation* in CGLS. Consequently, statistic methods partially compensate for errors in the source maps but are prone to replicate errors of the target.

The simple and machine learnt statistic groups often give almost identical results. Most of the differences are located on the edges of objects, as illustrated in the two last columns. For instance, edges between OCS-GEc 1111:*Built-up areas* and 222:*Herbaceous formations* are translated as CLC 211:*Non-irrigated arable land* by simple statistic methods while being translated as 242:*Complex cultivation patterns* by the machine learnt ones. This distinction makes sense as the surrounding of small cities are very often classified as 242 in the CLC product and is only obtainable by learning to perform the dual translation of the two classes simultaneously. We highlight that those two groups are highly dependent on the noise in the target. For instance, CGLS-LC100 tends to misclassify *Water* as *Open Forest* on rivers, and both groups replicate the same error in the translation as shown in the fourth column (MOS to CGLS-LC100).

A general weakness shared between all those methods is that they fail to predict many classes resulting in a low diversity of translated classes. Moreover, none of them can apply the CLC MMU of 25ha (25 pixels), resulting in this noisy single isolated pixels pattern.

### 4.1.3.2 Quantitative analysis

As for the qualitative review, quantitative metrics for soft association methods can only be computed for translation in which the target map is coarser resolved than the source one. Tables presenting the quantitative translation results hold empty cells for translation in which the target is identically or higher resolved. In this case we arbitrarily consider that **SoftSem** is identical to the **HardSem**. Similarly, other soft methods are considered identical to the **HardStat**.
<table><caption>Table 4.2: Agreement metrics between the prediction and the target map for all pairs of overlapping source/target maps. Cells are coloured in grey for methods based on statistical correspondences and in white for those based on semantics. Cells are kept empty for soft baselines when the target resolution is not coarser than the source. The average performance is computed (i) on the 26 translations, filling the empty cells with their corresponding **HardSem** or **HardStat** results (Total average) or (ii) on the nine translations when the target resolution is coarser (Average on soft).</caption><thead><tr><th rowspan="2">Source<br>Target</th><th colspan="5">CGLS (P)</th><th colspan="5">CLC (C)</th><th colspan="5">OSO (O)</th><th colspan="5">OCS-GEc (G1)</th><th colspan="5">OCS-GEu (G2)</th><th colspan="5">MOS (M)</th><th colspan="2">Total<br>average</th><th rowspan="2">Average<br>on soft</th></tr><tr><th>C</th><th>O</th><th>G1</th><th>G2</th><th>M</th><th>P</th><th>O</th><th>G1</th><th>G2</th><th>M</th><th>P</th><th>C</th><th>O</th><th>G1</th><th>G2</th><th>P</th><th>C</th><th>O</th><th>G1</th><th>G2</th><th>P</th><th>C</th><th>O</th><th>G1</th><th>G2</th><th>P</th><th>C</th><th>O</th><th>G1</th><th>G2</th><th>P</th><th>C</th><th>O</th><th>G1</th><th>G2</th></tr></thead><tbody><tr><td rowspan="5">OA<sub>gt</sub></td><td>HardSem</td><td>52</td><td>42</td><td>56</td><td><b>70</b></td><td><b>75</b></td><td>65</td><td>49</td><td>67</td><td>77</td><td><b>79</b></td><td>57</td><td>54</td><td>69</td><td>76</td><td>80</td><td>53</td><td>39</td><td>34</td><td>87</td><td>54</td><td>37</td><td>31</td><td>75</td><td>77</td><td>72</td><td>59</td><td>61</td><td>58</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftSem</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>62</td><td>59</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>HardStat</td><td>54</td><td>44</td><td>65</td><td>70</td><td>75</td><td>68</td><td>55</td><td>71</td><td>78</td><td>79</td><td>61</td><td>54</td><td>73</td><td>80</td><td>81</td><td>55</td><td>41</td><td>49</td><td>89</td><td>54</td><td>37</td><td>41</td><td>78</td><td>80</td><td>76</td><td>62</td><td>61</td><td>61</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftMaxProba</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>65</td><td>61</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftLearntFreq</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td><b>72</b></td><td><b>62</b></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftLearntGridPattern</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td><b>72</b></td><td><b>62</b></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="5">mF1<sub>gt</sub></td><td>HardSem</td><td>13</td><td>17</td><td><b>22</b></td><td>15</td><td>24</td><td>46</td><td><b>32</b></td><td><b>36</b></td><td><b>31</b></td><td><b>42</b></td><td>36</td><td>17</td><td><b>36</b></td><td><b>20</b></td><td>38</td><td>24</td><td>9</td><td>17</td><td>27</td><td>19</td><td>8</td><td>8</td><td><b>29</b></td><td>35</td><td>17</td><td><b>19</b></td><td>25</td><td>23</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftSem</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>38</td><td>19</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>HardStat</td><td>13</td><td>18</td><td>19</td><td>16</td><td>24</td><td>47</td><td>32</td><td>33</td><td>30</td><td>42</td><td>33</td><td>16</td><td>34</td><td>20</td><td>38</td><td>26</td><td>10</td><td>20</td><td>27</td><td>19</td><td>8</td><td>10</td><td>27</td><td>31</td><td>15</td><td>18</td><td>24</td><td>22</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftMaxProba</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>36</td><td>18</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftLearntFreq</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td><b>48</b></td><td><b>26</b></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftLearntGridPattern</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>47</td><td>24</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="5">EP1</td><td>HardSem</td><td>28</td><td>5</td><td>6</td><td>5</td><td>17</td><td>30</td><td>6</td><td>6</td><td>7</td><td>20</td><td>32</td><td>30</td><td><b>28</b></td><td><b>32</b></td><td>57</td><td>29</td><td>29</td><td>29</td><td>79</td><td>31</td><td>31</td><td>29</td><td><b>77</b></td><td>53</td><td>49</td><td><b>38</b></td><td>30</td><td>38</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftSem</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>36</td><td>36</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>HardStat</td><td>27</td><td>5</td><td>3</td><td>5</td><td>16</td><td>30</td><td>6</td><td>5</td><td>7</td><td>20</td><td>30</td><td>27</td><td>25</td><td>31</td><td>56</td><td>30</td><td>29</td><td><b>30</b></td><td><b>80</b></td><td></td><td>31</td><td>31</td><td><b>30</b></td><td><b>77</b></td><td>47</td><td>46</td><td><b>38</b></td><td>29</td><td>37</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftMaxProba</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>36</td><td>36</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftLearntFreq</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td><b>42</b></td><td><b>36</b></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftLearntGridPattern</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td><b>42</b></td><td><b>35</b></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>

<table><caption>Table 4.3: Quality of the translation computed on ground truth. Cells are coloured in grey for methods based on statistical correspondences and in white for those based on semantics. Cells are kept empty for soft baselines when the target resolution is not coarser than the source. The $OA_{gt}$ is evaluated on the 2300 random points ground truth, while the $mF1_{gt}$ is computed on the 2700 points enriched with rare classes.</caption><thead><tr><th rowspan="2">Source<br>Target</th><th colspan="5">P</th><th colspan="5">C</th><th colspan="5">O</th><th colspan="5">OCS-GEc</th><th colspan="5">OCS-GEu</th><th colspan="5">MOS</th><th colspan="2">Total<br>average</th><th rowspan="2">Average<br>on soft</th></tr><tr><th>C</th><th>O</th><th>G1</th><th>G2</th><th>M</th><th>P</th><th>O</th><th>G1</th><th>G2</th><th>M</th><th>P</th><th>C</th><th>O</th><th>G1</th><th>G2</th><th>M</th><th>P</th><th>C</th><th>O</th><th>G1</th><th>G2</th><th>M</th><th>P</th><th>C</th><th>O</th><th>G1</th><th>G2</th><th>M</th><th>P</th><th>C</th><th>O</th><th>G1</th><th>G2</th><th>M</th><th>P</th><th>C</th><th>O</th><th>G1</th><th>G2</th></tr></thead><tbody><tr><td rowspan="5">OA<sub>gt</sub></td><td>HardSem</td><td></td><td></td><td></td><td></td><td></td><td>47</td><td><b>46</b></td><td>62</td><td>79</td><td><b>81</b></td><td><b>72</b></td><td>51</td><td>76</td><td>84</td><td><b>85</b></td><td>68</td><td>64</td><td><b>86</b></td><td><b>86</b></td><td><b>92</b></td><td>72</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftSem</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>HardStat</td><td></td><td></td><td></td><td></td><td></td><td><b>52</b></td><td>45</td><td><b>68</b></td><td><b>80</b></td><td><b>81</b></td><td>68</td><td><b>57</b></td><td><b>77</b></td><td><b>85</b></td><td><b>85</b></td><td>64</td><td>63</td><td><b>86</b></td><td><b>89</b></td><td>91</td><td>73</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftMaxProba</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftLearntFreq</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftLearntGridPattern</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="5">mF1<sub>gt</sub></td><td>HardSem</td><td></td><td></td><td></td><td></td><td></td><td>12</td><td><b>18</b></td><td>29</td><td>22</td><td><b>26</b></td><td><b>62</b></td><td><b>42</b></td><td><b>56</b></td><td><b>55</b></td><td><b>60</b></td><td>45</td><td>20</td><td><b>51</b></td><td>30</td><td>41</td><td>38</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftSem</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>HardStat</td><td></td><td></td><td></td><td></td><td></td><td><b>13</b></td><td><b>18</b></td><td>22</td><td>25</td><td><b>26</b></td><td>59</td><td>37</td><td>50</td><td>48</td><td><b>57</b></td><td>35</td><td>17</td><td>48</td><td><b>31</b></td><td>41</td><td>35</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftMaxProba</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftLearntFreq</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftLearntGridPattern</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>

Table 4.2 resumes the agreement metrics of the various methods, Figure 4.8 resume the per-class agreement metrics, while Table 4.3 resumes the accuracy on the ground truth. The agreement metrics assesses the agreement between the target land-cover map and the translated map giving a detailed insight into predicted class diversity. However, it is maximised only when the translation method learns to replicate errors. Conversely, the ground truth enables us to determine the robustness to label noise but is not well suited for per-class metrics as the ground truth is too small. Lastly, agreement is computed on the target map extent, i.e. can not be computed between two maps with no overlap (MOS, OCSGE-cover and use). Conversely, accuracy on ground truth can only be computed on a
wide extent to avoid a too small sample size. Thus the translation results on the ground
truth are only provided when the source map is CLC, OSO or CGLS-LC100.

<figure><img src="image_28.png" /><figcaption>Figure 4.8: Per-class F1 agreement computed on the sum of the translation confusion matrices of all the sources to one target. See Appendix A for class label.</figcaption></figure>

The first observation is that textbfHardSem, which is by far the most used in literature,
gives worse results than **HardStat**. This underlines that in the case of a source class
with multiple possible translations, the closest semantically is often not the most probable
statistically. This observation is of utmost importance as current nomenclature translation
methods mainly focus on defining semantic similarity between class definitions rather than
defining a probabilistic matching between all the source and target classes. It pledges for
more research on automatic statistical matching methods such as those presented in this
manuscript.

A second observation is that when target is coarser than source, the four baselines perform-
ing soft associations (**SoftSem**, **SoftMaxProba**, **SoftLearntFreq**, **SoftLearntGrid-
Pattern**) give better results than **HardSem** and **HardStat**. This behaviour advocates
for the need to perform the resolution and nomenclature translation jointly rather than
separately, as most methods do.

The **SoftSem** and **SoftMaxProba** baseline comparison reveals that the latter exhibits a
significantly higher *OA<sub>ag</sub>* and *OA<sub>gt</sub>* than the first while achieving almost the same *mF1<sub>ag</sub>*
and *mF1<sub>gt</sub>*. It appears that statistic-based methods are biased towards achieving the
highest possible overall accuracy while neglecting the diversity of translated classes (see
Figure 4.8). The methods proposed in this manuscript should thus focus on ensuring
target class diversity.

The last observation is that the **SoftLearntFreq** and **SoftLearntGridPattern** methods outperform the others significantly (+6% OA<sub>ag</sub> compared to HardStat). We link this behaviour to the fact that they learn the direct translation of a combination of source classes instead of combining individual class translations. We note that the knowledge of the local spatial arrangement also increases slightly all metrics.

4.2 Spatial Context

This section investigates the spatial context potential for translation. First, we propose
a simple yet efficient way to evaluate spatial context based on manually defined shape
features (area, elongation, compactness...) , e.g. is the water pixel in a linear shaped
object or not. Once determined we train a random forest to use this shape information
to translated differently pixels with the same class but exhibiting to different shape e.g.
the random forest learns that a water pixel in a linear shape object should be translated
into river while other should be translated into lakes. Secondly, we propose a more
complex approach replacing those manually defined features with automatically learnt
ones, hoping that it could increase the translation quality. The proposed approach relies on
a convolutional neural network method that jointly performs nomenclature and resolution
translation.

4.2.1 Manually defined shape indicators

4.2.1.1 Motivation

This section is built on the observation that two pixels with the same source class but
a different target class often belong to segments with very different shapes. Figure 4.9
illustrates this idea by displaying segments with the same MOS classes but a different CLC translation. For instance, two pixels belonging to two different MOS *Forest* segments should be translated into (i) CLC *Non-irrigated arable land* when the Forest exhibits a thin linear-shaped structure, *i.e.* is a hedgerow or in (ii) CLC *Broad-leaved forest* when the Forest exhibits a more circular shape structure with a significantly larger area. We argue that the knowledge of the shape of source segments is crucial for translation.

<figure><img src="image_29.png" /><figcaption>Figure 4.9: Illustration of the relevance of the shape information for translation. For each of the MOS classes, *Forest*, *Water*, *Collective housing* and *Mine/dump/construction*, two different objects with different CLC translations are provided. Shape analysis of those MOS objects appears relevant to determine which CLC class to assign.</figcaption></figure>

Section 2.3.1 identified two approaches for extracting shape information in the literature: manually defined shape features (MDSF) or machine-learned shape features (MLSF). As no works have focused on incorporating shape in the land-cover translation framework, this section studies MDSF as a simple tool to improve translation. An MLSF method dealing with the shape and class spatial correlation is explored separately in Section 4.2.2.

Manually defined shape features have partly been explored in Geographic-object based image analysis (GEOBIA). In land-cover mapping, GEOBIA segmentations are often obtained from automatic image analysis methods aiming to group pixels in homogenous segments. The automatic algorithms are parametrized to achieve a tradeoff between (a) ensuring that contiguous pixels belonging to different classes belong to different segments and (b) that pixels belonging to the same class belong to the same segment. For instance, all contiguous pixels of a *Forest* must belong to a single segment, and this segment must only include *Forest*. As most approaches favorites (a), an over-segmentation is observed in most applications, *e.g.* a single *Forest* is composed of multiple segments. Consequently, few experiments on the use of MDSF for land-cover mapping have been conducted as the used segmentation often poorly reflects the true shape. Conversely, in the land-cover translation scenario, obtaining the source segmentation is straightforward and directly reflects the geometric properties of the object to translate. As the few works conducted on using those MDSF for land-cover mapping focus principally on urban area change
detection, we lack a proper analysis of the best MDSF to use. The experimental setup
section, first proposes a set of common shape indicators obtained from the literature that
could provide helpful information for translation.

Additionally, shape information is rarely included in land-cover map nomenclature def-
initions to ensure that nothing is misclassified due to a peculiar shape. Establishing
rule-based semantic translation based on shape information is unfeasible. A machine
learning (ML) strategy is explored to learn to translate using both the source label and the
MDSF. The choice of the ML method is discussed to both respects the characteristics of
the MDSF and offer the possibility to obtain a general intuition on which shape elements
are important for land-cover translation.

4.2.1.2 Experimental protocol

**Manually defined shape features** Ten shape features, commonly used in the GIS community [18] and easily analysable, are experimented with (refer to Table 2.2). We succinctly introduce them below and underline examples of classes for which they are sufficient alone to improve translation significantly. Combining those MDSF can help identify more classes not mentioned below. **Area** is used to evaluate the size of the segments and is particularly informative when the source resolution is finer than the target *e.g.* a 100m² grassland is probably a garden while a 10⁶m² is probably a pasture or a natural grassland. **Elongation** is used to evaluate the "thickness" of a segment using the ratio between area and perimeter (close to 0 for elongated segments). It helps translating segments involving thickness constraints *e.g.* translating water in a river or lake, or with cartographic generalisation rules *e.g.* a thin forest is probably a hedgerow between crops classified as *arable crops* in CLC. **Circularity** compares the segment area to the area of a circle with the same perimeter (1 for a circle). Especially useful to translate circular irrigated crops. **NestedPoly** counts the number of the segment within the considered segment and is the only selected MDSF rarely found in the literature. It enables translating classes characterized by density constraints *e.g.* an urban area segment holding many other segments of different land-cover classes is less likely to be a dense urban area. **Convexity** is the ratio between the area and the convex hull area of a segment. It indirectly evaluates the shape complexity. It is mainly used in urban mapping to distinguish different building's usage. **MBRH**, **MBRW**, **MBRArea**, **MBRFlatness** Height, width, area and Elongation of the minimum bounding rectangle of the segment. Complementary to **Area**, it helps to evaluate the Height, Width disproportion and, indirectly, the global spread of the land cover. **MBRArea=Area** implies that the shape is rectangle *e.g.* pastures often exhibit rectangle patterns while natural grassland do not. **MBRAngle** gives a rough estimate of the principal orientation of the segment. Especially useful to translated land-cover classes correlated to sun position and topography *e.g.* hillsides vineyards, or wind constraints *e.g.* hedgerows planted to limit wind influence.

Figure 4.10 presents the correlation matrix between the different shape indicators of
CGLS-LC100, CLC and MOS. The diagonal delivers the auto-correlation of the indicators for one map. For instance, the left top cell presents the auto-correlation of CGLS-LC100 shape indicators. A first observation is that the shape indicators of one map tend to be highly correlated, especially the Area, **MBRH**, **MBRW**, **MBRArea** and **NestedPoly**. This is easily explainable as wide area polygons necessarily have a wide **MBRArea** (thus a wide **MBRH** and **MBRW**) and is more likely to have multiple inserted sub-polygons. Additionally, the correlation between two features of the same map is highly dependent on the resolution and nomenclature of the map. For instance, The MBRAngle is mostly positively correlated with the other features for CGLS, negatively for CLC, and almost unrelated to MOS.

Interestingly, inter-correlation between two map features exhibits the same sort of correla-
tion as those observed inside one map. In particular, we observe that those inter-correlation
coefficient are high (most of the time superior to 0.5), denoting that source map segmenta-
tion shape indicators provide information on target segmentation shape.

<figure><img src="image_30.png" /><figcaption>Figure 4.10: Spearman’s correlation coefficient between the shape indicators of CGLS, CLC and MOS. The diagonal of the matrix denotes the autocorrelation between the indicators of the map. The rest of the matrix gives the correlation between the shape indicators of different maps.</figcaption></figure>

Unfortunately, as MDSF computation is particularly time-consuming for high-resolved
maps as it requires both i) to have access to a vectorised version of the map and ii) to
compute the indices for each of them, experiments are only conducted for the three coarser
of the six maps: CGLS-LC100, CLC, and MOS.
**Machine Learning** The selected machine learning algorithm must be robust to highly correlated features. Moreover, the model's decision should be interpretable to understand important shape characteristics. The random forest algorithm exhibiting these two characteristics [97] appears a natural choice. Source labels are one-hot encoded and combined with the MDSF features to train the random forest algorithm. This method is termed **SpaShape**. The **HardStat** method is used as a control to evaluate the role of shape indicators since its results are identical to a random forest trained using the source labels as features.

**Implementation details:** As pixels belonging to different patches might belong to the same segments, the independence between training and testing can not be ensured using the train/test based on patch separation presented in Section 3.1. We process the raw vector version of the map. Adjacent segments with the same class are merged to limit arbitrary segmentation artefacts, and MDSF are computed. Segments are randomly split in train/test using the same ratio as in the MLULC dataset (65,35) and rasterised to the source resolution. The random forest is trained pixel-wisely on truly independent train segments while preserving the fact that two pixels of a given segment have the same feature but can have a different corresponding target class. The confusion matrix and afferent metrics computed on the test are corrected by sample inclusion probability to be comparable with the results of the baselines, i.e. the random split in train/test/validation at the segment level might introduce a slight shift in the respective proportion of each class.

The same grid search cross-validation strategy as for **SoftLearntFreq** is used to get the best parameters for the random forest with almost identical results: 300 trees, five minimum samples per leaf and a max depth of 25.

### 4.2.1.3 Results

**Quantitative aspect** Qualitative aspect does not differ much visually as target classes benefiting from MDSF-aware translation are mostly rare classes. Therefore, we only present quantitative results in this section. Table 4.4 compares the results with the two hard-association baselines. A first observation is that the effect of adding shape information widely differs depending on the considered translation. When translating from a coarse map (CLC or CGLS-LC100) to a highly resolved one (MOS), adding spatial context does not improve the translation significantly (+0.5% OA<sub>ag</sub> and mF1<sub>ag</sub>). This is widely understandable because the translation is ill-defined. Additional high-resolved data is mandatory to obtain good results. Moreover, as very large objects usually encompass various classes due to the minimum mapping unit, their shapes give limited information on their actual content. On the contrary, when the source resolution is finer or comparable to the targeted one, the overall agreement and agreement f1-score are improved by a vast margin (+4% OA<sub>ag</sub>, +6% mF1<sub>ag</sub>) compared to the best baseline, HardStat). We underline that those results are even better than those observed with the soft-translation methods.
<table><caption>Table 4.4: Agreement metrics obtained using shape indicators compared to baselines</caption><thead><tr><th colspan="2">Source</th><th colspan="2">CGLS (P)</th><th colspan="2">CLC (C)</th><th colspan="2">MOS (M)</th><th rowspan="2">Total average</th></tr><tr><th colspan="2">Target</th><th>C</th><th>M</th><th>P</th><th>M</th><th>P</th><th>C</th></tr></thead><tbody><tr><td rowspan="3">OA<sub>ag</sub></td><td>HardSem</td><td>52</td><td><strong>75</strong></td><td>65</td><td>79</td><td>77</td><td>72</td><td>70</td></tr><tr><td>HardStat</td><td>54</td><td><strong>75</strong></td><td>68</td><td>79</td><td>80</td><td>77</td><td>72</td></tr><tr><td>SpaShape</td><td><strong>57</strong></td><td><strong>75</strong></td><td><strong>70</strong></td><td><strong>80</strong></td><td><strong>84</strong></td><td><strong>83</strong></td><td><strong>75</strong></td></tr><tr><td rowspan="3">mF1<sub>ag</sub></td><td>HardSem</td><td>13</td><td>24</td><td>46</td><td><strong>42</strong></td><td>35</td><td>17</td><td>30</td></tr><tr><td>HardStat</td><td>13</td><td>24</td><td>47</td><td><strong>42</strong></td><td>31</td><td>15</td><td>29</td></tr><tr><td>SpaShape</td><td><strong>19</strong></td><td><strong>25</strong></td><td><strong>52</strong></td><td><strong>42</strong></td><td><strong>39</strong></td><td><strong>24</strong></td><td><strong>34</strong></td></tr></tbody></table>

**Feature Importance** We study the importance of each MDSF indicator to provide insight into shape elements impacting land cover translation. Feature importance is assessed using a random feature permutation approach which is more robust to the highly correlated features than the more traditional mean decrease in impurity [30]. A random forest is first trained on the training set. Then, an iterative comparison is conducted on the test set between the obtained $mF1_{ag}$ and the $mF1_{ag}$ obtained with one randomly shuffled feature. This strategy for assessing feature importance tends to underestimate the importance of correlated features [30]. Consequently, the importance of Area, MBRh and MBRW features is probably underestimated. Figure 4.11 presents the results of the feature importance evaluation when MOS is used as the source map, but the same observation can be made using other source maps. The main observation is that the most important features tend to be either those assessing the spatial extent of the objects (Area, MBRarea, NestedPoly) or the shape compactness (Elongation). It appears important to ensure that any proposed spatial context-aware methods can assess the size of the object they are translating.

<figure><img src="image_31.png" /><figcaption>Figure 4.11: Feature importance for the $mF1_{ag}$ metric for the translation from MOS to either CLC or CGLS evaluated using a random feature permutation technique. The source label feature is removed for readability due to its high importance score (0.55). Some features importance might be underevaluated due to high feature correlation (refer to Figure 4.10)</figcaption></figure>
**Discussion** Our experiments demonstrate that the use of simple MDSF significantly improved the translation results when the source resolution is finer than the target one (+4% OA<sub>ag</sub>, +6% mF1<sub>ag</sub>). Conversely, when the source resolution coarse the observed improvement is low (± 0.5%). We link this behaviour to the fact that MDSF usage relevance directly depends on the assumption that segment shape accurately reflects its semantic content. However, land-cover maps with coarse resolution often mix multiple land-cover inside a single segment, especially when the minimum mapping unit is high (CLC). This effect is further amplified by the fact that low-resolution land-cover maps often exhibit complex shaped segments, including holes (NestedPoly>1) for which MDSF provides unreliable information e.g. **Circularity** does not reflect accurately segment compactness if the shape has a big hole in it.

The experiment design, i.e. a limited number of manually defined shape indicators, probably leads to an underestimation of the shape information potential for translation. MLSF could partially alleviate this problem by automatically learning translation-tailored shape features instead of arbitrarily chosen ones.

## 4.2.2 Machine Learnt contextual features

### 4.2.2.1 Motivation

The previous section underlined that the shape of source segments might significantly improve the land-cover map translation. However, the method relies on manually defined shape features, which cannot precisely estimate all possible shapes. This section investigates how to learn translation-tailored contextual features automatically. Unlike the previous section, the proposed method also incorporate the previously neglected neighbouring spatial context e.g. sand is usually near the sea and wetlands near rivers. Inspired by the literature review conducted in Section 2.3.1, a Convolutional Neural Networks (CNN) approach is experimented as they are perfectly tailored to integrate the semantics of the pixel with its spatial context.

### 4.2.2.2 Method

**Resolution gap** Unlike land-cover map classification, which usually preserves the input-data spatial resolution, translation is frequently confronted with a difference between source and target resolution. A first simple strategy to deal with the resolution gap is to either (i) resample the source to the target resolution before network processing or (ii) resample the output to the target resolution. (i) is mainly used when the source resolution is coarser than the target, while (ii) is used when the source is finer resolved. Resolution and nomenclature being highly intertwined, it appears detrimental to process them separately, especially when translating from a fine resolved to a coarse target map (see Figure 1.5). This section primarily focuses on the case of translating a fine-resolved
map to a coarse target by adapting the network architecture to translate nomenclature and resolution simultaneously. The reverse translation from a coarse to a highly resolved map involves using additional data, complexifying the distinction between the spatial context importance and the additional data. We address this issue separately in Section 5.2.2.

**A downsampling network** Image to land-cover map classification is traditionally cast as a semantic segmentation task, in which a set of remote sensing images are transformed into a class map [136, 206, 236]. Similarly, we observe that land-cover map translation can also be seen as a semantic segmentation task where the input pixels are not physical values (namely, the optical spectral bands or SAR polarized channels), but semantic classes, i.e., nominal categorical data with low cardinality. Therefore we base our network on a popular semantic segmentation network, the U-Net [257] introduced in Section 2.3.1 and presented in Figure 2.3. The main idea of U-Net is to encode the image input into a vector representation using successive down-sampling and convolution steps and then restore (decode) the image using successive up-sampling and deconvolution layers. Skip connections between the encoder and the decoder convolutions are used to avoid losing spatial information during the MaxPooling process. The standard U-Net skip connections impose aggregating features of the same scale in the encoder and decoder sub-networks resulting in output with the same resolution as the input. We propose a simple U-Net adaptation, termed Asymmetrical U-Net **A-Unet**, to achieve the desired simultaneous nomenclature and resolution translation. Figure 4.12 presents its implementation for a resolution gap of a factor 10. It consists in (i) removing some of the skip connections and (ii) choosing different MaxPooling ratios for the down-sampling and the up-sampling parts. Let $r$ be the resizing factor between the input and the output of the network, and $D = (d_1, d_2, ..., d_h), d_i \in \mathbb{N}$ the downsizing factor of the different pooling layers in the encoder. We need to ensure that:

$$r = \prod_{i}^{h} d_{i}. \qquad (4.1)$$

Pooling parameters must be as small as possible to reduce the loss of spatial detail.

$$D = \underset{d_i}{\operatorname{argmin}} \prod_{i}^{h} d_i. \qquad (4.2)$$

This problem has a unique solution obtainable by prime decomposition [355]. In the OSO to CLC translation case, the target map is ten times smaller than the source map. This leads to apply a five and a two pooling layers in the asymmetrical part of the encoder. To avoid information loss, we apply the two pooling layers first. In the case where the source and target resolutions are identical, this network is equivalent to a U-Net. In the case where the source resolution is coarser than the target, the A-Unet first resample the source to the target resolution using the nearest neighbour operator and provide the data to a classical U-Net. We remind the reader that this problem is ill-defined without additional data and that this solution is only proposed to provide reference results for Section 5.2.2.
<figure><img src="image_32.png" /><figcaption>Figure 4.12: Our U-Net adaptation. An asymmetrical encoder-decoder is designed to handle the downsampling of fine-resolved sources to coarse-resolve targets efficiently.</figcaption></figure>

**Implementation details:** As we rely on the basic U-Net architecture, the only hyper-parameters to tune are the: optimizer, learning rate, batch size and the number of feature maps of the first convolution block (the number of feature maps is simply increased by a two factor at each block in the U-Net architecture). We relied on Adam optimizer [163] as it offers a quick convergence with high robustness to learning rate value. Multiple loss are tested by reducing the loss of factor 0.5 starting from 0.02 using an "on plateau strategy". Experiments on the number of feature maps and batch size are conducted on the OSO to CLC translation as it involves the most resolved source map with the most target classes. 8, 16, 32, 64 feature maps were tested (the original implementation used 64). We highlight that 32 feature maps were sufficient to reach a performance plateau both in terms of OA<sub>ag</sub> and mF1<sub>ag</sub>. Various batch sizes ranged from 12 to 192 with a 2 factor step. The results from batch size 12 to 120 are almost identical, with less than 1% difference for all translations and started to decrease slightly above this number.

**Loss** The standard loss for semantic segmentation tasks is cross-entropy (CE), defined as the sum of the target entropy and the relative Kullback-Leibler divergence between the target and the prediction. Let $n$ be the number of predicted elements, $c$ the number of classes, $p$ a softmax prediction of the network and $y$ the ground truth. $p^i(k)$ denote the predicted probability that the $i^{th}$ element belongs to class $k$. $y^i(k) = 1$ if the true class of the $i^{th}$ elements is $k$ and $y^i(k) = 0$ otherwise. The CE loss is computed as:

$$L_{CE}(p, y) = \frac{1}{n} \sum_{i=1}^{n} \sum_{k=1}^{c} y^i(k) \log(p^i(k)). \qquad (4.3)$$

The principal limitation of CE is that it does not suitably handle imbalanced classes.
As stated previously, our case study holds highly imbalanced data, which prevents its adoption.

Many approaches have been proposed [257, 263] to cope with the class imbalance issue, such as the weighted cross-entropy [124] or focal loss [181]. We review below three losses for which we present the results in the following section.

The focal loss is built on the idea that rare classes are challenging to predict and thus result in low $p^i(k)$ values. It proposes to weight the cross-entropy inversely to the $p^i(k)$ value. A manually chosen $\alpha$ factor gives more or less importance to those low prediction values (when $\alpha = 0$, the focal loss is equivalent to CE).

$$L_{focal}(p, y) = \frac{1}{n} \sum_{i=1}^{n} \sum_{k=1}^{c} y^i(k)(1 - p^i(k))^{\alpha} \log(p^i(k)). \quad (4.4)$$

Additionally, region-based losses target to maximize the overlapping ratio between $p$ and $y$. Among all these losses, we select the Dice loss [216]: it computes an approximation of the F1-score metric, which is vastly used in the remote sensing community. The Dice loss is computed as follows:

$$L_{DICE} = t - \sum_{k=1}^{c} \frac{2 \sum_{i=1}^{n} p^i(k)y^i(k)}{\sum_{i=1}^{n} p^i(k) + \sum_{i=1}^{n} y^i(k)}. \quad (4.5)$$

We combine the CE and the Dice loss, as suggested in [140, 161, 240, 328], to incorporate benefits from finer decision boundaries (Dice) and accurate data distribution (CE). This alleviates the problem of high variance of the Dice loss.

$$L_{CE+DICE} = L_{CE} + L_{DICE}. \quad (4.6)$$

**Preprocessing** Unlike image-to-land-cover classification, which takes continuous variables as input (pixel Digital Number), Land-cover map translation takes categorical variables (classes). Distances computed between two input values convey meaning on images, i.e. a 2 difference in radiometry is smaller than a 4, while it does not on raw land-cover, i.e. the 2 and 4 difference between a forest labeled "1" and water "3" or a forest and crops "5" can not be compared. The first pre-processing is to transform the input to ensure that distances between classes convey meaning. In this section, we assume no prior knowledge of interclass distances and thus consider that all pairs of classes should be equidistant. One-hot encoding is used to achieve this goal: each pixel is encoded as a vector with each dimension encoding for the presence (1) or the absence (0) of a given land-cover class. For instance in the previous examples Forest = (1, 0, 0, 0, 0), Water = (0, 0, 1, 0, 0) and Crops = (0, 0, 0, 0, 1), thus the euclidean distance $D(D(Forest, Water) = D(Forest, Crops) = D(Water, Crops) = 1$. Chapter 6 explores another encoding paradigm for labels in which inter-class dis-
tances better reflect semantic differences. We underline that even though this step is not mandatory, it increases the network learning speed (≈ 3 times).

**Comparative assessment UNetBili**, a method that processes the source map through a U-Net and resamples the predicted logits using bilinear resampling, is used as a comparative baseline to evaluate if the common nomenclature and resolution translation of **A-Unet** improve translation quality. **DeepLabV3** a DeepLabV3 architecture (see Section 2.3.1.2) with a ResNet50 backbone is used to evaluate if a deeper network achieving better results in most semantic segmentation tasks can improve translation results. Unlike **UNetBili**, the resampling problem is directly addressed by the Spatial Pyramidal Atrous module (see Figure 2.4).

### 4.2.2.3 Results

**Qualitative analysis** As the 26 possible France-wide translations can not be displayed, we selected a set of illustrative patches that enables discussing the strengths and weaknesses of the proposed solution. The translation results of **HardSem**, **HardStat** and **A-Unet** are presented in Figure 4.13.

The A-Unet spatial context-wise translation method obtains significantly different results than the baselines. As discussing all the qualitative improvements is impossible, we categorize those improvements (nomenclature or resolution based), illustrate them with some examples visible in Figure 4.13, and try to highlight the reasons for the observed improvement in order to enable a better understanding of the method strengths and weaknesses.

From a nomenclature point of view, the use of spatial context is beneficial for classes with:

* A characteristic shape, e.g. in the fourth row (OSO to CLC) the CLC class 124: *Air-ports* is partially predicted using the particular "crossing road" pattern observed in the OSO map.

* A characteristic pattern, e.g. mixed forest, dense urban (not illustrated in Figure but well visible in the France-wide result)

* An inside segment translation gradient, e.g. Forest is more likely to be denser in the middle than on the edges. Forests are translated as *Open Forest* on the edges and *Closed Forest* in the middle in row 2 and 3. This observation also concerns distinction such as CLC 111: *Continuous urban areas* and 112: *Discontinuous urban areas*.

* A spatial co-occurrence pattern, e.g. OCSGEc 221: *Herbaceous Formation* is translated in OSO 13: *Pastures* near forest and in 6: *Cereals* (the dominant crop type in France) far from it in row 6.
<figure><img src="image_33.png" /><figcaption>Figure 4.13: Translation results of the HardStat, HardSem and A-Unet method for various source/target couples</figcaption></figure>
From a resolution point of view, spatial context contributes to:

* enable slight super resolution based on spatial patterns, *e.g.* in row 8, individual OCSGEc 1111:*Built-up areas* are partially retrieved from coarse OCSGEu 235:*Secondary or tertiary production and residential usage* based on 411:*Road networks* density.

* learn the CLC 25 adjacent pixels minimum mapping unit (MMU), *e.g.* A-Unet partially succeeds in learning MMU in rows 4 and 9.

The method exhibits the following qualitative limitations:

* The MMU is only partially learnt, *e.g.* single isolated pixels artefacts are still occasionally visible in rows 4 and 9.

* Linear shape structures are badly predicted, *e.g.* in the 5th row OSO 4:*Road surfaces* are only partially translated to MOS 10:*Transports*. In particular, even though baselines perform slightly better than the *A-Unet*. We link this behaviour to the fact that U-Net architectures are known to perform poorly on linear segment prediction.

**Quantitative Improvement**
To better understand the potential of context-wise methods, Table 4.5 presents the method's quantitative results regarding the agreement between the translation and the target map. Those first results reveal an average +5% OA<sub>ag</sub> and mF1<sub>ag</sub> improvement between the best baseline **SoftLearntGridPattern** and the A-Unet. Interestingly, this comes at the cost of reducing the average EPI by 3%. We link these paradoxical results to a qualitative observation that can be made in Figure 4.13, the A-Unet translation significantly alters the geometric shape compared to the baselines. For instance, the previously mentioned translation gradient of the forest to CGLS *Closed Forest* or *Open Forest* based on pixels distance to forest border results in a "buffer-like" translation agreeing highly to the target CGLS while losing precise edges delimitation.

Table 4.6 presents the quantitative results of the method between the translation and the manually built ground truth. A fine analysis requires the distinction of two groups of maps. **Group A** includes experiments where the source and target exhibit a France-wide spatial extent (CLC, OSO and CGLS). Conversely, **group B** includes those where the target covers a smaller extent (OCSGEc, OCSGEu, and MOS).

**Group A** $OA_{ag}$ and $OA_{gt}$ are computed on the same extent (France-wide) and should be identical (±1to2%) depending on the map. $OA_{ag} > OA_{gt}$ implies that the method is learning to replicate target map errors, which increases $OA_{ag}$ and decreases $OA_{gt}$. Conversely, $OA_{gt} > OA_{ag}$ implies the correct translation of pixels that are erroneous in the target map. When the source or target is CGLS, the most erroneous map with 72% accuracy, the **A-Unet** $OA_{ag}$ is on average superior to $OA_{gt}$ from 4% underlying that the
network is prone to replicate target errors. In other cases, no significant differences are observed. Interestingly, **A-Unet** translations between Group A land-covers is in average 11% better in OA<sub>ag</sub> than SoftSem ones while only 5% higher in OA<sub>gt</sub>. As the SoftSem method is insensible to target noise, this underlines **A-Unet** lack of label-noise robustness.

Conversely, **Group B OA<sub>ag</sub> and OA<sub>gt</sub> are not computed on the same extent (target extent vs France-wide) and thus not comparable. However, they enable studying the spatial generalization ability of the framework. As the OA<sub>gt</sub> of baseline methods exhibit less than 1% difference with those obtained by the A-Unet, we conclude that the methods perform poorly on spatial generalization. Especially when translating one of the maps of group A into MOS, the obtained France-wide MOS is significantly better using a baselines instead of A-Unet. This can mainly be imputed to unseen spatial patterns. For instance, the original MOS spatial extent does not include sea areas corresponding to patches entirely covered by water; translating such patches can result in strange predictions such as forest).**

The comparison between **A-Unet** and **UNetBili** reveals that performing jointly resolution and nomenclature translation performs slightly better (average +1% OA<sub>ag</sub>, +2.5% mF1<sub>ag</sub>).

The comparison with the agreement metrics obtained for the six translations using the manually defined shape feature (see Table 4.4) demonstrates that using a simple manually defined features procedure is sufficient to achieve results close from those obtained with the A-Unet when translating MOS to the coarse CLC or CGLS maps (1.5% OA<sub>ag</sub> and 5.5% mF1<sub>gt</sub>).

Interestingly, the A-Unet and its 3 million trainable parameters give almost identical results in most cases to the DeeplabV3 network (see Figure 2.4) on a ResNet50 backbone (40 million) parameters. It even performs significantly better when the source or target map is MOS, as the training set is especially tiny (around 250 patches) due to the quick overfitting of the Deeplab framework. Additionally, as achieving A-Unet performance does not benefit from increasing the number of feature maps, we choose to mainly focus on adding more information or changing the training characteristics (loss, goal) rather than increasing the architecture depth and complexity.

<table><caption>Table 4.5: Agreement between translation results and targeted maps, for the two best baselines, UNetBili A UNet followed by a resampling layer, A DeepLabV3 architecture and the A-Unet</caption><thead><tr><th rowspan="2">Source<br>Target</th><th rowspan="2"></th><th colspan="4">P</th><th colspan="4">C</th><th colspan="4">O</th><th colspan="4">G1</th><th colspan="4">G2</th><th colspan="4">M</th><th colspan="4">Average</th></tr><tr><th>C</th><th>O</th><th>G1</th><th>G2</th><th>M</th><th>P</th><th>O</th><th>G1</th><th>G2</th><th>M</th><th>P</th><th>C</th><th>G1</th><th>G2</th><th>M</th><th>P</th><th>C</th><th>O</th><th>G1</th><th>P</th><th>C</th><th>O</th><th>G2</th><th>P</th><th>C</th><th>O</th><th>G1</th><th>P</th><th>C</th><th>O</th><th>G2</th></tr></thead><tbody><tr><td rowspan="5">OA<sub>ag</sub></td><td>SoftSem</td><td>52</td><td>42</td><td>56</td><td>70</td><td>75</td><td>65</td><td>49</td><td>67</td><td>77</td><td>79</td><td>62</td><td>59</td><td>69</td><td>76</td><td>82</td><td>56</td><td>41</td><td>34</td><td>87</td><td>57</td><td>40</td><td>31</td><td>75</td><td>80</td><td>76</td><td>59</td><td>62</td><td></td><td></td><td></td><td></td></tr><tr><td>SoftLearntGridPattern</td><td>54</td><td>44</td><td>65</td><td>70</td><td>75</td><td>68</td><td>55</td><td>71</td><td>78</td><td>79</td><td>72</td><td>62</td><td>73</td><td>80</td><td>82</td><td>63</td><td>47</td><td>49</td><td>89</td><td>62</td><td>41</td><td>41</td><td>78</td><td>84</td><td>82</td><td>62</td><td>66</td><td></td><td></td><td></td><td></td></tr><tr><td>UnetBili</td><td>61</td><td>53</td><td>67</td><td>76</td><td>75</td><td>73</td><td>57</td><td>71</td><td>79</td><td>78</td><td>75</td><td>66</td><td>79</td><td>86</td><td>84</td><td>66</td><td>52</td><td>56</td><td>92</td><td>67</td><td>48</td><td>49</td><td>79</td><td>84</td><td>82</td><td>61</td><td>70</td><td></td><td></td><td></td><td></td></tr><tr><td>DeepLabV3</td><td>62</td><td>55</td><td>68</td><td>78</td><td>74</td><td>73</td><td>58</td><td>72</td><td>80</td><td>78</td><td>77</td><td>68</td><td>79</td><td>86</td><td>83</td><td>68</td><td>54</td><td>56</td><td>92</td><td>68</td><td>49</td><td>49</td><td>79</td><td>84</td><td>82</td><td>60</td><td>70</td><td></td><td></td><td></td><td></td></tr><tr><td>A-Unet</td><td>62</td><td>55</td><td>68</td><td>78</td><td>76</td><td>73</td><td>58</td><td>72</td><td>80</td><td>79</td><td>77</td><td>68</td><td>79</td><td>86</td><td>85</td><td>68</td><td>54</td><td>56</td><td>92</td><td>68</td><td>49</td><td>49</td><td>79</td><td>84</td><td>82</td><td>63</td><td>71</td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="5">mF1<sub>ag</sub></td><td>SoftSem</td><td>13</td><td>17</td><td>22</td><td>15</td><td>24</td><td>46</td><td>32</td><td>36</td><td>31</td><td>42</td><td>38</td><td>19</td><td>36</td><td>20</td><td>38</td><td>27</td><td>10</td><td>17</td><td>27</td><td>20</td><td>8</td><td>8</td><td>29</td><td>38</td><td>19</td><td>19</td><td>25</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftLearntGridPattern</td><td>13</td><td>18</td><td>19</td><td>16</td><td>24</td><td>47</td><td>32</td><td>33</td><td>30</td><td>42</td><td>47</td><td>24</td><td>34</td><td>20</td><td>42</td><td>37</td><td>15</td><td>20</td><td>27</td><td>28</td><td>12</td><td>10</td><td>27</td><td>43</td><td>25</td><td>18</td><td>27</td><td></td><td></td><td></td><td></td></tr><tr><td>UnetBili</td><td>26</td><td>25</td><td>26</td><td>18</td><td>31</td><td>56</td><td>34</td><td>35</td><td>24</td><td>41</td><td>55</td><td>34</td><td>45</td><td>27</td><td>51</td><td>44</td><td>21</td><td>29</td><td>44</td><td>37</td><td>16</td><td>22</td><td>40</td><td>45</td><td>27</td><td>20</td><td>35</td><td></td><td></td><td></td><td></td></tr><tr><td>DeepLabV3</td><td>26</td><td>26</td><td>27</td><td>19</td><td>29</td><td>53</td><td>36</td><td>35</td><td>30</td><td>40</td><td>60</td><td>37</td><td>45</td><td>27</td><td>50</td><td>48</td><td>24</td><td>29</td><td>44</td><td>38</td><td>19</td><td>22</td><td>40</td><td>45</td><td>26</td><td>18</td><td>34</td><td></td><td></td><td></td><td></td></tr><tr><td>A-Unet</td><td>26</td><td>28</td><td>27</td><td>19</td><td>32</td><td>56</td><td>36</td><td>35</td><td>30</td><td>41</td><td>60</td><td>37</td><td>44</td><td>27</td><td>53</td><td>48</td><td>24</td><td>29</td><td>44</td><td>40</td><td>20</td><td>22</td><td>40</td><td>46</td><td>29</td><td>21</td><td>35</td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="5">EPI</td><td>SoftSem</td><td>28</td><td>5</td><td>6</td><td>5</td><td>17</td><td>30</td><td>6</td><td>6</td><td>7</td><td>20</td><td>36</td><td>36</td><td>28</td><td>32</td><td>58</td><td>28</td><td>32</td><td>29</td><td>79</td><td>32</td><td>36</td><td>29</td><td>77</td><td>54</td><td>54</td><td>38</td><td>31</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SoftLearntGridPattern</td><td>27</td><td>5</td><td>3</td><td>5</td><td>16</td><td>30</td><td>6</td><td>5</td><td>7</td><td>20</td><td>42</td><td>35</td><td>25</td><td>31</td><td>57</td><td>35</td><td>31</td><td>30</td><td>80</td><td>35</td><td>32</td><td>30</td><td>77</td><td>57</td><td>55</td><td>38</td><td>31</td><td></td><td></td><td></td><td></td></tr><tr><td>UnetBili</td><td>22</td><td>4</td><td>3</td><td>3</td><td>11</td><td>27</td><td>5</td><td>4</td><td>5</td><td>17</td><td>39</td><td>35</td><td>25</td><td>28</td><td>49</td><td>30</td><td>28</td><td>24</td><td>77</td><td>28</td><td>30</td><td>21</td><td>70</td><td>50</td><td>54</td><td>28</td><td>28</td><td></td><td></td><td></td><td></td></tr><tr><td>DeepLabV3</td><td>22</td><td>4</td><td>3</td><td>4</td><td>9</td><td>27</td><td>6</td><td>5</td><td>6</td><td>15</td><td>39</td><td>36</td><td>25</td><td>28</td><td>46</td><td>31</td><td>29</td><td>24</td><td>77</td><td>29</td><td>30</td><td>21</td><td>70</td><td>48</td><td>53</td><td>17</td><td>28</td><td></td><td></td><td></td><td></td></tr><tr><td>A-Unet</td><td>22</td><td>4</td><td>3</td><td>4</td><td>12</td><td>27</td><td>6</td><td>5</td><td>6</td><td>17</td><td>39</td><td>36</td><td>25</td><td>28</td><td>49</td><td>31</td><td>29</td><td>24</td><td>77</td><td>29</td><td>30</td><td>21</td><td>70</td><td>50</td><td>55</td><td>29</td><td>28</td><td></td><td></td><td></td><td></td></tr></tbody></table>
<table><caption>Table 4.6: Comparison between translation results and ground-truth for the two best baselines, UNetBili (a UNet followed by a resampling layer), A DeepLabV3 architecture and the A-Unet. Unlike the agreement measure, which is computed on the target original spatial extent, this measurement is computed France-widely.</caption><thead><tr><th rowspan="2">Source<br>Target</th><th rowspan="2"></th><th colspan="5">P</th><th colspan="5">C</th><th colspan="5">O</th><th rowspan="2">Average</th></tr><tr><th>C</th><th>O</th><th>G1</th><th>G2</th><th>M</th><th>P</th><th>O</th><th>G1</th><th>G2</th><th>M</th><th>P</th><th>C</th><th>G1</th><th>G2</th><th>M</th></tr></thead><tbody><tr><td rowspan="5">OA<sub>gt</sub></td><td>SoftSem</td><td>47</td><td>46</td><td>62</td><td>79</td><td><strong>81</strong></td><td><strong>72</strong></td><td>51</td><td>76</td><td>84</td><td><strong>85</strong></td><td>71</td><td>65</td><td><strong>86</strong></td><td>86</td><td><strong>92</strong></td><td>72</td></tr><tr><td>SoftLearntGridPattern</td><td>52</td><td>45</td><td>68</td><td>80</td><td><strong>81</strong></td><td>68</td><td>57</td><td>77</td><td>85</td><td><strong>85</strong></td><td>73</td><td>68</td><td><strong>86</strong></td><td>89</td><td><strong>92</strong></td><td><strong>74</strong></td></tr><tr><td>UnetBili</td><td><strong>57</strong></td><td>50</td><td>69</td><td>81</td><td>76</td><td>69</td><td><strong>58</strong></td><td>77</td><td><strong>86</strong></td><td>76</td><td><strong>74</strong></td><td>69</td><td><strong>86</strong></td><td><strong>91</strong></td><td>87</td><td><strong>74</strong></td></tr><tr><td>DeepLabV3</td><td><strong>57</strong></td><td><strong>51</strong></td><td><strong>70</strong></td><td><strong>82</strong></td><td>73</td><td>69</td><td><strong>58</strong></td><td><strong>78</strong></td><td><strong>86</strong></td><td>74</td><td><strong>74</strong></td><td><strong>70</strong></td><td><strong>86</strong></td><td><strong>91</strong></td><td>82</td><td>73</td></tr><tr><td>A-Unet</td><td><strong>57</strong></td><td><strong>51</strong></td><td><strong>70</strong></td><td><strong>82</strong></td><td>76</td><td>69</td><td><strong>58</strong></td><td><strong>78</strong></td><td><strong>86</strong></td><td>78</td><td><strong>74</strong></td><td><strong>70</strong></td><td><strong>86</strong></td><td><strong>91</strong></td><td>87</td><td><strong>74</strong></td></tr><tr><td rowspan="5">mF1<sub>ag</sub></td><td>SoftSem</td><td>12</td><td>18</td><td>29</td><td>22</td><td><strong>26</strong></td><td><strong>62</strong></td><td><strong>42</strong></td><td><strong>56</strong></td><td><strong>55</strong></td><td><strong>60</strong></td><td>47</td><td>22</td><td>51</td><td>30</td><td><strong>43</strong></td><td><strong>38</strong></td></tr><tr><td>SoftLearntGridPattern</td><td>13</td><td>18</td><td>22</td><td>25</td><td><strong>26</strong></td><td>59</td><td>37</td><td>50</td><td>48</td><td>57</td><td>41</td><td>28</td><td>48</td><td>31</td><td>41</td><td>36</td></tr><tr><td>UnetBili</td><td><strong>21</strong></td><td>22</td><td>30</td><td><strong>26</strong></td><td>22</td><td>51</td><td>36</td><td>44</td><td>42</td><td>38</td><td>52</td><td><strong>39</strong></td><td><strong>53</strong></td><td><strong>34</strong></td><td>37</td><td>37</td></tr><tr><td>DeepLabV3</td><td><strong>21</strong></td><td><strong>24</strong></td><td><strong>31</strong></td><td><strong>26</strong></td><td>17</td><td>51</td><td>36</td><td>45</td><td>42</td><td>30</td><td>53</td><td>39</td><td><strong>53</strong></td><td><strong>34</strong></td><td>32</td><td>37</td></tr><tr><td>A-Unet</td><td><strong>21</strong></td><td><strong>24</strong></td><td><strong>31</strong></td><td><strong>26</strong></td><td>22</td><td>51</td><td>36</td><td>45</td><td>42</td><td>38</td><td><strong>53</strong></td><td><strong>39</strong></td><td><strong>53</strong></td><td><strong>34</strong></td><td>39</td><td>37</td></tr></tbody></table>

We compared the different loss functions on the OSO to CLC translation, which we considered the most interesting variety (44 labels in CLC) of predictable classes (OSO is highly resolved with 23 classes). Table 4.7 resumes the *mF1<sub>ag</sub>* and *OA<sub>ag</sub>* of the different loss functions introduced earlier while Figure 4.14 presents the F1-score per class. Our observation is that CE obtains the highest *OA<sub>ag</sub>* but the lowest *mF1<sub>ag</sub>*. All our attempts to increase the *mF1<sub>ag</sub>* score worsens the *OA<sub>ag</sub>* metrics; a compromise must be made between having a higher *OA<sub>ag</sub>*/*mF1<sub>ag</sub>*. A simple rule of thumb to compare the different losses is to use the CE as a reference for computing the ratio between the improvement of *mF1<sub>ag</sub>* and the decrease of *OA<sub>ag</sub>*. For instance, the focal loss with $\alpha=2$ increased the mF1 from 2% and decreased the accuracy of 1% resulting in a 2/1 ratio. With this strategy, we conclude that the combined CE and Dice loss offers the best compromise (5/2 ratio).

<table><caption>Table 4.7: OSO to CLC translation *OA<sub>ag</sub>* and *mF1<sub>ag</sub>* for different loss functions. The focal loss is computed with different alpha values.</caption><thead><tr><th></th><th>CE</th><th>focal α=0.5</th><th>focal α=1</th><th>focal α=2</th><th>focal α=5</th><th>Dice</th><th>CE+Dice</th></tr></thead><tbody><tr><td>OA<sub>ag</sub></td><td>68</td><td>68</td><td>67</td><td>67</td><td>65</td><td>65</td><td>66</td></tr><tr><td>mF1<sub>ag</sub></td><td>37</td><td>37</td><td>38</td><td>39</td><td>40</td><td>40</td><td>42</td></tr></tbody></table>

<figure><img src="image_34.png" /><figcaption>Figure 4.14: *mF1<sub>ag</sub>* computed between CLC translated from OSO and the original CLC map for various loss functions.</figcaption></figure>
4.2.2.4 Conclusion

This section presented experiments on how to consider the spatial context for translation. We separated our analysis into two parts, the first one studying the potential of manually defined shape features to help the translation. Although significant improvements are observed compared to the baselines, we highlighted that using manually defined features instead of learnt features might be detrimental to the quality of translations. We secondly proposed a deep learning-based method that jointly analyses the shape of the object a pixel belongs to and the classes of neighbouring pixels. We obtained significant improvement in most cases. However, we underlined that the current method lacks spatial generalization compared to Semantic baselines and is sensitive to target noise. Those problems is addressed in the next sections.

4.3 Geographic context

Section 3.2 observed that land-cover classes are heterogeneously distributed over France for
various anthropic and bio-physical reasons. The spatial heterogeneity is further increased
by the uneven distribution of errors on the territory resulting from annotation by different
human operators depending on the localisation or local fine-tuning of machine learning
models.

We argue that information about wide-scale spatial heterogeneity, which we refer to as *geographic context*, is valuable for translation as information such as *"this pixel is in a mountainous area"* drastically influences the probable translation *e.g.* a forest is probably composed of coniferous stands. This spatial heterogeneity could theoretically be taken into account directly by the network if all of France were processed as a single input. However, the patch-based approach used to circumvent memory limitation prevents it. Answering how to incorporate geographic context is mandatory.

Section 4.3.1 focuses on identifying potential sources of geographic context suitable for translation. Section 4.3.2, proposes a methodology to incorporate one of these potential sources in the A-Unet architecture.

4.3.1 Determining a source of geographic context

To simplify our analysis, the experiments conducted in this section only focus on the
CGLS to CLC translation. This choice is made based on the observation that this small
to high number of classes is a France-wide challenging translation for which the impact
of geographic context should be especially visible. Therefore it simplifies comparison
between methods. The impact of geographical context on other translations is detailed in
section 4.3.1.
4.3.1.1 Motivation

Section 2.3.2 points out that geographical context is difficult to apprehend as many
elements influence class distribution (topography, climate, anthropogenic factors, pedology,
error patterns ...). Additionally, abstract notions such as distance thresholds can help
translation, e.g. inland wetlands are far from the seashore while maritime wetlands are
not. Designing a geographical context-aware method using the vast amount of multimodal
data sources needed to learn such notions is challenging.

State-of-the-art techniques rely on stratified training in which a separate classifier is trained for areas with different geographical context [139]. This alleviates the need to incorporate multiple data sources in training but assumes prior knowledge of homogenous geographical context areas. For instance, the OSO map is generated by training independent classifiers on eight different ecoclimatic areas (see Figure 3.3) based on climate variables by [154]. An implementation, termed **HardStatEcoCli**, is presented in the following section.

From a theoretical aspect, stratified training introduces the idea that each area is completely
independent. For instance, if forests in an area are always composed of coniferous stands,
then translating forest to broad-leaved or water are comparable errors. Indeed resulting
models only use the information of their respective areas and thus do not have access to
more general knowledge. This results in sharp edges between two different areas, e.g. all
forests are translated into conifer in area 1 and broad-leaved in area 2. As geographic
context can be a more progressive gradient, we propose directly feeding a unique model
with using the area identifier as an input feature. An implementation of this method
termed **LearntClassEcoCli** is presented in the following section.

The core limitation of the two previous approaches is that they often focus on a single
source of geographical context, in the previous example, eco-climatic areas, neglecting all
the other sources. This drastically constrains the potential of geographical context to the
quality of the defined areas. In Section 2.3.2 we observe that land-cover may co-variate
with its spatial coordinates (latitudes and longitudes). Close-by coordinates generally
exhibit similar topography, climate and pedology. Conversely, those geographical variables
are highly correlated to the coordinates. For instance, in France, the temperature tends to
be higher for low latitude values while precipitation tends to be lower, the wind is more
pronounced on the West side of the country due to the proximity to the seashore, and
mountainous areas are only observed at specific latitude and longitude. One could directly
use the spatial coordinates as a proxy to assess geographical context. However, unlike the
eco-climatic areas, the link between the geographical context and the coordinates is not
easy to analyse. We can not easily define a stratified approach in which different models
are trained depending on the coordinates as we do not have direct knowledge of the links
between coordinates and geographical context. Therefore using a machine-learning strategy
to use geographical coordinates as features seems appropriate. A simple illustration is
that once trained to learn the main characteristic of the land-cover map of a country, one
can realise an educated guess on the nature of an element in an area only by knowing its
coordinates. For instance, if one is asked to guess what is the land-cover map at 48.8566° N, 2.3522° E, he would probably answer *buildings*, provided he previously learnt that some close-by coordinates belong to Paris. An implementation of this method termed **LearntClassLatLon** is presented in the following section. As the Stratified approach is the only common strategy in land-cover mapping literature, a comparison with the two other methods has to be conducted.

### 4.3.1.2 Experimental protocol

**HardStat** introduced in Section 4.1.2 assigns to each CGLS class the most frequently observed CLC correspondent on the training set, *i.e.* if CGLS *Closed forest* pixels are observed at the same place as CLC *Broad-leaved forest* pixels 80% of the time on the training set, then *"Broad-leaved forest"* is the most probable translation for *Closed forest*. We underline that **HardStat** method gives identical results as a random trained to translate using only one feature (the class of the considered pixel). **HardStat** is used as a control to evaluate the gain or benefits of the other methods.

**HardStatEcoCli** is very similar to **HardStat**. However, instead of assigning to each CGLS class the most frequently observed CLC correspondent on the whole training set, it assigns to each CGLS class the most frequently observed CLC correspondent in the considered ecoclimatic zone. For instance, if *Closed forest* pixels correspond at 90% to *Broad-leaved forest* in an ecoclimatic area A and at 70% to *Coniferous forest* in an ecoclimatic area B, the translation of *Closed forest* is different depending if the considered pixels is in area A or B. As predefined ecoclimatic areas are required, we rely on the same ecoclimatic map as OSO [154]. This method is used as a comparison tool to assess if learning to analyse the geographical context from features is better than using those features to perform stratified learning.

**LearntClassEcoCli** proposes to use the directly ecoclimatic information as a feature used in a machine learning model. A random forest model predicts the CLC class of a pixel based on the CGLS class and the ecoclimatic zone it belongs to. This approach offers the possibility to avoid training multiple independent models as for **HardStatEcoCli**.

textbf{LearntClassLatLon} is based on the same idea as **LearntClassEcoCli** but processes the geographical coordinates directly instead of the ecoclimatic zone as a feature. A random forest is trained to predict the CLC class based on the CGLS class and the latitude and longitude. Three sub-methods are also proposed to study the influence of the coordinates alone. **LearntLatLon** only use the coordinates with no information of the CGLS class. **LearntClassLat**, only have access to CGLS class and the latitude. **LearntClassLon** only have access to the CGLS class and the longitude.

**Implementation details:** Class used as input features for **LearntClassEcoCli** and **LearntClassLatLon**, are one-hot encoded. Similarly the ecoclimatic areas used as input features of **LearntClassEcoCli** are also one-hot encoded. We underline that those methods are trained on the training patch of the MLULC dataset after applying an
exclusion buffer on the patch's edges to limit the risk that the use of coordinates breaks the independence between the train and test. An arbitrary 2.5km buffer is used based on the observation that the 25ha and 100m width CLC MMU does not guarantee independence below, i.e. each patch being 6×6 km wide only a 1 times 1km area in the centre of the patch is used for training. Additionally, instead of using the exact coordinates of the considered pixel, each pixel is annotated with the pixel coordinates in the centre of the patch it belongs to.

### 4.3.1.3 Results

<table><caption>Table 4.8: Agreement measurement computed between CLC and the translation result of various geographic context aware methods</caption><thead><tr><th></th><th>HardStat</th><th>HardStatEcoCli</th><th>LeanrtEcoCli</th><th>LearntLatLon</th><th>LearntClassLat</th><th>LearntClassLon</th><th>LearntClassLatLong</th></tr></thead><tbody><tr><th>OA<sub>ag</sub></th><td>54</td><td>55</td><td>55</td><td>49</td><td>55</td><td>54</td><td>61</td></tr><tr><th>mF1<sub>ag</sub></th><td>13</td><td>16</td><td>17</td><td>16</td><td>18</td><td>17</td><td>28</td></tr></tbody></table>

Table 4.8 presents the results of each of those methods. First, we observe that geographical context-aware method obtains better results than **HardStat**. Note that the *mF1<sub>ag</sub>* remains unsurprisingly low as the translation from CGLS to CLC involves translating 12 classes into 44. Secondly, **LearntLatLon** perform globally worse than **HardStat** (-6% OA<sub>ag</sub>) while its *mF1<sub>ag</sub>* score is higher (+3%). This underlines that a higher class diversity can be obtained using a pure coordinate analysis rather than the CGLS classes. More interestingly, the results demonstrate that the simultaneous use of coordinates and the CGLS class (LearntClassLatLon) can significantly improve the translation into CLC compared to the **HardStatEcocli** and **LearntClassEcoCli** baselines. Lastly, by comparing those results with those presented in the spatial context section, we observed that the obtained OA<sub>ag</sub> and *mF1<sub>ag</sub>* for the **LearntLatLon** are identical between the **A-Unet** and this random forest with CGLS labels and coordinates. This is particularly interesting as it suggests that for this specific translation, the geographical context might have the same order of importance as the spatial context.

To better understand how the geographical context can improve the results, we provide the per-class F1-score histogram for the different methods in Figure 4.15. A first observation is that **HardStatEcocli** and **LearntClassEcoCli** give almost identical results except for the CLC *Bare rock* and *Water bodies*, for which the **LearntClassEcoCli** get slightly higher results. This underlines that learning to interpret geographical context, i.e., how to analyse ecoclimatic areas, might slightly improve translation compared to a stratification into multiple independent models. The second observation is that the LearntLatLon, which, unlike other models, has no access to the CGLS labels, outputs a significantly more expansive number of classes than **HardStat**, **HardStatEcocli** and **LearntClassEcoCli**. For instance, it predicts partially classes such as *Rice Fields*, which are difficult to translate using only CGLS labels or Climate based indicators. This is explained by the fact that
those classes are spatially constrained to a limited area, i.e. *Rice crops* are only observed in a particular region. Lastly, **LearntClassLatLon** widely outperforms any other methods.

<figure><img src="image_35.png" /><figcaption>Figure 4.15: Per class F1-score of the translation into CLC for different geographical context methods. For visibility, only the 26/44 CLC class (non-zero f1-score) are displayed.</figcaption></figure>

Figure 4.16 compares the *OA<sub>ag</sub>* of the geographical context methods with the *OA<sub>ag</sub>* of the **HardStat** method by summarising results on 30×30 km grid. Blue (reciprocally red) pixels denote the areas where the geographical context method obtains better (reciprocally worse) results than the **HardStat** method. As the aspect of the HardStatEcoCli and LearntClassEcoCli maps are identical, we only provide one for readability. The first observation is that adding the geographical context does not improve the translation homogeneously across France. Most of the improvements are observed in the Southern part of France, mainly in the mountainous areas and the Mediterranean coastline (which exhibits a specific climate). LearntClassLatLon improved the results in more areas. For instance, blue parts observed on the west coast are due to the successful translation from CGLS class *Closed Forest*" to CLC class *Coniferous forest* of a vast coniferous forest area, which is the only one of this size outside a mountainous area inside the French territory (otherwise it is translated into "Broad-leaved forest").

<figure><img src="image_36.png" /><figcaption>Figure 4.16: Spatial distribution of translation improvement using various techniques compared to the **HardStat** method. Blue areas denote the location where the considered method outperformed the **HardStat** method, and red areas give the opposite information.</figcaption></figure>
### 4.3.1.4 Conclusion

We conclude that the best feature to assess geographical context is the geographical coordinates as it substantially improves the stratified ecoclimatic training and does not require incorporating multi-modal data. In particular, our experiments underline that both latitude and longitude are needed to get a refined analysis of the geographical context. The achieved translation improvement is not distributed homogeneously across the territory.

Moreover, in the case of the CGLS to CLC translation, the improvement of using coordinates is equivalent to the one fostered by using the spatial context with A-Unet. Therefore developing ways to incorporate geographical context into the A-Unet appears important.

## 4.3.2 Incorporating geographical coordinates into a convolutional neural network

### 4.3.2.1 Motivation

Section 2.3.2 underlined that little work had been conducted on incorporating geographical coordinates in machine learning frameworks, especially on convolutional neural networks.

The few works addressing this issue underlined the difficulty of using GPS coordinates directly as the input of a neural network [348]. However, they did not formulate hypotheses on the underlying reasons and simply observed that very fine location indicators such as GPS coordinates are difficult to analyse for the classifier [294].

In natural language processing, the position of words in a sentence is often used to improve the translation. These word positioning problems share with geographical coordinates the idea that training and testing values might differ, i.e. No sentence in the training data is 24 words long, but some sentences of the testing data are. To tackle this problem, a positional encoding strategy was proposed by [318] (See Section 2.3.2). It transforms the coordinates using a fixed set of size 2d composed of cosine and sine functions with different frequencies. Let x be the considered position in an input sentence, $\vec{p}_x \in \mathbb{R}^d$ be its corresponding encoding, and d be the encoding dimension. Then $f: \mathbb{N} \rightarrow \mathbb{R}^d$ is the function that produces the output vector $\vec{p}_x$ and is defined as shown in Equation 2.9. The attracting aspect of this transformation is that it ensures that all positions have a unique encoding while ensuring that this encoding value always remains in a constant range between -1 and 1. Moreover, close by values remain close by in the encoding; for instance, the cosine similarity between the encoding of position one and position 2 is higher than between position 1 and 9. This is interesting as it might help the network to find relations of proximity between objects which we previously underlined as important for land-cover map translation.

Section 2.3.2 pointed out that positional encoding is also used in computer vision to break the translation invariance of convolution by encoding pixel location e.g. to distinguish the
upper left pixel of an image from the right bottom one. However, no guarantee exists on
the efficiency of positional encoding for geographical coordinates as the problem exhibits
widely different characteristics. First, the proposed approach should not break pixel
translation invariance as geographical context on 6k×6 km patches is homogeneous, e.g. a
forest in the upper part of a patch should be treated the same way as one in the bottom.
Consequently, all pixels of the same patch should have the same positional encoding.
Instead, the approach aims to break the patch invariance to translation, i.e. the same
patch in the North and South of France should be translated differently. Secondly unlike
pixel coordinates which are the same for training and testing patches, i.e. the first pixel
is a notion existing both for training and testing, geographical coordinates are different
between training and testing. It requires a generalisation ability that is not required for
pixel positional encoding. We present in the following sections the experiments conducted
to evaluate the potential of positional encoding on geographical coordinates.

4.3.2.2 Methods

Figure 4.17 resumes the different experimented methods concisely. All the presented methods only have access to the coordinates of the patch’s centre pixel instead of providing each pixel’s geographical coordinates. This directly stems from the observation that geographic context is defined as wide-scale context, i.e. is the same on a 6×6 km² patch, and that using per pixel location would break the translation invariance as discussed previously.

<figure><img src="image_37.png" /><figcaption>Figure 4.17: Presentation of four different methods (purple boxes) to add geographical coordinates to A-Unet. Each method is used independently.</figcaption></figure>

As a simple reference baseline, we propose to concatenate to the input map two new
dimensions, one with the latitude and one with the longitude value. This simple baseline
is referred to as the **ConcatCoord** method.

As a more evolved solution, we propose feeding the coordinates to a small multi-layer
perceptron to learn a coordinate embedding that can be added to the network. **MLPCoord** adds the output of the MLP to the bottleneck of A-Unet so that the positional information only influences a coarse scale using simple addition. **MLPCoordFilter** is identical to [49], one of the only approaches found in the literature using geographical coordinates. It adds the output of the MLP at the end of the network.

Lastly, we propose **MLPPosEnc** which use the same strategy as **MLPCoord** method, but using the positional encoding before the MLP discussed earlier. Unlike the traditional sequence-to-sequence architecture, the positional encoding must be in 2D to integrate both latitude and longitude. The authors of [235] proposed a strategy for image coordinates: rows and columns are independently encoded and then concatenated. We adopt the same strategy with the latitude and longitude coordinates. Let x and y be the longitude and latitude. $p_{x,y}$ is the corresponding positionally encoded matrix. d is the dimension of the encoded matrix, corresponding to the number of feature maps generated by the CNN layer where the positional encoding is added.

$$ p_x = \begin{bmatrix} \sin(x\omega_1) \\ \cos(x\omega_1) \\ \vdots \\ \sin(x\omega_{d/4}) \\ \cos(x\omega_{d/4}) \end{bmatrix}_{d/2} \quad p_y = \begin{bmatrix} \sin(y\omega_1) \\ \cos(y\omega_1) \\ \vdots \\ \sin(y\omega_{d/4}) \\ \cos(y\omega_{d/4}) \end{bmatrix}_{d/2} \quad p_{x,y} = \begin{bmatrix} \sin(x\omega_1) \\ \cos(x\omega_1) \\ \vdots \\ \sin(y\omega_{d/4}) \\ \cos(y\omega_{d/4}) \end{bmatrix}_{d} \quad \text{with } \omega_i = \frac{1}{10000^{2i/d}} \tag{4.7} $$

The resulting encoding $p_{x,y}$ is given to a one layer perceptron to preprocess the positional encoding. Adding more layers did not show significant improvement in our results.

### 4.3.2.3 Results

Table 4.9 presents the agreement measurements for the four different geographical coordinates encoding methods. We underline that for processing times concerns, we did not train each of the 26 models a dozen times to obtain a standard deviation estimation of each approach and thus can not conclude on the statistical significance of each value of the table. However from experience, we estimate the $OA_{gt}$ and $mF1_{gt}$ variations between training to be between ±0.3% for $OA_{gt}$ and ±1% for $mF1_{gt}$. We additionally provide in Figure 4.19 the per-class metrics of the OSO to CLC translations with error bars indicating the standard deviation over 10 independent training to give an insight into the training variability.

A first observation is that the **ConcatCoord** obtains slightly worse results than not using coordinates. This is directly linked to the fact that it enforces learning maps and coordinates features with the same first convolution layer despite being unrelated.

Interestingly, the **MLPCoord** approach obtains almost identical results as using the A-Unet without coordinates confirming that using geographical coordinates directly is
not efficient as explained in other studies [294]. Even though the **MLPCoordFilter** proposed by [49], slightly improve the results, the proposed **MLPPosEnc** achieve a more significant improvement (+1% OA<sub>ag</sub> and +3% mF1<sub>gt</sub>) encompassing wide per translation differences from -2% to +8% OA<sub>ag</sub> and -3% to +12% mF1<sub>ag</sub>. Explaining the reason for each improvement/non-improvement for each translation is unfeasible as it depends on the source and target nomenclature complementarity, but some general rules are observed.

In general, the biggest improvements are observed when translating OCSGE cover and use into CGLS, CLC and OSO. This can widely be imputed to the peculiar spatial extent of OCSGE (see Figure 3.4), i.e. divided into two areas, one in the North and one in the South. The North and South parts belong to widely different ecoclimatic areas conditioning many translations. For instance, OCSGE use **Forestry** mainly concerns Broad-leaved trees in the North and Coniferous in the South, and OCSGE cover **Herbaceous vegetation** mainly includes pastures in the North and croplands in the South. Geographical context incorporation help to translate each part of the map independently. Even though this difference in geographic context also helps to translate CGLS, CLC and OSO into one another, their France wide-extent partially reduces the quantitative impact of this improvement. For instance, trees in the south part of OCSGE account for approximately 10-15% of CGLS, CLC and OSO trees (France wide maps) but for more than 50% of OCSGE. Thus the maximum improvement on tree translation by adding geographical context in the south part is 10-15% for France-wide maps while being close to 50% for OCSGE.

Conversely, using coordinates brings no improvement or worsens the results when the source map or target is the MOS map. We link this behaviour to the fact that the MOS is by far the one with the smallest spatial extent (around 2% of France's territory). The geographical context is almost the same in all areas, making the geographical coordinate encoding useless. Moreover, we observed some overfitting of the network due to the small number of coordinates which downgrades the results on the test set.

<table><caption>Table 4.9: Agreement between the translation results of the different geographical coordinate aware models and the target maps. The backbone network is A-Unet.</caption><thead><tr><th rowspan="2">Source</th><th rowspan="2">Target</th><th colspan="5">CGLS (P)</th><th colspan="5">CLC (C)</th><th colspan="5">OSO (O)</th><th colspan="5">OCSGE cover (G1)</th><th colspan="5">OCSGE use (G2)</th><th colspan="5">MOS(M)</th><th rowspan="2">Average</th></tr><tr><th>C</th><th>O</th><th>G1</th><th>G2</th><th>M</th><th>P</th><th>O</th><th>G1</th><th>G2</th><th>M</th><th>P</th><th>C</th><th>G1</th><th>G2</th><th>M</th><th>P</th><th>C</th><th>O</th><th>G2</th><th>P</th><th>C</th><th>O</th><th>G1</th><th>P</th><th>C</th><th>O</th><th>Average</th></tr></thead><tbody><tr><td rowspan="6">OA</td><td>NoCoord</td><td>62</td><td>55</td><td>68</td><td><strong>78</strong></td><td><strong>76</strong></td><td>73</td><td>58</td><td>72</td><td><strong>80</strong></td><td><strong>79</strong></td><td><strong>77</strong></td><td>68</td><td>79</td><td><strong>86</strong></td><td><strong>85</strong></td><td>68</td><td>54</td><td>56</td><td>92</td><td>67</td><td>49</td><td>49</td><td>79</td><td>84</td><td>82</td><td>63</td><td>71</td></tr><tr><td>ConcatCoord</td><td>62</td><td>54</td><td>68</td><td><strong>78</strong></td><td><strong>76</strong></td><td>72</td><td>58</td><td>72</td><td>79</td><td><strong>79</strong></td><td><strong>77</strong></td><td>67</td><td>79</td><td><strong>86</strong></td><td><strong>85</strong></td><td>68</td><td>54</td><td>55</td><td>92</td><td>66</td><td>48</td><td>49</td><td>79</td><td>83</td><td>82</td><td>63</td><td>70</td></tr><tr><td>MLPCoord</td><td>63</td><td>55</td><td>68</td><td><strong>78</strong></td><td><strong>76</strong></td><td>73</td><td>58</td><td>72</td><td><strong>80</strong></td><td><strong>79</strong></td><td><strong>77</strong></td><td>68</td><td>79</td><td><strong>86</strong></td><td><strong>85</strong></td><td>69</td><td>55</td><td>57</td><td>92</td><td>67</td><td>51</td><td>50</td><td>79</td><td>85</td><td>82</td><td>63</td><td>71</td></tr><tr><td>MLPCoordFilter</td><td>63</td><td>56</td><td>68</td><td><strong>78</strong></td><td><strong>76</strong></td><td>73</td><td>58</td><td>72</td><td><strong>80</strong></td><td><strong>79</strong></td><td><strong>77</strong></td><td>68</td><td>79</td><td><strong>86</strong></td><td><strong>85</strong></td><td>69</td><td>56</td><td>57</td><td>92</td><td>67</td><td>52</td><td>51</td><td>79</td><td>85</td><td>82</td><td>63</td><td>71</td></tr><tr><td>MLPPosEnc</td><td><strong>65</strong></td><td><strong>57</strong></td><td><strong>69</strong></td><td><strong>78</strong></td><td><strong>76</strong></td><td><strong>75</strong></td><td><strong>59</strong></td><td><strong>73</strong></td><td><strong>80</strong></td><td><strong>79</strong></td><td><strong>77</strong></td><td><strong>69</strong></td><td><strong>80</strong></td><td><strong>86</strong></td><td><strong>85</strong></td><td><strong>71</strong></td><td><strong>58</strong></td><td><strong>58</strong></td><td><strong>93</strong></td><td><strong>69</strong></td><td><strong>54</strong></td><td><strong>53</strong></td><td><strong>79</strong></td><td><strong>86</strong></td><td><strong>84</strong></td><td><strong>64</strong></td><td><strong>72</strong></td></tr><tr><td>NoCoord</td><td>26</td><td>28</td><td>27</td><td>19</td><td>32</td><td>56</td><td>36</td><td>35</td><td><strong>30</strong></td><td><strong>41</strong></td><td>60</td><td>37</td><td>44</td><td><strong>27</strong></td><td><strong>53</strong></td><td>48</td><td>24</td><td>29</td><td><strong>44</strong></td><td>40</td><td>20</td><td>22</td><td><strong>40</strong></td><td><strong>46</strong></td><td>29</td><td>21</td><td>35</td></tr><tr><td>ConcatCoord</td><td>26</td><td>28</td><td>27</td><td>19</td><td>32</td><td>56</td><td>34</td><td>35</td><td><strong>30</strong></td><td><strong>41</strong></td><td>60</td><td>37</td><td>44</td><td><strong>27</strong></td><td><strong>53</strong></td><td>47</td><td>25</td><td>29</td><td><strong>44</strong></td><td>41</td><td>20</td><td>22</td><td><strong>40</strong></td><td><strong>46</strong></td><td>28</td><td>21</td><td>35</td></tr><tr><td rowspan="4">mF1</td><td>MLPCoord</td><td>27</td><td>28</td><td>27</td><td>19</td><td>32</td><td>56</td><td>36</td><td>36</td><td><strong>30</strong></td><td><strong>41</strong></td><td>60</td><td>38</td><td>44</td><td><strong>27</strong></td><td><strong>53</strong></td><td>48</td><td>27</td><td>29</td><td><strong>44</strong></td><td>43</td><td>22</td><td>23</td><td><strong>40</strong></td><td>45</td><td>29</td><td>21</td><td>35</td></tr><tr><td>MLPCoordFilter</td><td>28</td><td>28</td><td>28</td><td>19</td><td><strong>33</strong></td><td>56</td><td>36</td><td>36</td><td><strong>30</strong></td><td><strong>41</strong></td><td>60</td><td>38</td><td>44</td><td><strong>27</strong></td><td><strong>53</strong></td><td>49</td><td>29</td><td>29</td><td><strong>43</strong></td><td>45</td><td>23</td><td>24</td><td><strong>40</strong></td><td>45</td><td>29</td><td>22</td><td>36</td></tr><tr><td>MLPPosEnc</td><td><strong>30</strong></td><td><strong>29</strong></td><td><strong>30</strong></td><td><strong>20</strong></td><td><strong>33</strong></td><td><strong>58</strong></td><td><strong>37</strong></td><td><strong>38</strong></td><td><strong>30</strong></td><td><strong>41</strong></td><td><strong>61</strong></td><td><strong>40</strong></td><td><strong>45</strong></td><td><strong>27</strong></td><td><strong>53</strong></td><td><strong>52</strong></td><td><strong>34</strong></td><td><strong>31</strong></td><td><strong>43</strong></td><td><strong>52</strong></td><td><strong>29</strong></td><td><strong>25</strong></td><td><strong>40</strong></td><td><strong>45</strong></td><td><strong>30</strong></td><td><strong>23</strong></td><td><strong>38</strong></td></tr></tbody></table>

Table 4.10 presents the results of the different geographical coordinates encoding methods on the ground truth. Translation learnt on France-wide source to local extent target can not be used to produce France-wide target using geographical coordinates encoding methods as only local extent coordinates are seen during training. Therefore metrics computed on the France-wide ground truth can only be computed for the six France-wide sources to France-wide target translations. This, of course, represents a major limitation
to the method that is addressed in the next chapter. The observations of the ground truth results confirm the improvement observed in agreement metrics, ensuring that observed improvements are not due to a higher ability of the network to replicate target errors when using coordinates.

<table><caption>Table 4.10: Comparison between the different geographical coordinates aware models translations and the ground truth. The backbone network is **A-Unet**. Only translations for which the source and target spatial extent are France-wide can be computed.</caption><thead><tr><th>Source</th><th></th><th colspan="2">CGLS (P)</th><th colspan="2">CLC (C)</th><th colspan="2">OSO (O)</th><th rowspan="2">Average</th></tr><tr><th>Target</th><th></th><th>C</th><th>O</th><th>P</th><th>O</th><th>P</th><th>C</th></tr></thead><tbody><tr><td rowspan="5">OA</td><td>NoCoord</td><td>57</td><td>51</td><td>69</td><td>58</td><td>74</td><td>70</td><td>63</td></tr><tr><td>ConcatCoord</td><td>56</td><td>51</td><td>69</td><td>57</td><td>73</td><td>69</td><td>63</td></tr><tr><td>MLPCoord</td><td>58</td><td>52</td><td>69</td><td>58</td><td><strong>74</strong></td><td>70</td><td>64</td></tr><tr><td>MLPCoordFilter</td><td>58</td><td>53</td><td>69</td><td>58</td><td>74</td><td>70</td><td>64</td></tr><tr><td>MLPPosEnc</td><td><strong>60</strong></td><td><strong>55</strong></td><td><strong>70</strong></td><td><strong>59</strong></td><td><strong>75</strong></td><td><strong>71</strong></td><td>65</td></tr><tr><td rowspan="5">mF1</td><td>NoCoord</td><td>21</td><td>24</td><td>51</td><td>36</td><td>53</td><td>39</td><td>37</td></tr><tr><td>ConcatCoord</td><td>22</td><td>24</td><td>51</td><td>36</td><td>53</td><td>40</td><td>38</td></tr><tr><td>MLPCoord</td><td>22</td><td>25</td><td>52</td><td>36</td><td>53</td><td>41</td><td>38</td></tr><tr><td>MLPCoordFilter</td><td>24</td><td>25</td><td>52</td><td>36</td><td>53</td><td>42</td><td>39</td></tr><tr><td>MLPPosEnc</td><td><strong>27</strong></td><td><strong>28</strong></td><td><strong>53</strong></td><td><strong>36</strong></td><td><strong>56</strong></td><td><strong>45</strong></td><td>41</td></tr></tbody></table>

To evaluate the ability of the network to interpolate geographical context from a limited number of training patches, Figure 4.18 presents the evolution of the agreement metrics depending on the training size for **NoCoord** and **MLPPosEnc**. The validation and test patches are fixed and represent each 5% of patches. A first observation is that **NoCoord** and **MLPPosEnc** results stabilized at the same training size (between 70-80% of all patches). It underlines that using the coordinates does not imply increasing the training size compared to a no-coordinate approach. We also observe that even when only 5% of the patches are in the training set, the **MLPPosEnc** gives slightly better results than the **NoCoord** one. We conclude that using coordinates is never detrimental. Lastly, using 40% of patches to train **MLPPosEnc** gives the same results as using 90% with **NoCoord** underlying the vast potential of coordinates to reduce training size.

<figure><img src="image_38.png" /><figcaption>Figure 4.18: Agreement for OSCGEu to CLC translation depending on training set size.</figcaption></figure>
<figure><img src="image_39.png" /><figcaption>Figure 4.19: Per class agreement F1-score of the OSO to CLC translation using the different coordinate encoding methods. Errors bars are computed as the standard deviation over 10 independent training. Only the 37/44 CLC classes with non zero F1-score are displayed for readability.</figcaption></figure>

#### 4.3.2.4 Conclusion

We first focused on determining potential data sources of contextual information. Instead of adding all the multi-modal data that could provide geographic context information, we propose to use the geographical coordinates directly to improve the translation. By comparing the result obtained by a random forest trained to use coordinates with one trained using Ecoclimatic areas, we obtained an intuition of the high potential of using a coordinate approach.

The second section investigates how to incorporate those coordinates to a convolutional neural network. Inspired by recent advances in natural language processing, we proposed to use positional encoding to ensure that the coordinate feature extraction learnt on the training set transfer well to the test set. The results demonstrate the effectiveness of the method in improving translation accuracy. However, this solution suffers one main limitation: it requires the training data to be more or less homogeneously dispatched over the studied area. This is not an issue as long as no spatial generalisation is needed but it becomes problematic if one wants to increase the spatial extent of a target map using a wide extent source. We address this issue in Section 5.1. Additionally, we point out that since no comparison between the MLPPosEnc method and a stratified training based on EcoClimatic areas was conducted during this PhD, no conclusion is made on the hypothetical superiority as results obtained using the random forest algorithm are not transferable to those obtained with a CNN. However, we underline that unlike the current
state of the stratified art training, the MLPPosEnc does not require pre-defined knowledge
of the area and only needs to train a single model.

## 4.4 Temporal context

This section focuses on the impact of temporal context and how to integrate it. We distinguish two underlying kinds of temporal context. The semantic temporal context, discussed in Section 2.3.3.1, is the one directly implied by the source and target nomenclature definitions. For instance, CLC *Rice fields* definition states, "As part of regular cultivation cycle, rice fields are occasionally left fallow for 1-3 years. These parcels are considered to be rice fields, too." and the "Pastures" definition states "Lands that are permanently used (at least five years) for fodder production.". Conversely, the "Rice crops" definition for OSO only includes rice fields cultivated in the considered year. Thus the translation from OSO to CLC requires three years of OSO maps to perform the correct translation.

The second one, we refer to as the temporally constrained context describes the potential temporal gap between the source and target used for learning. For example, translation from OSGEc to CLC involves learning the translation from a 2014 map to a 2012 or 2018 map. This temporal gap constraint is not a problem for traditional semantic-based methods as they do not learn the translation on existing maps. Conversely, it could represent a considerable limitation if this temporal gap significantly affects the translation results. This section investigates those two temporal contexts separately. We exclude from this study the use of image time-series to preserve a pure land-cover translation approach.

The following experiments focus on the OSO to CLC translation, as (i) CLC is the maps with the highest number of classes with temporal context (ii) 3 consecutive years of OSO maps (2016, 2017, 2018) can be used for CLC 2018 translation. The CGLS-LC100 to CLC translation could have also been studied as four consecutive years of CGLS are available but would have been more challenging to analyse due to the lower diversity of predictable classes using CGLS and the high error rate of the original CGLS product. Eventually, since the OSO 2016 and 2017 nomenclature merges most of the agricultural classes into two superclasses, *Summer crops* and *Winter crops*, we perform similarly for the OSO 2018 products, i.e. all the OSO products included 17 classes.

### 4.4.1 Multitemporal source translation

#### 4.4.1.1 Motivation

Section 2.3.3.1 underlines that state-of-the-art for incorporating multitemporal information into a classical land-cover map generation problem involves analysing a large time series of images. The literature shows the current superiority of correctly encoding the temporal context provided by satellite image time-series into a deep-based architecture (using
Transformers or Recurrent Neural Networks with adapted cells) w.r.t. manually defined features based on the variation of given spectral indices.

However, taking into account temporal context in the land-cover map translation case is widely different due to the difference in nature of the problem and data availability:

* Temporal ordering does not matter: nomenclature definition such as CLC "Rice Field": "[...] rice fields are occasionally left fallow for 1-3 years. [...]" does not include a notion of temporal ordering, i.e. it does not matter if the field was occupied by rice the last year, two years ago or three years ago but only that it was occupied by rice once during the three years.

* We do not have access to substantial temporal stacks of land-cover maps: most land-cover maps have been produced for less than ten different periods, many of them only once.

The limited and irregular temporal sampling through decades of LC maps prevents from designing a specific method for temporal information extraction. We assume that using a simple approach, such as concatenating multiple versions of the source maps, is sufficient to achieve improvement. We instead focus on understanding the reason behind the potential improvement. Especially as the fusion of multiple dates of a source requires frequently updated land-cover maps, the method discussed here is mainly intended to be used on automatically classified land-cover maps (ACLC). Section 2.3.4 pointed out that those ACLC exhibit particular spatial error patterns that might vary depending on the year due to changes in the classification algorithm or in the data used. When attempting a multitemporal analysis on such source, learning different features per year could improve translation results by better detecting source errors rather than exploiting temporal context. We argue that understanding the reason behind potential improvements is crucial for future studies on multi-temporal map denoising. Refer to Section 5.2.1 for a more elaborate way than concatenation to incorporate multiple sources of data information (such as images, maps, Digital terrain model).

### 4.4.1.2 Experimental protocol

We compare three methods with different temporal information extraction power:

**Concat** feeds the network with a temporally ordered concatenation of OSO 2016, 2017, and 2018. Since input is one-hot encoded, we underline that the first 17 channels describe OSO 2016, channels 17 to 34 describe OSO 2017, and the remaining 34 to 51 OSO 2018. In this setup, the network might learn different features for the same land-cover class depending on its year of observation.

**Shuffle** randomly exchanges the temporal order of the concatenation for each patch, e.g. in one patch, the first 17 pixels describe OSO 2018 while in the other patch, the first 17
channels describe OSO 2017. In this setup, the same features are learnt for all years for a given land-cover class. However, the learnt features can be oriented towards detecting some error patterns specific to a given year, e.g. focus on distinguishing OSO 2018 roads which are far less erroneous than other years and exhibits different spatial pattern.

**Mean** replaces the concatenation by the temporal average of one-hot encoding. For instance, in a two-class setup a pixel classified as class one during two years and as class two during one year is encoded as $e = [\frac{2}{3}, \frac{1}{3}]$. This encoding is independent from temporal ordering making it difficult for the network to learn year-specific features.

Theoretically in a scenario where a multi-temporal analysis only benefits from the temporal semantic context, the results of this three methods should be identical, i.e. temporal ordering should not matter. A higher score of the **Concat** compared to other methods would indicate that learning custom feature per year is beneficial for the translation, which could be linked to a higher ability to identify source errors.

To comfort the distinction between improvement fostered by semantic context from those fostered by a higher ability to compensate source errors, CLC classes with annual temporal context elements are manually identified (see definitions in Appendix D). This results in the selection of 10/44 CLC classes which should be the only to one to benefit from the multitemporal analysis under a no-noise compensation scenario. It mainly concerns, crop oriented classes such as *Non-Irrigated arable land* and *Rice fields* (crop rotation with fallow land period) or *Pastures* (used for fodder production at least for 5 years) and natural landscapes such as *Natural Grasslands* (no human influence for long period), *Moors and heathland* and *Sclerophyllous vegetation* (includes crops left fallow for 3 years and more), *Transitional woodland/shrub* (includes recently cut/reforested areas), *Burnt areas* (recently burnt) and *Glaciers and perpetual snow* (permanent over several years). Those classes are displayed in green in the per-class histograms of the results section.

Lastly, OSO 2016 to CLC 2018, OSO 2017 to CLC 2018 and OSO 2018 to CLC 2018 translation results are presented as control values to evaluate the benefit of a multi-temporal analysis over a mono-temporal one.

### 4.4.1.3 Results

<table><caption>Table 4.11: Average over ten independent training of the agreement metrics for the OSO to CLC translation in different temporal configurations</caption><thead><tr><th></th><th>OSO2016</th><th>OSO2017</th><th>OSO2018</th><th>Mean</th><th>Shuffle</th><th>Concat</th></tr></thead><tbody><tr><th>OA<sub>ag</sub></th><td>68</td><td>68</td><td>68</td><td>68</td><td>68</td><td><strong>69</strong></td></tr><tr><th>mF1<sub>ag</sub></th><td>37</td><td>39</td><td>37</td><td>39</td><td>40</td><td><strong>42</strong></td></tr></tbody></table>

Table 4.11 presents the translation results from OSO to CLC using the multiple methods described above. A first observation is that the **Concat** multi-temporal analysis improves translation quality, especially in *mF1<sub>ag</sub>* (+5%). The fact that the **Mean** and **Shuffle**
methods perform worse than Concat underlines that temporal ordering does matter. As we pointed out that CLC class definitions don't include temporal ordering notions, we assume that the main benefit of ordered temporal sequences is to enable the network to learn year dependent features describing year specific noise patterns rather than exploiting the temporal semantic context. To obtain a rough estimate of the proportion of improvement due to better noise detection or temporal semantic context, we provide Figure 4.20 presenting per class results. CLC classes, including a multi-year temporal criterium in their definition, are displayed in green.

<figure><img src="image_40.png" /><figcaption>Figure 4.20: Agreement F1-score per class for the OSO to CLC translation in different temporal scenarios. CLC classes including multi-year temporal criteria in their definition, are in green. Red values indicate CLC classes with no such criteria but a high gap between one of the multitemporal method and the per-year methods. Only CLC classes with a non-zero F1 score are displayed for visibility. Error bars are computed as the standard deviation of 10 independently trained models.</figcaption></figure>

First, we observe that the three proposed method gives almost always better results than mono-temporal analysis. However, in most cases, the improvements are small, e.g. 112:Discontinuous urban fabric mF1 is, on average, 2% higher using Concat than only one date. Focusing on the "green" classes reveals that only three of the 8 classes benefit significantly from a multitemporal analysis: 211:Non permanently irrigated crops, 231:Pastures, 321:Natural Grasslands. In the remaining 5 classes, the incorporation of multitemporal data did not increase the results. Additionally, looking at red classes reveals that the most significant improvements are observed on classes with no temporal constraints, such as 124:Airport or 313:Mixed Forest. We highlight that in those cases the Concat method usually performs better than the two other methods. We directly link those improvements to special error patterns varying across years of the OSO map, e.g. Airports roads in OSO are widely imprecise, and a change in data used for training fostered a difference in the road's spatial distribution in 2018 compared to other dates.

We comfort the constatation that learning custom per-dates features counteracts the noise in the source map by observing that only 4 out of the eight predicted "green" classes exhibit the same results for the three implemented methods: 231:Pastures, 311:Natural grassland, 313:Scelorypyllous vegetation, 314: Transitionnal Woodland/Shrubs. In four
other cases, the **Concat** method performs better than the two other methods.

To sum up, we conclude that conducting multitemporal source land-cover translation is beneficial for the quality of translation. We underline that the main improvement is not due to a higher ability to analyse the temporal semantic context, as all three methods should perform equally (only ≈ 1-2% mF1 improvement). Instead, we link most of the improvement to a higher ability to identify source errors by enabling learning custom features per date (+5% mF1).

## 4.4.2 Temporal gap effects

### 4.4.2.1 Motivation

Ideally, source and target used to train our models should represent the same time period. Unfortunately, under operational uses, many land-cover translations can not be learnt on pairs of temporally matching sources and targets. This temporal gap inherently affects the training procedure by adding some label noise, *i.e.* each source pixel is learnt to be translated into what was or will be its corresponding target class in the past/future.

The amount of temporal gap induces noise correlated to the land cover change proportion observed between the two dates. This proportion depends, of course, on the temporal gap size and the spatial extent, *i.e.* some areas are prone to land-cover change. Additionally, land cover change proportion also highly correlated to the resolution, *i.e.* small changes are more likely, and nomenclature, *i.e.* the most probably changing classes over time, are crops-oriented classes.

In this section, we assume that land-cover map changes slightly between two dates as long as the period considered remains modest (within ten years), the considered spatial extent is wide enough (country scale), and the nomenclature is not focused on temporally changing classes. For instance, less than 2% changes are observed for CLC 2012/2018, MOS 2012/2017 and CGLS-LC100 2015/2018 over France. We hypothesise that under those assumptions learning land-cover map translation with a temporal gap between source and target should not bring detrimental changes to the translation quality as it induces a reasonable amount of noise (2%) compared to the noise inherently included in the land cover maps (*e.g.* CGLS product is only 72% accurate). This section focuses on testing this hypothesis by proposing three different operational setups with different temporal gaps to evaluate the robustness of the method to this temporal induced noise.

We underline that in the special case of the OSO, those assumptions are not met. Interestingly a stable 25% ±1% change is observed between all the different combinations of periods between 2016 and 2019, *i.e.* the 25% change between OSO 2016 and OSO 2017 is also observed between 2016 and 2019 or 2017 and 2018. We link this behavior to OSO crop nomenclature with an annual rotation between *summer crops* and *winter crops* explaining 15%. The remaining 10% is mainly observed on erroneous areas or edges
between different land-cover types. As those circumstances add a considerable amount of noise, we consider that a specific method robust to label noise must be designed and address this issue separately (see Section 4.5).

### 4.4.2.2 Experimental protocol

To evaluate the robustness of the method under a reasonable amount of changes, we present three scenarios corresponding to different time gaps between source and target land-cover maps and, subsequently, three specific operational use cases. Two reference years for CLC (2012 and 2018) and two for OSO (2016 and 2018) are selected. The experiments are conducted on CLC at level 2, which exhibits a 15 classes nomenclature, to simplify the analysis.

**Scenario 1** corresponds to a scenario aiming for the automatic extension of CLC to a broader area, assuming that CLC has not yet been generated over a full area of interest. It could be particularly relevant for the forthcoming editions of CLC: one would only need to generate a high-quality sample version on specific areas, and our framework could fill the gaps. This methods trains and tests on OSO 2018 and CLC 2018. This scenario represents a no-temporal gap setup.

**Scenario 2** corresponds to the updating operational setting. First, a translation model is trained on a pair of pre-existing OSO and CLC products. Then, the model is applied to the OSO product of the year for which the new CLC map is to be produced. We assume that the most recent OSO product is 2018 and that we want to produce CLC 2018. The translation model is trained using CLC 2012. We choose to pair it with OSO 2016 to minimize the disagreement in the training data caused by land-cover map changes. In this scenario, OSO 2018 is translated with the learnt model, and CLC 2018 is used as reference data for validation.

One limitation of **Scenario 2** is that if the learning algorithm can cope with discrepancies in the training data, it may be better to use the most recent OSO map in the training phase. This would allow taking into account the evolution of land-cover map, which does not affect the translation itself. For instance, one could assume that climate change makes wetland areas dryer. It would not introduce a change between CLC 2012 and CLC 2018 (dryer wetlands are still wetlands) but could introduce an evolution in OSO. Indeed, OSO does not have wetland classes, and wetlands in CLC correspond to water, sand, grasslands or moorlands in the OSO nomenclature. In this situation, some CLC wetlands could transition from water to sand, grassland or moorland. This would mean that the translation rule learned between OSO 2016 and CLC 2012 could not be applied to OSO 2018. In order to assess this situation, we propose **Scenario 3**, in which the translation model is trained by pairing CLC 2012 with OSO 2018. This model is then applied to OSO 2018. CLC 2018 is used for validation. The underlying assumption is that the model fails to learn some of the associations imposed by the training data that correspond to real changes from the point of view of the CLC nomenclature.
4.4.2.3 Results

As the scenarios gives almost identical results visually this section focuses on quantitative
assessment. We provide a web interface to visualize results at a France-wide scale for
scenario 2 at https://oso-to-clc.herokuapp.com/.

<figure><img src="image_41.png" /><figcaption>Figure 4.21: mF1 obtained by performing ten different initializations of the network for each scenario. Error bars indicate the standard deviation over the 10 trials.</figcaption></figure>

Figure 4.21 presents the per-class results of the different scenarios computed on the ground truth. The original CLC 2018 level 2 map is also compared to our ground truth (achieves a 86% OA<sub>gt</sub>, 85% mF1<sub>gt</sub>). The first scenario, which translates OSO 2018 into CLC 2018 in both the train and test phase, obtains an 81% OA<sub>gt</sub> and a 63% mF1<sub>gt</sub>. The second scenario, training a model to translate OSO 2016 into CLC 2012 and then using it in the test phase to translate OSO 2018 into CLC 2018, achieves a 79% OA<sub>gt</sub>, 60% mF1<sub>gt</sub>, slightly under-performing the first scenario, especially on urban classes. This could partly be explained by the difference between the OSO 2016 and 2018 products in per class thematic accuracy. Additionally, it could also be imputed to the difference between the regularisation needed to correct land-cover map changes between OSO 2016 and CLC 2012 versus OSO 2018 and CLC 2012. The third scenario translates OSO 2018 into CLC 2012 with the underlying hypothesis that changing areas between 2012 and 2018 are only considered as additional noise achieves a performance qualitatively and quantitatively similar to the first scenario (81% OA<sub>gt</sub>, 61% mF1<sub>gt</sub>). This scenario obtains reasonably good results while not needing a common time stamp of the source and target for learning (scenario 1) and no older source map (scenario 2).

The analysis of the three scenarios reveals that, under the mentioned constraint of learning
the translation with a limited temporal gap between the source and the target and on a
wide spatial extent, the temporal gap does not deteriorate the results. Interestingly this
statement holds for all classes.
## 4.5 Cartographic Context

### 4.5.1 Motivation

The image to the landcover classification paradigm often faces reasonable amount of noise in the input image while the label noise in the target used for training is often unknown. Conversely, translation between cartographic products involves working with significant label-noise in both source and target (5 to 30% errors) for which the noise distribution is most of the time known in the form of a confusion matrix provided by the land-cover maps producer.

Precisely estimating the impact of source and target label noise on the translation training is unfeasible at large scale as it would require noise-free data. However, we still can estimate the proportion of errors replicated by the translation as the gap between $OA_{ag}$ and $OA_{gt}$ for translation between France-wide maps (-2% to +7% taking into account a ±2% uncertainty on $OA_{gt}$). We argue that developing a method that could reduce this overfitting to target errors could increase the results by approximately up to 7%.

This section focuses on how to tackle target label noise which we believe more impactful than source label noise. Source errors are already partly compensated by learnt translation as i) they define translation on real data, i.e. they also learn to translate erroneous source pixels into the expected target classes ii) the approach relies on spatial context analysis, e.g. roads misclassified as urban areas can be identified based on their linear shape and correctly translated. Moreover source errors can also partially be compensated by the use of multi-temporal inputs (previous section), or multi-modal data (next chapter).

### 4.5.2 Idea

As underlined in the literature review, learning with noisy target labels has been a core subject in the machine learning community for decades. However, limited work has been conducted on the specific case of semantic segmentation (per-pixel classification) and land-cover mapping field. Using the classification of [284] mentioned in Section 2.3.4, we underline that current methods used in the land-cover field are mostly based on building architecture performing sample selection [5, 179, 338] (detect noisy elements and corrects or remove them from training) or regularization [132]. We believe this is detrimental as some interesting specificities of land-cover mapping and semantic segmentation could help build a custom loss adjustment-based solution to improve the quality of the translation.

First, semantic segmentation exhibits a considerable difference from other classification problems. When performing a per-image classification, the network can only access a very constrained sample of classes per iteration. For instance, the current state of the art on the Image-Net dataset used a 128 batch size on a dataset with 120 classes. This implies that, at best, each class has a unique sample at each iteration. This is not the case in
semantic segmentation. For instance, in our case, with a batch size of 30 and a 600×600 patch size, we might have access to multiple thousands (even millions) of examples for almost all classes (except the very spatially constraint one) as each pixel corresponds to a single classification.

Secondly, when classifying standard computer vision datasets such as image-net, no information is provided on the per class noise as it was never assessed. Conversely, one specificity when working with land-cover maps is that most of them are provided with a confusion matrix to assess the quality of the product. We refer to the confusion matrix as a noise transition matrix for coherence with works on noisy label correction [237]. Interestingly we did not find previous works that integrate those noise transition matrices in the loss function in the land-cover field, while this strategy was proposed several years ago in computer vision. Those strategies apply the idea that the loss function can be corrected using the noise transition matrix as per-instance information, i.e. if one element is $T_i$ in the target map but is 80% of the time truly $T_i$ and 20% $T_j$, then the network should predict this element as 80% $T_i$ and $T_j$ 20%.

Those strategies appear widely sub-optimal as we have access to numerous examples of each class. Indeed the knowledge that $T_i$ is 80% of the time truly $T_i$ and 20% $T_j$ does not truly reflect per-element information. In reality $T_i$ elements should either be $T_i$ at 100% in 80% of cases and $T_j$ at 100% in 20% of cases. Instead of performing a per element loss correction, the loss correction should use the distribution of target labels. For instance, if there are $10^5$ pixels of class $T_i$ in the target map, then the network should predict $8 \times 10^4 T_i$ and $2 \times 10^4 T_j$ each with a 100% confidence. To our knowledge, this strategy, which involves knowing the noise transition matrix and having access to large enough class distribution at each iteration, has never been explored.

### 4.5.3 Experimental Protocol

As we evaluate the quality of the translation on our ground truth the source map usable in this experiment only includes CGLS, OSO and CLC. Experiments are conducted on the OSO to CGLS translation as translating into CGLS is the most complicated from a noise point of view (28% noise) and is not ill-defined due to a higher target spatial resolution (as the CLC to CGLS translation).

**Implementing a distribution-based loss correction error** The proposed solution is inspired by [202] consist in matching distribution for label super-resolution. We process in three steps.

First, we need to evaluate the class distribution from the network output in a differentiable way. We first apply a softmax layer to obtain per-class confidence between 0 and 1, with all the per-class confidence summing to one for each pixel. Let $p_j(X = k)$ be the predicted confidence value for the $i^{th}$ pixel to be of class $k$ while being annotated as class $j$ in the target, then $O_j(k)$ defined in Equation 4.8 approximates the number of pixels labeled as $j$
in the target and k by the network:

$$O_j(k) = \sum_{i=0}^{n_j} p_j^i(X = k) \quad \text{with } n_j \text{ the total number of pixels of class } j \text{ in the target.} \tag{4.8}$$

As the number of pixels labeled j might vary from one iteration to the other, we normalize $O_j(k)$ by the n number of pixels labeled j, which gives the probability that a pixel labeled j in the target is labeled k by the network (Equation 4.9):

$$P_j(k) = \frac{O_j(k)}{n}. \tag{4.9}$$

Secondly, we need to have access to the probability that an element labeled j in the target is, in truth k. This is given by the noise transition matrix available with most land-cover maps. We denote this value $N_j(k)$. In our case, we can either evaluate this noise transition matrix directly using the manually built ground truth or the official noise transition matrix provided with the dataset. Using the same ground truth for obtaining $N_j(k)$ than the one used for evaluating the results might overestimate the proposed method's potential by informing the network of the ground truth distribution used at test time. We always use the official confusion matrix unless explicitly mentioned otherwise.

Thirdly, we evaluate the difference between the previously defined discrete probability functions $P_j$ and $N_j$. We compute Kullback-Leibler divergence between the predicted class distribution $P_j$ and the expected one $N_j$ for all pixels denoted j in the target as defined in Equation 4.10:

$$KL_j(P_j || N_j) = \sum_{k=1}^{c} N_j(k) \log \left( \frac{N_j(k)}{P_j(k)} \right) \quad \text{with } c \text{ the total number of classes.} \tag{4.10}$$

This loss could theoretically be computed for each target class, outputting a divergence for each class distribution and averaged. However, this method would give the same weight to all class distributions, which would be widely detrimental to the $OA_{gt}$ by giving equal weight to rare and frequent classes. We propose to proceed to a weighted by class proportion mean of those Kullback-Leibler divergence measurements giving the Noise Divergence loss defined as:

$$L_{ND} = \sum_{j=1}^{c} \frac{KL_j(P_j || N_j)n_j}{\sum_{l=1}^{c} n_l}. \tag{4.11}$$

Our first experiment consists in using the Noise Divergence loss to train a network and compare the obtained $OA_{gt}$ and $mF1_{gt}$ with the one obtained with the Cross-Entropy.

**Studying the importance of the noise quality matrix** As this method relies on a noise matrix obtained from the land-cover map provider, the quality of this transition
matrix is uncertain. More specifically, as most confusion matrices are computed on size-limited ground truth, per-class confusion might be widely inaccurate.

Moreover, the confusion matrix provided by the land-cover map is not necessarily computed to the same spatial extent as the one on which the translation is performed. For instance, the CGLS official confusion matrix for level 2 is only available at global scale. As error patterns can vary depending on the regions, it might be significantly different from one computed on France only. Thus it appears crucial to test the method's robustness to the quality of the noise confusion matrix.

We designed two experiments. The first compares the OA<sub>gt</sub> and mF1<sub>gt</sub> obtained when the L<sub>ND</sub> use the official noise transition matrix or the ground truth noise transition matrix. L<sub>ND</sub> using the ground truth should provide an upper bound of possible results as it partially breaks the independence between training and testing (should only be used as a comparison tool). The results obtained by the L<sub>ND</sub> using the official CGLS noise transition matrix, which appears significantly different from the one obtained on ground truth (Figure 4.22), provide a first insight into the method robustness to errors in the noise matrix.

<figure><img src="image_42.png" /><figcaption>Figure 4.22: Probability for each CGLS class (in row) to be confused with one another (in column) according to the official confusion computed worldwide (left) or on France-wide ground truth (right). The Black line indicates that there is no "Moss and Lichens" on the France-wide extent.</figcaption></figure>

A second experiment introduces various noise levels in the official noise transition matrix. Noise in land-cover maps is directional, i.e. all errors are not equiprobable. To preserve the erroneous class distribution, we redistribute the additional noise in the same proportion as the one observed in the original matrix, i.e. if we increase the CGLS official noise transition matrix by a 5% ratio, CGLS 1:"Closed Forest" will have $N_1'(1) = 0.95N_1(1) = 0.95 * 0.82$. Let $c_{CGLS}$ denotes the number of CGLS classes (12), the remaining transitions are computed
according to Equation 4.12:

$$N'_{1}(k) = N_{1}(k) + (N_{1}(1) - N'_{1}(1)) \frac{N_{1}(k)}{\sum_{i=2}^{n} N_{1}(i)} \quad (4.12)$$

**Comparison with other methods** We propose comparing four loss-based strategies to improve results when training on noisy labels.

The Mean Absolute Error (MAE) has been mathematically demonstrated to be a noise-robust loss function [99]. As its implementation is easy and not computationally intensive, most current loss-based noise correction compares to it, which makes it an interesting baseline to compare to other works.

$$L_{MAE} = \frac{1}{n} \sum_{i=1}^{n} \sum_{k=1}^{c} |y^{i}(k) - (p^{i}(k))|. \quad (4.13)$$

The Symmetric cross-entropy [326] (SCE) has also been proposed to tackle label noise. Unlike cross-entropy, reverse cross-entropy is a noise-robust loss function. The authors propose a linear combination of cross-entropy (good convergence) and reverse cross-entropy (noise robust) to achieve better results than cross-entropy alone. Using the same notation as those used for Equation 4.3 we obtain Equation 4.14.

$$L_{SCE} = \alpha L_{CE} + \beta L_{RCE} = \frac{1}{n} \sum_{i=1}^{n} \sum_{k=1}^{c} \alpha y^{i}(k) \log(p^{i}(k)) + \beta p^{i}(k) \log(y^{i}(k)). \quad (4.14)$$

We obtain the best results for this technique using $\alpha = 0.1$ and $\beta = 1$. Solely those results are presented.

The simplest loss correction approach to take into account the per-pixel classification uncertainty is to consider a uniform class noise. This strategy refered as the Label Smoothing regularisation [242] (LSR), uses the cross-entropy loss (Equation 4.3) but replaces the one-hot encoded label by a soft encoding version i.e. instead of having $y^i(k)$ if the i-th element true class is k and 0 otherwise, $y^i(k)$ follow Equation 4.15 with smooth < 0.5.

$$p^i(k) = \begin{cases} 1 - smooth, & \text{if } i^{\text{th}} \text{ true class is } k \\ \frac{smooth}{c-1}, & \text{else} \end{cases}. \quad (4.15)$$

The last experimented strategy relies on the noise transition matrix to compensate for errors at a pixel level. Known as the Forward Correction [237] (FC), this strategy multiplies the network prediction with the ground truth noise matrix during the training phase to obtain a corrected prediction (see Equation 4.16).

$$L_{FC} = \frac{1}{n} \sum_{i=1}^{n} \sum_{k=1}^{c} y^{i}(k) \log(p^{i}(k) N_{j_i}(k)) \quad \text{with } j_i \text{ target class of i.} \quad (4.16)$$
4.5.4 Results

Table 4.12 presents the effects of the different loss both in terms of agreement and on the GT. In terms of agreement with the target, the cross-entropy is, as expected, the best-performing function. Interestingly the noise correction loss tends to give comparable results with the CE regarding the result/target agreement except for the noise transition matrix aware losses (FC and ND). Our solution is by far the worst in terms of resemblance to the target with -8% in OA<sub>ag</sub> and -6% in OA<sub>mF1</sub>. Conversely, compared to the ground truth, the best method for OA<sub>gt</sub> is the SCE and the best in mF1<sub>gt</sub> is our ND loss. Even though the results obtained are stable across multiple independent trainings (mF1<sub>gt</sub> standard deviation is around 2% for each method), we underline that those results are not statistically significant due to the limited ground truth size (2300 points for OA<sub>gt</sub> and 2700 mF1<sub>gt</sub>). The results presented here should be taken cautiously, as only global tendencies can be observed. Comparing the results of ND<sub>off</sub> and ND<sub>gt</sub> is also interesting as despite the vast difference between the official and the ground truth confusion matrix, the results are not significantly different. This demonstrates partial robustness to an imperfect confusion matrix as the observed drop down in mF1<sub>gt</sub> is only 2%.

<table><caption>Table 4.12: Comparison of various noise correction methods.</caption><thead><tr><th></th><th>CE</th><th>MAE</th><th>SCE</th><th>LSR</th><th>FC</th><th>ND<sub>off</sub></th><th>ND<sub>gt</sub></th></tr></thead><tbody><tr><th>OA<sub>ag</sub></th><td><strong>77</strong></td><td>76</td><td><strong>77</strong></td><td><strong>77</strong></td><td>73</td><td>69</td><td>68</td></tr><tr><th>OA<sub>gt</sub></th><td>72</td><td>72</td><td><strong>73</strong></td><td>72</td><td>69</td><td>72</td><td>72</td></tr><tr><th>mF1<sub>ag</sub></th><td><strong>61</strong></td><td>38</td><td>60</td><td>60</td><td>39</td><td>55</td><td>48</td></tr><tr><th>mF1<sub>gt</sub></th><td>53</td><td>36</td><td>55</td><td>56</td><td>39</td><td><strong>60</strong></td><td>62</td></tr></tbody></table>

Figure 4.23 compares the per class F1-score of the different method. We observed a global tendency to achieve higher results using the **ND loss** than any other baselines in seven (out of eleven) of the reported classes. The largest improvement is observed on the 60: "Bare / Sparse vegetation", reaching nearly a 40% F1-score while being close to 0% for all other methods. However, we can not conclude further due to sample size limitation.

<figure><img src="image_43.png" /><figcaption>Figure 4.23: Per-class F1<sub>gt</sub> for the OSO to CGLS translation. Error bars are estimated using the Equation F.14 presented in Appendix F.</figcaption></figure>
Figure 4.24 presents the evolution of the *mF1<sub>gt</sub>* depending on the noise level inserted inside the official confusion matrix. Increasing the noise matrix with a 15% noise ratio, the *ND<sub>off</sub>* loss remains better than the traditional CE loss. This demonstrates the usability of our method even with imperfect approximations of the noise matrix.

<figure><img src="image_44.png" /><figcaption>Figure 4.24: *mF1<sub>gt</sub>* obtained by training the network with the *ND<sub>off</sub>* with an increasing directional noise ratio. We provide the mF1 score for the CE and *ND<sub>gt</sub>* as respective expected lower and upper bounds of the method.</figcaption></figure>

4.5.5 Conclusion

We introduced a Noise Divergence loss built on the idea that land-cover translation is a
specific problem in which we have access at each iteration to a vast amount of labels for all
classes and an accurate transition matrix. We propose to compute the divergence between
the per-target class distribution of network prediction and the expected distribution (the
noise transition matrix). The results demonstrate a global tendency of the method to
outperform other loss correction methods in terms of mF1gt. However, due to the small
ground truth size, we can not conclude on statistical significance. We underline that a
common strategy would be to work on simulated label noise. Albeit results obtained from
a noise label simulation are not transferable to real noise scenarios as noise simulation
techniques are unable to reproduce accurately real land-cover noise, which is directional,
often systemic, and location dependent (i.e. edges of object), class dependent and not
homogeneously distributed geographically. We also observed that our method is quite
resilient to errors in the noise translation matrix used for learning, making it usable in real
operational cases. Lastly we point out that more experimentation should be conducted to
determine the amount of noise required to make such loss correction approach better than
cross entropy. Some preliminary experiment conducted on the translation from CGLS to
CLC suggest that the method is not suitable for small amount of noise as under the 10%
noise of CLC cross entropy seems to give better results than our noise corrected loss.
## 4.6 Conclusion

The translation of land-cover maps is traditionally performed at the scale of the map nomenclature by associating each source class and target class. Starting from the observation that two elements of the same source class might have highly different translations according to their spatial, geographical and temporal context, we proposed to replace the nomenclature-levelled translation with methods performing translation at the scale of the pixel. Pixel-scale translation should allow two source pixels of the same class to have a different translation in the target nomenclature. To assess the quality of the proposed methods, we first proposed six baselines inspired by the current literature on map translation. The first two **HardSem** and **HardStat** are the most commonly found in the literature and associate a single target class with each source class. The other four methods are based on the idea that when the target map has a lower resolution than the source, it is possible to carry out a combined translation of several source pixels, slightly improving the translation quality. The key takeaway of this section is that statistical translation generally achieves better results than semantics. Indeed, when a source class can be translated in several ways, the semantic translation does not guarantee that the translation carried out is the most probable. This effect is all the stronger as the translation is carried out in an area of limited spatial extent. Thus, the apparent semantic translation of OSO's *Broad-leaved forest* class in CLC nomenclature is *Broad-leaved forest*. However, if the translation is performed on the Paris area, the statistically most probable translation is *Green urban areas*. This observation is crucial because current methods such as LCCS are based on the idea that the translation from one class to another must rely on proximity between semantic attributes of source and target classes rather than an on a probability of association. It leads to a considerable deterioration in the quality of map translations. The second lesson of these experiments was to emphasise that statistical methods favouring the global resemblance to the target map tend to provide a lower diversity of classes than semantic methods. This underlines the importance of considering solutions to preserve this diversity as much as possible.

Secondly, we explored the potential of spatial context for translation. We first hypothesise that knowing the shape of the object to which the pixel to be translated belongs increases the quality of the results of a translation significantly. To confirm this hypothesis, we started by estimating the shape of the objects using shape indicators commonly used in the GIS community but rarely used on land-cover maps. As the nomenclature does not include in the definitions information on the expected object shape (e.g. it does not indicate that a river is a thin elongated shape) we proposed to learn directly on the dataset how to use these indicators to perform a translation. Results demonstrated that the shape information allows a significant increase in the diversity of the predicted classes when the target resolution is equivalent to or lower than the source. Starting from the observation that these shape indicators are probably not perfectly tailored to the translation, we introduce a method capable of learning by itself to exploit the shape of objects and include