# Seasonal semi-supervised domain adaptation for linking population studies and Local Climate Zones

Basile Rousse, Sylvain Lobry, Géraldine Duthé, Valérie Golaz, Laurent Wendling

► To cite this version:

Basile Rousse, Sylvain Lobry, Géraldine Duthé, Valérie Golaz, Laurent Wendling. Seasonal semi-supervised domain adaptation for linking population studies and Local Climate Zones. Joint Urban Remote Sensing Event - JURSE 2023, May 2023, Heraklion, Greece. pp.4. hal-04394041

HAL Id: hal-04394041

https://hal.science/hal-04394041v1

Submitted on 15 Jan 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Seasonal semi-supervised domain adaptation for linking population studies and Local Climate Zones

Basile Rousse*†, Sylvain Lobry*, Géraldine Duthé†, Valérie Golaz†‡, Laurent Wendling*

*Université Paris Cité, LIPADE, F-75006 Paris, France, name.surname@u-paris.fr

†French Institute for Demographic Studies (INED), Aubervilliers, France

‡Aix-Marseille Univ., IRD, LPED, Marseille, France

**Abstract**—Environment and demographic dynamics are strongly linked. However, relevant data to study this interaction may be scarce especially in sub-Saharan Africa where it is not always possible to perform such studies with a high temporal frequency. Satellite imagery, when linked to demographic data, can be a significant asset to estimate missing data as it covers every country with both high spatial and temporal resolution. We aim to take advantage of satellite data to characterize the environment in inter-tropical areas. This environment is regulated by the changing of two seasons that are essential to consider. We introduce a semi-supervised domain adaptation strategy for neural networks based on seasonal changes. This strategy can be used to produce land cover maps in regions of the world where limited labeled datasets are available. We apply this method to produce environmental indicators and link them to malaria rates from the Malaria Indicator Survey of Burkina Faso. We show that malaria rates are correlated not only to urbanisation but also to the environmental characterisation of studied areas.

**Index Terms**—Semi supervised domain adaptation, deep learning, contrastive learning, demography, land cover, health, malaria

## I. INTRODUCTION

Demographic studies often provide approximate locations of interviewed households which allows to link demographic data to their corresponding environment using derived spatial data. Demographic analysis would gain from comprehensive environmental descriptors. The Local Climate Zones (LCZ) classification scheme (Figure 1) can be used to describe the environment based on the surface structures, their material and human activity. It was originally designed for global-scale heat island detection, without cultural consideration [1]. This standardized descriptor can be predicted at large scale using remote sensing images and can be linked to various demographic studies. LCZ mapping from single and multi sources spatial features is an active research topic. The So2Sat dataset is a large-scale dataset of 32× 32 Sentinel-1 and Sentinel-2 images from various parts of the world for training deep neural networks [2]. LCZ maps of 1642 cities have been generated using this dataset for urban studies [4] with a resolution of 320m. A Global LCZ map [3] was produced with a 100m resolution built from 46 spatial features, processed with random forest algorithms. Using the method developed in [13] and random forests classifiers, [12] links LCZ to Malaria prevalence in sub-Saharan cities. Considering

<figure><img src="image_2.png" /><figcaption>Fig. 1. Local Climates Zones classes</figcaption></figure>

local environmental characteristics for demographic analysis requires generating accurate maps of the country under study during the period of the survey. Mapping methods should also be robust to local environmental specifics and not require additional labeled data. In sub-Saharan countries, models should produce accurate maps during wet and dry seasons, that can occur several times in a year.

Semi-Supervised Domain Adaptation (SSDA) is meant to benefit from the increasing amount of unlabeled data. SSDA methods aim to reduce the divergence between a source domain (with labeled data) and a target domain (without labeled data) which have different distributions. Among them, consistency regularization techniques force a model to predict the same class for an image from the target dataset and its perturbations. Recent developments took advantage of the contrastive loss instance representation learning for reducing the domain gap [14]. Such techniques are well suited for remote sensing applications where global datasets have already been created [6], [7] and data is produced on a regular basis. This contribution aims to provide a training strategy to produce LCZ maps of every part of the world, using Sentinel-2 products, without additional annotated data. To this end, we introduce a seasonal-SSDA (s-SSDA) approach combining supervised and contrastive learning to extract useful global and more specific local features using unlabeled data. The model is expected to transfer spatial features learnt from the source dataset as well as being robust to the target seasonal variations. The generated LCZ maps can provide useful data to contextualise localised demographic information. As a study case, we mapped the whole of Burkina Faso to link LCZ environmental descriptors to demographic indicators through the Malaria Indicator Survey (MIS) of 2017-2018 [15]. We show that the proposed s-SSDA improves the performances of LCZ mapping. With these results, we characterized studied areas according to their urbanization rate, their LCZ characterisation and their malaria rates.

Work supported by the Data Intelligence Institute of Paris (DiiP), and IdEx Université Paris Cité (ANR-18-IDEX-0001). This work was performed using HPC resources from GENCI-IDRIS (Grant 2021-AD011013527).
## II. METHOD

We define a neural network $F(.)$ as our LCZ classifier which takes as input an image $x$ and returns a vector $s = [s_1, s_2, ..., s_{17}]^T$ of prediction scores for each LCZ class. The prediction of the model is the class with the highest score. As mentioned in section I, existing global datasets may need to be supplemented by additional data related the country under study. This method aims to take advantage of existing labeled datasets and of the large amount of unlabeled Sentinel-2 images to produce maps at the country-level. We define $D_S = (x_i, y_i)_{i \in [1, n_S]}$ as the labeled dataset where $x_i$ is a Sentinel-2 image, $y_i$ its associated label and $n_S$ the number of the samples in the dataset. The proposed method is based on seasonal perturbations to make the model robust to these seasonal changes. Then, we supplement $D_S$ with an unlabeled dataset $D_T = (z_i^{s1}, z_i^{s2})_{i \in [1, n_T]}$ made of $n_T$ pairs of images $z_i^{s1}, z_i^{s2}$ from the same area at different seasons $s_1$ and $s_2$. These two datasets are combined using a SSDA approach and contrastive learning.

### A. Contrastive learning

Contrastive learning is a machine learning technique used to teach a model features from a dataset without any labelling. At each step, the model is taught to group together similar images (two similar images are a positive pair) into the latent space by increasing similarity within positive pairs, and decreasing similarity within negative pairs though the contrastive loss:

$$L_{i,j} = - \log \frac{\exp(\text{sim}(F(z_i), F(z_j))/\tau)}{\sum_{k=1, k \neq i}^{2N} \exp(\text{sim}(F(z_i), F(z_k))/\tau)} \quad (1)$$

where $N$ is the size of the batch, $\text{sim}(\cdot, \cdot)$ is a similarity measure, $\tau$ is the temperature, $(i, j) \in [1, N]^2$, $(z_l)_{l \in [1, 2N]}$ samples from the batch, and $(z_i, z_j)$ a positive pair.

### B. Semi-supervised domain adaptation

SSDA is based on two datasets extracted from two different domains, referred as the labeled source dataset $D_S$ and the unlabeled target dataset $D_T$. The deep learning model $F(.)$ will be optimized for image classification according to two simultaneous yet different processes, using $D_S$ and $D_T$. The first process is a regular supervised training on $D_S$ in which a supervised loss $L_S$ (here the cross-entropy) should be minimized. The regularization process is similar to [9] and [10], originally used for self-supervised learning. The model is fed with images from the same area but different seasons, referred as $z_i^{s1}$ and $z_i^{s2}$. $z_i^{s2}$ can be considered as a seasonal perturbations of $z_i^{s1}$. The regularization term is computed using the contrastive loss between the output of the model for $z_i^{s1}$ and $z_i^{s2}$, where $\text{sim}$ is the Cosine similarity.

$$L_T = - \log \frac{\exp(\text{sim}(F(z_i^{s1}), F(z_i^{s2}))/\tau)}{\sum_{k=1, k \neq i}^{2N} \exp(\text{sim}(F(z_i^{s1}), F(z_k^{s2}))/\tau)} \quad (2)$$

The total loss used for back-propagation is a weighted sum of the supervised and unsupervised losses. The process is il-

<figure><img src="image_3.png" /><figcaption>Fig. 2. Training procedure</figcaption></figure>

ustrated in figure Figure 2. With the regularization coefficient $\alpha \in [0, 1]$:

$$L = \alpha \times L_S + (1 - \alpha) \times L_T \quad (3)$$

### C. Temporal regularization using Markov chains

The high availability of Sentinel-2 images enables taking into account years prior to the year of interest to ensure a consistent temporal continuity in the LCZ predictions. By assuming that LCZ classification follow a Markov process, maps from the different years and seasons can be linked using a Markov chain. Let's define:

* $I_N \in \mathbb{R}^{32 \times 32}$ be the prediction of $F(.)$ at time $N$.

* $LCZ_N$ the LCZ class ($c_N \in [1, 17]$) of the patch at time $N$.

* $M \in \mathbb{R}^{17 \times 17}$ a matrix where $m_{i,j} \in [1, 17]^2$ is the coefficient of M at row i and column j. $m_{i,j}$ is the probability in the 1st Markov process to go from $LCZ_{N-1} = i$ to $LCZ_N = j$, $(i, j) \in [1, 17]^2$. These probabilities have been set empirically according to observed changes in the region of interest during previous years.

If $(LCZ_N)$ follows a 1st order Markov process. Then, for all $N$:

$$P(LCZ_N = c_N) = m_{c_{N-1}, c_N} \times P(LCZ_{N-1} = c_{N-1}) \quad (4)$$

The probability of a given input to belong to the LCZ class according to the model predictions at time $N$ is $c_N$ is $p = P(LCZ_N = c_N | I_N)$. Then, according to the Bayes theorem:

$$p = \frac{P(I_N | LCZ_N = c_N)}{P(I_N)} \times m_{c_{N-1}, c_N} \times P(LCZ_{N-1} = c_{N-1}) \quad (5)$$

Where $P(I_N | LCZ_N = c_N)$ is the output of the model.

## III. EXPERIMENTS AND RESULTS

### A. Training datasets

The Sentinel-2 images from So2Sat dataset is used as a source dataset $D_S$. Sub-Saharan countries have several dry and rainy seasons in a year that may not be included in So2Sat dataset. Land cover, especially for vegetation, may greatly vary according to this seasonal change. To build positive pairs for the contrastive loss, we adopt a strategy similar to [10]: an image of a region of interest at time $t + \Delta t$ is considered as a temporal perturbation of an image of this specific region at
<figure><img src="image_4.png" /><figcaption>Fig. 3. Resulting LCZ map of Burkina Faso and zoom on Ouagadougou (left: Supervised model, middle: s-SSDA + Markov model, right: OpenStreetMap) for February 2018. Legend is shown in Figure 1.</figcaption></figure>

time *t*. Burkina Faso experiences one dry and one wet season each year. To create the target dataset *D*<sub>*T*</sub>, regions of interest (ROIs) such as cities, lakes, industries or national parks have been selected over Burkina Faso. Images of ROIs are extracted at two different times: January to March 2022 for the dry season, and late August to November 2021 for the wet season. Input images are created by splitting ROIs from both date into 32 × 32 × 10 patches (height × width × spectral bands) to match the So2Sat template. This selection process results in 225K pair of patches used during unsupervised training.

## B. Experimental settings

In this paper, we use Resnet-50 [5] as our classifier. The model was first trained on the entire So2Sat dataset. This pre-training is used to initialize weights for the semi-supervised training step. We used Adam optimizer with a learning rate of 0.001. We set the batch size to 256 for both the supervised and unsupervised phases, the temperature for the contrastive loss $\tau$ to 0.5 and the regularization term $\alpha$ to 0.9.

## C. LCZ map generation and ablation study

We collected Sentinel-2 tiles (bands at 10m and 20m resolution) over Burkina Faso for early 2018 and early 2017 with a cloud percentage under 5%. After training our model with the training strategy described in section II, we performed a LCZ classification on each of the Sentinel tiles followed by a Markov Chain. The resulting map (resolution 320m × 320m upsampled to the image resolution, 10m × 10m), in Figure 3, is a concatenation of the generated tiles using the Markov Chain. Houses surrounded by vegetation may not be detectable, especially if building materials are light and similar to the environment (e.g. wood). In that case, the type of environment structure does not change and remains very rural. Such ambiguities may not occur for larger villages and cities. To validate our method, we collected 4 early 2018 Sentinel-2 images and cropped them over Ouagadougou, Bobo-Dioulasso, Fada-Ngourma and Ouahigouya. We gridded these resulting images according to 32 × 32 pixels areas

<table><caption>TABLE I
VALIDATION RESULTS ON 494 MANUALLY LABELED PATCHES.</caption><thead><tr><th></th><th>OA</th><th>AA</th><th>IoU</th></tr></thead><tbody><tr><td>Supervised baseline</td><td>0.245</td><td>0.270</td><td>0.175</td></tr><tr><td>s-SSDA</td><td>0.427</td><td>0.402</td><td>0.278</td></tr><tr><td>s-SSDA + Markov</td><td>0.561</td><td>0.538</td><td>0.389</td></tr></tbody></table>

and labeled them using very high resolution satellite images. Overall accuracy (OA), Average accuracy (AA) and Intersection over Union (IoU) for a model trained only on So2Sat, a model trained using the s-SSDA method and its regulation using a Markov chain are shown in Table I. Both s-SSDA models outperforms the supervised baseline in our validation set. In addition to increasing the results metrics, the temporal regularization adds continuity.

# IV. APPLICATION: MALARIA INDICATOR

## A. Malaria data

The objective of the MIS 2017-2018 was to estimate basic demographic and health indicators about malaria and knowledge related to this disease. Interviewed households were sampled so that results on malaria prevalence for 6-59 months old children are representative for each of the 17 study areas. As a result, 252 Enumeration Zones (EZs) were selected. 245 EZs were visited and the remaining 7 EZs could not be visited for safety reasons. In total, 6322 households were visited. Malaria rapid tests, giving results in 15 minutes were performed on children with the consent of their legal representatives. When positives, more reliable tests in laboratory were done to confirm the rapid test results and determine the specific parasite involved. We computed the malaria rate of each EZ by dividing the number of positive 6-59 months old children by the total number of 6-59 months old children.

## B. Linking EZs to Local Climates Zones

Besides general information about each EZ, the survey provides the geo-location of their centroids, with a random offset added for confidentiality reasons. This displacement is randomly generated so that it is within a circle of 2 kilometers radius for urban areas, and 5 kilometers for rural EZs except for 1% of them with an offset that can reach under 10 kilometers. Within the 245 visited EZs, 224 were accurately geo-located and used to conduct the following study. Although approximate, these centroids provided with MIS data can be used to link EZs, and thus households, to their local environment using the LCZ classification, as shown in Figure 3. Each geo-location is associated to a squared area of S pixels (Sx10m x Sx10m squares). S is intentionally small against the offset considering values in a small area near the coordinates does not alter the results [11]. We set S = 64 which corresponds to a ground resolution of 640mx640m. We characterise the environment by the counts (in the 10m × 10m resolution map) of each LCZ class within this area.
<figure><img src="image_5.png" /><figcaption>Fig. 4. Mean LCZ distribution of clusters with clusters indices from Figure 1</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 5. Malaria rates and Built population index of each cluster of EZs</figcaption></figure>

### C. Linking malaria to Local Climates Zones

To study the link between malaria and LCZ, we first grouped EZs in clusters according to their LCZ distributions. Mean distributions for these clusters are shown in Figure 4. They represent different types of environment, with distributions gradually shifting from rural areas to urban classes. Cluster 1 is mostly made of bush/scrub and scattered tree areas without urban areas. Cluster 2 is dominated by bush/scrub, sparsely built areas and few low plants. Cluster 3 is halfway between urban and rural areas with sparsely built areas, compact low-rise, open low-rise and also bush/scrub. Cluster 4 is mostly made of urban areas with high densities, and few rural classes. We secondly linked these clusters with their associated malaria rates computed using MIS data. Figure 5 shows the distribution of malaria rates within each cluster of EZs, with their corresponding Built population index provided by the MIS survey. The Built population index is produced by the Global Human Settlement Layer (GHSL) from satellite image texture, morphology and patterns [8]. It is the ratio of occupied footprint, in each cell. As expected when only considering LCZ distributions, clusters represent different environmental structures with different urbanisation rates. Interestingly, malaria rates are not equivalent for different clusters in both urban and rural areas. Bush/scrub areas have similar but lower malaria rates than mixed vegetation areas. The LCZ classification enables distinguishing different types of areas where the propagation of malaria is higher.

## V. CONCLUSION

In this paper, we introduce a seasonal-SSDA strategy for mapping sub-Saharan countries using the Local Climate Zones classification scheme. This strategy is based on one supervised phase using the So2Sat dataset for training the model to classify LCZ, and a second strategy using consistency regular-

ization using unlabeled images from 2 different seasons (wet and dry) and the contrastive loss. Produced maps are regularized using a Markov chain to ensure temporal continuity. This strategy enables generating maps at any time to match them with demographic surveys period. Finally, we linked the produced Burkina Faso's LCZ map of early 2018 and malaria rates. Our results allow to differentiate several environmental structures according to their malaria rates.

## REFERENCES

[1] I.D. Stewart and T.R. Oke, "Local Climate Zones for Urban Temperature Studies," Bulletin of the American Meteorological Society, vol. 93, pp. 1879-1900, 2012.

[2] X.X. Zhu, J. Hu, C. Qiu, Y. Shi, J. Kang, L. Mou et al., "So2Sat LCZ42: A Benchmark Data Set for the Classification of Global Local Climate Zones", IEEE GRSS Magazine, vol. 8, pp. 76-89, 2020.

[3] M. Demuzere, J. Kittner, A. Martilli, G. Mills, C. Moede, I.D. Stewart et al., "A global map of Local Climate Zones to support earth system modelling and urban scale environmental science", Earth System Science Data Discussions, vol. 14, pp.3835-3873, 2022.

[4] X.X. Zhu, C. Qiu, J. Hu, Y. Shi, Y. Wang, M. Schmitt et al., "The urban morphology on our planet - Global perspectives from space", Remote Sensing of Environment, vol. 269, 2022.

[5] K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition", 2016 IEEE CVPR, pp. 770-778, 2016.

[6] M. Schmitt, L. Hughes, C. Qiu and X. Zhu, Xiao, "SEN12MS - A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusion", ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. IV-2/W7, pp 153-160, 2019.

[7] G. Sumbul, M. Charfuelan, B. Demir and V. Markl, "BigEarthNet: A Large-Scale Benchmark Archive for Remote Sensing Image Understanding", IEEE IGARSS, pp. 5901-5904, 2019.

[8] M. Pesaresi, D. Ehrlich, A. Florczyk, S. Freire, A. Julea, T. Kemper et al. "GHS built-up grid, derived from Landsat, multitemporal (1975, 1990, 2000, 2014).", European Commission, Joint Research Centre, 2015.

[9] T. Chen, S. Kornblith, M. Norouzi and G. Hinton, "A Simple Framework for Contrastive Learning of Visual Representations", Proceedings of the 37th ICML, vol. 119, pp. 1597-1607, 2020.

[10] O. Mañas, A. Lacoste, X. Giró-i-Nieto, D. Vazquez and P. Rodríguez, "Seasonal Contrast: Unsupervised Pre-Training From Uncurated Remote Sensing Data", Proceedings of ICCV, pp. 9414-9423, 2021.

[11] K. Grace, N.N. Nagle, C.R. Burgert-Brucker, S. Rutzick, D.C. Van Riper, T. Dontamsetti et al., "Integrating Environmental Context into DHS Analysis While Protecting Participant Confidentiality: A New Remote Sensing Method", Population and Development Review, pp. 197-218, 2019.

[12] O. Brousse, S. Georganos, M. Demuzere, S. Dujardin, M. Lennert, C. Linard et al., "Can we use Local Climate Zones for predicting malaria prevalence across sub-Saharan African cities?", Environmental Research Letters, 2020.

[13] M. Demuzere, B. Bechtel and G. Mills, "Global transferability of local climate zone models", Urban Climate, vol. 27, pp. 46-63, 2019.

[14] A. Singh, "CLDA: Contrastive Learning for Semi-Supervised Domain Adaptation", NeurIPS, vol. 34, pp. 5089-5101, 2021

[15] Institut National de la Statistique et de la Démographie et al., "Enquête sur les indicateurs du paludisme au Burkina Faso", 2018, [Online] Available: https://dhsprogram.com/pubs/pdf/MIS32/MIS32.pdf