# Interpretability of a multi-source Land Use classification machine learning model using a SHAP approach

Martin Cubaud, Arnaud Le Bris, Laurence Jolivet, Ana-Maria Olteanu-Raimond

<figure><img src="image_1.png" /></figure>

## To cite this version:

Martin Cubaud, Arnaud Le Bris, Laurence Jolivet, Ana-Maria Olteanu-Raimond. Interpretability of a multi-source Land Use classification machine learning model using a SHAP approach. Conférence Spatial Analysis and GEOmatics (SAGEO) 2025 - La Science de l'Information Géographique dans tous ses états, May 2025, Avignon, France. hal-05351684

HAL Id: hal-05351684

https://hal.science/hal-05351684v1

Submitted on 6 Nov 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Interpretability of a multi-source Land Use classification machine learning model using a SHAP approach

Martin CUBAUD, Arnaud LE BRIS,
Laurence JOLIVET, Ana-Maria OLTEANU-RAIMOND

Univ Gustave Eiffel, ENSG, IGN, LASTIG, F-77420 Champs-sur-Marne,
France

(martin.cubaud,arnaud.le-bris,laurence.jolivet,ana-maria.raimond)@ign.fr

RÉSUMÉ. Une connaissance millésimée et actualisée de l'Usage des Sols est essentielle pour la planification urbaine et la gestion des ressources naturelles. Cependant, l'intégration de données multi-sources et l'interprétation des modèles d'apprentissage automatique dans ce contexte restent des défis majeurs. Cet article propose une méthode s'appuyant sur la méthode SHAP pour améliorer l'interprétabilité des modèles d'apprentissage automatique d'Usage des Sols en analysant l'impact des différentes sources sur le résultat. Cette dernière permet d'expliciter les contributions des plus de 250 attributs utilisés dans un processus de classification basé sur des données multi-sources (et multi-modales). Nos résultats montrent l'importance relative de différents types d'attributs, ainsi que leurs rôles pour des classes d'usage spécifiques. L'analyse des valeurs SHAP permet de mieux comprendre les interactions entre les différentes sources de données, de proposer des pistes d'amélioration pour le modèle et facilite la reproductibilité du processus dans d'autres zones géographiques avec des sources disponibles différentes.

MOTS-CLÉS : Classification de l'Usage des Sols, explicabilité de l'apprentissage automatique, fusion de données, valeurs SHAP

KEYWORDS: Land Use classification, explainable machine learning, data-fusion, SHAP values
# 1. Introduction

Land Use (LU) classifies the socio-economic activities of an area (e.g., agricultural, residential), and in conjunction with Land Cover (LC) is crucial for monitoring urban development and effectively managing land and natural resources. Despite the advancements in machine learning, automatic LU classification out of remote sensing data remains a challenge. However, better accuracies are achieved by recent approaches exploiting multi-source data such as imagery, LC, topographic data or Volunteered Geographic Information (VGI) (Tu et al., 2020; Liu et al., 2021; Li et al., 2023). Notably, (Cubaud et al., 2024a) developed an XGBoost model integrating attributes from diverse sources, achieving robust performances across heterogeneous rural and urban study areas.

Interpreting interactions within machine learning models built from multi-source data is challenging but crucial to understand how individual predictions are made, the impact of each source or attribute on performance, and how the models can be improved. It is particularly important in LU classification, as it ensures transparency, supports decision-making, and fosters trust in the automated workflows. Recent works on machine learning explainability have proposed various methods to address these challenges. For instance, (Lundberg, Lee, 2017) introduced SHapley Additive exPlanation (SHAP) values, which provide a consistent, mathematically grounded way to attribute the contribution of each input feature to a model's predictions. SHAP has shown advantages over other explainability techniques, such as LIME (Ribeiro et al., 2016) or feature permutation methods, by offering a unified framework that supports both local (individual predictions) and global (dataset-wide) interpretability. SHAP handles interactions among features, a key requirement for analyzing multi-source datasets as used in LU classification.

To address the need for explainability in LU classification workflows, this paper proposes a method for interpreting SHAP values to provide deeper insights into feature importance, enabling an improved understanding of the contribution of diverse data sources and attributes to the model's predictions.

# 2. Materials and methods

## 2.1. LU classification workflow and used data sources

Given a partition of a study area into polygons, the workflow described in (Cubaud et al., 2024a) aims to assign to each a unique LU class. The OCS GE (Occupation des Sols à Grande Echelle) produced by IGN was used as LU ground truth. The workflow has four steps. First, 254 attributes are constructed from 9 data sources of different natures: the intersection of the LU polygons with BD ORTHO, CLC, OSO, BD TOPO, RPG, INSEE, Land Files and OSM, and the geometry of the LU polygons themselves. The dataset with the ground truth and the constructed attributes is published in (Cubaud et al., 2024b). In
this paper, the attributes are classified into three types: explicit, implicit and neighboring. The first type provides explicit information about LU classes, i.e., their name or values directly reference one or more LU classes (e.g., number of residential buildings). The second provides implicit information that the model may use to distinguish the classes (e.g., area of the polygon). The neighboring attributes average the values of each attribute in the adjacent polygons. The second step involves some preprocessing: attributes normalization, an 80%-20% train-test split, and balancing the classes in the train set using random undersampling and the SMOTENC algorithm. Third, an XGBoost model is trained on all attributes. Finally, the performances of the model are evaluated on the test set. In (Cubaud et al., 2024a), the LU classification achieved a mean F1 score of 86% for 11 classes in the department of Rhône (2020).

## 2.2. Interpretability proposed method based on SHAP

In the context of multi-class classification, the objective of SHAP values (Lundberg, Lee, 2017) is to explain, for each sample of the test set and for each class, the extent to which each attribute contributes positively or negatively to the likelihood of that class being predicted as the sample's class.

Let $z_{i,k}$ be the logits of class $k$ for the sample $i$, linked to the probabilities of class $k$ by $p_{i,k} = \frac{e^{z_{i,k}}}{\sum_{k'=1}^{K} e^{z_{i,k'}}}$, with $K$ the number of classes. Let $x_i$ be the value of an attribute $X$ for sample $i$ and $F$ the set of all attributes:

$$SHAP(i, k, X) = \sum_{S \subseteq F \setminus \{X\}} \omega(S) \mathbb{E}[z_k | X' = x'_i \forall X' \in S \cup \{X\}] - \mathbb{E}[z_k | X' = x'_i \forall X' \in S] \quad (1)$$

with $\omega(S)$ a weight depending on the size of $S$. Thus, $SHAP(i, k, X)$ quantifies how much $x_i$ increases the logits and probability of class $k$ relative to a baseline.

SHAP values are aggregated in several ways to extract meaningful information. The importance of an attribute $X$ for a class $k$ can be computed by:

$$SHAP(k, X) = \frac{1}{N} \sum_{i=1}^{N} |SHAP(i, k, X)| \quad (2)$$

The total importance of an attribute can then be computed as:

$$SHAP(X) = \sum_{k=1}^{K} SHAP(k, X) = \frac{1}{N} \sum_{i=1}^{N} \sum_{k=1}^{K} |SHAP(i, k, X)| \quad (3)$$

To interpret the influence of each attribute, we propose the following method: (1) Formalize hypotheses for each attribute on how it will affect the predictions, based on its type and on the values it takes. For instance, we hypothesize that explicit attributes will primarily contribute to their associated
LU classes; (2) Compute the SHAP values to verify these hypotheses, by aggregating the values globally and per class and analyzing attributes ranking; (3) Analyze real-life examples of individual good and bad classifications.

### 3. Results and discussion

The proposed method yields several types of results, including individual explanation, insight into the model's strengths and weaknesses, and global and class-specific attribute importance. The latter are presented here as an example. Figure 1 presents the SHAP values for the 15 most important attributes with respect to the LU classes. The width of the link between the attributes and the classes is proportional to SHAP(class, attribute) values.

<figure><img src="image_3.png" /><figcaption>FIGURE 1. SHAP values between the 15 most important attributes and the LU classes. (N) indicates neighboring attributes.</figcaption></figure>

Each source has at least one attribute among the 50 most important ones, except RPG (ranked 77th). Some attributes are not used at all (e.g., points from OSM). Most of the implicit attributes have higher SHAP values than most of the explicit ones. This can be explained by the fact that implicit attributes contribute to more classes. Although some neighboring attributes are among the most important attributes (e.g., neighborhood of commercial areas or of railways), 76% of the neighboring attributes have a lower SHAP value than the corresponding non-neighboring attribute.

The most important attribute of nearly half (7 out of 11) of the classes is an explicit attribute. For 4 out of 11 classes, it is a neighboring attribute. 66% of the attributes with explicit information for one class have their highest SHAP value for this class. The others have their highest SHAP value for another class for which the model has confusions with this class.
[ISBN: 978-2-910545-13-X, Mai 2025, Avignon]

Interpreting Land Use Classification with SHAP

## 4. Conclusion

The proposed method represents a step forward in understanding the predictions of LU classification models by explaining how each attribute contributes to the prediction of classes, depending on its type. SHAP proved to be a robust and versatile tool for interpreting complex interactions in multi-source datasets. Here, only a subset of the results are presented, but additional insights were also gained by interpreting individual predictions and by comparing the SHAP values for good and bad predictions with respect to the values of the attributes. We are extending the SHAP approach to other tasks, such as the transferability of an LU classification model between two study areas or LU change detection. Finally, we plan to work on modeling the derived knowledge in a machine-readable format to improve the reuse of the LU classification workflow.

## 5. Acknowledgment

This project was provided with computing AI and storage resources by GENCI at IDRIS thanks to the grant AD011014471R1 on the supercomputer Jean Zay's CSL partition.

## Bibliography

Cubaud M., Le Bris A., Jolivet L., Olteanu-Raimond A.-M. (2024a). Assessing the transferability of a multi-source land use classification workflow across two heterogeneous urban and rural areas., vol. 17, n° 1, p. 2376274.

Cubaud M., Le Bris A., Jolivet L., Olteanu-Raimond A.-M. (2024b). OCS GE Land Use dataset including the French departments of Gers and Rhône.

Li J., Liu H., Li J., Chen X., Tao Z. (2023). A knowledge-based approach for estimating the distribution of urban mixed land use., vol. 16, n° 1, p. 965-987.

Liu L., Olteanu-Raimond A.-M., Jolivet L., Le Bris A., See L. (2021). A data fusion-based framework to integrate multi-source VGI in an authoritative land use database., vol. 14, n° 4, p. 480-509.

Lundberg S. M., Lee S.-I. (2017). A unified approach to interpreting model predictions. In I. Guyon et al. (Eds.), *Advances in neural information processing systems 30*, p. 4765-4774. Curran Associates, Inc.

Ribeiro M. T., Singh S., Guestrin C. (2016). "Why Should I Trust You?": Explaining the Predictions of Any Classifier. In *Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining*, p. 1135-1144. Association for Computing Machinery.

Tu Y., Chen B., Zhang T., Xu B. (2020). Regional Mapping of Essential Urban Land Use Categories in China: A Segmentation-Based Approach., vol. 12, n° 7, p. 1058.