DOI: 10.5281/zenodo.11246630
title: Integrated Approach to Global Land Use and Land Cover Reference Data Harmonization

description: INTRODUCTION

This document details the process of creating a global inventory of reference samples and Earth Observation (EO) / gridded datasets for the Global Pasture Watch initiative. This inventory is used for training and validating machine-learning models for the GPW grassland mapping. This documentation outlines methodology, data sources, workflow, and results.

Keywords: Grassland, Land Use, Land Cover, Gridded Datasets, Harmonization.

OBJECTIVES





Create a global inventory of existing reference samples for land use and land cover (LULC).




Compile global EO / gridded datasets that capture LULC classes for a point or region and harmonize them to match the Global Pasture Watch classes.




Develop automated scripts for data harmonization and integration.



REPOSITORY STRUCTURE





data/: Contains raw datasets and its metadata.




workflow/: Methodology and reference ontology used for harmonizing datasets




scripts/: Includes Python codes used for downloading, processing, and harmonizing datasets, and data analysis/visualization.




results/: Final harmonized datasets in Parquet format.



 

DATA COLLECTION 

The following datasets were incorporated :





Land Change Monitoring, Assessment, and Projection (LCMap) (U.S.)




MapBiomas samples (Brazil)




Land Use/Land Cover Area Frame Survey LUCAS samples (Europe)




GeoWiki G-GLOPS training dataset (Global)




PREDICTS database samples (Global)




WorldCereal samples (Global)




DynamicWorld samples (Global)




CropHarvest samples (Global)




EuroCrops samples (Global)




Global Land Cover Mapping and Estimation (GLanCE) samples (Global)






Datasets


Spatial distribution

Time period
Number of individual samples


WorldCereal
Global
2016-2021
38,267,911


GLanCE
Global
1985-2021
31,061,694


EuroCrops
Europe
2015-2022
14,742,648


GeoWiki G-GLOPS training dataset
Global
2021
11,394,623


MapBiomas Brazil
Brazil
1985-2018
3,234,370


Land Use/Land CoverArea Frame Survey (LUCAS)
Europe
2006-2018
1,351,293


Dynamic World
Global
2019-2020
1,249,983


Land Change Monitoring,Assessment, and Projection (LCMap)
U.S. (CONUS)
1984-2018
874,836


GeoWiki 2012
Global
2011-2012
151,942


PREDICTS
Global
1984-2013
16,627


Crop Harvest
Global
2018-2021
9,714




Data files and metadata

https://drive.google.com/drive/folders/1JNncgbxoW3-OeU-QCkRNnPGC6T-0efe_?usp=sharing

 

WORKFLOW

Harmonization Process

This section details harmonizing global reference samples and EO/gridded datasets to align them with GPW classes, optimizing their integration into the GPW machine learning workflow. 

We considered reference samples derived by visual interpretation with spatial support of at least 30 m (Landsat and Sentinel), that could represent LULC classes for a point or region. 

For every dataset, we have an automated Python script to download the vector files and convert the original LULC classes into the following Global Pasture Watch Project classes:

0. Other land cover





Natural and Semi-natural grassland




Cultivated grassland




Crops and other related agricultural practices



We also empirically assigned a weight to each sample based on the original dataset's class descriptions, reflecting the level of mixture within the class. This weighting system ranged from 1 (Low) to 3 (High), with higher weights indicating a greater mixture of different landscapes within the same class. Samples with low mixture levels are more accurate and will be more effective for differentiating typologies and for validation purposes.

The harmonized dataset includes these columns:




Attribute Name
Definition


dataset_name
Original dataset name


reference_year
Reference year of samples from the original dataset


original_lulc_class
LULC class from the original dataset


gpw_lulc_class
Global Pasture Watch LULC class


sample_weight
Sample's weight based on the level of mixture of landscapes compressed within the original LULC class




 

Reclassification Process

We developed a specifically tailored grassland ontology. This ontology serves as a reference system for classifying different grassland types, facilitating the harmonization of reference samples obtained from various sources.

Our ontology likely focuses on differentiating between two primary grassland classes relevant to the project: cultivated grasslands (1) and natural or semi-natural grassland (2).

Using the ontology as a guide, we reclassified samples from original datasets. Here's how we approached the different scenarios:





Broader Definitions: For datasets with a single, broad classification level (e.g., grassland or herbaceous vegetation), we relied on the description provided within the dataset's metadata to determine the most appropriate mapping to our ontology's classes. In those cases, the level of mixture of different grassland typologies was higher, meaning the samples might not effectively differentiate between cultivated and natural grasslands.




Multiple Classification Levels: Some datasets might have more detailed classifications. For example, a class named "Temporary grass crops" would likely be reclassified as "cultivated grassland (class 1)" based on the ontology's definition; while a class like "Savannah" would be reclassified as "natural grassland (class 2)".






Level
Class
GPW class
Description


5
GRASSLAND
1 or 2
Land (and the vegetation growing on it) devoted to the production of introduced or indigenous forage for harvest by grazing, cutting, or both. The vegetation of grassland in this context is broadly interpreted to include grasses, legumes and other forbs, and at times woody species may be present


5.1
Cultivated grassland
2


Forage is established with domesticated introduced or indigenous species that may receive periodic cultural treatment such as renovation, fertilization or weed control.



5.1.1
Annual grassland
2
Forage established annually, usually with annual plants, and generally involves soil disturbance, removal of existing vegetation, and other cultivation practices.


5.1.2
Temporary grassland
2
Land on which vegetation is composed of annual, biennial, or perennial forage species kept for a short period of time (usually only a few years). It can also be integrated in a crop rotation (ley).


5.2
Natural and Semi-Natural grassland
1
Areas dominated by herbaceous vegetation (grasses, sedges, rushes) with minimal woody plant cover, shaped by natural processes (fire, grazing by wild animals) or influenced by traditional human activities (grazing livestock, mowing) that maintain the natural character of the vegetation.


5.2.1
Campos
1
Grassland consisting mainly of grasses, along with herbs, small shrubs and occasional trees; on undulating and hilly landscape, with variable soil fertility.


5.2.2
LIanos
1
Extensive system of grasslands, seasonally flooded, with infertile and acidic soils.


5.2.3
Meadow
1
A natural or semi-natural grassland often associated with the conservation of hay or silage






Full ontology here: https://docs.google.com/spreadsheets/d/1rRbM63flbizg6eP0PtogePM4K6ZnEDoFauCrV3TxZtA/edit?usp=sharing


Reclassification Example

MapBiomas






Original WorldCereal Class

Description (from Metadata)
GPW Class
Weight
Justification


Grassland
Vegetation with a predominance of graminoid herbaceous stratum, including herbaceous and subshrub dicotyledons. The botanical composition is influenced by soil and topographic gradients as well as grazing management (livestock)
1
2
Definition emphasizes influence of grazing but doesn't specify planting. Potential for natural origin.


Pasture
Predominantly planted pasture areas, directly relatedto agricultural activity. Natural pasture areas, in turn, arepredominantly characterized as grasslands or wetland, and may or may not be subjected to grazing practices.
2
1
Highlights "planted" areas for grazing. Suggests active management practices.




Look-up tables 

Datasets lookup tables https://docs.google.com/spreadsheets/d/1i9mQefIW_eOJNW29mf9EqB6BcZr1UAM0HbP9iCE1icI/edit?usp=sharing

 

Quality and Assurance Analysis

Our Quality Assurance (QA) analysis focused on data accuracy and consistency. We ensured the integrity of our dataset through the following approach:



Duplicate Removal: We identified and excluded duplicate rows. This involved checking for rows with identical coordinates and corresponding years. These duplicates likely represent redundant entries within the original dataset or within different datasets.

Outlier Detection: We screened for outliers by identifying data points with values exceeding the geographic boundaries of the original source's study area. Such values could indicate errors in data collection or processing within the original dataset.


By eliminating duplicates and outliers, we ensure that the training and validation processes rely on a clean and accurate dataset.

 

FILES

Harmonizing the Datasets

Add scripts

 

Quality Assurance Analysis

Add scrips

 

RESULTS

The final harmonized reference samples and documented processes are uploaded to Google Drive and GitHub for sharing and reproducibility.

Link to the data on Google Drive

Link to GitHub

 

ACKNOWLEDGMENTS

The development of this global inventory of reference samples and EO/gridded datasets relied on valuable contributions from various sources. We would like to express our sincere gratitude to the creators and maintainers of all datasets used in this project.

LICENSE

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. 

 

REFERENCES





Van Tricht, K. et al. Worldcereal: a dynamic open-source system for global-scale, seasonal, and reproducible crop and irrigation mapping. Earth Syst. Sci. Data 15, 5491–5515, 10.5194/essd-15-5491-2023 (2023)




Schneider, M., Schelte, T., Schmitz, F. & Körner, M. Eurocrops: The largest harmonized open crop dataset across the european union. Sci. Data 10, 612, 10.1038/s41597-023-02517-0 (2023)




Souza, C. M. et al. Reconstructing Three Decades of Land Use and Land Cover Changes in Brazilian Biomes with Landsat Archive and Earth Engine. Remote. Sens. 12, 2735, 10.3390/rs12172735 (2020)




Stanimirova, R. et al. A global land cover training dataset from 1984 to 2020. Sci. Data 10, 879 (2023)




d’Andrimont, R. et al. Harmonised lucas in-situ land cover and use database for field surveys from 2006 to 2018 in the european union. Sci. data 7, 352, 10.1038/s41597-019-0340-y (2020)




Stehman, S. V., Pengra, B. W., Horton, J. A. & Wellington, D. F. Validation of the us geological survey’s land change monitoring, assessment and projection (lcmap) collection 1.0 annual land cover products 1985–2017. Remot Sensing environment 265, 112646, 10.1016/j.rse.2021.112646 (2021).




Tsendbazar, N. et al. Product validation report (d12-pvr) v 1.1 (2021)