Product Validation Report(D12-PVR)
Document Ref:
Version:Contract No:
Creation Date:
Last Modified:WorldCover_PVR_v1.1
1.1
4000128231/19/I-LG
2021-06-21
2020-10-22

Submitted byWorldCover

Product Validation Report | V1.1

Page | 2

DOCUMENT RELEASE SHEET
Name

Signature

Book Captain

Nandika Tsendbazar (WU)

Contributing Authors

Linlin Li (WU)
Myke Koopman (WU)
Sarah Carter (WU)
Martin Herold (WU)
Ivelina Georgieva (IIASA)
Myroslava Lesiv (IIASA)
Ruben Van De Kerchove (VITO)
Olivier Arino
PublicApproval (internal)
Approval (ESA)
DistributionDate
10-22-2021

10-22-2021

DOCUMENT CHANGE LOG
Issue

Issue datePages affected

Relevant information

0.1
1.0

2021-08-31
2021-10-15

All
All

1.1

2021-10-22

All

First draft on statistical validation
Updates on statistical validation and othersections are added.
Minor changes after review

WorldCover

Product Validation Report | V1.1

Page | 3

Table of Content
TABLE OF CONTENT................................................................................................................................... 3
LIST OF FIGURES ........................................................................................................................................ 5
LIST OF TABLES .......................................................................................................................................... 6
APPLICABLE DOCUMENTS ......................................................................................................................... 7
REFERENCE DOCUMENTS .......................................................................................................................... 7
ACRONYMS AND ABBREVIATIONS ............................................................................................................ 8
1.

INTRODUCTION ................................................................................................................................. 9
PROJECT OBJECTIVES AND APPROACH .......................................................................................... 9
PURPOSE OF THE DOCUMENT ....................................................................................................... 9
CONTENT OF THE DOCUMENT ..................................................................................................... 102.

METHODS ........................................................................................................................................ 11
STATISTICAL ACCURACY ASSESSMENT ......................................................................................... 11
2.1.1.

CGLS-LC validation dataset ........................................................................................................ 122.1.2.

Adopting the CGLS Validation dataset for validating the WorldCover 2020 product ............... 14

2.1.3.

Accuracy estimation of the WorldCover 2020 product ............................................................. 16

COMPARISON OF GLC MAPS ....................................................................................................... 16
2.2.1.

GLC products used for comparison ........................................................................................... 16

2.2.2.

Visual comparison ..................................................................................................................... 19

SPATIAL ACCURACY ASSESSMENT ............................................................................................... 193.

2.3.1.

Definition and input data sets ................................................................................................... 19

2.3.2.

Geographically weighted logistic regression ............................................................................. 19

RESULTS........................................................................................................................................... 21
STATISTICAL ACCURACY ASSESSMENT ......................................................................................... 21
VISUAL COMPARISON WITH EXISTING PRODUCTS. ..................................................................... 223.3. SPATIAL ACCURACY ASSESSMENT ................................................................................................... 29
4.

LIMITATIONS ................................................................................................................................... 29

WorldCover

Product Validation Report | V1.1

Page | 4

5.

CONCLUSIONS ................................................................................................................................. 31

6.

REFERENCES .................................................................................................................................... 32

ANNEX .................................................................................................................................................... 34

WorldCover

Product Validation Report | V1.1

Page | 5

List of Figures
Figure 1: Spatial distribution of all validation sample sites for different (sub) continents ............................... 13
Figure 2: Screenshot of the Geo-Wiki based interface for validation data collection ...................................... 14
Figure 3: A screenshot of an example sample interpretation (green – trees, orange – shrubs, yellow –grassland) .......................................................................................................................................................... 14
Figure 4. Snippets of validation data locations depicting shifts between the images sourced from Google map
and Bing map. .................................................................................................................................................... 16
Figure 5 The legend of the WorldCover 2020 product ...................................................................................... 23
Figure 6 Comparison of Esri 2020 landcover (A), FROM-GLC10 (B), WorldCover 2020 (C), and Google Earth (D)for urban areas. 1. New York City (Coordinates: 40°46'26.86"N 73°58'20.57"W), 2. Singapore (Coordinates:
1°22'18.0"N 103°57'22.2"E), see Figure 5 for legend........................................................................................ 23
Figure 7 Comparison of Esri 2020 landcover (A), FROM-GLC10 (B), WorldCover 2020 (C) and Google Earth (D)for wetland areas.1.Russia (Coordinates: 62°20'46.6"N 69°23'26.3"E) 2. Zambia (Coordinates: 16°09'24.2"S23°14'12.5"E), see Figure 5 for legend .............................................................................................................. 24
Figure 8 Comparison of Esri 2020 landcover (A), FROM-GLC10 (B), WorldCover 2020 (C) and Google Earth (D)for areas with tree cover/ mangrove. 1. Malaysia (Coordinates: 2°24'36.8"N 111°31'00.1"E) 2. Suriname(Coordinates: 5°52'38.6"N 56°53'30.7"W) 3. Australia (Coordinates: 32°47'07.9"S 116°37'42.0"E ), see Figure
5 for legend ....................................................................................................................................................... 25
Figure 9 Comparison of Esri 2020 landcover (A), FROM-GLC10 (B), WorldCover 2020 (C) and Google Earth (D)for dryland areas. 1. Mongolia (Coordinates: 48°02'26.6"N 92°35'13.0"E) 2. Argentinia (Coordinates:
45°47'27.8"S 69°30'28.6"W), see Figure 5 for legend ....................................................................................... 25
Figure 10 Comparison of Esri 2020 landcover (A), FROM-GLC10 (B), WorldCover 2020 (C) and Google Earth(D) for cropland areas. 1. USA (Coordinates: 35°11'41.5"N 119°06'20.8"W ) 2.Uganda (Coordinates:
0°22'20.9"N 33°00'19.4"E ), see Figure 5 for legend ......................................................................................... 26
Figure 11 Comparison of Esri 2020 landcover (A), FROM-GLC10 (B), WorldCover 2020 (C), and Google Earth(D) for a snow/ice area in Jostedalsbreen National Park, Norway. (Coordinates: 61°43'59.9"N 7°18'19.3"E),
see Figure 5 for legend ...................................................................................................................................... 26
Figure 12 Comparison of Esri 2020 landcover (A), FROM-GLC10 (B), WorldCover 2020 (C), and Google Earth(D) for Lake Lefroy, Australia. (Coordinates: 31°18'46.5"S 121°43'30.1"E), see Figure 5 for legend................ 27
Figure 13. Comparison of the CGLS-LC100 2019 and WorldCover 2020 products (A: Area east of Edmonton,
Canada, B: Montpellier and surroundings in France, C: Port Harcourt, Nigeria, D. Area west of Maracaibo,
Venezuela), see Figure 5 for legend .................................................................................................................. 28
Figure 14 Spatial accuracy layer. Note: results are highly driven by the input reference data set and may not
capture all the mistakes .................................................................................................................................... 29
Figure 15 Examples of artefacts in the WorldCover 2020 product with sharp boundaries between land covertypes (A) Nigeria -Coordinates: 12°58'05.2"N 8°24'32.6"E; (B) Western Mongolia- Coordinates 50°39'05"N
93°31'32"E; (C) Canada - Coordinates 57°45'40"N 94°10'51"W; see Figure 5 for legend ................................ 30

WorldCover

Product Validation Report | V1.1

Page | 6

List of Tables
Table 1: CEOS WGCV stage 3 and 4 validation requirements http://lpvs.gsfc.nasa.gov/ ................................. 12
Table 2. Definition of land cover types in FROM-GLC10 product that is based on FROM-GLC30 legend (Gonget al. 2013)......................................................................................................................................................... 17
Table 3. Land use/cover class definitions of the Esri 2020 Land Cover............................................................. 17
Table 4: Confusion matrix (%) for the WorldCover 2020 product at a global scale, corrected by sample
inclusion probabilities. ...................................................................................................................................... 21
Table 5: Overall accuracy for the WorldCover 2020 product at the continental level and sample sites used
(SSU- secondary sample units, PSU – primary sample units) ............................................................................ 22
Table 6. Confusion matrix (%) for the WorldCover 2020 product for Africa, corrected by sample inclusion
probabilities. ...................................................................................................................................................... 34
Table 7. Confusion matrix (%) for the WorldCover 2020 product for Europe, corrected by sample inclusion
probabilities. ...................................................................................................................................................... 34
Table 8. Confusion matrix (%) for the WorldCover 2020 product for Eurasia, corrected by sample inclusion
probabilities. ...................................................................................................................................................... 35
Table 9. Confusion matrix (%) for the WorldCover 2020 product for Asia, corrected by sample inclusion
probabilities. ...................................................................................................................................................... 35
Table 10. Confusion matrix (%) for the WorldCover 2020 product for Oceania, corrected by sample inclusion
probabilities. ...................................................................................................................................................... 36
Table 11. Confusion matrix (%) for the WorldCover 2020 product for North America, corrected by sample
inclusion probabilities. ...................................................................................................................................... 37
Table 12. Confusion matrix (%) for the WorldCover 2020 product for South America, corrected by sample
inclusion probabilities. ...................................................................................................................................... 37

WorldCover

Product Validation Report | V1.1

Page | 7

Applicable Documents[AD 1][AD 2][AD 3][AD 4]The General Clauses and Conditions for ESA Contracts (herein referred to as GCC),
reference ESA/REG/002, rev. 2 not attached hereto but known to both Parties and available
on http://emits.sso.esa.int/emits/owa/emits.main ) “reference documentation” –“administrative documents”, as amended by the Draft Contract
The Statement of Work, ref. EOP-SD-SOW-2018-030, Issue 1, Revision 0, dated 15
November 2018;
The Standard Requirements for Management, Reporting, Meetings and Deliverables (rev 3:
2015-11) and its Annex A: Layout for Contract Closure Documentation (in its latest version)
The Contractor’s Proposal reference TAP/16974/060219, issue 1, Revision 0, dated
06/02/2019

Reference Documents
For clarity, all references used are in the text using the APA style. A bibliography is provided at the end of the
document.

WorldCover

Product Validation Report | V1.1

Acronyms and Abbreviations
AD
CCI
CEOS
CGLS
ESA
GLC
GOFC-GOLD
GWR
IIASA
LCCS
LPV
MMU
PA
PSU
PVR
RB
RD
SDGs
SoW
SSU
UA
URD
UTM
VHR
WGCV

Applicable Document
Climate Change Initiative
Committee on Earth Observation Satellites
Copernicus Global Land Service
European Space Agency
Global land cover
Global Observation of Forest Cover and Land Dynamics
Geographically Weighted Regression
International Institute for Applied Systems Analysis
Land Cover Classification System
Land Product Validation
Minimum Mapping Unit
Producer Accuracy
Primary Sampling Unit
Product Validation Report
Requirement Baseline
Reference Document
Sustainable Development Goals
Statement of Work
Secondary Sampling Unit
User Accuracy
User Requirement Document
Universal Transverse Mercator
Very High Resolution
Working Group on Calibration and validation

Page | 8

WorldCover

Product Validation Report | V1.1

Page | 9

1. Introduction
Project objectives and approach
In an ever-changing environment, the need for accurate, timely, and high-resolution information onland use/land cover and its changes has increased tremendously. However, until now, regional orcontinental land cover maps either used low-resolution images (>100 m) as input or were based
exclusively on high-resolution optical Earth observation data, such as Sentinel-2 or Landsat. The useof SAR data such as Sentinel-1 to produce large-area land cover maps is still in its infancy.
For this purpose and inspired by the 2017 WorldCover conference (http://worldcover2017.esa.int,
attended by more than 400 participants), the European Space Agency (ESA) initiated the WorldCoverproject. The key outcome of this project was the release in October 2021 of a freely accessible global
land cover (GLC) product at 10 m resolution for 2020, based on both Sentinel-1 and Sentinel-2 data,containing 11 land cover classes and independently validated with a global overall accuracy of 74.4%.
A crucial aspect for WorldCover was the involvement of several end-users active in different domains
who provided primary input for all engineering aspects and followed the whole project workflow fromdesign up to validation and uptake. Consequently, WorldCover intends to provide a substantial benefit
to various user communities and expands the established GLC base of users and the development ofnovel services.

Purpose of the document
The Product Validation Report (PVR) is a deliverable of contract N°4000128231/19/I-LG. The objectiveof this document is to describe the validation results of the WorldCover 2020 product. The validation
results include statistical accuracy assessment, map comparison, and spatial accuracy assessment.

WorldCover

Product Validation Report | V1.1

Page | 10

Content of the document
This document is structured as follows: Section 2 details the methods for validating the WorldCover 2020 product. Here, the methods
used for statistical accuracy assessments, map comparison, and spatial accuracy are detailed.
 Section 3 details the main results on validation of the WorldCover 2020 product, with details
on qualitative map comparisons and statistical accuracy assessment.
 The observed limitations of the WorldCover 2020 product are included in Section 4.
 The final conclusions of the validation results are included in Section 5.

WorldCover

Product Validation Report | V1.1

Page | 11

2. Methods
Validation is an intrinsic part of satellite-derived land product generation as it informs on the productquality, consistency and builds the product users' confidence in using the land product. In addition,
validation should inform on the “fitness for use” to a variety of end-users that use land products.
Information on these aspects of a map can be achieved through map validation assessments.
In this project, we performed the following steps to provide information on map quality, consistency,
and fitness for use:

Statistical accuracy assessment following Committee on Earth Observation Satellites (CEOS)
Working Group on Calibration and Validation (WGCV) Land Product Validation (LPV)
requirements.
Visual comparison with other products

 Spatial uncertainty assessment.
Statistical accuracy assessment aimed to provide information on the product accuracy using astatistically rigorous method which is based on independent validation data selected using probability
sampling. Here we made use of the world’s most advanced (CEOS-WGCV Stage 4) and multi-purpose
independent GLC validation systems that have been developed, scientifically published (Tsendbazar
et al. 2018), and operationally used and regularly updated as part of the Copernicus Global Land
Service (CGLS).
For comparison, we assessed visually two other 10m resolution GLC maps and a 100m resolution
Copernicus GLC map. Spatial uncertainty assessment was carried out using more than 200 000
reference data locations. In the following subsection, we detail methods used to conduct theassessments towards the validation of the WorldCover 2020 product.

Statistical Accuracy Assessment
The statistical accuracy assessment of the WorldCover 2020 product follows closely the best practices
guidelines for GLC validation supported by the international communities such as the Global
Observation of Forest Cover and Land Dynamics (GOFC-GOLD) and the Committee on Earth
Observation Satellites (CEOS).
The “validation” is defined according to the definition of the CEOS Working Group on Calibration andValidation (CEOS-WGCV):“The process of assessing, by independent means, the quality of the data products derived from thesystem outputs”
The statistical accuracy assessment of the WorldCover 2020 product meets the requirements of the
CEOS WGCV Validation Stage 3, while we also have capabilities to meet the Validation Stage 4 in casethe product is updated (Table 1: CEOS WGCV stage 3 and 4 validation requirements
http://lpvs.gsfc.nasa.gov/).

WorldCover

Product Validation Report | V1.1

Page | 12

Table 1: CEOS WGCV stage 3 and 4 validation requirements http://lpvs.gsfc.nasa.gov/
Stage 3

Uncertainties in the product and its associated structure are well-quantified fromcomparison with reference in situ and higher resolution airborne and satellite data.
Uncertainties are characterized in a statistically robust way over multiple locations andtime periods representing global conditions. The spatial and temporal consistency of the
product and consistency with similar products has been evaluated over globally
representative locations and periods. Results are published in the peer-reviewed literature.

Stage 4

Validation results for stage 4 are systematically and operationally updated by independent
actors for comparative assessment of existing products when new products are released
and as the time-series expands.

The multi-purpose Global Land Cover Validation dataset developed for the Copernicus Global Land
Service- Land Cover product (henceforth called CGLS-LC validation dataset) was used. The following
characteristics of the CGLS-LC validation dataset made it suitable to assess statistically the WorldCover2020 product meeting the requirements mentioned above:
1. A global stratification that is independent of any land cover maps and uses the Sentinel 2
Universal Transverse Mercator (UTM) grid as a geographic base
2. Globally more than 21 000 primary sampling units (PSUs) with each containing one hundred
10x10 m reference pixels, are suitable for assessing accuracy per continent (minimum of 3000
PSUs per continent).
3. Stage 4 validation dataset: the PSUs are updated every year by focusing on areas that went
under change since 2015 as part of the CGLS (until 2019).
4. A multi-purpose validation data suitable for validating 10m land cover maps, which allows
the validation for WorldCover 2020 product.
5. A dataset containing high-quality land cover elements information at 10m resolution
contributed and reviewed by more than 30 international and regional experts from aroundthe world.
6. The 10m land cover element information fully corresponds to Sentinel-2 10m resolution
pixels and is collected on the Geo-Wiki reference data collection platform
The following subsections include a description of the CGLS-Validation data including a wellestablished method to estimate map accuracies with statistical rigorousness for validating theWorldCover 2020 product.

2.1.1.

CGLS-LC validation dataset

The CGLS-LC validation dataset is based on probability sampling to allow a design-based inference ofmap accuracies. The criterion of statistical probability sampling with known and non-zero inclusion
probabilities was followed. The validation dataset is based on stratified random sampling, employing
a global stratification (Olofsson et al. 2012). Globally there are 149 strata divided over seven(sub)continents. The validation dataset consists of 21 752 PSUs. The available PSUs per continent(minimum 3000 PSUs per continent) allows to carry out a statistical accuracy assessment for eachcontinent with a high level of precision (Tsendbazar et al. 2021). The spatial distribution of thevalidation sites and (sub)continent divisions are shown in Figure 1.

WorldCover

Product Validation Report | V1.1

Page | 13

Figure 1: Spatial distribution of all validation sample sites for different (sub) continents
The CGLS-LC validation dataset is multi-purpose data compatible for validating maps with different
resolutions and legends. Each PSU (covering an area of 100m × 100m) was divided into 10×10 small
blocks (henceforth called SSU: secondary sampling unit). Since the reference land cover elements
were collected at 10m × 10m SSU level, the dataset is compatible for assessing land cover maps with
10-100m resolutions. For the thematic representation, the generic land cover elements include trees(phenology and leaf types), shrubs, grass, crops, built-up areas, bare area, lichens/mosses, open
water, snow & ice, and regularly flooded areas were recorded at each SSU. The land cover elements
were defined according to the United Nations Land Cover Classification System (LCCS) (Di Gregorio2005). The validation data was collected using a dedicated web-interface through the Geo-Wikiplatform (Fritz et al. 2011). The interface provided access to different remote sensing data and allowedlabelling land cover (Figure 2). An example of labelling the land cover of SSUs within a PSU is provided
in Figure 3. The reference land cover was visually interpreted by 30 regional experts remotely.
The validation dataset contains reference land cover information for the years 2015-2019. Moredetailed information on the validation data can be found in Tsendbazar et al. (2021).

WorldCover

Product Validation Report | V1.1

Page | 14

Figure 2: Screenshot of the Geo-Wiki based interface for validation data collection

Figure 3: A screenshot of an example sample interpretation (green – trees, orange – shrubs, yellow –grassland)

2.1.2.

Adopting the CGLS Validation dataset for validating the WorldCover 2020 productWe used the CGLS-LC validation data for the reference year 2019 as input for validating theWorldCover 2020 product. We then updated the validation data for 2020 by following the samerevisiting procedure used in Tsendbazar et al. (2021). For this, we randomly revisited 10% of the 2019validation dataset. Further, we used the BFAST Monitor change detection algorithm to identify
validation sites with high possibility of land cover change between 2019 and 2020 (Verbesselt et al.
2010). In total, about 2860 sample sites were revisited and checked for land cover change. In case ofa change in land cover since 2019, the land cover information was updated.

WorldCover

Product Validation Report | V1.1

Page | 15

Note that the CGLS-LC validation data is used as an independent validation dataset for the CGLS-LC
mapping effort and to continue to support independent validation of maps, the dataset will not bereleased to the public.
To assess the WorldCover 2020 product, we used the SSUs that have an area of 10x10m. All 100 SSUs
within a PSU were considered for validating the WorldCover 2020 product. Thematically, thevalidation dataset is compatible with the legend of the WorldCover 2020 product other than the
mangrove class. The validation dataset includes flooded trees and shrubs. We used these sample
locations as the mangrove class. In addition, the sample sites that overlap with three mangrove
products were visually checked to confirm the locations are indeed mangroves. These datasets are
the Global distribution of Mangrove Forests (Giri et al. 2011), the Global Database of Continuous
Mangrove Forest Cover for the 21st Century (Hamilton 2015), and the Global Mangrove Watch
product (Bunting et al. 2018).
The CGLS-LC validation dataset was collected by visual interpretation of VHR images. VHR images canhave some level of geolocation errors which can result in shifts between VHR base maps. Figure 4
illustrates examples of shifts between the images sourced from Google map and Bing maps. In case ofsuch a shift, Google map locations were preferred for the CGLS-LC validation data collection. This wasdue to the larger availability of VHR maps in the Google map for recent years. However, in cases where
no Google VHR images were available, Bing map and ESRI VHR images were used, supported by
Sentinel 2 imagery. As 10m resolution is considered a high resolution for land cover maps, possible
geolocation errors between VHR images (e.g., Google map and Bing map) used for visualinterpretation can have an impact on the accuracy of land cover products. In addition, Sentinel-1 and-2 imageries used to produce the WorldCover 2020 product have a geolocation error of approximately
one pixel at 10m resolution (Clerc 2021). Therefore, there can be co-registration issues between theWorldCover 2020 product, and the validation data labels.
To account for the possible impact of geolocation errors of the VHR images used for validation datacollection and the WorldCover 2020 product, we have filtered out the SSUs that are isolated in termsof land cover. Here, SSUs having the same land cover type with at least two direct neighbours were
included and the rest were not included in the validation. In other words, if an SSU with urban land
cover type is surrounded by one urban SSU and three trees SSUs as direct neighbours, the said SSU
was not included in the validation. This filtering was done to reduce the impact of geolocation errorsof VHR images on the validation, as the impact of the geolocation error would be less in a morehomogeneous area in terms of land cover. However, considering the potential geolocation errors of
the VHR images and the Sentinel-1 and 2 data, this is a conservative approach to deal with thegeolocation error issue.
In total, 1 935 650 SSUs across the world were used for the statistical accuracy estimation of theWorldCover 2020 product.

WorldCover

Product Validation Report | V1.1

Page | 16

Figure 4. Snippets of validation data locations depicting shifts between the images sourced from
Google map and Bing map.

2.1.3.

Accuracy estimation of the WorldCover 2020 productFor the accuracy estimation, we used a stratified one-stage cluster approach (Pengra et al. 2015). This
cluster approach is suitable since multiple SSUs within the PSUs are used for the assessment.
Calculation of inclusion probabilities for the PSUs and SSUs follows the methods described in Pengra
et al. (2015) and Tsendbazar et al. (2018). Based on the estimation weight (the inverse of inclusion
probability) per sampling unit, mapped and reference land cover types, a confusion matrix wasconstructed accounting for unequal sample inclusion probabilities (Stehman et al. 2003; Wickham etal. 2010). The estimation weights for each sample unit were available for this validation dataset.
Accuracy estimates namely overall accuracies, class-specific accuracies, and their confidence intervals(at 95% confidence level) were calculated following the stratified one-stage cluster approach (Pengra
et al. 2015; Stehman et al. 2003; Tsendbazar et al. 2018).
Next to global level accuracy estimates, the accuracy estimation was also done per continent (7 subcontinents) following the initial design of the validation dataset (Figure 1).

Comparison of GLC maps
2.2.1.

GLC products used for comparison

Two available GLC products at 10m were selected for comparison with the WorldCover 2020 product.
These are FROM-GLC10 for 2017 and ESRI 2020 GLC maps.
(1) The FROM-GLC10 is a 10m GLC map for the reference year 2017, produced by Tsinghua
University, China (Gong et al. 2019). Training data consisted of 340 000 sample units locatedat around 93 000 sites worldwide. Random Forest algorithm was used to classify Sentinel-2
images acquired in 2017. FROM-GLC10 provides 10 generic land cover classes consisting ofcropland, forest, grassland, shrubland, wetland, water, tundra, impervious, barren, andsnow/ice. The map has a reported overall accuracy of 72.76% based on an independentWorldCover

Product Validation Report | V1.1

Page | 17

validation dataset consisting of 140 000 sample units at over 33 000 locations. The land coverclass definition of this product is included in Table 2.
(2) Esri 2020 Land Cover is a 10 class global land use/land cover map for the year 2020 at 10m
resolution (ESRI 2021). This product was recently produced by ESRI in collaboration with
Microsoft’s Planetary Computer and scaled using Microsoft Azure Batch. The map was
produced using Sentinel-2 images acquired in 2020. A deep learning model was trained using
5 billion hand-labelled Sentinel-2 pixels at over 20 000 sites distributed over the world. The
map depicts different land use and land cover classes, namely water, trees, grass, flooded
vegetation, crops, scrub, built area, bare, and snow/ice class. Using the validation set, the map
has a reported overall accuracy of 86% for 8 land cover classes (excluding snow/ice andclouded areas). The land use and land cover class definition of this product is included in Table
3.
Table 2. Definition of land cover types in FROM-GLC10 product that is based on FROM-GLC30 legend(Gong et al. 2013)
Classes
Cropland

Code
10

Forest20

Grasslands

30

Shrublands

40

Wetlands

50

Waterbodies

60

Tundra70

Impervious

80

Barren Land

90

Snow and ice

100

Description
Land that has clear traits of intensive human activity. It varies from bare field, seeding,crop growing to harvesting. Fruit trees are classified into forests. Pasture could betransitional from croplands to natural grasslands. Lands for rice cultivation, arable andtillage lands, greenhouse farming.
Trees observable in the landscape from the images. Parcels planted with fruit trees or
shrubs: single or mixed fruit species, fruit trees associated with permanently grassed
surfaces. Tree cover percentage classification to >15% and tree height classification to >3mGrasslands for grazing as well as natural grasslands. Herbaceous cover percentage
classification >15%
Shrub cover identifiable in the image. Has a texture finer than tree canopies but coarser
than grasslands. With height between 0.3-5m and cover percentage >15%.
Marshlands with distinctively high reflectivity in the NIR band. Low relief areas withperched bogs, playas, and patholes may also be included. Forested wetland is not included.
Aquatic and hydrophytic herbaceous plants.
All inland waterbodies. Patches of fish ponds are included in this category. Natural ordammed water bodies, natural or artificial water-courses serving as water drainage
channels and salinity water
Located at high mountains above tree line and high latitude regions with low height
vegetation. The growing season is between 1 and 2 months. Dominated by low shrubs,
grasses, lichens, and mosses at the background as well as various sedges and forbs.
Primarily based on artificial cover such as asphalt, concrete, sand and stone, bricks, glasses,
and other cover materials. Impervious road cover with high albedo materials as well as
Impervious roof tops covered by low albedo materials (e.g. asphalts, black shingles).
Lands not covered by vegetation or vegetation is hardly observable but dominated byexposed soil, sand, gravel, and rock backgrounds. Dry salt flats occurring on the flat flooredbottoms of interior desert basins; Sandy areas composed primarily of dunes; gravel landand bare rocks; other types of land not covered by vegetation such as lake/river bottoms
in dry season.
Distributed in the polar areas and high mountains. Lands under perennial or nonperennial ice or snow cover.

Table 3. Land use/cover class definitions of the Esri 2020 Land CoverLand cover
types
Water

Code

Definition

1

Areas where water was predominantly present throughout the year; may not cover areas
with sporadic or ephemeral water; contains little to no sparse vegetation, no rock outcrop
nor built up features like docks; examples: rivers, ponds, lakes, oceans, flooded salt plains.

WorldCover
Trees

2

Grass

3

Floodedvegetation

4

Crops

5

Scrub/shrub

6

Built Area

7

Bare ground8

Snow/ice

9

Clouds10

Product Validation Report | V1.1

Page | 18

Any significant clustering of tall (~15-m or higher) dense vegetation, typically with a closed
or dense canopy; examples: wooded vegetation, clusters of dense tall vegetation withinsavannas, plantations, swamp or mangroves (dense/tall vegetation with ephemeral wateror canopy too thick to detect water underneath).
Open areas covered in homogenous grasses with little to no taller vegetation; wild cereals
and grasses with no obvious human plotting (i.e., not a plotted field); examples: natural
meadows and fields with sparse to no tree cover, open savanna with few to no trees,parks/golf courses/lawns, pastures.
Areas of any type of vegetation with obvious intermixing of water throughout a majorityof the year; seasonally flooded area that is a mix of grass/shrub/trees/bare ground;
examples: flooded mangroves, emergent vegetation, rice paddies and other heavily
irrigated and inundated agriculture.
Human planted/plotted cereals, grasses, and crops not at tree height; examples: corn,
wheat, soy, fallow plots of structured land.
Mix of small clusters of plants or single plants dispersed on a landscape that shows exposed
soil or rock; scrub-filled clearings within dense forests that are clearly not taller than trees;
examples: moderate to sparse cover of bushes, shrubs, and tufts of grass, savannas withvery sparse grasses, trees or other plants
Human made structures; major road and rail networks; large homogenous impervious
surfaces including parking structures, office buildings and residential housing; examples:houses, dense villages / towns / cities, paved roads, asphalt.
Areas of rock or soil with very sparse to no vegetation for the entire year; large areas ofsand and deserts with no to little vegetation; examples: exposed rock or soil, desert andsand dunes, dry salt flats/pans, dried lake beds, mines.
Large homogenous areas of permanent snow or ice, typically only in mountain areas or
highest latitudes; examples: glaciers, permanent snowpack, snow fields.
No land cover information due to persistent cloud cover.

As detailed in Table 2 and Table 3, both the maps depict the GLC at 9-10 general land cover types
which are mostly in line with the class definition of the WorldCover 2020 product. However, there arealso discrepancies between the products in terms of the thematic definition of the land cover/ landuse definitions. Overall, the FROM-GLC10 map has a more similar thematic representation to theWorldCover 2020 product as this map is based on the LCCS. The main discrepancies are the tundraclass of the FROM-GLC10 map (Table 2) which is a mix of shrubs, grass, and lichen classes in theWorldCover 2020 product. Similarly, greenhouse farming is included in crop class while this type ofarea is included in the “built-up” class of WorldCover 2020 as per the land cover type definition.
The Esri 2020 Land Cover has discrepancies with the WorldCover 2020 product in terms of thethematic representations. Firstly, the flooded vegetation class includes rice paddies and irrigated/inundated agriculture. This type of cropland is included in the cropland class of the WorldCover 2020
product. Secondly, the scrub/shrub class includes clusters of plants that are dispersed over an area
without any specification of cover percentage. The clusters of plants can be bushes, shrubs, and tuftsof grass. The latter is included in the grassland class of the WorldCover 2020 product. Third, mangroves
are included in both trees and flooded vegetation, which makes the land cover types not mutually
exclusive.
Regardless of the discrepancies, both the products are at 10m resolution and temporally similar to theWorldCover 2020 product. Therefore, these two products were visually compared with theWorldCover 2020 product.
In addition, we visually compared the WorldCover 2020 product with the 100m resolution CGLS-LC
2019 product (Buchhorn et al. 2020). The CGLS-LC100 2019 product is a GLC map produced by the
CGLS (Buchhorn et al. 2020). This product is at 100m resolution, depicting 10 land cover types at a
generic level (Level 1), with further detailed classes related to tree cover at Level 2. The product isbased on the Proba-V satellite data.

WorldCover2.2.2.

Product Validation Report | V1.1

Page | 19

Visual comparison

We compared the WorldCover 2020 product visually with Google Earth VHR images and the two 10m
resolution GLC products (e.g. FROM-GLC10 and Esri 2020 Land Cover). The FROM-GLC10 map wasdownloaded from http://data.ess.tsinghua.edu.cn. The Esri 2020 Land Cover product was accessedfrom the Living Atlas of ESRI.
Next, a visual comparison of the WorldCover 2020 product with the CGLS-LC100 2019 product wasalso done (https://lcviewer.vito.be/2019).

Spatial accuracy assessment
Spatially explicit accuracy estimates of land cover products are of considerable value for user
communities in various fields, from biodiversity to agricultural monitoring. Well-known accuracy
values such as kappa, or overall accuracy give a general idea of the thematic accuracy of the map, but
don’t provide information on the spatial differences in map quality. Spatial accuracy estimates,
however, inform users regarding the degree of uncertainty in land cover mapping across space, thusbuilding confidence in using different land cover products. From a user’s perspective, these spatially
explicit accuracy estimates help when comparing different land cover products in order to choose thebest product for a region of interest.

2.3.1.

Definition and input data sets

We define spatial accuracy as a probability that a map is correct in a certain area. The map isconsidered to be correct if it is in agreement with a reference data set, which is assumed to be thetruth.
To calculate a spatial accuracy layer of the WorldCover 2020 map, we aggregated the map to a 100m
resolution for two reasons, (1) adjust to spatial parameters of the reference data set; (2) minimise
computational resources. The size and quality of the reference data set are very important. Therefore,
we collected the reference data set from the most reliable sources, taking into account temporal andthematic consistency. The reference data set includes the validation data set, used for the statistical
validation (section 2.1); the CGLOPS training data set at 100m (Buchhorn et al. 2020); CrowdVal
reference data set for a few African countries (IIASA 2019), and additionally collected data forGreenland and the most famous desert areas. The reference data set has its limitations that should beconsidered while interpreting the spatial accuracy layer. The first limitation is that majority of the data
has been collected for the year 2015 but the validated map is for the year 2020. The second limitationis that the CGLOPS training data set has been used in map production. However, in the map
production, the CGLOPS data has been used at 10m resolution, it has been filtered and modified
following a set of rules (described in the product user manual). So, it differs from the original data at
100m but, anyway, it adds some positive bias while interpreting the spatial accuracy layer. The
agreement between the reference data set and the WorldCover 2020 product is 72% (independentvalidation accuracy is above 74%), which shows that the mentioned above bias is small. Also, during avisual inspection, we found that the CGLOPS data contributed a lot to mapping uncertainty hotspots(or low accuracy areas), due to its higher density compared with other sources of data.
The reference data set consisted of 203 073 sample sites at 100m resolution.

2.3.2.

Geographically weighted logistic regression

To develop a spatial accuracy layer, we applied a geographically weighted logistic regression (GWR)
since it is computationally less expensive than other approaches (e.g. kriging). Logistic regression is aWorldCover

Product Validation Report | V1.1

Page | 20

commonly-used classification method and is a generalised linear model employed when the response
variable is binary. GWR estimates model parameters at each geographical location by using a distanceweighted kernel, so that the observations closer to the studied location have more influence on theparameter estimates than the observations further away (Fotheringham 2002). GWR was developed
by Brunsdon et al. (1998) as a spatial statistical method that allows regression parameters to vary overspace.
As input data, binary information if the map agrees with the reference data was used; as a searchradius, an adaptive moving kernel was applied, searching for 100 nearest neighbours, which is theminimum number of observations required to calculate logistic regression. The calculations were done
for a 1-degree grid. The resolution of the grid depends on reference data set density. In our case, thesearch radius of 100 nearest neighbour corresponds to a distance of a few degrees. This means that
calculations at finer resolutions do not provide users with more information. All calculations have been
done in R statistical environment, using such packages as spgwr, sp, and raster.

WorldCover

Product Validation Report | V1.1

Page | 21

3. Results
Statistical Accuracy Assessment
Overall accuracy estimates of the WorldCover 2020 product can be found in Table 4. On a global scale,using 1.93 million SSUs at 21 624 PSU locations, the overall map accuracy was 74.4±0.1.
In terms of class-specific accuracies, tree cover and snow/ice, cropland, water body, and bare/sparsevegetation classes had high accuracies. Grassland and built-up classes had moderate accuracies, while
shrubs, wetlands, and moss/lichen classes had lower accuracies.
In general, at the global scale, there was a slight underestimation of moss and lichen and grasslandclass while a slight overestimation of trees was observed when comparing against the validation
dataset.

Cropland

Built-up

Bare / sparse veg

Permanent water
bodiesHerbaceous
wetland

Mangroves

Moss and Lichen

Total

User's accuracy

Confidence
interval ±

25.5

2.1

3.0

0.4

0.1

0.1

0.1

0.1

0.0.

0.1

31.5

80.8

0.1

Shrubland

1.5

3.3

2.9

0.2

0.0

0.6

0.0

0.0

0.0

0.0

0.0

8.6

38.6

0.3

Grassland

1.0

1.6

16.2

1.3

0.0

1.8

0.0

0.1

0.4

0.0

0.9

23.4

69.3

0.2

Cropland

0.2

0.1

1.2

7.5

0.0.

0.1

0.0

0.1

0.0

9.2

81.1

0.2

Built-up

0.0

0.0

0.1

0.0

0.5

0.1

0.0

0.0

0.7

67.7

0.9

Bare / sparsevegetation

0.1

0.2

1.1

0.3

0.1

15.1

0.0

0.1

0.0

0.3

17.3

87.5

0.1

2.2

0.0

0.0

2.4

93.9

0.0

2.3

0.0

0.0

0.1

2.6

88.5

0.4

1.7

27.8

0.1

68.6

1.2

2.5

48.6

100.0

0.0

Snow & ice

Snow & ice

GrasslandTree cover

Tree cover

Shrubland

Table 4: Confusion matrix (%) for the WorldCover 2020 product at a global scale, corrected by sample
inclusion probabilities.

0.0

Permanent
water bodies

0.0

0.0

0.0

0.0

0.0

0.1

Herbaceous
wetland

0.0

0.1

0.6

0.0

0.0

0.0

0.1

0.5

0.0

Mangroves

0.0

0.0

0.0

0.0

0.0

0.0

0.1

Moss and
Lichen

0.0

0.1

0.7

0.0

Total

28.3

7.5

25.8

9.7

Producer's
accuracy

89.9

44.1

63.0

0.1

0.3

0.2

Confidence
interval ±

0.4

0.0

0.0

0.0

0.7

18.6

2.3

2.7

1.2

0.1

3.1

76.7

67.9

81.4

97.9

85.0

40.6

51.5

40.0

0.2

0.8

0.2

0.1

0.3

0.7

1.9

0.5

0.2
0.2
0.3
0.5
1.9
0.5

74.4
0.1

The accuracy of the WorldCover 2020 product at the continental level is listed in Table 5. Generally,
most continents had accuracies around 70% or above, except Oceania. The overall accuracy was
highest for Asia (80.7%) followed by South America, Europe, and Africa. The lower accuracy for
Oceania could be due to high shrubland and grassland confusions in the open shrublands of Australia.

WorldCover

Product Validation Report | V1.1

Page | 22

Table 5: Overall accuracy for the WorldCover 2020 product at the continental level and sample sites
used (SSU- secondary sample units, PSU – primary sample units)
Overallaccuracy

Total SSU

Total PSU

Global74.4 ± 0.1

1,935,650

21,624Africa73.6±0.2

313,7093,599

Europe76.8±0.2

282,703

3,118

Eurasia70.2±0.2

274,280

3,003

Asia

80.7±0.1

284,4813,064

Oceania67.5±0.2

248,205

2,961

North America

72.2±0.2

258,4862,880South America

76.1±0.2

273,786

2,999

The class-specific accuracies and the confusion matrices for each continent are listed in Tables 6-12.
Similar to the global scale accuracy, tree cover was mapped with high accuracy for most continents.
For this class, the commission error (100-user’s accuracy) tended to be higher than the omission error(100-producer’s accuracy) for most continents except Africa. In Africa, the tree cover class had a higher
omission error than commission error.
Cropland class was mapped with higher accuracies for all the continents except Africa where this class
had higher omission error. In Africa, the cropland class was mostly confused with the grassland class.
Mangroves class was mapped with moderate accuracies for continents in which this class has apresence. In South America and North America, this class tended to have higher omission errors thancommission errors.
Similar to the global level accuracy, permanent water bodies and snow ice classes were mapped withhigher accuracy, except for South America, where the snow/ice class was tended to be omitted.
Similarly, shrubs, herbaceous wetland, and moss and lichen classes showed lower class accuracies for
all the continents consistent with the global level accuracy.
Bare /sparse vegetation class had varied accuracies among the continents. This can be mainly due tothe differences in area coverage of this class in the continents. For example, this class had highaccuracy in Africa and Asia which have significant coverage of bare/sparse vegetation class. In
contrast, in Europe where bare/sparse vegetation does not cover large areas, this class was mappedwith lower accuracies.

Visual comparison with existing products.
Comparison with 10m products.
The WorldCover 2020 product has been visually compared with other 10m products namely: FROMGLC10, Esri 2020 landcover, and VHR images of Google Earth. For easy visual comparison, the legendsof the Esri 2020 landcover and FROM-GLC10 products were adapted to match the legend of theWorldCover 2020 product (Figure 5).

WorldCover

Product Validation Report | V1.1

Page | 23

Figure 5 The legend of the WorldCover 2020 product
Built-up area
Figure 6 depicts the visual comparison of two urban areas, parts of New York and Singapore. In thiscomparison, it was clear that both the WorldCover 2020 and the FROM-GLC10 had a higher level ofdetail compared to the Esri 2020 landcover product. It was for instance possible to detect multiple
green areas (tree and grass cover) within the cities and to identify linear features such as bridges whenusing these two products. However, when comparing the WorldCover 2020 and the FROM-GLC10
products it was noticeable that there have been some misclassifications within the FROM-GLC10
product. There has been a misclassification of water bodies within the city of New York which shouldhave been built-up areas and around Singapore airport, there has been an overestimation of cropland.
Better urban /built-up characterization of the WorldCover 2020 product compared with the FROMGLC product might be related to the use of Sentinel-1 data in the WorldCover 2020 productgeneration.

Figure 6 Comparison of Esri 2020 landcover (A), FROM-GLC10 (B), WorldCover 2020 (C), and Google
Earth (D) for urban areas. 1. New York City (Coordinates: 40°46'26.86"N 73°58'20.57"W), 2. Singapore(Coordinates: 1°22'18.0"N 103°57'22.2"E), see Figure 5 for legend
Wetlands

The visual comparison of wetlands in Figure 7 showed the difference between the products. It showedagain a higher level of detail in the WorldCover 2020 and the FROM-GLC10 products compared to theWorldCover

Product Validation Report | V1.1

Page | 24

other 10m resolution product; the Esri 2020 landcover. However, the FROM-GLC10 product did mostly
classify wetland areas as grassland and very little with the wetlands class. Overall, the WorldCover2020 product was able to detect more accurately the wetland areas within the depicted locations.

Figure 7 Comparison of Esri 2020 landcover (A), FROM-GLC10 (B), WorldCover 2020 (C) and Google
Earth (D) for wetland areas.1.Russia (Coordinates: 62°20'46.6"N 69°23'26.3"E) 2. Zambia(Coordinates: 16°09'24.2"S 23°14'12.5"E), see Figure 5 for legend
Mangrove/tree cover
The comparison made in Figure 8 focused on the classification of mangroves and tree cover. At thefirst location (the Rajang river delta, Malaysia) there were large differences visible between the
products. This area is characterized by palm oil plantations as can be seen on the VHR image (Figure
8D). The Esri 2020 landcover product identified some plantations as cropland and others as trees. The
FROM-GLC10 product characterized the majority of the area as cropland, whereas the WorldCover2020 product classified it as tree cover. The differences in the products were mainly related to classdefinitions; Esri 2020 landcover classified oil palms as crops and others as trees.
The second locations depict the mangrove area near Nieuw Nickerie in Suriname. The higher level ofdetail of the WorldCover 2020 and the FROM-GLC10 products was again clearly visible. However, boththe WorldCover 2020 and Esri 2020 landcover showed more accurately the presence of mangroves.
The separate mangrove class from the WorldCover 2020 provides the user with additional information
regarding the type of vegetation compared to the flooded vegetation or trees classes used in the Esri2020 landcover product.
The third location shows a forested area in Western Australia. The largest difference between the
products was the depicted forested area. The FROM-GLC10 product showed a smaller forest area thanthe other two land cover products. When comparing this with the satellite image it showed that the
FROM-GLC10 product underestimated the forested area.

WorldCover

Product Validation Report | V1.1

Page | 25

Figure 8 Comparison of Esri 2020 landcover (A), FROM-GLC10 (B), WorldCover 2020 (C) and Google
Earth (D) for areas with tree cover/ mangrove. 1. Malaysia (Coordinates: 2°24'36.8"N 111°31'00.1"E)
2. Suriname (Coordinates: 5°52'38.6"N 56°53'30.7"W) 3. Australia (Coordinates: 32°47'07.9"S116°37'42.0"E ), see Figure 5 for legend
Grassland and shrubs areas
Figure 9 shows a recurring pattern within the products when visualising shrubs, grassland and bareareas. The Esri 2020 landcover product identified much of the area as shrubs where the WorldCover2020 and FROM-GLC10 classified these areas as a combination of bare/sparse vegetation andgrassland. The WorldCover 2020 product overestimated the extent of grassland in these areas slightly.
There was sparse vegetation visible in these areas during some of the months, however likely not tothe extent which is currently visualized. In general, the Esri 2020 landcover product identified vast
areas of grassland such as in Mongolian Steppe and Patagonia as shrubs or scrubs. Although this classincludes some grass areas, it is still a large overestimation of this class.

Figure 9 Comparison of Esri 2020 landcover (A), FROM-GLC10 (B), WorldCover 2020 (C) and Google
Earth (D) for dryland areas. 1. Mongolia (Coordinates: 48°02'26.6"N 92°35'13.0"E) 2. Argentinia(Coordinates: 45°47'27.8"S 69°30'28.6"W), see Figure 5 for legend
CroplandFigure 10 shows two agricultural areas, one in the USA and the other one in Uganda. The Esri 2020
landcover product categorized most of the area as cropland, with additional in both examples anoverestimation of built-up area. Both the WorldCover 2020 and the FROM-GLC10 products provided
a level of detail that enabled the recognition of individual agricultural plots. For the example of theUSA, it was possible to identify center pivot irrigation activities. The WorldCover 2020 productidentified more tree cover, which represents orchards in this case, than the other 10m resolution

WorldCover

Product Validation Report | V1.1

Page | 26

products. The FROM-GLC10 product did identify a few plots with orchards as tree cover but classified
most as cropland.
For the location in Uganda, there has been an overestimation of tree cover in the region and anunderestimation of cropland for the WorldCover 2020 product.

Figure 10 Comparison of Esri 2020 landcover (A), FROM-GLC10 (B), WorldCover 2020 (C) and Google
Earth (D) for cropland areas. 1. USA (Coordinates: 35°11'41.5"N 119°06'20.8"W ) 2.Uganda(Coordinates: 0°22'20.9"N 33°00'19.4"E ), see Figure 5 for legend
Snow/Ice

There was a main agreement among the 10m resolution products on the snow/ice cover in the regionof Jostedalsbreen National Park displayed in Figure 11. The WorldCover 2020 and FROM-GLC10
products provided more details regarding the edges of the snow and ice area, with the tundra class.

Figure 11 Comparison of Esri 2020 landcover (A), FROM-GLC10 (B), WorldCover 2020 (C), and Google
Earth (D) for a snow/ice area in Jostedalsbreen National Park, Norway. (Coordinates: 61°43'59.9"N
7°18'19.3"E), see Figure 5 for legend
Waterbody/salt lake
Figure 12 shows an example of a salt lake. The 10m resolution products all gave a different
interpretation of the presence of water, where the FROM-GLC10 indicated the largest water area. The
Esri 2020 landcover and WorldCover 2020 gave a more accurate representation of the actual situation.

WorldCover

Product Validation Report | V1.1

Page | 27

Figure 12 Comparison of Esri 2020 landcover (A), FROM-GLC10 (B), WorldCover 2020 (C), and Google
Earth (D) for Lake Lefroy, Australia. (Coordinates: 31°18'46.5"S 121°43'30.1"E), see Figure 5 for legend
To summarize, the visual comparison between three 10m resolution GLC maps revealed a depictionof a high level of details with land cover characterization in the WorldCover 2020 and the FROM-GLC10
products as compared with the ESRI 2020 LandCover product. The WorldCover 2020 and FROM-GLC
product were in agreement with characterizing main land cover types, however, the WorldCover 2020
product performed better in areas such as Australia and Oceania. Regarding the ESRI 2020 LandCoverproduct, although this product depicts land cover at 10m resolution, the level of details was lesscompared to the other two products.
Note that this visual comparison is not a spatially explicit comparison of the WorldCover 2020 productwith other products, as the comparison was done for a limited number of examples. As such theexamples used for visual comparison may not capture all the strengths and weaknesses of theWorldCover 2020 product, compared with the other two 10m resolution maps.
Comparison with a 100m resolution product
The WorldCover 2020 product was visually compared with the CGLS-LC100m 2019 map. Figure 13
shows four visual examples for comparison. The legend colours of the two maps are the same. As canbe seen in Figure 13, with the 10m resolution of the WorldCover 2020 product, the landscapes were
mapped in great detail. Areas with fragmented forest stands, cities, and areas with linear objects suchas roads and rivers were mapped in greater detail in the WorldCover 2020 product. Figure 13 A
exemplifies the level of details with the WorldCover 2020 depicted as opposed to the CGLS-LC100
2019 map at 100m resolution.
Next, compared with the CGLS-LC100 2019 map, the WorldCover 2020 product better-depictedwetland areas (Figure 13B and C). The CGLS-LC100 2019 map had less area as wetlands near the
Camargue National Park and Montpellier, while the WorldCover 2020 product mapped more areas aswetlands. In Figure 13B, the CGLS-LC100 2019 map depicted shrubland areas in surrounding
Montpelier city in France. When compared visually, the WorldCover 2020 product corresponded
better with the VHR images of Google as there are more grassland and cropland areas rather thanshrubs.
Mangrove areas near Port Harcourt of Nigeria were mapped as herbaceous wetland in the CGLS-LC100
2019, while this area was correctly mapped in the WorldCover 2020 as mangroves (legend colouraquamarine)(Figure 13C). In areas east of Port Harcourt city, cropland areas were mapped correctly in
the CGLS-LC100 2019 product. This area was mapped as mostly grassland in the WorldCover 2020
product. As was observed in the statistical validation of the WorldCover 2020 product (Table 6), theWorldCover 2020 product tended to underestimate cropland with higher confusion with grasslandclass.
Similarly, in Figure 13D, in areas west of Maracaibo of Venezuela, cropland areas were correctly
mapped in the CGLS-LC100, in contrast, the WorldCover 2020 product mapped this area as mostlyWorldCover

Product Validation Report | V1.1

Page | 28

grassland. However, the WorldCover 2020 product provided a better characterization of other land
cover types such as urban and more detailed forest area depictions.

Figure 13. Comparison of the CGLS-LC100 2019 and WorldCover 2020 products (A: Area east of
Edmonton, Canada, B: Montpellier and surroundings in France, C: Port Harcourt, Nigeria, D. Area westof Maracaibo, Venezuela), see Figure 5 for legend

WorldCover

Product Validation Report | V1.1

Page | 29

3.3. Spatial accuracy assessment
Figure 14 gives more detailed information on the quality of the WorldCover 2020 product comparedto the confusion matrices at the continental level (Table 6-12). It illustrates the spatial accuracy layer
calculated for the WorldCover 2020 product, aggregated to 100m resolution. The layer provides
information on the overall agreement between the map and the reference data set and highlights themost uncertain areas. This agreement has been calculated as a probability that the map agrees withthe reference data set, with values ranging from 0 to 99.

Figure 14 Spatial accuracy layer. Note: results are highly driven by the input reference data set andmay not capture all the mistakes
By analysing the spatial accuracy layer, users would see that the map has issues in the following areas: In Russia, there is high confusion between lichens and bare, and lichens and grassland.
 In Canada, wetlands are considerably underestimated and there is one artefact(Figure 15).
 In the USA, there is confusion between tree cover and shrubs, as well as, bare landand shrubs.
 In Central America, wetlands are underestimated.
 In Brazil, croplands are underestimated In Argentina, confusion between tree cover and shrubs
 The problematic areas in Africa are well known. Those are very fragmented areas,
with mixed croplands, tree cover, and shrubs.
 In Europe, issues with shrubs and tree cover delineation (Table 7).
 In Indonesia, issues related to the delineation of tree cover and shrubs.
 In Australia, confusion between grassland and shrubs, and also bare land and shrubs.
Users should note that the results displayed in Figure 14 are highly driven by the input reference data
set and may not capture all the mistakes. To improve it, there is a need to increase the reference data
set in size.

4. Limitations
Our statistical validation and visual comparison with other products revealed that the WorldCover2020 product characterizes the land cover at a global scale with higher quality and good spatial details
making use of 10m resolution Sentinel-1 and Sentinel-2 data. However, our assessment also revealed
that this product has some limitations which can be tackled in future mapping efforts.
As can be seen in Figure 15, it can also occur that there are some anomalies in the WorldCover 2020
product. Figure 15A, there is a sharp divide between cropland and grassland while similar anomalies

WorldCover

Product Validation Report | V1.1

Page | 30

can be found in Western Mongolia related to the moss/lichen and grassland classes. Figure 15 C shows
sharp boundaries between wetland and grassland classes. These artifacts can be tackled in future
mapping efforts.

Figure 15 Examples of artefacts in the WorldCover 2020 product with sharp boundaries between land
cover types (A) Nigeria -Coordinates: 12°58'05.2"N 8°24'32.6"E; (B) Western Mongolia- Coordinates
50°39'05"N 93°31'32"E; (C) Canada - Coordinates 57°45'40"N 94°10'51"W; see Figure 5 for legend
As identified in the spatial accuracy assessment and continental accuracy estimates, the separation
between tree cover and shrubs areas could be improved, particularly the accuracy of the shrubs classis lower for all the continents. Due to the similarity between natural vegetation classes, it is challenging
to accurately separate shrublands. Furthermore, underestimation of cropland areas has been notedin South America and Africa due to the challenges in the separation between grasslands and croplands.
Although the comparison with other products revealed improvements in characterizing aquatic land
cover types such as mangrove and wetlands, the wetland class was mapped with lower accuracy andthis class has been underestimated in Canada and Central America. Since the wetland class is achallenging class due high water dynamics and water influence, further attention can be paid toimprove this class, also for improving this class in the training and validation datasets.
The validation of the WorldCover 2020 product was done at 10m resolution. At this high resolution,
the geolocation errors of VHR images used for validation data collection and the geolocation errors of
the Sentinel-1 and Sentinel-2 data used for map generation can influence the accuracy estimation(Figure 4). We used SSUs that have similar land cover types with at least 2 of their direct neighbours
for validating the 10m resolution WorldCover 2020 product. However, further assessments could bedone to assess the accuracy of high-resolution products taking into consideration of the possible
geolocation errors. For example, accuracy assessments at coarser level e.g., 3x3 SSUs or at the PSU
level (100x100m area) could be investigated, next to the 10m resolution assessment.

WorldCover

Product Validation Report | V1.1

Page | 31

5. Conclusions
This document reports the validation process of the WorldCover 2020 product. The 10m resolution
WorldCover 2020 product depicting the GLC for the year 2020 was validated. The validation was basedon the CGLS-100 validation dataset, a statistical validation dataset developed as part of the Copernicus
Global Land Service- Dynamic Land cover product. To validate the WorldCover 2020 product, thisvalidation dataset was updated to the year 2020 by revisiting a random subset of the dataset and thesample sites which have higher possibility of land cover change identified using a time series
algorithm, BFAST. This allowed to provide an independent and statistically-robust global andcontinental accuracy assessment at the time of the release of the WorldCover 2020 map in October2021.
Our 10m resolution validation showed that the overall accuracy of the WorldCover 2020 product is74.4 ±0.1%. In terms of land cover types, tree cover and snow/ice, and cropland classes had highaccuracies, while shrubs, herbaceous wetland, and moss/lichen classes were mapped with low
accuracies. Overall accuracy at the continental level is mostly above 70%, with the highest accuracy of
80.7% for Asia and the lowest accuracy of 67.5% for Oceania. The lower accuracy for Oceania could
be due to high shrubland and grassland confusions in the open shrublands of Australia.
Our visual comparison revealed that the WorldCover 2020 and the FROM-GLC products captured
landscapes at a higher level of detail than the ESRI 2020 Landcover product. Further, the WorldCover2020 product had better characterization of land cover in areas such as Australia and Oceania.
Compared to the CGLS-LC100 2019 product, the WorldCover 2020 product underestimates croplandin areas such as South America and Africa, which could be focused on future efforts of improving themap characterization.
Spatial accuracy assessment was conducted for the WorldCover 2020 product using more than 200
000 reference data locations to highlights areas that are mapped with good quality and areas that
require attention for future efforts.
Overall, the WorldCover 2020 product shows promising results in characterizing the World’s land
cover at 10m resolution making use of Sentinel 1 and Sentinel 2 data for the 2020. Users of the map
encouraged to make use of the statistical accuracy analysis at global and continental level, and thespatial accuracy assessment to best apply the WorldCover 2020 product for their purposes.

WorldCover

Product Validation Report | V1.1

Page | 32

6. References
Brunsdon, C., Fotheringham, S., & Charlton, M. (1998). Geographically Weighted RegressionModelling Spatial Non-Stationarity. Journal of the Royal Statistical Society. Series D (The Statistician),47, 431-443
Buchhorn, M., Bertels, L., Smets, B., De Roo, B., Lesiv, M., Tsendbazar, N.-E., & Li, L. (2020).
Copernicus Global Land Service: Land Cover 100m: version 3 Globe 2015-2019: Algorithm Theoretical
Basis Document. https://doi.org/10.5281/zenodo.3938968
Bunting, P., Rosenqvist, A., Lucas, R.M., Rebelo, L.-M., Hilarides, L., Thomas, N., Hardy, A., Itoh, T.,
Shimada, M., & Finlayson, C.M. (2018). The Global Mangrove Watch—A New 2010 Global Baseline ofMangrove Extent. Remote Sensing, 10, 1669
Clerc, S. (2021). Sentinel 2 data product quality reports.
https://sentinels.copernicus.eu/web/sentinel/data-product-quality-reports
Di Gregorio, A. (2005). Land cover classification system: classification concepts and user manual:
LCCS. Rome, Italy: Food and Agriculture Organization of the United Nations
ESRI. (2021). Esri 10-Meter Land Cover.2021,July 5, https://livingatlas.arcgis.com/landcover/
Fotheringham, A.S.B., C.; Charlton, M. (2002). Geographically Weighted Regression: The Analysis of
Spatially Varying Relationships; . John Wiley & Sons: Chichester, UK,
Fritz, S., McCallum, I., Schill, C., Perger, C., See, L., Schepaschenko, D., van der Velde, M., Kraxner, F.,& Obersteiner, M. (2011). Geo-Wiki: An online platform for improving global land cover.
Environmental Modelling & Software, 31, 13
Giri, C., Ochieng, E., Tieszen, L.L., Zhu, Z., Singh, A., Loveland, T., Masek, J., & Duke, N. (2011). Status
and distribution of mangrove forests of the world using earth observation satellite data. Global
Ecology and Biogeography, 20, 154-159
Gong, P., Liu, H., Zhang, M., Li, C., Wang, J., Huang, H., Clinton, N., Ji, L., Li, W., Bai, Y., Chen, B., Xu,
B., Zhu, Z., Yuan, C., Ping Suen, H., Guo, J., Xu, N., Li, W., Zhao, Y., Yang, J., Yu, C., Wang, X., Fu, H., Yu,
L., Dronova, I., Hui, F., Cheng, X., Shi, X., Xiao, F., Liu, Q., & Song, L. (2019). Stable classification withlimited sample: transferring a 30-m resolution sample set collected in 2015 to mapping 10-m
resolution global land cover in 2017. Science Bulletin, 64, 370-373
Gong, P., Wang, J., Yu, L., Zhao, Y., Zhao, Y., Liang, L., Niu, Z., Huang, X., Fu, H., Liu, S., Li, C., Li, X., Fu,
W., Liu, C., Xu, Y., Wang, X., Cheng, Q., Hu, L., Yao, W., Zhang, H., Zhu, P., Zhao, Z., Zhang, H., Zheng,Y., Ji, L., Zhang, Y., Chen, H., Yan, A., Guo, J., Yu, L., Wang, L., Liu, X., Shi, T., Zhu, M., Chen, Y., Yang,G., Tang, P., Xu, B., Giri, C., Clinton, N., Zhu, Z., Chen, J., & Chen, J. (2013). Finer resolution
observation and monitoring of global land cover: first mapping results with Landsat TM and ETM+
data. International Journal of Remote Sensing, 34, 2607-2654
Hamilton, S. (2015). CGMFC-21, MFW, Global Mangrove Levels by Pixel (~30m), 2000 - 2012. In S.
Hamilton (Ed.): Harvard Dataverse

WorldCover

Product Validation Report | V1.1

Page | 33

IIASA. (2019). CrowdVal project.2021,October 15,
https://iiasa.ac.at/web/home/research/researchPrograms/EcosystemsServicesandManagement/Cro
wdVal.html
Olofsson, P., Stehman, S.V., Woodcock, C.E., Sulla-Menashe, D., Sibley, A.M., Newell, J.D., Friedl,
M.A., & Herold, M. (2012). A global land-cover validation data set, part I: fundamental designprinciples. International Journal of Remote Sensing, 33, 5768-5788
Pengra, B., Long, J., Dahal, D., Stehman, S.V., & Loveland, T.R. (2015). A global reference databasefrom very high resolution commercial satellite data and methodology for application to Landsat
derived 30m continuous field tree cover data. Remote Sensing of Environment, 165, 234-248
Stehman, S.V., Wickham, J., Smith, J., & Yang, L. (2003). Thematic accuracy of the 1992 National
Land-Cover Data for the eastern United States: Statistical methodology and regional results. Remote
Sensing of Environment, 86, 500-516
Tsendbazar, N., Herold, M., Li, L., Tarko, A., de Bruin, S., Masiliunas, D., Lesiv, M., Fritz, S., Buchhorn,
M., Smets, B., Van De Kerchove, R., & Duerauer, M. (2021). Towards operational validation of annual
global land cover maps. Remote Sensing of Environment, 266, 112686
Tsendbazar, N.E., Herold, M., de Bruin, S., Lesiv, M., Fritz, S., Van De Kerchove, R., Buchhorn, M.,
Duerauer, M., Szantoi, Z., & Pekel, J.F. (2018). Developing and applying a multi-purpose land covervalidation dataset for Africa. Remote Sensing of Environment, 219, 298-309
Verbesselt, J., Hyndman, R., Newnham, G., & Culvenor, D. (2010). Detecting trend and seasonalchanges in satellite image time series. Remote Sensing of Environment, 114, 106-115
Wickham, J., Stehman, S., Fry, J., Smith, J., & Homer, C. (2010). Thematic accuracy of the NLCD 2001
land cover for the conterminous United States. Remote Sensing of Environment, 114, 1286-1296

WorldCover

Product Validation Report | V1.1

Page | 34

Annex

Bare / sparse veg

Permanent water
bodiesHerbaceous
wetland

Mangroves

Correct

Total

User's accuracy

Confidence
interval ±

0.02

0.03

15.1

18.3

82.6

0.3

0.75

0.01

0.04

0.01

7.55

19.8

38.1

0.6

0.01

1.83

0.01

0.48

14.7

21.4

68.8

0.5

4.06

0.01

0.15

0.05

4.06

5.69

71.4

0.7

0.18

0.01

0.27

0.12

0.27

0.6

45.5

2.5

1.06

0.35

0.08

30.1

0.05

30.1

31.9

94.4

0.3

0.07

1.16

0.03

1.16

1.28

90.6

0.01

0.03

0.5

0.5

0.82

60.5

1.9

0.03

0.02

0.13

0.13

0.19

66.3

3.5

1.16

0.5

0.13

Cropland

0.03

Grassland

0.01

Shrubland

0.01

Tree cover

Built-up

Table 6. Confusion matrix (%) for the WorldCover 2020 product for Africa, corrected by sample inclusion
probabilities.

Tree cover

15.1

1

1.71

0.39

Shrubland

3.9

7.6

6.65

0.89

Grassland

0.71

1.4

14.7

2.24

Cropland

0.14

0.2

1.1

Built-up

0.02

Bare / sparse veg
Permanent water
bodiesHerbaceous wetland

0.04

0.2

0.02
0.01

0.22

Mangroves

0.06

0.02

Correct

15.1

Total
Producer's accuracy
Confidence interval
±

7.6

14.7

4.06

0.27

30.1

20

10

25.7

7.99

0.38

33.1

1.31

1.14

0.18

75.8

73.2

57.4

50.8

71.1

91.1

88.7

43.4

71.6

0.4

0.7

0.5

0.7

3.2

0.3

0.9

1.6

3.4

0.9

73.63
0.2

Shrubland

0.26

0.5

0.71

0.02

Grassland

1.39

1.12

16.5

3.95

0.24

0.63

Cropland

0.22

0.26

1.76

22.08

0.03

0.04

Built-up

0.03

0.13

0.03

1.72

0.1

0.44

0.33

0.17

0.69

0.03

0.11

0.61

Bare /
sparse veg
Snow andice
PermanentwaterbodiesHerbaceous
wetlandMoss and
Lichen

0.07

0.09

0.02

0.03

Confidence interval ±0.16

User's accuracy

0.5

Total

Bare / sparse veg

0.69

Correct

Built-up

4.69

Moss and Lichen

Cropland

1.86

Herbaceous wetland

Grassland

31.53

Permanent water
bodiesShrublandTree cover

Snow and ice

Tree cover

Table 7. Confusion matrix (%) for the WorldCover 2020 product for Europe, corrected by sample
inclusion probabilities.

0.09

0.05

0.02

31.53

39.58

79.7

0.3

0.04
0.09

0.27

0.5

1.54

32.7

1.5

16.5

24.74

66.7

0.4

22.08

24.42

90.4

0.2

1.72

2

85.8

0.9

0.07

0.69

1.94

35.4

0.01

0.61

0.74

83.2

2.37

2.47

96.1

0.53

0.02

0.04

0.05

2.37

1
1.2

0.4
0.01
0

0.34
0.02

0.58

0.04

0.01
0.6

0.06

0.02

0.26

0.01

0.02

0.56

0.26

0.71

37.1

0.56

1.87

30.2

1.7
1.3

WorldCover

Product Validation Report | V1.1

Correct

31.53

0.5

16.5

22.08

1.72

0.69

0.61

2.37

0.26

0.56

Total

33.52

3.88

25.17

27.13

2.67

2.42

0.71

2.64

0.64

1.21

94.1

13.0

65.6

81.4

64.4

28.3

86.1

89.7

41.4

46.8

0.2

0.7

0.4

0.3

1.1

0.9

1.2

0.7

1.9

1.7

Producer's
accuracy
Confidence
interval ±Page | 35

76.8
0.2

Cropland

Built-up

Herbaceous wetland

Moss and Lichen

Correct

Total

User's accuracy

4.33
0.66

5.77
0.39

0.05

0.02

0.04
0.27

0.15

0.11
0.03

0.33
0.02

32.25
0.66

43.06
1.5

74.9
44.1

0.3

Grassland
Cropland

0.87
0.02

1.33
0.01

21.29
1.02

0.69
4.94

0.01

1.96
0.02

0.08

0.5
0.04

2.1

21.29
4.94

28.84
6.05

73.8
81.6

0.3

Built-up
Bare / sparseveg
Snow and ice

0.01

0.04

0

0.11

0.04

0.11

0.2

55.8

0.47

0.02

0.02

3.82

0.01

3.82

4.59

83.2

0.01

0.2

0.2

0.21

95.7

Permanent
water bodies
Herbaceous
wetlandMoss and
LichenCorrect
Total
Producer's
accuracy
Confidence
interval ±

0.01

0.02

0.02

0.04
0.01

0.06

0.01

0.16

Snow and ice

0.06

3.39

0.06

0.08

3.39

3.66

92.6

0.04

0.24

1.24

1.68

1.24

5.81

21.3

0.87

0.02

0.06

2.33

2.33

6.07

38.4

0.12

0.26

2.23

0.08

0.28

2.44

32.25
33.49

0.66
6.9

21.29
33.69

4.94
5.72

0.11
0.17

3.82
7.12

0.2
0.21

3.39
3.94

1.24
2.05

2.33
6.71

96.3

9.6

63.2

86.3

66.2

53.6

95.1

86.0

60.4

34.8

0.1

0.4

0.3

0.6

4

0.6

0.6

0.7

1.4

0.7

Confidence interval±

Grassland

32.25
0.13

Permanent water
bodiesShrublandTree cover
Shrubland

Bare / sparse veg

Tree cover

Table 8. Confusion matrix (%) for the WorldCover 2020 product for Eurasia, corrected by sample
inclusion probabilities.

1.5
0.6
3.6
0.6
1.3
0.5
0.6
0.8

70.23
0.2

Grassland

Cropland

Built-up

Bare / sparse veg

Permanent water
bodiesHerbaceous wetland

Mangroves

Correct

Total

User's accuracy

Confidence interval ±

24.25

1.34

2.19

0.93

0.11

0.13

0.09

0.03

0.05

24.25

29.1

83.3

0.2

Shrubland

0.2

0.3

0.62

0.18

0.01

0.28

0.01

0.01

0.01

0.3

1.61

18.5

1.2

Grassland

0.82

0.9

9.78

0.9

0.02

1.58

0.04

0.13

9.78

14.2

68.9

0.5

Cropland

0.66

0.13

1.45

12.77

0.06

0.37

0.19

0.24

12.77

15.9

80.5

0.4

Moss and Lichen

ShrublandTree cover

Snow and ice

Tree cover

Table 9. Confusion matrix (%) for the WorldCover 2020 product for Asia, corrected by sample inclusion
probabilities.

0.01
0.01

WorldCover
Built-up
Bare /
sparse veg
Snow andice
Permanentwaterbodies

Product Validation Report | V1.1

0.07
0.22

0.38

0.01

0.1

0.06

0.98

0.19

2.17

0.61

0.23

30.09

0.03

0.03

0.37

0.01

0.06

1.89

0.06

0.1

0.02

0.98

1.4

69.6

30.09

34

88.5

0.37

0.4

91.5

1.89

2.21

85.5

1.4
0.2
0.9

0.8

Herbaceous
wetland

0.07

0.05

Mangroves

0.03

Moss and
Lichen

0.02

0.02

0.52

Correct

24.25

0.3

9.78

12.77

Total

26.29

3.07

16.9

92.3

9.7

0.2

0.6

Producer's
accuracy
Confidence
interval ±

0.21

0.12

Page | 36

0.01

0.06

0.1

0.01

0.13

0.13

0.01

0.01

0.98

30.09

0.37

1.89

0.1

0.13

0.01

15.55

1.41

33.02

0.41

2.41

0.6

0.2

0.1

57.9

82.1

69.1

91.1

89.9

78.7

16.1

66.4

8.1

0.5

0.4

1.4

0.2

1

0.8

1.5

3.8

2.7

0.1

0.29

33.5

0.13

0.18

72.9

0.01

0.71

1.6

2.9
3.2
0.5

80.7
0.1

Total

User's accuracy

Confidence interval ±

28.9

0.8

44.14

54.64

80. 8

0.3

0.03

3.26

3.92

83.2

0.7

0.1

0

0.1

0.12

85.2

3.3

0.01

2.97

0.03

2.97

4.99

59.6

0.02

0.26

0.26

0.31

84.0

12.1

1.34

5.98

0.03

0.02

0.22

0.02

0.01

Shrubland

1.27

4.63

8.23

Grassland

1.14

2.25

44.14

0.57

Cropland

0.01

0.01

0.61

3.26

Built-up

1.88

0.01

Bare /
sparse veg

0.02

Permanentwaterbodies0.01

Herbaceous
wetland

0.02

0.06

1.82

0.05

0.03

0.01

6.29

0.03

0.13

Mangroves

Tree cover

Moss and Lichen

16.01

Herbaceous wetland

4.63

Permanent water
bodies0Bare / sparse veg

0.5

Built-up

61.4

Cropland

19.72

Grassland

12.1

Shrubland0Tree cover

Correct

Table 10. Confusion matrix (%) for the WorldCover 2020 product for Oceania, corrected by sample
inclusion probabilities.

0

0.06

0.02
0.01

1.2

1.9
0.01

Mangroves

0.06

0.07

0.01

0.01

0.01
0.05

Moss and
Lichen0.01

Correct

12.1

4.63

44.14

3.26

0.1

2.97

0.26

0.01

0.05

0.01

Total

14.57

8.3

60.87

3.91

0.16

11.53

0.35

0.16

0.05

0.1

Producer's
accuracy

83.04

55.79

72.51

83.46

61.38

25.79

75.47

3.78

88.19

5.3

0.01

0.17

3.5

0.05

0.07

69.8

0.01

0.01

100

67.5

0.4
4
0

WorldCover

Product Validation Report | V1.1

Confidence
interval ±

1.1

0.5

0.3

0.7

3.8

0.7

2.7

0.4

4.1

Page | 37

2.6

0.2

0.68

0.11

0.07

4.14

0.09

0.66

11.76

0.2

0.03

0.02

0

0.06

0.03

0.05

0.39

0.06

0

0.04

0.08

0

0.16

0.98

0.01

24.07
26.87

3.11
8.09

12.12
19.54

6.75
7.99

89.6

38.5

62.1

0.2

0.7

0.4

0.01

0.18
0.11
3.19

24.07
3.11
12.12
6.75
0.44

29.45
7.15
20.85
7.56
0.61

81.7
43.6
58.1
89.2
72.7

0.3
0.7
0.4
0.4
2

1.37

4.14

6.93

59.8

0.05

11.76

12.47

94.3

4.59

0.06

0.32

4.59

5.28

87.0

0.14

0.44

0.89

0.44

2.03

21.5

0.02

0.02

77.4

4.76

7.65

62.3

0
0.1

Confidence interval ±

0.32

0.11
0.05
0.3
0.02

User's accuracy

0.06

0.12
0
0.15
0.01

Total

0.11
0.58
0.56
0.03
0.05

Correct

0.1
0.01
0.06
0
0.44

Moss and Lichen

Bare / sparse veg

0.1
0.02
0.94
6.75
0.01

Mangroves

Built-up

2.65
1.87
12.12
0.7
0.08

Herbaceous wetland

Cropland

2.02
3.11
2.4
0.01
0

Permanent water
bodiesGrassland

24.07
1.39
1.13
0.04
0.03

Snow and ice

ShrublandTree cover
Shrubland
Grassland
Cropland
Built-up
Bare /
sparse veg
Snow andice
PermanentwaterbodiesHerbaceous
wetland
Mangroves
Moss and
LichenCorrect
Total
Producer's
accuracy
Confidence
interval ±Tree cover

Table 11. Confusion matrix (%) for the WorldCover 2020 product for North America, corrected bysample inclusion probabilities.

0.02

1.44

0.01

0.14

0.04

4.76

0.44
0.68

4.14
7.81

11.76
11.89

4.59
5.23

0.44
1.01

0.02
0.03

4.76
10.87

84.5

65.3

53.1

98.9

87.7

43.2

55.6

43.8

0.5

1.9

0.7

0.1

0.6

1.6

6.1

0.6

0.7
0.2
0.6
0.9
4.5
0.7

72.21
0.2

1.24

2.35

15.38

1.11

0.03

1.82

Cropland

0.12

0.06

1.71

6

0.01

0.11

Built-up
Bare /
sparse veg
Snow and
ice

0.01

0.26

0.03

0.03

5.74

0.07

0.01

0.15

0.08

0.03
0.2

0.72

0.06

0.18

0.17

0.14

0.02

0.05
0.08

1.04
0.04

0.03

0.02

0.07

0.14

Confidence interval ±Grassland

0.14
0.99

User's accuracy

0.03

0.08

Total

0.38

2.47

Correct

2.1

4.67

Moss and Lichen

3.24

1.7

Mangroves

42.1

Shrubland

Herbaceous wetland

Tree cover

Permanent water
bodiesBuilt-up

Snow and ice

CroplandBare / sparse veg

Grassland

ShrublandTree cover

Table 12. Confusion matrix (%) for the WorldCover 2020 product for South America, corrected bysample inclusion probabilities.

42.1

48.52

86.8

0.2

4.67

9.96

46.8

0.7

15.38

23.26

66.1

0.4

6

8.04

74.6

0.6

0.26

0.34

78.0

2.1

5.74

6.96

82.4

0.15

0.16

92.3

0.6
0.5

WorldCover
PermanentwaterbodiesHerbaceous
wetland
Mangroves
Moss and
LichenCorrect
Total
Producer's
accuracy
Confidence
interval ±Product Validation Report | V1.1

0.02

0.01

0.01

0.06

0.04

0.41

0.01

0.14

Page | 38

1.23

0.03

0.01

1.23

1.46

83.9

0.04

0.43

0.08

0.43

1.06

40.3

0.02

0.03

0.09

0.09

0.14

63.6

0.01

0.09

16.0

0.02

0.03

0.02

0.01

42.1

4.67

15.38

6

0.26

5.74

0.15

1.23

0.43

0.09

0.01

45.32

10.6

22.87

7.64

0.37

9.01

0.22

1.58

1.82

0.39

0.19

92.9

44.0

67.3

78.5

72.2

63.7

67.2

77.7

23.5

23.4

7.1

0.1

0.6

0.4

0.5

2.3

0.6

2.4

1.1

1.1

2

2.6

1
1.8
3.3
5.6

76.1
0.2