Ver2. Cryosphere Products

1

Evaluation Summary
Product

Snow and Ice
covered area (incl.
cloud detection)
Okhotsk sea-ice
distribution
Snow and ice
surfaceTemperature

Snow grain size ofshallow layer

ReleasethresholdStandard
accuracy

Targetaccuracy

10%

7%

10%

5%

5K

100%

2K

50%

Status*1

Evaluation Methods

5%○Comparison with other satellites data (e.g.
MODIS, VIIRS, Sentinel-3…).

3%○Comparison with other satellites data (e.g.
MODIS, VIIRS, Sensinel-3…).

◎Comparison with in-situ observation(Automatic weather station thermalradiometer data) and other satellites data
(e.g. MODIS, VIIRS Sentinel-3…).

○→◎Comparison with climatology (relationshipbetween snow surface temperature andsnow grain size) for the release accuracy
threshold. In addition, comparison with insitu data for the standard and target
accuracy thresholds.

1K

30%

*1 Symbols denote as follows; 〇: the release threshold achieved, ◎: the standard accuracy achieved, ☆: the target accuracy achieved.

2

Validation results of Cryosphere products - SICE- Version 2 Major changes and validation data detailsMajor Change for the C1AB - Snow and Ice cover area algorithm- Revised cloud detection/surface classification algorithm from ordinary threshold method to Neuralnetwork machine learning method- All training data were simulated by DISORT radiative transfer model

Major change for the C1C - Okhotsk sea-ice distribution algorithm- Revised cloud detection/surface classification algorithm from ordinary threshold method to Neuralnetwork machine learning method partly.
- All training data using Neural network were simulated by DISORT radiative transfer model
Validation data for the C1AB/ SICE - Snow and Ice cover Extent product (snow/ice fraction > 15%*)- Snow area: MOD10C2 Snow Cover Extent Product- Sea ice area: MOD29E1D Sea Ice Product
Validation data for the C1C/OKID - Okhotsk sea-ice distribution product (sea ice fraction > 15%*)- Sea ice area :MOD29E1D Sea Ice Product*NSIDC defines sea ice exists in case of the ice fraction/ice concentration more than 15%.

Validation results of Cryosphere products - SICE
- Snow and Ice cover extent product validation results using other satellite products
Ver.2 sample image
2019.05.09 - 2019.05.24

Validation result of Ver. 1
Snow Fraction > 10 %2018.03.14 - 2018.08.20
8 days composite

Validation result of Ver. 2
Snow Fraction > 15 %2018.03.22 - 2019.12.31
16 days composite

N = 21r = 0.99 (p < 0.01)
Rel. Err. = 9.4 %Bias = 744,065 km2

N = 40r = 0.99 (p < 0.01)
Rel. Err. = 8.5 %Bias = 462,283 km2

1. Validation period was expanded: 0.5 year to 1.5 year.
2. Composite period and valid snow/ice fraction were revised: 8 day to 16 days, 10% to 15% .
Validation result

Release accuracyStandard accuracy

Target accuracy

Ver.1: 9.4 % (Mar. 2018 - Aug. 2018)
Ver.2: 8.5 % (Mar. 2018 - Dec. 2019)10 %7%

5%Accuracy improved and SICE product is likely to achieve the standard accuracy

Validation results of Cryosphere products - OKID
- Version 2 Major changes and validation data details
Major Change for the C1AB - Snow and Ice cover area algorithm- Revised cloud detection/surface classification algorithm from ordinary threshold method to Neuralnetwork machine learning method- All training data were simulated by DISORT radiative transfer model

Major change for the C1C - Okhotsk sea-ice distribution algorithm- Revised cloud detection/surface classification algorithm from ordinary threshold method to Neuralnetwork machine learning method partly- All training data using Neural network were simulated by DISORT radiative transfer model
Validation data for the C1AB/ SICE - Snow and Ice cover Extent product (snow/ice fraction > 15%*)- Snow area: MOD10C2 Snow Cover Extent Product- Sea ice area: MOD29E1D Sea Ice Product
Validation data for the C1C/OKID - Okhotsk sea-ice distribution product (sea ice fraction > 15%*)- Sea ice area :MOD29E1D Sea Ice Product*NSIDC defines sea ice exists in case of the ice fraction/ice concentration more than 15%.

Validation results of Cryosphere products - OKID
-Okhotsk sea-ice distribution product validation results using other satellite products

False Color Image

Validation result of Ver. 2

Validation result of Ver. 1

Ver.2 sample image
2020.02.27
OKID classification
snow

Ice Fraction > 10 %2018.03.14 - 2018.08.20
8 days composite

Ice Fraction > 15 %2018.02.10 - 2020.03.05
8 days compositecloudsea icelandocean

N = 30r = 0.99 (p < 0.01)
Rel. Err. = 9.0 %Bias = -5,986 km2

N = 21r = 0.99 (p < 0.01)
Rel. Err. = 9.1 %Bias = 91,081 km2

1. Validation period was expanded: 0.5 year to 2 year.
2. Surface classification was improved from visual evaluation compared with False color image.
Validation result

Release accuracyStandard accuracy

Target accuracy

Ver.1: 9.1 % (Mar. 2018 - Aug. 2018)
Ver.2: 9.0 % (Feb. 2018 - Mar. 2020)

10 %5%3%

OKID product needs more detail evaluation using high resolution satellite images

Validation results of Cryosphere products - SIPR
- Version 2 Major changes and validation data details
Major Change for the C2AB - Snow grain size of shallow layer- Revised snow grain size estimation algorithm using Neural network machine learning method- Add the layer numbers of Neural-net: improved the processing speed- Revised the training data set (BRDF data set) using Neural-net: improved inversion accuracy

Major change for the C2AB - Snow and Ice surface temperature- Revised the emissivity tableValidation data for the C2AB/ SGSL - Snow grain size of shallow layer- Field campaign carried out on the Greenland Ice Sheet East-GRIP site (Jul. 2018) andJapan/Hokkaido Nakasatsunai site (Feb. 2020)- Surface Specific Area (SSA) measured by IceCube and HISSGraS and converted to optical equivalentsnow grain size
- All data match-up conditions are in 10 minutes and 250 meters from nearest point of satellites
Validation data for the C2AB/ SIST - Snow and Ice surface temperature- Automatic Weather Station (AWS) installed by PROMICE (Fausto and van As, 2019) have measured
the Upward/Downward Longwave Radiation Flux- Ground surface temperature was converted from Longwave radiation Flux observation- All data match-up conditions are in 10 minutes and 250 meters from nearest point of satellites
Fausto, R.S. and van As, D., (2019). Programme for monitoring of the Greenland ice sheet (PROMICE): Automatic weather station data.
Version: v03, Dataset published via Geological Survey of Denmark and Greenland. DOI: https://doi.org/10.22008/promice/data/aws

Validation results of Cryosphere products - SIPR
Snow grain size of shallow layer product validation results using in-situ observation data (only E-GRIP)
Ver.2 sample image
2019.05.09 - 2019.05.24

Snow grain size estimation accuracy evaluation by comparison between
Ver.1 and Ver.2 results in same quality control condition using Ver.2 SICE result
500

EGRIP

★
EGRIP

SGLI Snow Grain Size (µm)-Ver.1

400

100%

Ver.2

50%
50%

300

100%
200

N=8r = 0.43 -> 0.31
Rel. Er. = 44.9% -> 28.5%
bias = 19.5 -> 12.4 µm

10000100
200
300
400
In-situ Snow Grain Size (µm)

500

Validation resultRelease accuracyStandard accuracy

Target accuracy

Ver.1: 45 % (E-GRIP, QC: Ver.2)
Ver.2: 29 % (E-GRIP, QC: Ver.2)

100 %

50 %

30 %

Quality control was improved by cloud detection algorithm revision of SICE.
Snow grain size estimation accuracy improved by revision of C2 algorithm.

Validation results of Cryosphere products - SIPR
Snow grain size of shallow layer product validation results using in-situ observation data (wide-range)
Ver.2 sample image
2019.05.09 - 2019.05.24

Adding the in-situ data measured in Nakasatsunai, Hokkaido, JapanSGSL was evaluated in a wide-range of arctic and mid-latitude region
500
Ver.2 EGRIP

★
EGRIP

SGLI Snow Grain Size (µm)

-400

100%

Ver.2 NSN

50%50%

300

100%200

N = 11r = 0.50
Rel. Er. = 34.1%
bias = 19.1 µm

10000

100
200
300
400
In-situ Snow Grain Size (µm)

500

Validation result

Release accuracyStandard accuracy

Target accuracy

Ver.1: 45 % (only arctic, QC: Ver.2)
Ver.2: 34 % (wide-range, QC: Ver.2)

100 %

50 %

30 %Accuracy improved and SGSL product achieved the standard accuracy

Validation results of Cryosphere products - SIPR-

Snow and Ice surface temperature product validation results using AWS observation data (only E-GRIP)
Ver.2 sample image
2019.05.09 - 2019.05.24

Snow and Ice surface temperature estimation accuracy evaluation bycomparison between Ver.1 and Ver.2 results using E-GRIP observation dataEGRIP★
EGRIP
N = 33r = 0.97 (p < 0.01)
RMSE = 1.5 K -> 1.4 K
bias = 1.2 K -> 1.0 K

Validation resultRelease accuracyStandard accuracy

Target accuracy

Ver.1: 1.5 K (EGRIP)
Ver.2: 1.4 K (EGRIP) & 2 K (GrIS, next page)

5K

2K

1K

Accuracy improved and SIST product achieved the standard accuracy

Validation results of Cryosphere products - SIPR-

Snow and Ice surface temperature product validation results using AWS observation data (whole GrIS)
Snow and Ice surface temperature estimation accuracy evaluation
using all AWSs installed whole region of the Greenland Ice Sheet

ALLKPC_U

N = 135
r = 0.93 ( p < 0.01)
RMSE = 1.98 K
bias = 0.9 K

RMSE = 1.39 K

SCO_UTHU_U
RMSE = 1.00 K

RMSE = 0.90 K

KAN_U

UPE_U

SIST: R/S/G = 5K/2K/1K
RMSE = 1.57 K

RMSE = 4.38 K