ESA-JAXA Pre-Launch EarthCARE
Science and Validation Workshop
13 – 17 November 2023 | ESA-ESRIN, Frascati (Rome), Italy

Performance Evaluation of the Rain Mask Algorithm for Global
Precipitation Monitoring Using NASA MPLNET Lidar Network
MPLNET Team: Ellsworth J. Welton (NASA GSFC), Jasper R. Lewis (UMBC) , James R. Campbell (NRL),
Erica K. Dolinar (NRL) , Simone Lolli (CNR-IMAA)
Collaborators: Jason Tackett (NASA LARC), Erica Mc-Grath Spangler (MSU), Michäel Sicard (UR),Rob Koopman (ESA), Stephanie Rusli (ESA), Gemine Vivone (CNR-IMAA)

Rain Masking Algorithm (RMA)• We present the results of testing the rain mask
algorithm, developed for potential use in the NASAMPLNET lidar network over a period of several years.
• The rain mask algorithm aims to detect precipitation
events from lidar measurements of the volume
depolarization ratio (VDR) under cloud base.
• The algorithm will help in validating satellite cloudprofiling radar observations of the European Space
Agency EarthCARE mission.
ESA-JAXA Pre-Launch EarthCARE Science and Validation Workshop | 13 – 17 November 2023 | ESA-ESRIN, Frascati (Rome), Italy

EarthCARE-MPLNET Cal/Val NNH20ZDA001N-EUSPIIntercomparison• The Rain Masking Algorithm using lidar observations as input, categorically outputs a rain or
no-rain mask which has 60-seconds temporal resolution.
• Data is then aggregated into 60-minute bins for comparison.

• The intercomparison is carried out considering a co-located disdrometer as reference. The
protocol involves transforming the disdrometer rainfall intensity at a 60-second resolution,
aggregated into a 60-minutes bin,. into two classes: class 1 for "rain" and class 0 for "no-rain”.
• Data where lidar data are completely attenuated, or where the algorithm indicates that theprecipitation does not reach the ground, are excluded• For a bin, to be accepted for comparison, it must have rain detected for a minimum of fiveminutes within sixty minutes period.
ESA-JAXA Pre-Launch EarthCARE Science and Validation Workshop | 13 – 17 November 2023 | ESA-ESRIN, Frascati (Rome), Italy

EarthCARE-MPLNET Cal/Val NNH20ZDA001N-EUSPIIntercomparison Protocol- Two MPLNET sites: Goddard Space FlightCenter (GSFC), USA and Universitat
Politecnica de Catalunya (UPC), Spain.
- Intercomparison based on categoricallydistinguishing between rain and no-rain
events using different statistical variables
such as accuracy, error, sensitivity,specificity, precision, false positive rate, F1score, Matthews Correlation Coefficient,and Kappa- -- Database Homogenization: no rain eventsoccur more frequently than rain events, thedatabase is normalized to include an equal
distribution of both types, with 50% rain
events and 50% no rain events. The no rain
events are randomly selected to match thenumber of rain events.
- Precipitation not reaching the ground is notconsidered in the intercomparison

ESA-JAXA Pre-Launch EarthCARE Science and Validation Workshop | 13 – 17 November 2023 | ESA-ESRIN, Frascati (Rome), Italy

EarthCARE-MPLNET Cal/Val NNH20ZDA001N-EUSPIRMAStatistics

Disdrometer

FPR: the false positive rate measures the ratio of false positives within the negative samples
Specificity: When it's actually no-rain, how often does it predicts no-rain?
MCC takes into account all four values in the confusion matrix, and a high value (close to 1) means that bothclasses are predicted well, even if one class is disproportionately under- (or over-) represented.
ESA-JAXA Pre-Launch EarthCARE Science and Validation Workshop | 13 – 17 November 2023 | ESA-ESRIN, Frascati (Rome), Italy

EarthCARE-MPLNET Cal/Val NNH20ZDA001N-EUSPIIntercomparison at NASA Goddard Space Flight Center
GSFC; 38.99N; 76.84W; 55 m a.s.l., Feb 2020-May 2022, 398 rain events
The accuracy is 0.81. RMA correctly
detects around 81% of rain events.
The rain mask algorithm has a highsensitivity of 0.96, accurately
recognizing 96% of occurrences.
The F1-score of 0.82,
the Matthews Correlation Coefficientof 0.62, and the Kappa coefficient of0.58 provide balanced measures of
the algorithm’s overall performance.
ESA-JAXA Pre-Launch EarthCARE Science and Validation Workshop | 13 – 17 November 2023 | ESA-ESRIN, Frascati (Rome), Italy

EarthCARE-MPLNET Cal/Val NNH20ZDA001N-EUSPIIntercomparison at NASA Goddard Space Flight Center
GSFC; 38.99N; 76.84W; 55 m a.s.l., Feb 2020-May 2022, 398 rain events

ESA-JAXA Pre-Launch EarthCARE Science and Validation Workshop | 13 – 17 November 2023 | ESA-ESRIN, Frascati (Rome), Italy

EarthCARE-MPLNET Cal/Val NNH20ZDA001N-EUSPIIntercomparison at Universitat Politecnica de Catalunya, Barcelona Spain
UPC; 41.38N 2.11E, 125 m a. s. l., 2019-2020, 519 rain events
RMA correctly detects around 78% ofrain events
RMA exceptional sensitivity of 0.99,
Specificity of 0.55 indicates moderate
performance in correctly identifying norain events.
Precision is 0.70, good level inpredicting positive rain events.
The F1-score of 0.81, the MCC of 0.60,
and the Kappa coefficient of 0.55
provide balanced measures of thealgorithm’s overall performance.
ESA-JAXA Pre-Launch EarthCARE Science and Validation Workshop | 13 – 17 November 2023 | ESA-ESRIN, Frascati (Rome), Italy

EarthCARE-MPLNET Cal/Val NNH20ZDA001N-EUSPIIntercomparison at Universitat Politecnica de Catalunya, Barcelona Spain
UPC; 41.38N 2.11E, 125 m a. s. l., 2019-2020, 519 rain events

ESA-JAXA Pre-Launch EarthCARE Science and Validation Workshop | 13 – 17 November 2023 | ESA-ESRIN, Frascati (Rome), Italy

EarthCARE-MPLNET Cal/Val NNH20ZDA001N-EUSPICases in Disagreement at GSFC

04 May 2021

Time

21-22

MPLNET(2=1 min precip.)112

Disdrometer(mm/hr)

0ESA-JAXA Pre-Launch EarthCARE Science and Validation Workshop | 13 – 17 November 2023 | ESA-ESRIN, Frascati (Rome), Italy

EarthCARE-MPLNET Cal/Val NNH20ZDA001N-EUSPICases in Disagreement at GSFC

26 Feb 2021

Time
23-24

MPLNET(2=1 min precip.)

Disdrometer (mm/hr)

01.13

ESA-JAXA Pre-Launch EarthCARE Science and Validation Workshop | 13 – 17 November 2023 | ESA-ESRIN, Frascati (Rome), Italy

IMERG(mm/hr)
1.045EarthCARE-MPLNET Cal/Val NNH20ZDA001N-EUSPIMay 2021 at GSFC
The range of rain events detectableby the RMA (sub 1 to > 20-mm/hr
based on Disdrometer).
RMA (and MRR) often appear todetect rain slightly ahead of theDisdrometer.

Thanks for your attention!

ESA-JAXA Pre-Launch EarthCARE Science and Validation Workshop | 13 – 17 November 2023 | ESA-ESRIN, Frascati (Rome), Italy

EarthCARE-MPLNET Cal/Val NNH20ZDA001N-EUSPI