CALIPSO Validation Lessons Learned
Dave Winker,Mark Vaughan
and the CALIPSO team

2nd EarthCARE Validation Workshop, 25-28 May 2021

1

Validation Objectives• Verify instrument performance- calibration, SNR, linearity, transient recovery• Verify geolocation- pointing, altitude registration• Quantify the accuracy and precision of Level 2 science dataproducts- identify sources of random errors and biases• Examine underlying assumptions in retrieval algorithms- Sa, Sc, spectral independence of cirrus backscatter• Supports quality assurance, algorithm improvement activities

2

From yesterday: “… calibration is a never-ending process … regular
reprocessing campaigns … improvements of the Level 1 dataset”

Each data release involves anew round of validation:3

Example: Stratospheric Aerosol Product• Improved calibration in Version 4 (2014) allowed aerosol retrievals to 30 km• First stratospheric aerosol product released in 2018
Validation against solar occultation and limb scattering sensors
CALIOPSAGE IICALIOPOMPSCALIOPOSIRIS

Time-height cross sections of CALIOP stratosphericaerosol extinction, Jan 2007 to Dec 2017 (Kar et al.,2019)

Zonally averaged CALIOP extinction coefficient profiles fordata acquired between 30°N and 30°S compared to datafrom SAGE III-ISS, OMPS, and OSIRIS (Kar et al. 2019)4

Initial Level 1 Validation•Early validation of Level 1 profiles was critical•

Targeted airborne campaign in Aug 2006 (CC-VEX)•Payload on NASA ER-2:– Goddard Cloud Profiling Lidar (CPL)– JPL W-band radar (CRS)– MODIS Airborne Simulator (MAS)•Initial CALIOP Level 1 validation objectives:– Sanity check on Level 1 lidar profiles• Do they ‘look right’? Unexpected artifacts?

– Verify predicted detection sensitivity– Radiometric calibration– Relative calibration of perpendicular channel (PGR)• Performed using on-board pseudo-depolarizer

5

CC-VEX Results (Aug 2006)

11 August 2006

CALIOPCALIOP (black)
CPL (blue)
CPL

Rayleigh backscatter profiles:
CALIOP – red
CPL – blue
rawinsonde - black6

Recent Improvements in Polarization Calibration
Polarization Gain Ratio Trend(using on-board calibration hardware)

Recently identified a long-suspectedday/night depolarization bias (~ 3%)using special operations:
daynight

Will improve daytime polarization
calibration in next data release7

Level 2 Validation
Uncertainty in Particulate Backscatter Coefficients at Altitude ns2 ( bp,n )bp,n 2222
ææ 22
æ s2 ( bp,n -1 ) ö ö ö
æ s2 ( bm,n ) öö2 æ s ( S)sX
s2 ( h) ö æç s ( Tp,n -1 )
æö
()
1n2ç
2
÷÷÷
= An ç+ç ÷ ç+ Bn ç
÷ + ( 2 htp,n ) ç 2 + 2 ÷ +÷2222ç è X n ø è R n ø ç bm,n ÷ç bp,n -1 ÷ ÷ ÷Sh ø ç (T2 )èèøp,n1èøøøèèMeasurementUncertainty

Molecular Number Density
Uncertainty
Lidar Ratio Uncertainty

Includes errors due toð
ðð
ð
ð
ð
ð

Calibration

SNR
molecular density (again)
offset calculations
polarization gain ratio
polarization cross-talkranging

Accumulated Aerosol
Attenuation Uncertainty
Multiple Scattering Uncertainty

LEGEND
S= lidar ratio b = backscatter coefficient
R= scattering ratio s 2(x) = variance of xT= transmittance t = optical depth
m= molecular p = particulate (e.g., aerosol)P= measured data C = calibration constanth= multiple scattering factor

rn 2 × P ( rn )X n = X ( rn ) =C × Tm2 ( rn )ö
æ R öæ
1An = ç n ÷ × ç÷÷çè R n - 1 ø è 1 - R n ×bm,n × S ×h×Drn ø

B n = S ×h×Drn ×bp,n -1

8Unique Challenges for Active Profilingl

70 meter 'swath' – spatial matching to ground sites is rarel

Few validation datasets available to validatell

profile measurements

lnighttime aerosol retrievals

laerosol retrievals above (low SNR) or below (attenuation correction) clouds

Global data products means global validation

1-day coverage

9Validation Resources– Ground-based networks• Aeronet• Earlinet, ADnet– Satellite comparisons• MODIS, MLS, AIRS
• CALIOP vs. IIR– Targeted, continuing aircraft campaigns• LaRC HSRL (King Air)Jun 2006 - 20012• NOAA ESRL (Cessna)

Jul-Aug 2006-2009– Large field campaigns• NASA AMMA (Cape Verde)

Aug 2006• SAMUM

2006, 2008• CIRCLE-2May 2007

• NASA TC4 (Costa Rica)

Jul-Aug 2007

• ASTAR/PAM-ARCMIP

April 2007/09/11/12• ARCTAS/PolarCat

April, July 2008

• SEAC4RS

Aug-Sep 2012

• etc
10

Comparisons with Aeronet•

Useful, but need to accumulate samples over several years– Typically find one or two usable samples/station/year– Only provide column properties
1.5

Dhadnah (all - 22)

CALIOP AOD

1.0

0.5

0.0
0.0

0.2

0.4

0.6

0.8

1.0

Aeronet AOD
0.6

1.50.5

Abu Al Bukhoosh (all - 26)

0.4

CALIOP AOD

1.0

CALIOP AODOmar et al, 2012: 1081 samples over 4years (600 cloud-free)
Identified problems due to:
Spatial mismatch
Topography
Aeronet cloud contamination

Sede Boker (all - 36)

0.5

0.3

0.2

0.1

0.0
0.0

0.5

1.0

Aeronet AOD

1.5

0.0
0.0

0.2

0.4

Aeronet AOD

0.6

in addition to CALIOP retrieval errors
11532 nm Calibration Assessment
CALIPSO

4/17/2009

HSRL

Bias =
3.4%

12

HSRL used as independent
check on CALIOP calibration
Over 120 underflights since 2006:
during various campaigns
and dedicated flights

Biases and uncertainties in method estimated at 4.5%±3.2%(Rogers et al, ACP, 2011)13

Validation of Level 2 aerosol from HSRL
Direct characterization of extinction
sensitivity:• histogram of HSRL aerosol inside layers
detected by CALIOP vs. histogram of all
HSRL aerosol

Validation of CALIOP aerosol typing:
HSRL measured lidar ratio vs. CALIOP aerosol type (daytime)(Rogers et al, 2012)14

Final Thoughtsl“Truth” is elusivel

Validation is never finishedl

It is only approached asymptoticallyl

Validation of products globally is challengingl

Large field campaigns useful, but:llNot exactly “validation” but critically importantlll

Validation tends to be one objective of many
Consistency checks (does it 'look right’)
Intercomparison of multiple retrieval algorithms based on
independent assumptions

Satellite sensors may have limited accuracy, but are globallDepending on parameter and sensor, comparisons can varyfrom ‘sanity check’ to true validation

15