NWP model evaluation using
ACTRIS cloud profiling
Ewan J. O’ Connor(FMI, Finland)
EARTHCARE 2nd Validation Workshop
online25-28 May 2021

Climate and NWP Model Evaluation• Challenge: Compare 3D volume to 2D slice- Satellite§

2D slice in space

Y direction (km)X direction (km)EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Climate and NWP Model Evaluation• Challenge: Compare 3D volume to 2D slice- Satellite§

2D slice in space- Ground-based§

1D – use winds to advect -> 2D

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Climate and NWP Model Evaluation• Challenge: Compare 3D volume to 2D slice- Satellite§

2D slice in space- Ground-based§

1D – use winds to advect -> 2D§Re-bin to model grid§

Always along-wind

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Climate and NWP Model Evaluation• Challenge: Compare 3D volume to 2D slice
ModelObs

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Climate and NWP Model Evaluation• Challenge: Compare 3D volume to 2D slice- Options are:
Statistical§ Probabilistic§- Climatologies§

Mean profiles, distributions

https://doi.org/10.1175/BAMS-88-6-883

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

New NWP models• HARMONIE NWP model data from KNMI and SMHI
- Multiple versions to test different parametrizations

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Conditional sampling• Challenge: Compare 3D volume to 2D slice- Options are:
Statistical§ Probabilistic§- Climatologies§

Mean profiles, distributions- Issues:§

Conditional sampling– Varies with parameter of interest

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Conditional sampling• Challenge: Compare 3D volume to 2D slice- Options are:Statistical§ Probabilistic§- Climatologies§

Mean profiles, distributions- Issues:§

Conditional sampling– Varies with parameter of interest- Treat model in a similar manner

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Observation

ModelComposites: ECMWF, Lindenberg

Cloud fraction

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Observation

ModelComposites: ECMWF, Lindenberg

Cloud fraction

Ice water content

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Observation

ModelComposites: ECMWF, Lindenberg

Cloud fraction

Ice water content

Liquid water content

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Observation

ModelComposites: ECMWF, Lindenberg

Cloud fraction

Skill score SEDI

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Mace HeadChilbolton
Diurnal composite of cloud
fraction in summer
ECMWF model (on the right) now
Lindenberg has a good representation of thediurnal cycle.
Still not enough mid-level cloud
Potenza

Mace HeadChilbolton

Skill scores show that convection ishard to forecast:
Lindenberg• For Sc/Cu at the top of the BL• For deeper convection at Potenza

PotenzaClimate and NWP Model Evaluation• Routine metrics for evaluating clouds in Climate and NWP models- Climatologies§

Mean profiles, distributions- Evaluate NWP forecast skill§

Forecast the correct cloud at the right time

• Progress during ACTRIS– Annual composites– Can be on height or temperature grid
– Diurnal composites– Can be on height or temperature grid– Seasonal comparisons (Define your season or regime)– Extend skill score comparison– Long-term (interannual variability)– Forecast lead time
– Model version

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Solar forecast uncertainty
Impact of ECMWF cloudforecast on solar radiation
forecast

Tuononen, M., O'Connor, E. J., and Sinclair, V. A. (2019): Evaluating solar radiation forecast uncertainty,
Atmos. Chem. Phys., 19, 1985-2000, doi.org/10.5194/acp-19-1985-2019, 2019

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Solar forecast uncertainty
Impact of ECMWF cloudforecast on solar radiation
forecast
Expected to be noisy – low
clouds are inhomogeneous
However bias in clear sky andin fully overcast. Why?

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Solar forecast uncertainty
Impact of ECMWF cloudforecast on solar radiation
forecast
Expected to be noisy – low
clouds are inhomogeneous
However bias in clear sky andwhen fully overcast. Why?

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Solar forecast uncertainty
Impact of ECMWF cloudforecast on solar radiation
forecast
Expected to be noisy – lowclouds are inhomogeneous
However bias in clear sky andin fully overcast. Why?
Clear sky -> aerosol
climatology
Overcast -> thin clouds havewrong LWP
Tuononen, M., O'Connor, E. J., and Sinclair, V. A. (2019): Evaluating solar radiation forecast uncertainty,Atmos. Chem. Phys., 19, 1985-2000, doi.org/10.5194/acp-19-1985-2019, 2019

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Climate and NWP Model Evaluation• Challenge: Compare 3D volume to 2D slice- Satellite§

2D slice in space- Ground-based§

1D – use winds to advect -> 2D§

Always along-wind

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

Persistent cloud features

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

EARTHCARE Validation• Challenge: Compare 1D profile to 1D profile- Colocation issue• Evaluation:- Options are:Evaluation

Statistical (CFADs etc)
§ Probabilistic§- Issues:§

Conditional sampling different§

Spatial vs temporal sampling– aircraft• Check meteorological input dataEARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021

24Forecast lead-time

Model spin up
Assimilation preferred state

Model preferred state

EARTHCARE 2nd Validation Workshop – virtual – 25-28 May 2021