Harnessing the power of forward models:past, present and future
EarthCARE Science & Validation Workshop, November 2023

Robin Hogan, Shannon Mason & Mark Fielding (ECMWF)
Fabian Jakub, Alessandro Battaglia & Chris Westbrookr.j.hogan@ecmwf.int© ECMWF December 6, 2023

Overview• Fast yet accurate forward models are crucial for:– Variational retrieval algorithms– Satellite simulators in weather & climate models (e.g. COSP)– Data assimilation• In this talk I will:– Illustrate how good forward models can be used to extract unexpected informationfrom the observations, using the CAPTIVATE (ACM-CAP) synergistic retrieval scheme– Highlight how they might offer new retrieval approaches and potentially suggest newsatellite concepts– Hack the two-stream equations mercilessly2

Two-stream source function (TSSF) method for infrared & microwave radiances• TSSF Underpins RTTOV-SCATT used for all-sky microwave assimilation at ECMWF• Used in CAPTIVATE for simulating infrared radiances, but can also simulate 94 GHz brightness
temperature

Change in up/downfluxes along path

Gain byscattering

Loss by scattering& absorption

Gain byemission

𝐹+𝐹−Increasing optical depth t

𝑑𝐹 +−= −𝛾1 𝐹 + + 𝛾2 𝐹 − + 𝐵𝑑𝜏
𝑑𝐹 −= −𝛾1 𝐹 − + 𝛾2 𝐹 + + 𝐵𝑑𝜏

Solve for fluxes
up and downToon et al. (1989), Bauer et al. (2006)

Computeradiance using
fluxes as thescattering source
function (treatspecularreflection in themicrowave)

3

Infrared
radiances, TSSF
Lidar

Solar radiance,
FLOTSAM“multiscatter”

Radar“multiscatter”

Radar PIA

94 GHz BT, TSSFCAPTIVATE forward models vs. A-Train observations

4How should CAPTIVATE
interpret the 94 GHzbrightness temperature?

• Warm or shallow rain

Radar PIA

• Rain with ice aloft• Dense ice aloft?
• Ice scattering cancels
rain emission?

94 GHz BT• Need to relax retrieval
assumptions to fully exploit
new measurements

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

5

Shortwave radiances• Shortwave TSSF lessaccurate for high opticaldepth & sun near thehorizon• FLOTSAM (forward lobe
two-stream radiancemodel): model a separate
stream for light scatteredinto the large forward lobe
in the phase function

𝜏 = 100

𝜏 = 100

𝜏 = 31

𝜏 = 31

𝜏 = 10𝜏 = 10𝜏 = 3.1

𝜏 = 3.1
𝜏=1
𝜏 = 0.1

𝜏=1
𝜏 = 0.1

Solve for fluxes
up and down
Computedirect solar
beam

𝐹+𝐹−

Compute radianceusing the diffuse
fluxes and the
direct beam as thesource function
https://github.com/rjhogan/flotsam
6

“Halifax” simulated EarthCARE scene• If we retrieve liquid cloud only where seen directly by lidar,
cannot match solar radiances• Solar radiances only matched if weretrieve liquid cloud in the presence ofrain, agreeing with model “truth”

True LWC

True rain rate

Retrieved LWCRetrieved rain rate

• This reduces retrieved rain rate by a factor of three,also better matching model “truth”!
• Need Cal/Val data to check inferences in the real world
Mason et al. (2023)

7

Time-dependent two-stream method (TDTS) for radar & lidar multiple scattering• Add time-dependent terms to two stream equations, part of“multiscatter” package for radar and lidar multiple scattering
1 𝜕𝐹 + 𝜕𝐹 +−= −𝛾1 𝐹 + + 𝛾2 𝐹 − + 𝑆 +𝜇1 𝑐 𝜕𝑡𝜕𝜏
−1 𝜕𝐹
𝜕𝐹 −+= −𝛾1 𝐹 − + 𝛾2 𝐹 + + 𝑆 −𝜇1 𝑐 𝜕𝑡𝜕𝜏
• TDTS is used in CAPTIVATE and 2C-RAIN-PROFILE• At night, CALIPSO alone can be used to estimate LWP and cloud base

Retrieved LWC

Compare toCloudSat PIA

Hogan (2008), Hogan & Battaglia (2008)https://www.met.reading.ac.uk/clouds/multiscatter/8

Experimental extension to simulate depolarization
Apparent backscatter

Depolarization

Wide-angle
scattering dominates

Small-angle
scattering
dominates

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS• Example of lidar
depolarization dueto multiple scattering• TDTS models anadditional variable:“co-polar”
backscatter, which isreduced with eachscattering event• Good agreementwith Monte Carlocalculations from
Alessandro Battaglia• Next radar in spaceshould have linear
depolarization ratio!
9

Poor Man’s Radiative Transfer in 3D (POMART3D)• What is the cheapest way to include 3D effects accurately and differentiably?
• POMART3D adds no new variables, just horizontally advects and diffuses two-stream fluxes
between columns as the solver progresses• Can be used for shortwave and longwave fluxes in a cloud-resolving model, or (in principle)
as the forward model in a 3D retrieval

Direct beam
is horizontally
advected
betweenlayers

𝐹+𝐹+𝐹+

𝐹−

𝐹−

𝐹−

Diffuse fluxes
are horizontally
diffused during
up/down passes
Radiances could
be advectedhorizontally

Column 1

Column 2

EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS

Column 3
10A cumulonimbus near Darwin at 200-m resolution• POMART3D captures the shadows and bright patches seen in Monte Carlo calculations(missing with the independent column approximation), at a tiny fraction of thecomputational cost

Model simulation: Charmaine Franklin, Monte Carlo calculations:
FabianJakub
October29, 2014

11

Outlook• Can we use the Cal/Val campaigns to evaluate ourinferences in complex scenes?
– How well can we infer liquid clouds embedded in rain andimprove rain rates via solar radiances & PIA?
– To what extent are 94 GHz brightness temperatures providing
additional information in mixed-phase clouds?

• What new forward models are needed to extract additional
information from novel measurements?

• Do sophisticated forward models open up possibilities ofnew satellite instruments (e.g. wide field-of-view lidar?)• Can we use fully 3D forward models to do a fully 3D
retrieval combining active sensors with (multi-view) imager?
• Can we answer these questions without resorting toMachine Learning? ☺

October 29, 201412