# Presentation: 686936

## Assimilation of AIRS & IASI at ECMWF 

**Andrew Collard**

- Acknowledgement to Tony McNally

## Overview

**Introduction **
**Assimilation Configuration**
**Channel Selection**
**Cloud Detection**
**IASI First Guess Departures**
**AIRS & IASI Forecast Impacts**
**Water Vapour**
**Data Compression**
**Conclusions**

## Introduction

## Current and Future High-Spectral Resolution InfraRed Sounders on Polar Orbiters

## An IASI Spectrum

## IASI vs HIRS: The Thermal InfraRed

- IASI vs HIRS: The Thermal InfraRed

## AIRS vs HIRS Jacobiansin the 15μm CO2 band

**HIRS-4**

**HIRS-5**

**HIRS-6**

**HIRS-7**

**HIRS-8**

- Selected AIRS Channels:        82(blue)-914(yellow)

**1000 hPa**

**100 hPa**

## HIRS vs IASI: Temperature Retrieval Accuracy

## HIRS vs IASI: Response to Important Atmospheric Structure

- IASI

- HIRS

**Response to a structure the observation of which would have improved the forecast of the reintensification of Hurricane Floyd over SW France and SW England on 12****th**** September 1993.  (Rabier et al., 1996) **

## Assimilation Configuration

## Current Operational Configurations

**AIRS**
**Operational at ECMWF since October 2003**
**324 Channels Received in NRT**
**One FOV in Nine**
**Up to 155 channels may be assimilated (CO****2**** and H****2****O bands)**
**IASI**
**Operational at ECMWF since 12****th**** June 2007**
**8461 Channels Received in NRT**
**All FOVS received; Only 1-in-4 used**
**366 Channels Routinely Monitored**
**Up to 168 channels may be assimilated (CO****2**** band only)**

## Assumed Noise for AIRS & IASI Assimilation

- AIRS Instrument Noise
- Assumed Observational Error
- Clear Sky Background Departures

- Noise or First Guess Departure (K)

- Wavelength (μm)

## Assimilation Configuration:Channel Selection

## Why Select Channels?

**The volume of IASI data available is such that we do not have the computational resources to simulate and assimilate all these data in an operational timeframe**
**Not all channels are of equal use when assimilated into an NWP system**
**We choose channels that we wish to monitor (often with a view to future use)**
**We choose a subset of these channels which we actively assimilate**

## Pre-screened channels

## Selected Channels (2)

## AIRS 324 vs IASI 366

**IASI 366**
**AIRS 324**

## Comparison of Actively Assimilated Channels (1) 

## Comparison of Actively Assimilated Channels (2) 

- Comparison of Actively Assimilated Channels (2)

## Jacobians of 15μm CO2 Band

**Temperature Jacobian (K/K)**

**Pressure (hPa)**

## Assimilation Configuration:Cloud Detection

## Cloud detection scheme for Advanced Sounders

**CLOUD**

**AIRS channel 226 at 13.5micron**
**(peak about 600hPa)**

**AIRS channel 787 at 11.0 micron**
**(surface sensing window channel)**

**temperature jacobian (K)**

**pressure (hPa)**

**unaffected **
**channels **
**assimilated**

**contaminated **
**channels **
**rejected**

**Cloud detection scheme for Advanced Sounders**

**A non-linear pattern recognition algorithm is applied to departures of the observed radiance spectra from a computed clear-sky background spectra**.
**This identifies the characteristic signal of cloud in the data and allows contaminated channels to be rejected**

**obs-calc (K)**

**Vertically ranked channel index**

- The large number of AIRS or IASI channels
- allows improved measurement of the
- cloud-top height compared to HIRS

## Number of Clear Channels

**High Peaking Channels**

**Window**
**Channels**

## IASI First Guess Departures

## First –Guess Departure Standard Deviations in 15μm CO2 Band

- First –Guess Departure Standard Deviations in 15μm CO2 Band

**Calculated**
**Std. Dev.**

**Observed **
**Std. Dev.**

## First Guess Departure Biases in 15μm CO2 Band

**Ordered by**
**Wavenumber**

**Ordered by**
**Jacobian Peak Pressure**

## First Guess Departure Standard Deviations and Biases in the Longwave Window

**Bias**

**Standard**
**Deviation**

## First-Guess Departure Biases in Water Band

## First –Guess Departure Standard Deviations in Water Band

**Calculated**
**Std. Dev.**

**Observed **
**Std. Dev.**

## First –Guess Departure Standard Deviations in Shortwave Band

**Calculated**
**Std. Dev.**

**Observed **
**Std. Dev.**

## AIRS & IASI Forecast Impacts

## AIRS Impact at ECMWF

- 500hPa Geopotential Anomaly Correlation

**Southern Hemisphere**

**Northern Hemisphere**

## IASI Impact on SH Geopot. AC 

**No IASI**
**With IASI**

## IASI Forecast Scores Again: 500hPa Geopot. AC

**NH**

**SH**

**IASI**
**Better**

**IASI**
**Worse**

**8****th**** March-5****th**** May 2007**

## IASI Forecast Scores Again: 500hPa Geopot. AC

**NH**

**SH**

**IASI**
**Better**

**IASI**
**Worse**

**11****th**** July-26****th**** August 2007**

## AIRS/IASI Impact at ECMWF in Context: N. Hemis.

- No satellite experiments

## Anomaly correlation of 500hPa height for the Southern Hemisphere

- Anomaly correlation of 500hPa height for the **Southern Hemisphere**
- (average of 50 cases summer and winter 2003 verified with OPS analyses)

- Single instrument experiments

## 168 channels assimilated ... 8293 to go

## Using the IASI SpectrumLongwave CO2 Band

- 156 of 1st 500 channels are active
- Adjacent channels not used because of oversampling
- Other channels omitted because of ozone and humidity contamination

## Using the IASI SpectrumShortwave CO2 Band

- Short wave temperature bands: 350 channels
- Solar contamination, High Noise, Interfering species

## Using the IASI SpectrumChannels Primarily Sensitive to the Surface

- 2900 Window Channels

## Using the IASI SpectrumTrace Gases and RT Challenges

- Using the IASI SpectrumTrace Gases and RT Challenges

- 700 channels affected by non-LTE during the day

- 350 channels in the
- main ozone band

- Many other channels
- (not shown) affected by
- O3, CH4, NO2, CO etc.

## Using the IASI SpectrumThe 6.3μm Water Band

- Water band
- 3800 channels

## Water Vapour

## Use of Water Vapour Channels at ECMWF

**We get a small positive impact from using the water vapour channels**
**We use a cloud detection scheme that uses the first guess departures in the water band itself**
**The signal from water vapour can mimic cloud**
**The resulting clear channels also tend to be those where water vapour departures are smallest**
**We also assume 2K observation errors (c.f. instrument noise is ~0.2K).**

**(For AIRS)**

## In-band vs Cross-Band Cloud Detection

**AIRS 1583**
**1402cm****-1**

**Cross-Band**

**In-Band**

## Choosing 10 IASI Water Vapour Channels

- Grey channels are the 120 H2O
- channels distributed via the
- GTS

## Fit to other observations: 10 IASI Water Channels

- Fit to other observations: 10 IASI Water Channels

- Best value at ~1.5K

**Normalised**
**to unity here**

## Fit to other observations: Other Platforms

**NOAA-17 HIRS S.Hemis.**

**EOS-Aqua AIRS S.Hemis.**

**Black is expt. with IASI **
**humidity channels**
**Red is Control**

**The addition of the IASI water **
**band improves the analysis**
**fit to HIRS on NOAA-17 and increases usage of AIRS data.**

## Fit to other observations: Sondes

**Spec. Hum. S.Hemis. Sondes**

**Black is expt. with IASI **
**humidity channels**
**Red is Control**

**Rel. Hum. S.Hemis. Sondes**

**Spec. Hum. **
**N.Hemis. Sondes**

**The addition of the IASI water **
**band improves the analysis**
**fit to radiosondes**

## RH500 Forecast Impact Root Mean Square Error verified vs Operational Analysis

- 1st-23rd August 2007

- Expt
- Better

- Cntrl
- Better

**N.Hemis.**

**S.Hemis.**

## RH500 Forecast Impact: RMS Error verified vs Experiment’s Analysis

- RH500 Forecast Impact: RMS Error verified vs Experiment’s Analysis

- Expt
- Better

- Cntrl
- Better

**N.Hemis.**

**S.Hemis.**

**CY32R3**
**Control is Analysis of ****Experiment with H****2****O channels**
**1****st**** Aug.-9****th**** Sept. 2007**

## Operational Implementation of IASI Humidity Channels

**To be implemented in next operational upgrade (CY35R2)**
**10 IASI channels with 1.5K assumed error**
**7 AIRS channels with 1.5K error for consistency**

## Water Discussion

**We have to greatly inflate water vapour errors to avoid degrading the model**
**This is because of the large number of channels with error correlations between them (including bias)**
**By assuming greatly increasing the  water vapour observation errors we are negating the influence of inter-channel differences that allow us greater vertical resolution**
**Can we do a better job?**

## Data Compression

## Data Compression

**Advanced IR sounder radiances contain a lot of information (~30pieces) ...**
**...****but there are two orders of magnitude more channels.**
**Hence there is a large amount of redundancy**
**How can we use these data more efficiently?**

## Why is data compression important?

**Very large data volumes need to be communicated in near-real time (e.g., EUMETSAT to NWP centres)**
**Simulation of spectra (needed for assimilation) is costly**
**Assimilation is costly**
**Data storage**

##  Spectral data compression with PCA*

**of covariance of spectra from **
**(large) training set**

-  Spectral data compression with PCA*

- The complete AIRS spectrum can be compressed using a truncated principal component analysis (e.g. 200PCAs v 2300 rads)

- This allows data to be transported efficiently

**Original**
**Spectrum**

**Mean spectrum**

**Coefficients**

**N.B. This is usually performed in noise-normalised radiance space**

***Principal Component Analysis**

- To use PCs in assimilation requires an efficient RT model to calculate PCs directly
- PCs are more difficult to interpret physically than radiances

## Loop of Jacobians of PCs

##  Spectral data compression and de-noising

**of covariance of spectra from **
**(large) training set**

-  Spectral data compression and de-noising

- The complete AIRS spectrum can be compressed using a truncated principal component analysis (e.g. 200PCAs v 2300 rads)

- Each reconstructed channel is a linear combination of all the original channels and the data is  significantly de-noised.

**Original**
**Spectrum**

**Mean spectrum**

**Reconstructed**
**spectrum**

**Coefficients**

**N.B. This is usually performed in noise-normalised radiance space**

- If *N* PCs are used all the information is contained in
*N* reconstructed channels (theoretically)

## AIRS Reconstructed Radiances

**Data are supplied in near-real time by NOAA/NESDIS in the same format as the “real” radiances.**
**The same channels are supplied, except some “popping” channels are missing**
**Based on 200 PCs**
**QC Flag supplied**

## First Guess Departures for AIRS are Reduced

- Wavelength (μm)

- 4

- 14

- 0

- 1.0

- First Guess Departure Std. Dev. (K)

**Instrument noise is the main contributor to FG departure in the 15****μ****m CO****2**** band only**

- Original

- Reconstructed

## A look at Reconstructed Radiances’ Errors

**Reconstructed Radiances**

**Covariances of background departures for clear observations in 15μm CO****2**** band**

**Instrument noise is dominant and diagonal.  Correlated noise is from background error**

**Instrument noise is reduced (std. dev. Is approximately halved) but has become correlated.  **

**15μm**

**14.5μm**

**14μm**

**13μm**

**Original Radiances**

## Improvements in Cloud Detection

- Original

- Reconstructed Radiances

**First Guess Departure (K)**

**First Guess Departure (K)**

**Height-ranked Channel Index**

**Height-ranked Channel Index**

- < 2% of Channels are
-  flagged differently for
- RR vs Normal radiances

- Last Clear = 70

- Last Clear = 75

- ECMWF Scheme:
- Ranks Channels Height
- Applies low-pass filter
- Tests for non-zero gradient

## Reconstructed Radiances Result in Different Increments...

**Mean Temperature **
**Increments over Antarctic**
**Plateau: March-April 2005**

**Reconstructed Radiances**
**Control**

## ...and can improve the analysis

**Control**
**Reconstructed Radiances**

**Damping of stratospheric **
**Oscillations before use of GPSRO **
**and AMSU-A Ch 14 fix**

## Forecast Impact of Reconstructed Radiances

- Essentially Neutral

## Future Dissemination and Use of Advanced Sounder Radiances

**Is it acceptable to receive radiances in the form of Principal Component Amplitudes?**
**While forecast impact on changing to reconstructed radiances has been neutral, the detailed increments and forecast changes are not identical**
**Would we lose information we could use?**
**Can we make better use of the observations?**
**RR still seems more promising than direct use of PC scores as Jacobians of PCs are far less localised.**
**Exploitation of information in IASI spectrum is not currently limited by the number of channels that can be handled by the minimisation**
**See water example**
**Good coverage of LW temperature channels**

## Conclusions

**AIRS and IASI have been operational at ECMWF since October 2003 and June 2007 respectively**
**IASI and AIRS have demonstrated positive impacts on the ECMWF NWP model and form an important part of the assimilation system**
**We are in the process of better understanding our use of the humidity channels.  IASI humidity channels will be assimilated with the next operational upgrade.**
**Advanced IR Sounder radiances may be efficiently compressed through Principal Component Analysis.  This may be used for NRT dissemination.  The advantages of this datatype in NWP remain to be demonstrated.**

## Questions?

## IASI Spectral Correlation

- Correlations
- from forecast
- model

- Nearly
- diagonal
- instrument
- noise

- Covariance
- of first-guess
- departures
- 1st 200 channels

**IASI Channel Number**

**IASI Channel Number**

## IASI Spectral Correlation

- Expected correlation
- structure from
- apodisation of
- IASI spectrum

**IASI Channel Number**

## Sensitive areas and cloud cover

**Sensitive areas and cloud cover**

- Location of sensitive
- regions
- Summer-2001
- (no clouds)

- monthly mean
**high** cloud cover

- monthly mean
**low** cloud cover

- sensitivity surviving
**high** cloud cover

- sensitivity surviving
**low** cloud cover

- From* McNally (2002) QJRMS ****128***

## O-A Stats for NOAA-16 AMSU-B

- Larger Assumed AIRS H2O Errors = Better Fit to AMSU-B

## Choosing 84 IASI H2O Channels

- Chosen from the the 366 Channels
- Black channels have stratospheric
- contribution

## Fit to other observations84 IASI Water Channels

- Fit to other observations84 IASI Water Channels

- Best value
- at ~4K

**Normalised**
**to unity here**

**MetOp **
**HIRS-12 **
**Tropics**

## Cloud

## Cloud

**Cloud in the field of view can greatly affect our ability to use IR radiances**
**Studies have show that the most important areas to measure for accurate forecasts often have cloud**
**We need to identify strategies to deal with cloud**

## Dealing with Cloud

**Use only clear fields of view**
**Low (~5%) yield**
**Use only channels unaffected by cloud**
**Low yield in lower tropospheric channels**
**Cloud clearing**
**Simulate a clear observation by using multiple fields of view and assume that only cloud fraction changes between them**
**Simultaneous analysis of cloud optical properties**
**These observations are rich in cloud information**

**Used now operationally**

**Under Investigation**

## Cloud Cleared Radiances

**“****clear” spectrum from an array of partially**
**cloudy fields-of-view (9 in the case of AIRS)**
**Assumes the cloud height in each FOV is **
**identical and only cloud fraction varies between the FOVs.**
**Initialised with regression from AMSU-A radiances**
**Can calculate a noise amplification factor which is the basis of **
**the QC flag**
**Supplied by NOAA/NESDIS in near-real time but with poorer **
**timeliness than conventional radiances**

## Cloud-Cleared Radiances

**AIRS Ch. 221.  QC Flag Used.  No Cloud Detection**

## Cloud Cleared Radiances

**AIRS Ch. 221.  QC Flag Used.  With Cloud Detection**

**Still have positive tail**

## Cloud Cleared Radiance Current Status		

**The geographical location of outliers around regions which are flagged as of poor quality indicates that the supplied QC flag may need revising.**
**Referred back to data suppliers before looking at revising cloud detection**
**Recent degradation of AMSU-A Ch 4 may affect quality**
** **

## Simultaneous Analysis of Cloud Properties(Tony McNally)

**Derive a cloud-top pressure (CTP) and cloud fraction from observed radiances with a 2-D least-squares fit in the screening run**
**For overcast FOVs use all channels (that are currently used for clear sky case)**
**This has the advantage of reducing the degrees of freedom **
**For other FOVs revert to operational cloud-detection scheme to identify clear channels**
**Assimilate these radiances with CTP as a sink variable**

## Using data in cloudy areas

- Clear data coverage of mid/lower tropospheric sounding radiances:
- IASI 434 (METOP-A)
- AIRS 355 (AQUA)
- HIRS 7 (NOAA-17 / METOP-A)
- Colour indicates first guess departure

- Additional overcast locations where cloudy radiance analysis fills gaps due to cloud detection rejections:
- IASI 434 (METOP-A)
- AIRS 355 (AQUA)
- HIRS 7 (NOAA-17 / METOP-A)
- First guess departures similar to clear data after QC of *complex* clouds

**Tony McNally**

## Temperature increments at the cloud top

**Temperature increments at the cloud top**

- Cell of very high overcast clouds off the coast of PNG

- All channels collapse to near delta-functions at the cloud top giving very high vertical resolution temperature increments just above the diagnosed cloud

- Temperature increments (IASI)

**blue=ops**
**red=ops+ cloudy IR**

**Tony McNally**

## Temperature increments

- Monthly averaged RMS temperature increments at 500hPa
- CTRL minus EXPT

- Possibly some reduced increments at isolated stations

**Tony McNally**

## Cloudy Retrievals Current Status		

**Technically works**
**The restriction to overcast and heavy QC currently yields < 10% extra data **
**The small amount of additional data to not influence the bulk characteristics of the analysis or departure statistics **
**The extra cloud top increments look reasonable at overcast locations – have very high vertical resolution**
**Forecast performance essentially neutral**
**Will continue to search for individual cases of forecast impact using singular vectors**
**Next steps ...use the data in more complex cloud conditions / Marine Stratocumulus / use NWP cloud information ?**

**Tony McNally**

## Land Surface

## Land and Sea Ice

**AIRS data is used over land for channels not sensitive to the surface**
**Addition of similar IASI channels had neutral impact and was not pursued**
**We do use these data over sea ice – treating the surface as sea.  Not ideal but...**
**Removing channels results in negative impact**
**Modelling surface emissivity results in negative impact**
**In the future we aim to implement an emissivity analysis using leading EOFs of the emissivity spectrum**
**the interaction with the cloud detection scheme will be challenging**