# Presentation: 511556

## NASA Simulation Experiment for Airborne Remote Observations (completed Nov 2005)

- <date/time>

- A global Observing System Simulation Experiment (OSSE) was recently conducted at NASA’s Goddard Laboratory for Atmospheres for the Boeing Company.  The following slides describe the experiment design, illustrate the results of the experiment, and provide conclusions based on these results.
- The purpose of this effort was to design and execute an OSSE to evaluate the potential impact of adding two airborne remote observing systems to the current observing network.  The sensors, proposed by Boeing, include a direct detection Doppler Wind Lidar (DWL) and a Global Positioning System limb occultation receiver (GPS).  Both observing systems will furnish remotely-sensed atmospheric profiles from commercial aircraft platforms.

**NASA Simulation Experiment for Airborne Remote Observations (completed Nov 2005)**

## Basic Steps in the Performance of the OSSE

- <date/time>

**Basic Steps in the Performance of the OSSE**

- Simulate DWL and GPS observations
- Execute global assimilations and forecasts using simulated observations from current observing network (Control)
- Execute global assimilations and forecasts using Control observations  and observations from DWL and GPS (Control + DWL + GPS)
- Evaluate results using metrics designed to highlight the potential impact of these new observing systems on key large-scale and synoptic scale atmospheric analysis and forecast fields

## Core Components of OSSE

- <date/time>

**Core Components of OSSE**

- _Nature run_

- Model forecast data set from which all simulated observations are extracted.  Also used as the verification or “truth” data set.
- Spatial resolution:  0.5° latitude x 0.625° longitude x 35 pressure levels
- Temporal resolution:  6 hours
- Forecast range:  06 UTC 11 Sep 1999 – 18 UTC 5 Nov 1999
- Full set of diagnostics with varying sea surface temperature field.

- _Data Assimilation System (DAS)_

- FVSSI (hybrid system using NASA FVGCM model coupled with NCEP SSI      analysis)
-   FVGCM:  1° horizontal resolution, 55 eta level model
-   SSI:  T62/64 level Spectral Statistical Interpolation system

- _Forecast model_

- FVGCM:  1° horizontal resolution, 55 eta level model

## Experiments with FVSSI

- <date/time>

**Experiments**** ****with FVSSI**

- 1.  Control

- Assimilation from 00 UTC 12 Sep 1999 – 18 UTC 31 Oct 1999  (50 days)
- Observations include all conventional, satellite cloud motion wind, Quikscat, and NOAA-12 and NOAA-14 HIRS and MSU satellite temperature retrievals.
- 5-day free forecasts initiated every 2 days starting 00 UTC 17 Sep 1999  (total of 20 forecasts).   Prognostic fields are saved every 6 hours.

- 2.  Control + DWL + GPS

- Assimilation from 00 UTC 12 Sep 1999 – 18 UTC Oct 1999  (50 days)
- Observations include all the observations used in the Control and DWL and GPS observations.
- 5-day free forecasts initiated every 2 days starting 00 UTC 17 Sep 1999  (total of 20 forecasts).   Prognostic fields are saved every 6 hours.

## DWL and GPS Simulation Methodology

- <date/time>

**DWL and GPS Simulation Methodology**

- The simulation of the observations was based on the following:
- Personal communication with Boeing and HRL scientists
- Available literature on space-borne or airborne DWL and GPS sensors
- 3.    Limitations of the DAS (limit to the number of observations, no GPS observation operator)

- DWL simulation

- U and V components of the wind are simulated
- Existing ACARS in situ observation locations, times, and flight levels provide the point at which profiles are generated
- 50km maximum horizontal resolution
- 250m vertical resolution from the surface to 3km altitude, 1km resolution above 3km
- Full profile from aircraft to surface assumed in cloud-free locations

## DWL and GPS Simulation Methodology (continued)

- <date/time>

**DWL and GPS Simulation Methodology (continued)**

- GPS simulation

- Profile length in locations of cloud cover is determined by probability of beam attenuation through nature run cloud cover beneath aircraft.  Standard FVGCM overlapping cloud algorithm used to compute total cloud fraction down to each potential observation level
- Random noise is applied to the observations with an estimated 1.0 meter per second error standard deviation
- Final profiles are formatted to resemble rawinsonde profiles for inclusion to DAS

- Temperature is the only simulated quantity; moisture is expected to be available from this observing system however it is NOT simulated in the current OSSE due to uncertainties regarding error levels and ability to properly utilize this quantity in current DAS (based on AIRS experiments).
- Existing ACARS in situ observation locations, times, and flight levels provide the point at which profiles are generated
- 85km maximum horizontal resolution

## DWL and GPS Simulation Methodology (continued)

- <date/time>

**DWL and GPS Simulation Methodology (continued)**

- An estimated along-beam resolution of 150m is adjusted to 6km to achieve a manageable number of observations for current DAS.  This method of thinning is consistent with application to real data of this nature.
- Profiles are simulated only after aircraft has reached a relatively stable cruising altitude.  400hPa level is chosen as a nominal lower cutoff.
- Assume no cloud attenuation
- Random noise is applied to the observations with an estimated 1.5 Kelvin error standard deviation

- Random selection of incoming radio beam direction is made for each profile to simulate the maximum possible reception angle lying 90° on either side of the aircraft’s flight path
- Random selection of incoming radio beam elevation is made for each profile in the range of 1° above aircraft to 4° below aircraft.  This is to simulate the possible reception of radio beam above and below aircraft relative to the flight path

## DWL and GPS Simulation Methodology (continued)

- <date/time>

**DWL and GPS Simulation Methodology (continued)**

- Maximum 200km profile length (≤10% profiles reach the ground)
- Final profiles are formatted to resemble ACARS in situ observations for inclusion to DAS

**Notes:**
The first week of the OSSE is affected by “spin-up” (the system getting used to the introduction of the new data) neverthless this plot still clearly shows the positive impact (blue contours) by the introduction of the DWL+GPS observations in the North Atlantic and, in a somewhat more broken pattern, in the North Pacific.  Negative impacts (red contours) are apparent in Canada and in scattered pockets elsewhere.

**Notes:**
By October, the OSSE has settled-down and we now see well-defined clustering of the positive impact in the North Atlantic and Pacific and a little less noise overall.  Negative impacts have become focused over Canada and around the North Pole, away from the location of the DWL+GPS observations.

**Notes:**
As with SLP, this 500 mb geopotential height field impacts indicates a somewhat noisy pattern, likely due to spin up, but generally positive impact in areas where there are DWL+GPS observations.

**Notes:**
The 500 mb geopotential height impact has become less noisy in October like the SLP impact field.  Positive impacts still mostly prevail over the Pacific and Atlantic Oceans and the U.S. with negative impacts concentrating in Canada and near the North Pole.

**Notes:**
All of the anomaly correlation line plots (next 11 slides) are based on an average of 23 five-day forecasts initiated every two days from the assimilation output.  The first forecast starts on 9/17/99 to avoid problems with spin up.
In the above plot, there is very slight, but positive improvement in the experiment which includes DWL+GPS beyond day 4 for the northern hemisphere as a whole.  Note:  higher anomaly correlation values indicate a better forecast.

**Notes:**
Somewhat more significant improvement is seen in the North Atlantic region, but still slight.

**Notes:**
On average, virtually no affect of the DWL+GPS in the North Pacific on the SLP. 

**Notes:**
Odd result over the U.S., which looks slightly positive but until day 4.  The cross-over of lines in day 4 may be an indication that the region is too small for this statistic.

**Notes:**
The following 4 geopotential height slides are similar to the SLP statistics only smaller, essentially negligible.

**Notes:**
The following 3 slides of 400 mb zonal wind anomaly correlations indicate slight improvement similar to the SLP.

**Notes:**
This plot is more for internal use.  The fields are averages over the September 11-30 period.  The top two panels show that in areas where DWL+GPS observations exist, the observations are drawing the analysis closer to the nature run.  The bottom left panel indicates the degree at which the DWL+GPS observations are contributing to the analysis relative to all other observing systems and clearly shows a significant contribution over the U.S. and coastal waters.

**Notes:**
This is the same as the last slide but for the month of October.  The results are similar.

## Conclusions

- RMS impacts (slides 11-14) indicate an improvement in the standard SLP and Z500 analysis fields with the inclusion of DWL and GPS observations in most areas where these new observations exist.
- The largest impact, especially in SLP, is over the North Atlantic (the maximum impact being about twice as large as that of the North Pacific).  This North Atlantic impact is likely from two sources:  local effects of the concentrated aircraft flight paths over the North Atlantic along the typical storm track; advection from upstream, where even greater sampling is being performed over the U.S.  Primarily, local effects are likely responsible for the smaller positive impact in the North Pacific.
- Over the continental U.S. the impact is almost negligible in SLP but is maximum over the southeast U.S. in Z500.  However, the average Z500 impact values are all quite small (maximum of only 6 meters over Florida in October).  Even though the vast majority of DWL and GPS observations are located over the U.S., this already well-sampled region leaves very little room for improvement and therefore little or no impact would be expected there.

## Conclusions (continued)

**   **

- Persistent negative impacts are very apparent over Canada and around the North Pole.  Little or no DWL and GPS observations were simulated in these regions so some of this effect may not be local.  Many of the fields that were examined (some of which are not included in this PPT file) show a very sharp boundary, near the U.S.-Canadian border, where impacts change sign.  This very closely delineates between a data-rich region, the U.S., where there is an immense concentration of conventional and DWL and GPS observations, and a data-sparse region, Canada, where relatively few conventional and DWL and GPS observations exist.  We believe that the negative impacts may be due to enhanced error growth of certain features due to artificial gradients created by the sharp data cutoff.  The DAS would then propagate these errors zonally and poleward.  It’s unclear as to whether this is entirely as a result of the very asymmetrical distribution of such a large number of DWL and GPS observations or a manifestation of the DAS.  In general, however, the addition of virtually any new observing system, real or simulated, results in both positive and negative impacts with all global data assimilation systems that have been used at NASA.  Still, this will require further investigation.
- The anomaly correlation figures (slides 15-25) show the forecasts with DWL and GPS observations included in the initial conditions to be, on average, the same or slightly better than the control  forecasts.  As in the RMS analysis impact figures, the forecast improvement appears to be greatest over the North Atlantic.  Small, incremental, forecast improvements from the addition of new and proposed observing systems to the existing observing network is typical, so the results of this OSSE are consistent with past real data and simulated data experiments for this metric.

## Glossary

- Nature run
- A long (generally one month or longer), high resolution model forecast having suitable outputs from which to create synthetic observations.  This forecast is used as verification or truth when evaluating DAS analyses and model free forecasts.

- Data Assimilation System (DAS)
- A method of blending large numbers of observations from a diverse set of observing systems with model background fields (ie model first guess) to obtain the best analysis possible.  The analysis is used as initial conditions for model free forecasts.

- Model first guess (or background fields)
- A 6-hour model forecast designed to extend an assimilation 6 hours forward in time prior to the introduction of observations.  Utilizing a model first guess when generating an analysis provides continuity between analysis times, fills gaps in data sparse regions, and reduces field imbalances that may be created by observations.

- Model free forecast
- A 5-day model forecast starting from an analysis.  The term ‘free’ is used to distinguish this type of forecast from the 6-hour first guess forecast that is run as part of the DAS.

## Glossary (continued)

- DAS cycle
- One 6-hourly fixed length of assimilation which involves generating the 6 hour first guess, gathering observations in a 6 hour block centered on the synoptic time (00,06,12, or 18 UTC), and using these observations to correct the first guess for that synoptic time.   Example:  Start with an analysis from the previous cycle at, say, 12 UTC; generate a first guess valid at 18 UTC; introduce the observations gathered from 15 UTC to 21 UTC; correct the first guess using these observations;  the result is an analysis at 18 UTC.

- Anomaly Correlation
- A commonly-used OSSE metric that compares a forecast with the nature run while using climatology as an anchor.  It’s also known as a pattern correlation since it emphasizes similarities in weather patterns, that is, it rewards the forecast if its weather patterns are similar to the nature run weather patterns.  Pattern consistency is an important aspect when evaluating a forecast.  Like the standard correlation, values range from –1 to 1; anomaly correlation values below 0.6 indicate no forecast skill.

- FVSSI – NASA hybrid DAS composed of FVGCM model and NCEP’s SSI analysis.

- FVGCM – Finite Volume General Circulation Model.

- SSI – NCEP’s Spectral Statistical Interpolation analysis system.

- SLP – Sea Level Pressure in units of millibars (mb) or hectopascals (hPa).

- Z500 – 500 mb geopotential height.  This and SLP are the most common fields examined.

**Notes:**
I’ve added this old slide to help describe the internal process of the DAS.  IC means initial conditions which is synonymous with analysis.   The first box ‘’IC From DAS’ at 00 UTC could be initial conditions from the previous DAS cycle of the OSSE or if it’s the beginning of the experiment (in our case 9/12/99 00 UTC) then the initial conditions may be from a real DAS  spin-up.  The essence of this is that the first guess field is corrected using the simulated observations collected at 06 UTC.