# Presentation: 262526

## Enabling the Transition of CPC Products to GIS Format

- Brian Doty
- Jennifer Adams
- Michael Halpert
- Viviane Silva

## Strategy

- Enhance the GrADS software package to:
- Output GIS formats
- Improve analysis and data management for multi-ensemble multi-model data sets
- CPC delivers products via the ArcGIS server

## The Grid Analysis and Display System (GrADS)

- Integrated environment for data access, analysis, and display
- Automation and batch processing
- Open source; active open source community
- Handles many data formats:  binary, netCDF, HDF, GRIB1, GRIB2, BUFR
- Distributed data via GDS and openDAP (GDS: GrADS Data Server)

## GIS Data Formats

- Shapefile
- Vector format
- Enhance the GrADS graphics package to produce vector data compliant with Shapefile standards
- Attribute database (metadata)
- Use metadata already internal to GrADS where possible; provide interface to provide additional metadata
- GeoTIFF
- Raster format
- Produce analytic grid products with GrADS and output directly to GeoTIFF format

## GrADS, Ensembles, Multi-model

-  Biggest design issue:  How to handle time
-  Examples follow

**Notes:**
Now that the ensemble handing is working for all data types including GRIB2, GrADS is ready to handle a lot of important 5D data sets. 

## Ensemble Forecast Time Series at a fixed location

- Ensemble Forecast Time Series at a fixed location

- Forecast Time --->

**Notes:**
Overlay the forecasts from 21 ensemble members, each drawn with a different color. The members all agree for the first 2-3 days, then begin to diverge in the 3-7 day period,and after that there is little coherence to the forecast. 

## Ensemble Forecast Grid  at a fixed location

- Ensemble Forecast Grid  at a fixed location

- Forecast Time --->

- Ensemble Member

**Notes:**
Here is the same data drawn as a 2-D Time v. Ensemble plot Each row in the grid represents one of the colored lines drawn in the previous plot. Pixels are colored according to the data values. I can draw it this way because the ensembles are handled as the 5th dimension in my gridded forecast data set. The well-defined  purple stripe is the event all members agree on at the beginning of the forecast, and the image gets noisier as the forecast evolves.

## Two Ensemble Forecasts at a fixed location

- Two Ensemble Forecasts at a fixed location

- Forecast Time --->

- Ensemble Member

**Notes:**
Now we expand our ensemble set to include the same data from the forecast initialized 12 hours earlier. Note the time axis has expanded a bit to accommodate the shift in the temporal coverage of the second ensemble forecast. The dominant characteristics of the data values don’t appear to have changed much from one forecast to the next. 

## Ten Ensemble Forecasts at a fixed location

- Ten Ensemble Forecasts at a fixed location

- Forecast Time --->

- Ensemble Member

**Notes:**
Now we expand our ensemble set again to include five days worth of forecasts (10 forecasts at 12hr intervals). Note the time axis has expanded even more to accommodate all 210 members.Now you can see how the purple stripe feature has gradually coalesced over the five-day period from something incoherent and noisy into a well-defined event. 

## 33 Ensemble Forecasts at a fixed location

- 33 Ensemble Forecasts at a fixed location

- Forecast Time --->

- Ensemble Member

**Notes:**
How many more forecasts can we add? As many as hardware and patience will allow. 
The data in this image are taken from a single GrADS data set: 33 forecasts, each with 21 members and 65 time steps. That’s 45,045 individual grib2 data files aggregated together. 550 mb of grib2 data unpacked into 3.5 Gb in memory.It took an hour to draw this plot, which included indexing, data I/O, and image rendering.GRIB2 does have a downside.

## Examples of Ensemble Data Sets

-  NCEP GFS Ensembles (GENS)
-  NCEP Short Range Ensemble Forecasts (SREF)
-  NCEP Climate Forecast System (CFS)
-  ESRL MRF Re-Forecasting Experiment
-  WCRP CMIP3 Multi-Model Data (IPCC AR4)
-  ENSEMBLES
-  THORPEX Interactive Grand Global Ensemble (TIGGE)

**Notes:**
Now that the ensemble handing is working for all data types including GRIB2, GrADS is ready to handle a lot of important 5D data sets. 

## Ensemble Data Sets Behind GDS 2.0

-  Data become more usable and accessible
-  Subsets over all dimensions
-  Server-side analysis
-  File aggregation and format translation
-  Metadata standards:

**Notes:**
Putting 5-D data sets behind GDS makes the data even more usable and accessible for a variety of reasons. 
Developing standards for ensemble metadata has opened a can of worms. 
Things I don’t like about proposed metadata standards: 1) casting attribute metadata as gridded data variables of type “char”2a) two-dimensional time axis, which separates reference time and forecast offset2b) unitless, abstract time axis, with reference time and forecast offset as separate data variables to be pieced together by the user
Our approach is to present metadata in such a way so that GrADS will understand it, and so that it shouldn’t interfere with other clients

## Ensemble Forecast Time Series at a fixed location

- Ensemble Forecast Time Series at a fixed location

- Forecast Time --->

**Notes:**
Overlay the forecasts from 21 ensemble members, each drawn with a different color. The members all agree for the first 2-3 days, then begin to diverge in the 3-7 day period,and after that there is little coherence to the forecast. 

## Ensemble Forecast Time Series at a fixed location

- Ensemble Forecast Time Series at a fixed location

- Forecast Time --->

**Notes:**
Overlay the forecasts from 21 ensemble members, each drawn with a different color. The members all agree for the first 2-3 days, then begin to diverge in the 3-7 day period,and after that there is little coherence to the forecast. 

## TIGGE Multi-Member Multi-Model Ensemble 500mb Geopotential Height valid August 30, 2008

- TIGGE Multi-Member Multi-Model Ensemble 500mb Geopotential Height valid August 30, 2008

- 7-day Lead

- 3-day Lead

- 5-day Lead

- 1-day Lead

**Notes:**
Here’s an example of what can be done with TIGGE data behind GDS -- 
Each color is an ensemble mean forecast of 500mb Height from 8 different TIGGE providers.The white contours show the multi-model ensemble average (198 members).The four panels show the forecasts with the same valid time but with lead times of 1, 3, 5, and 7 days. Note how the models 

## TIGGE MME Forecast Error and Ensemble Spread 500mb Geopotential Height valid August 30, 2008

- TIGGE MME Forecast Error and Ensemble Spread 500mb Geopotential Height valid August 30, 2008

- 7-day Lead

- 3-day Lead

- 5-day Lead

- 1-day Lead

**Notes:**
The gray-shaded background field is the standard deviation of the multi-model ensemble mean. The white blobs are areas where the models disagree, I.e., where the colored contours in the previous panel diverge. Note how these “clouds of variance” dissipate as lead time shrinks. 
The colored contours show the absolute value of the forecast error, which is the multi-model ensemble mean minus the analysis (the mean of the 00hr forecasts from the control run of each model). 

## TIGGE Multi-Member Multi-Model Ensemble 500mb Geopotential Height valid August 30, 2008

- TIGGE Multi-Member Multi-Model Ensemble 500mb Geopotential Height valid August 30, 2008

- 7-day Lead

- 3-day Lead

- 5-day Lead

- 1-day Lead

**Notes:**
Here’s an example of what can be done with TIGGE data behind GDS -- 
Each color is an ensemble mean forecast of 500mb Height from 8 different TIGGE providers.The white contours show the multi-model ensemble average (198 members).The four panels show the forecasts with the same valid time but with lead times of 1, 3, 5, and 7 days. Note how the models 

## TIGGE MME Forecast Error and Ensemble Spread 500mb Geopotential Height valid August 30, 2008

- TIGGE MME Forecast Error and Ensemble Spread 500mb Geopotential Height valid August 30, 2008

- 7-day Lead

- 3-day Lead

- 5-day Lead

- 1-day Lead

**Notes:**
The gray-shaded background field is the standard deviation of the multi-model ensemble mean. The white blobs are areas where the models disagree, I.e., where the colored contours in the previous panel diverge. Note how these “clouds of variance” dissipate as lead time shrinks. 
The colored contours show the absolute value of the forecast error, which is the multi-model ensemble mean minus the analysis (the mean of the 00hr forecasts from the control run of each model). 

## The End