# Presentation: 107517

## (TIGGE and) the EU Funded BRIDGE project

**Baudouin Raoult**
**Head of Data and Services Section**
**ECMWF**

- GO-ESSP Paris. June 2007

**THORPEX was established in May 2003 by the 14th WMO  Congress as a ten-year international global atmospheric research and development programme**
**THORPEX is under the auspices of the WMO Commission for Atmospheric Sciences (CAS)**
**THORPEX Goals:**
**To reduce and mitigate natural disasters**
**To fully realise the societal and economic benefits of improved weather forecasts**

- GO-ESSP Paris. June 2007

## TIGGE

**THORPEX Interactive Grand Global Ensemble**
**Key objectives of TIGGE:**
**Enhanced collaboration on development of ensemble prediction, internationally and between operational centres and universities**** **
**New methods of combining ensembles from different sources and of correcting for systematic errors **
**A deeper understanding of the feasibility of interactive ensemble system responding dynamically to changing uncertainty **
**The development of a prototype future Global Interactive Forecasting System****  (GIFS)**

- GO-ESSP Paris. June 2007

## The TIGGE core dataset 

**Global ensemble forecasts to around 14 days generated routinely at different centres around the world**
**Outputs collected in near real time and stored in a common format for access by the research community**
**Easy access to long series of data is necessary for applications such as bias correction and the optimal combination of ensembles from different sources **

- GO-ESSP Paris. June 2007

## Homogeneity of the TIGGE database

**Homogeneity is paramount for TIGGE to succeed**
**The more consistent the archive the easier it will be to develop applications**
**There are three aspects to homogeneity:**
**Common terminology (parameters names, file names,...)**
**Common data format **
**Definition of an agreed list of products (Parameters, Steps, levels, ...)**

|  | NCEP | ECMWF |
| --- | --- | --- |
| precipitation  | precip | TP |
| Mean sea level pressure | prmsl | MSL |
| Temperature  at 2 meters | t2m | 2T |

- GO-ESSP Paris. June 2007

## Completeness

**The objective is to have 100% complete datasets at the Archive Centres**
**Completeness may not be achieved for two reasons: **
**The transfer of the data to the Archive Centre fails**
**Operational activities at a data provider are interrupted and back filling past runs is impractical**
**Incomplete datasets are often very difficult to use**
**Most of the current tools used for ensemble forecasts assume a fixed number of members from day to day **
**These tools will have to be rewritten**

- GO-ESSP Paris. June 2007

## Strong governance

**Precise definition of:**
**Which products: list of parameters, levels, steps, units,...**
**Which format: GRIB2**
**Which transport protocol: UNIDATA’s LDM**
**Which naming convention: WMO file name convention**
**Only exception: the grid and resolution**
**Choice of the data provider**
**Best possible model output**
**One must do what one requires from others**
**Sample dataset available**
**Various GRIB2 tools, “tigge_check” validator, ...**
**Scripts that implement exchange protocol**

- GO-ESSP Paris. June 2007

## Phased implementation of the archive

**Phase 1: multiple instances, low development effort**

**Phase 1**
**Archive**
**Centres**

- GO-ESSP Paris. June 2007

## Building the TIGGE database

**Three archive centres: CMA, NCAR and ECMWF**
**Ten data providers:**
**Already sending data routinely: ECMWF, JMA (Japan), UK Met Office (UK), CMA (China), NCEP (USA)**
**Coming soon: BOM (Australia), CPTEC (Brazil), KMA (Korea), MSC (Canada), M****é****t****é****o-France (France)**
**Exchanges using UNIDATA LDM, HTTP and FTP**
**Firewall issues with China**
**Operational since 1st of October 2006**
**43 TB, growing by ~ 1 TB/week**

- GO-ESSP Paris. June 2007

## Quality Assurance

- GO-ESSP Paris. June 2007

## Accessing TIGGE

**Portals at NCAR and ECMWF. CMA portal to be developed**
**ECMWF portal offers:**
**Access to offline data**
**Aggregation along any axis (date, level, parameter, origin, ensemble, ...) **
**Provision of multi-model data on a single grid  (regridding to any lat/lon grid)**
**Sub-area selection**
**Reduces volumes to be downloaded by many order of magnitude**

- GO-ESSP Paris. June 2007

## Phased implementation of the archive

**Phase 2: distributed approach, higher development effort**

- GO-ESSP Paris. June 2007

## BRIDGE

**Bridge is a 2 years project funded by the EC under the FP6-IST programme. **
**It will demonstrate the benefits of GRID technology for international cooperation, in particular between Europe and China**
**This work focuses on the development of interoperable Grid infrastructures (CNGrid, GRIA) **
**Three applications: Aircraft Design, Meteorology, Drug Design**

- GO-ESSP Paris. June 2007

## Meteo scenario: exploring TIGGE phase 2

**Distributed processing on distributed data, across the two GRID middleware**
**Each site hosts only part of the data**
**Each site offers basic operations on the data (e.g. computing an average)**
**Strategy: minimize data transfers**
**Run operations at data location**
**Decompose operations in simpler ones **
**Most of the time intermediate results are much smaller**

- GO-ESSP Paris. June 2007

## EPS Products

**Examples**
**Ensemble mean**
**Standard deviation**
**Clustering**
**Probability of weather events**
**Extreme Forecast Index**
**EPSgram**
**Some products can be decomposed in simpler operations on a subset of members**
**Ensemble means**
**Some products need all the members **
**Clustering**

- GO-ESSP Paris. June 2007

## Example 1: ensemble mean

**Data requests describes 10 fields**
**6 are available from ECMWF**
**4 are available from CMA**
**S1 = sum(6 ECMWF fields) performed at ECMWF**
**S2 = sum(4 CMA fields) performed at CMA**
**Intermediate results and associated metadata moved to site were user invoked request: (S1,6) and (S2,4)**
**Final result A=(S1 + S2)/(6+4) computed locally and returned to user**
** **

- GO-ESSP Paris. June 2007

## Example 2: EPSGram

**Send lat/lon location at each sites**
**Receive list of values back**
**Compute distributions**
**Generate plot**

- GO-ESSP Paris. June 2007

## Non-decomposable operation

- GO-ESSP Paris. June 2007

## Decomposable operation

- GO-ESSP Paris. June 2007

## Operations as Workflows

**Study a selection of EPS products**
**Classify them into decomposable and non-decomposable **
**For decomposable operations**
**Implement sub-operations as web services **
**Deploy several instances of these web services **
**Describe full operation as a workflow**
**User will not see the workflow, but the high level operation**
**There may be several ways to execute a workflow**
**Because the same sub-operations are offered by several sites**
**Choose the “cheapest” workflow**
**Cost of a workflow based on amount of data moved**

- GO-ESSP Paris. June 2007

## Conclusion

**TIGGE Phase 1 is progressing well**
**Strong governance**
**Very good working relationship between CMA, NCAR and ECMWF**
**BRIDGE project gives us the funding to explore how to implement TIGGE Phase 2**

- GO-ESSP Paris. June 2007