# Presentation: 440661

## CF Metadata Conventions: Governance, Support, and Future

- Karl E. Taylor
- Program for Climate Model Diagnosis and Intercomparison
- Lawrence Livermore National Laboratory
- Presented to the
- GO-ESSP Community Workshop
- Paris, France
- 11 June 2007

**Notes:**
Introductory remarks

## Intro to CF metadata conventions

- Climate and Weather Forecast Metadata that makes data files self-describing
- Examples of CF metadata
- Coordinate information needed to locate data in space and time
- Further grid information (e.g., grid cell bounds, area)
- Standard names – in conjunction with other attributes helps users determine whether data from different sources are the same physical quantity and helps distinguish variables within archives
- Some processing info. (e.g., zonal mean, climatological processing)

## Brief history of CF

- Started with “COARDS” standard, and developed through efforts of a handful of volunteers
- 2003: CF 1.0 released
- 2005: CF white paper discussing future governance circulated
- 2006: White paper revised and presented to WCRP WGCM
- 2006: Current governance structure established
- 2003-present: increasing acceptance of CF and adoption by several community coordinated projects:
- IPCC AR4 archive conforms
- PMIP, HTAP, regoinal groups, seasonal groups, ENSEMBLES ...

## Governance structure

- Original CF “authors” turned over control to two working committees:
- Conventions Committee
- Standard Name Committee
- Committee work done via email and web discussion (“Trac”)
- Support provided by:
- BADC: Alison Pamment (50%) – standard names
- PCMDI: Kyle Halliday (20%) – web site support
- WCRP/WGCM has been approached to appoint a CF Governance Panel charged with responsibility for stewardship of CF.

## What happened at last GO-ESSP meeting?

- Discussed CF governance & procedure for making changes
- Some pressing CF issues were identified and in some cases individuals volunteered to try to make progress
- Augmented grid information, especially for less structured grids.
- Relationship between CF and GIS grids & metadata
- Standard name issues (including dialects, profiles, hierarchy structures)
- Relationship between CF and netCDF4
- CF handling of in situ observations
- How to handle discovery information
- Sample datasets and reference implementations

## Agenda for today

- Governance issues – what needs fixing?
- Underlying principles – any adjustments needed?
- Technical issues – what’s been done? Can we make progress on any of the discussion topics?
- Supportive software – status reports
- Any additions? Other concerns?

## Conventions Committee: Support and Governance

- Agenda Item
- GO-ESSP Community Workshop
- Paris, France
- 11 June 2007

## Current committee members

- Karl Taylor (chair)
- Kyle Halliday (secretary)
- Balaji
- John Caron
- Jonathan Gregory
- Tom Gross
- Steve Hankin
- Jamie Kettleborough
- Russ Rew
- Rich Signell

## Terms of reference

- Further develop (revise) CF conventions
- Consider implementation of CF metadata conventions in other file formats (besides netCDF)
- Determine what is meant by CF conformance

## Current procedure for modifying/extending CF

- Anyone can propose a change
- Discussion opens & a member of the committee volunteers to moderate
- A provisional resolution is reached by consensus and summarized by moderator
- Reference files are produced illustrating the new feature
- If not a trivial change, trial implementations are carried out.
- If problems emerge iterate
- When consensus reached, the proposed change is provisionally accepted.   (advanced implementers may begin adoption)
- Collection of changes evaluated as a package leading to new CF versions released by consensus
- Test reference files become part of the “test data”

## Problems encountered.

- Progress on some proposals, but few have been carried through to implementation.
-  Contrast with standard names committee
- Few actions have taken place
- With a couple of exceptions, the volunteer moderation of discussion topics has floundered
- Procedure for introducing changes outlined in white paper:
- Email
- Web-based Trac system
- Jonathan’s thoughts

## Issues Concerning Underlying Principles of CF

- Agenda Item
- GO-ESSP Community Workshop
- Paris, France
- 11 June 2007

## Underlying principles

- Special needs of observations
- Should the scope of metadata accommodated by CF include:
- Description of experiment
- Description of model
- Description of higher-order grid-characteristics (e.g., relationship between u & v grids, which grid cells are connected to which)
- Competing needs of data producers & application developers
- CF extensions can be implemented faster by producers who sometimes are impatient for changes to be formally adopted
- Should requirement be relaxed that all CF metadata reside in the same file as the data itself?
- Bundling of common definitions of a quantity into I.D.’s (defined by a combination of attributes including the standard name + others)
- Aggregation layers
- CF profiles

## Miscellaneous Technical Issues

- Agenda Item
- GO-ESSP Community Workshop
- Paris, France
- 11 June 2007

## Miscellaneous technical issues

- Ensembles (including realization_weight and other issues): Alison
- Time axis “issues” for forecasts: Jonathan
- Definition of the “geoid”:  Alison
- Use of “anomaly”: Alison or Jonathan
- Subgrid variation description issues (use of “where”): Jonathan
- Time and calendar issues for paleoclimate simulations: Jean-Yves? Balaji?

## The Future of CMOR

- Karl E. Taylor
- Program for Climate Model Diagnosis and Intercomparison
- Lawrence Livermore National Laboratory
- Presented to the
- GO-ESSP Community Workshop
- Paris, France
- 11 June 2007

## The Climate Model Output Rewriter (CMOR)

- Provides a uniform interface for contributing CF-compliant data (adhering to “good practices”) to the growing number of MIP’s (e.g., CMIP, AMIP, CFMIP, PMIP, APE, HTAP)
-  For IPCC AR4, nearly all model output in the archive was written through CMOR.
- Assures CF-compliance and all MIP-established output requirements are met.
- Traps many metadata errors when files are produced.
- See http://www-pcmdi.llnl.gov/software/about_software.php

## Project-specific CMOR tables facilitate and ensure consistency of model output from contributing groups

- Proper specification of several coordinate attributes, including:
- Correct standard name
- “axis”, “positive”, and “formula_terms” attributes, as appropriate
- Proper specification of several variable attributes, including:
- Correct standard name
- Required dimensions
- “cell_methods” attribute
- A capability to
- Reorder axis order
- Reverse axis direction (or translate longitude dimension)
- Convert units (through udunits)

## CMOR flagged common errors, including

- Pointing out when required metadata are omitted.
- Rejecting incorrect metadata (wrong units, inadmissible attribute values, etc.)
- Rejecting inconsistent coordinate dimensions passed by user to CMOR.
- Rejecting non-monotonic coordinate values or inconsistent boundary values, as passed by user.
- Rejecting values that are clearly unrealistic (likely indicating improper units conversion or incorrect sign).

## What changes will be implemented in the next 6 months?

- Station data (needed, for example, by the HTAP project), including
- Station names
- Station locations
- Output from regional models
- With logically rectangular grids
- But with non-cartesian longitudes and latitudes
- i.e., data(i,j), lon(i.j), and lat(i,j)

## Longer term prospects for CMOR

- Difficult to see extending CMOR to write the grid-description files mandated by the GridSpec proposal.
- Probably possible, however, to write the actual data files through CMOR (and leave it to other software to produce the special GridSpec files).