# Presentation: 917660

## Towards improvednetCDF-GIS interoperability:

- Potential utility of the “Well-Known Model” concept

- Phil Bentley, Met Office Hadley Centre

## Key Topics

- This presentation visits the following topics:
- Problem: The netCDF-GIS disconnect
- Partial Solution: Use of the WKM concept
- Examples of WKM usage
- Conclusions & Further Work

## Problem: The netCDF-GIS disconnect

## The Scenario

**Notes:**
User requirement: to view and analyze netCDF data within mainstream GI Systems.
The particular focus here is commercial off-the-shelf GIS packages.
As opposed to community-specific, netCDF-aware GIS packages such as IDV, CDAT, Ferret.

## The view from ncdump

- netcdf tasmean-1960-90 {
- dimensions:
- 	time = UNLIMITED; // (1 currently)
- 	height = 1;
- 	latitude = 73;
- 	longitude = 96;
- 	bound = 2;
- variables:
- 	float time(time);
- 	float bounds_time(time, bound);
- 	float height(height);
- 	float bounds_height(height, bound);
- 	float latitude(latitude);
- 	float bounds_latitude(latitude, bound);
- 	float longitude(longitude);
- 	float bounds_longitude(longitude, bound);
- 	float air_temperature1(time, height, latitude, longitude);
- 		air_temperature1:standard_name = "surface_temperature";
- 		air_temperature1:long_name = "Temperature at 1.5M";
- 		air_temperature1:units = "K";
- 		air_temperature1:cell_methods = "time: mean within years ...";
- 		air_temperature1:_FillValue = -1.e+30f;
- }

**Notes:**
Fairly standard 4D netCDF file.
Single surface temperature variable with 4 familiar dimensions.
30-year average of annual mean values.
Based on a global grid geometry model.
Cell bounds are specified.

## The view from CF Checker

- CHECKING NetCDF FILE: tasmean-1960-90-ll.nc
- =====================
- Using Standard Name Table Version 7 (2007-11-20T02:00:00Z)
- ------------------
- Checking variable: time
- ------------------------------------
- Checking variable: bounds_time
- ------------------------------------
- Checking variable: height
- ------------------------------------
- Checking variable: bounds_height
- ------------------------------------
- Checking variable: latitude
- ------------------------------------
- Checking variable: bounds_latitude
- ------------------------------------
- Checking variable: longitude
- ------------------------------------
- Checking variable: bounds_longitude
- ------------------------------------
- Checking variable: air_temperature1
- ------------------------------------
- ERRORS detected: 0
- WARNINGS given: 0

**Notes:**
Our netCDF file validates just fine using the latest version of the CF checker.

## The view from Panoply

**Notes:**
Displays just fine in Panoply.

## The view from CDAT

**Notes:**
Displays just fine in CDAT.
Likewise in a number of other netCDF-capable packages: ncView, IDV, ToolsUI, and so on.

## The view from ESRI ArcGIS

**Notes:**
This image is a snapshot from ESRI ArcGIS 9.2.
File imported using their netCDF visualisation tool, MultiDimension, which claims to be CF-compliant.
This was the best result we could achieve after trying numerous different options.
Appear to be grid rendering errors at both ends of the image.
Rendering errors arise from the fact that the ArcGIS fails to read and use some key CF attributes.
Notably the bounds attribute and associated cell bounds variables.
Consequently ArcGIS does not know how to correctly construct and position the grid cells.

## The view from ESRI Tech Support

- Quote paraphrased from ESRI Technical Support email:
***“******Shift your longitude coordinates half a grid cell to the east”***

**Notes:**
It took us several months of email dialogue to convince ESRI that our netCDF files were valid and that the ArcGIS netCDF viewer was 'broken as designed'.
ESRI has since acknowledged that the MultiDimension tool does not recognise and use key CF attributes, and in particular the bounds attribute..

## So what's going on?

- CF convention lacks an attribute for specifying explicitly and unambiguously the spatial data model used by a variable in a netCDF file.
- Is it (the data model) a regular, structured grid?
- Is it an unordered collection of point locations?
- Is it a series of vertical profiles or trajectories?
- Hence, GIS software developers & applications are forced to make inferences from esoteric combinations of CF/netCDF variables and attributes.
***Why do we make this so complicated?!***

**Notes:**
Yes, CF (and other) metadata attributes exist for describing aspects of the spatial data model.
However, their use and interpretation is somewhat arcane.
Need therefore to have in-depth knowledge of CF specification.
This is okay for software developers working in the climate science community.
But not so easy for mainstream GIS vendors that do not possess specialist CF/netCDF domain knowledge.

## Partial Solution: Use of the "well-known model" (WKM) concept?

**Notes:**
This is intended as a potential and partial solution.
By no means a full solution to the disjoint between the netCDF and GIS data models.

## Potential Utility of the Well-Known Model (WKM) Concept

- Mirrors the Well-Known Text (WKT) and Well-Known Binary (WKB) concepts used in e.g. OGC specifications
- Provides a **hint** to client applications as to how to interpret the spatial data model(s) encoded in a netCDF file.
- WKM values would be based on *de facto* and *de jure* geospatial data standards such as...
- OGC/ISO 19107 (spatial schema - aka feature geometry)
- OGC/ISO 19123 (schema for coverage geometry)
- Gridspec (Balaji’s schema for climate model grids)

## Proposed WKM Syntax

- New CF metadata attribute called, say, data_model or spatial_data_model
- Used as a global-scope attribute: defines the *default* spatial data model for a netCDF file, e.g.
- :data_model = "continuous coverage: rectified grid"
- Used as a variable-scope attribute: defines the spatial data model for that variable, e.g.
- variables:
- 	float temp(time, regions);
- 		temp:data_model = "feature_collection: polygons";
- Attribute values would be drawn from a controlled vocabulary, as per CF standard names.

**Notes:**
Alternative names for WKM attribute: spatial_geometry, geometry_model, well_known_model.

## Vector Data Models

- Based on OGC/ISO 19107 standard for spatial schema (aka feature geometry).
- Example values of data_model attribute:
- feature_collection: points
- feature_collection: multipoints
- feature_collection: curves/lines
- feature_collection: multicurves/lines
- feature_collection: polygons
- feature_collection: multipolygons

**Notes:**
GML standard should provide a good base from which to define feature geometry.
But additional work required to define more complex geometries such as vertical profiles and trajectories.

## Coverage/Grid Data Models

- Based upon combination of OGC/ISO 19123 (coverage geometry) and Gridspec standards
- Example values of data_model attribute:
- // from OGC/ISO 19123...
- discrete coverage: points
- discrete coverage: curves
- discrete coverage: surfaces
- continuous coverage: thiessen polygon network
- continuous coverage: hexagonal grid
- continuous coverage: quadrilateral grid
- continuous coverage: rectified grid
- continuous coverage: georectified grid
- // from GFDL Gridspec...
- continuous coverage: logically rectangular
- continuous coverage: structured triangular
- continuous coverage: unstructured polygonal

**Notes:**
Again, GML standard provides a good base from which to define coverage geometry types.
Work required to synthesise coverage geometry elements from ISO 19123 and Gridspec.
Envisaged that majority of gridded netCDF datasets will be based upon fairly straightforward quadrilateral / rectified grids.

## Examples of WKM Usage

## Ex 1: Time-series of station data

- // default data model for this netCDF file
**:data_model = "feature_collection: points";**
- dimensions:
- 	stations = 100;
- 	time = UNLIMITED;
- 	...
- variables:
- 	float lat(stations);
- 	float lon(stations);
- 	float time(time);
- 	float temp(time, stations);
- 		temp:standard_name = "surface_temperature";
- 		temp:coordinates = "lat lon";
- 		// following attribute is optional
- 		**temp:data_model = "feature_collection: points"**;
- 		...

**Notes:**
Since a coverage is a sub-type of feature, the temp variable could also be modelled as a discrete point coverage.
Choice depends on the intended application / end use of dataset.

## Ex 2: Time-series of max wind speed within geographic regions

- // default data model for this netCDF file
**:data_model = "feature_collection: points";**
- dimensions:
- 	regions = 10;
- 	verts = 1000;
- 	time = UNLIMITED;
- 	...
- variables:
- 	char region_names(regions, 80);
- 	int nverts(regions); // num vertices making up each region
- 	float lat(verts);
- 	float lon(verts);
- 	float time(time);
- 	float windspeed(time, regions);
- 		windspeed:standard_name = "wind_speed";
- 		windspeed:coordinates = "lat lon";
- 		windspeed:cell_methods = "lat: lon: maximum";
- 		// specific data model for this variable
- 		**windspeed:data_model = "feature_collection: polygons"**;
- 		...

**Notes:**
In this example the default data model is overridden by the specific data model for the windspeed variable.

## Ex 3: 4D Global Model Grid

- dimensions:
- 	lat = 96;
- 	lon = 73;
- 	level = 38;
- 	time = UNLIMITED;
- 	...
- variables:
- 	float lat(lat);
- 	float lon(lon);
- 	float level(level);
- 	float time(time);
- 	float humidity(time, level, lat, lon);
- 		humidity:standard_name = "specific_humidity";
- 		// data model based on OGC/ISO 19123...
- 		**humidity:data_model = "discrete coverage: grid points"**;
- 		// ...or based on Gridspec.
- 		**humidity:data_model = "discrete coverage: logically rectangular"**;
- 		...

**Notes:**
There are various information models for describing coverage-type and grid-type geometries.
Work is required to synthesise / harmonise these models.
One option would be to use namespace prefixes to distinguish different data models.

## Conclusions

- The Well-Known Model concept...
- ...exploits existing geospatial standards
- ...is intended to act as a **simple hint** to client applications (and humans too) regarding the spatial data model(s) encoded in a netCDF file
- ...is aimed particularly at GIS software community where CF metadata conventions appear to be less familiar / understood
- ...should hopefully facilitate better manipulation of netCDF data within GIS applications
- ...shouldn't conflict with existing CF attributes and usage

**Notes:**
WKM is not a panacea.
GIS vendors will still need to correctly implement netCDF and CF standards.
Hopefully a mechanism such as the data_model attribute proposed here will facilitate that compliance.
WKM is intended to supplements rather than replace existing CF attributes.

## Further Work?

- If WKM seen as a Good Thing then...
- Undertake further analysis of current geospatial data model standards
- Elicit input/feedback from GIS community
- Synthesise ideas from existing geometry models (OGC/ISO, Gridspec, netCDF, CDM)
- Identify/refine an initial list of data_model definitions. Firm up syntax.
- Compile usage examples for commonly encountered netCDF data models
- Submit proposal to CF community

**Notes:**
Syntax of attribute values as proposed here follows the syntax used by other CF attributes, e.g. cell_methods.
Appear to be plenty of related initiatives exploring the potential integration of FES and GIS world-views.
These could usefully be consolidated / harmonised.

## That’s all folks!

- Q&A