# Presentation: 858898

## Metadata (for the data users downstream)

- RFC GIS Workshop
- July 2007

- NOAA/NESDIS/NGDC
- Anna.Milan@noaa.gov

**Documentation**

**Notes:**
I was invited here to talk with you about “metadata”.
The first point I would like to make is that metadata is just another word for documentation. I am not here to beat you into submission so that you will always use the FGDC content standard for your geospatial data. Instead, I hope that you will leave this talk with ideas about standard documentation and how it contributes to data management. I also hope to learn from you. What are your current data management practices? Why do you care about metadata?
What are your documentation needs?
(Who are the users and what are their requirements? Who are your future or unknown users and what are their requirements?)

## What is documentation?

- Why is it important and how is it changing?

**Notes:**
Magnetogram Data w/o any metadata. Stored at National 
Archives, but had to throw away... Oct 2005.

## Metadata Team Partnership

**Data**
**Provider**

**Direct**
**Users**

**Unknown**
**Users**

- User communities should be able to understand the information without needing the assistance of the experts who produced the information.

**Notes:**
Metadata/Documentation team is comprised of a partnership between data providers, archives and users.

Most data providers have an established and ongoing relationship with the users. Data exchange requirements are understood by both parties (and the need for standard documentation is less imperative.)
Near/Real-time data stream
Data quality i.e. sum checks, data flags, etc are important for real-time users.
Documentation tends to not be standard, i.e. headers and complex details
Data archive provides data to unknown users
archived data
general, standard documentation 
Partnerships between the archive and data provider is needed to create high quality metadata. 

OAIS Mandatory Responsibility: Ensure that the information to be preserved is Independently Understandable to the Designated Community. In other words, the community should be able to understand the information without needing the assistance of the experts who produced the information.

## What is Documentation?

**What** is the content of the data?
**Who** created the data?
**When** was it created?
**Where** is it geographically?
**How** was the data developed?
**Why** was the data developed?

## Standard Documentation

- Provides a common set of terminology and definitions
- Enables usability of data for broad communities
- Enables communities to archive, search and access data easily
- Critical to preserving the understanding of data

**Notes:**
When data managers use standard documentation. It is creating an apples to apples comparison for data shoppers.  For instance, when users are comparing two documents it is easy for them to discover the title, distribution formats of multiple and disparate datasets. 

## Standard Documentation Helps...

- Data producers and providers to manage and **evaluate** data
- Data consumers to **discover**, **assess** and **obtain** data
- Enable computer based **interoperability**

- to

## Evolution of Geospatial Standards

**ISO 19115 – 2**

**ISO 19115/NAP**

**FGDC Remote **
**Sensing Extensions**

**FGDC**

**NOAA**
**Supplemental**

**DIF**

- OGC Sensor
- Web Enablement?

**Notes:**
In the beginning (well the 80’s) the DIF (Directory Interchange Format) was established. It is comprised of the basics – title, summary, keywords... 
FGDC Content Standard for Digital Geospatial Metadata (CSDGM) adopted in 1994, revised in 1998
 In 1994 Clinton signed an executive order that requires federal agencies to use the FGDC standard to document geospatial data 
“NOAA Supplemental” was a set of local extensions in the NOAA metadata community to create common metadata fields for URLS, instruments, etc
In 2002, the Remote Sensing Extensions were endorsed by FGDC. For standard documentation of satellites, instruments, missions, user guides, science papers, algorithms, etc. ISO 19115 is an international metadata standard for geospatial data. It was created in Europe. International adoption of this standard is growing. United States and Canada are creating a profile for North America. International counterpart to FGDC. More comprehensive, more detailed quality reports.
Eventually FGDC will not be a supported or endorsed standard. ISO/NAP will be the mandatory standard. U.S. joining the international metadata community. 
ISO 19115 – 2 for gridded imagery is the international counterpart to FGDC/RSE.

As metadata standards evolve they will  be more and more likely to fulfill the documentation needs for your data. 

## Geospatial Content Standards

- Content Standard for Digital Geospatial Metadata
- Extensions for Remote Sensing Metadata
- Biological Data Profile
- Profile for Shoreline Data
- ISO 19115:2003 Geographic Information – Metadata
- North American Profile of ISO 19115:2003
- ISO 19115-2: Extensions for imagery and gridded data
- WMO Profile of ISO 19115

**Notes:**
- content standards -
The original FGDC Content Standard for Digital Geospatial Metadata and the ISO 19115 Metadata Standard are general standards designed to apply to many different types of data. The FGDC Remote Sensing Extensions and the ISO 19115-2 are extensions that are designed to hold metadata for remotely sensed and gridded datasets that are not described in the base standards. Many of these extensions are also useful for general metadata requirements.

According to Executive Order 12096 all Federal agencies are ordered to use this standard to document geospatial data created as of January, 1995 (FGDC)

## OGC Sensor Web Enablement 

- Source: _[http://portal.opengeospatial.org/files/?artifact_id=15540](http://portal.opengeospatial.org/files/?artifact_id=15540)_

**Notes:**
- content standards -
The original FGDC Content Standard for Digital Geospatial Metadata and the ISO 19115 Metadata Standard are general standards designed to apply to many different types of data. The FGDC Remote Sensing Extensions and the ISO 19115-2 are extensions that are designed to hold metadata for remotely sensed and gridded datasets that are not described in the base standards. Many of these extensions are also useful for general metadata requirements.

## OGC Sensor Web Enablement 

- Observations and Measurements (Application Schema)
- For encoding sensor observations
- OpenGIS® Sensor Model Language (SensorML) Implementation Specification
- For describing sensor characteristics  (geolocation , response)
- OpenGIS® Sensor Observation Service (SOS) Implementation Specification
- How to access SML and O&M

**Notes:**
- content standards -
The original FGDC Content Standard for Digital Geospatial Metadata and the ISO 19115 Metadata Standard are general standards designed to apply to many different types of data. The FGDC Remote Sensing Extensions and the ISO 19115-2 are extensions that are designed to hold metadata for remotely sensed and gridded datasets that are not described in the base standards. Many of these extensions are also useful for general metadata requirements.

## Metadata  Management Workflow

## Creation to User Discovery

**Documentation**

**Distribution**
**Website**

## At NGDC... 

- Compile existing resources
- User guides, scientist expertise, papers etc.
- Create standard documentation with NMMR
- import XML or use edit interface
- Validate against FGDC w/ RSE Schema in NMMR
- Publish documentation
-  rsync folders, Z39.50 server and websites
- GOS, GCMD and FGDC Clearinghouse harvest XML files
- GOS ingests and translates to html with their stylesheet
- GCMD converts to DIF and then to HTML
- Documentation is searchable in GOS and GCMD

## Management Tools

- Standard documentation enables usage of standard management tools
- Some Resources
- NOAA Metadata Manager and Repository (NMMR)
- Metadata Enterprise Resource Management Aid (MERMAid)
- ESRI ArcGIS Metadata
- ISO Metadata Editor (IME)
- More resources listed at FGDC website

**Notes:**
Standards enable usage of standard tools. 

## Discovery Tools

- Standard documentation enables usage of standard discovery tools
- Some resources
- Geospatial One-Stop
- _[http://gos2.geodata.gov/wps/portal/gos](http://gos2.geodata.gov/wps/portal/gos)_
- NASA Global Change Master Directory
- _[http://gcmd.nasa.gov/](http://gcmd.nasa.gov/)_
- NSDI FGDC Clearinghouse
- _[http://clearinghouse3.fgdc.gov/](http://clearinghouse3.fgdc.gov/)_
- NBII Clearinghouse
- _[http://mercury.ornl.gov/nbii/](http://mercury.ornl.gov/nbii/)_

## Discovery Tools

## Discovery Tools

## Published Documentation

## Published Documentation

**Published Documentation**

**Notes:**
Web Map Viewer of multibeam dataset with links to published standard metadata records. 

## Best Practices

## Best Practices 

- Standard documentation should be recorded throughout the life of a dataset.
- Planning > processing > analysis > publication and subsequent revisions
- Institutionalize standard documentation production and maintenance in your centers
- Make standard documentation a key component of data development and management process
- Use standard vocabulary!

## FGDC Base Standard

- _[http://www.fgdc.gov/csdgmgraphical/index.html](http://www.fgdc.gov/csdgmgraphical/index.html)_

## Best Practices

- Metadata records usually include the basics –
- Identification – title, originator, abstract, keywords
- Distribution – Contact, output formats, links
- Spatial Organization
- Raster or vector data
- Spatial Reference Information
- Data Quality

## Spatial Referencing in FGDC

## Spatial Referencing in FGDC

## Spatial Reference Sample

## Data Quality

## ISO 19115 UML

## RFC Monthly Precipitation Metadata

- RFC Monthly Precipitation metadata record published by NCDC
- _[http://ngdc.noaa.gov/nmmr/xmls/fgdc.jsp?id=gov.noaa.ncdc:C00389&view=classic](http://ngdc.noaa.gov/nmmr/xmls/fgdc.jsp?id=gov.noaa.ncdc:C00389&view=classic)_

## RFC Monthly Precipitation Metadata

## RFC Monthly Precipitation Metadata

## Collection Versus Granule

| Granule Level | Collection Level  |
| --- | --- |
| dynamic | static/quasi-static |
| real-time | archive |
| time stamp of a data file | temporal coverage spans dataset lifespan |
| specific location | general bounding area |
| for data use | for data discovery |
| syntactic descriptions | semantic descriptions |
| Rich Inventory (Specialized) | FGDC Metadata (Standard) |

## Examples

## Rich Inventory for AOT Data

**Notes:**
Graph of the mean aerosol optical depth from 2001 to 2007. 
In 2002, there was a significant change in the mean value. We think this is due to an algorithm change in the data production. 
This demonstrates, how a real-time granule metadata can contribute to understanding the data.  This information should also go into the collection level metadata record.

## Rich Inventory for GOES Winds

**Notes:**
This is another example, of how our Rich Inventory shows us anomalies in the data stream.

## MADIS Quality Map Viewer

**Notes:**
Meteorological Assimilation Data Ingest System 
Ingested their quality database into a mapviewer based on network and station and quality. 

## Wrap Up

## Be Good Data Stewards

**Be Good Data Stewards**

## RFC and Documentation?

- Current practices?
- Issues/ideas?
- Questions?
- Other?

## Resources

- FGDC Website
- Geospatial One Stop
- NASA Global Change Master Directory
- Open Geospatial Consortium

## Resources (Tools)

- Small sample of metadata editing tools available:
- NOAA Metadata Manager and Repository (NMMR)
- Web-based interface link for managing FGDC metadata hosted by NOAA/NGDC
- Metadata Enterprise Resource Management Aid (MERMAid)
- Web-based interface for managing FGDC metadata hosted by NOAA/NCDDC
- ISO Metadata Editor (IME)
- Downloadable software for creating ISO metadata files

## Standards & Specifications

- _[Content Standard for Digital Geospatial Metadata (version 2.0), FGDC-STD-001-1998 ](http://www.fgdc.gov/standards/projects/FGDC-standards-projects/metadata/base-metadata/v2_0698.pdf)_(PDF)
- Content Standard for Digital Geospatial Metadata: Extensions for Remote Sensing Metadata, FGDC-STD-012-2002 (PDF)
- North American Profile of ISO 19115 (PDF draft)

## Standards & Specifications

- ISO 19115:2003 Geographic Information - Metadata
- ISO 19115-2: Geographic information — Metadata — Part 2: Extensions for imagery and gridded data
- unpublished work, in development
- Sensor Web Enablement Overview from Open Geospatial Community (SWE)