# Presentation: 472870

## Global Earth ObservationIntegrated Data Environment (GEO-IDE) 

**Presentation to **

***Presenter name and affiliation***

## The information revolution

- Need a quote that conveys the significance of the period of history we are in -- the Information Revolution.  "Get with it or become fossilized and irrelevant."
- maybe some reference to how much the nation as a whole is investing in integrating systems.  (Contrast with NOAA ...)
- Anyone have a gem to go here?

## NOAA’ Top Ten Challenges1

- 10) Alphabet Soup
- 9) Stove Pipes
- 8) Integration
- 7) Architecture
- 6) Data Sharing

- 5) User Needs
- 4) Maximizing Benefits
- 3) Communication
- 2) Data Management
- 1) Execution

- Vice Admiral Lautenbacher in address to American Meteorological Society Annual Meeting, January 30, 2006

## Data Management - a top priority

- Improving data management is among the highest priority challenges facing NOAA ... integrated data management is at the heart of the GEOSS concept

**Notes:**
During he presentation at the AMS, the Vice Admiral also noted that data management is the key to better integrating data and information to better meet societal needs. 
Integrated Earth Observations and products are used for:
Detecting and forecasting weather and climate variability
Facilitating safe and efficient marine operations 
Ensuring national security 
Managing resources for sustainable use 
Preserving and restoring healthy marine ecosystems 
Mitigating natural hazards 
Ensuring public health 

Full suite of monitoring systems and effective data management systems are needed to reap the benefits of earth observations.

## Why is improved integration needed?

***Why is improved integration needed?***
***Important societal issues require data from many observing systems***

| Atmospheric Observations |  |  |
| --- | --- | --- |
| Land Surface Observation |  |  |
| Ocean Observations |  |  |
| Space Observations |  |  |
| Data Systems |  | Coordinated, efficient,integrated, interoperable |

**Discipline Specific View**

**Whole System View**

**Current systems are program specific, focused, individually efficient.**
**But incompatible, not integrated, isolated from one another and from wider environmental community**

**Notes:**
Comprehensive and integrated observations are at the heart of improving our understanding of the Earth System.
We need an integrated, whole system perspective to provide the information that society needs.

## Example Problem:  Coastal Inundation & Erosion involves multiple systems

**Temperature**

**Bathymetry**

**Reanalysis & Coupled ****Climate ****Models**

**Coastal Inundation & Erosion Scenarios**

**Biochemical Impacts****(Ecosystems)**

**Human Population Distribution Scenarios**

**Societal Infrastructure Coastal Scenarios**

**Sea Ice**

**Glaciers**

**Ocean**
** **Volume
-  Circulation
-  Quantity

**Sea Level Change**

**River****Run-off**

**Tides**

**Winds**

**Winds**

**Temperature**

**Temperature**

**Observations (~75)**

**Human Impacts**

**Models (~25)**

**Physical Impacts**

**Land Mass & Ocean Properties (~20)**

**Sea Level Change (~12)**

**Permafrost**

**(# in parenthesis represents data sets which must be integrated)**

**Notes:**
# in parenthesis represents data sets which must be integrated in order to fully understand, predict and mitigate this issue.  This is extremely difficult today because of disparate formats, protocols and terminology.

## Data Management – a top Federal Priority

- Congressional mandate
- Public Law 102-567, section 106 directs NOAA to (among other things):
- Develop effective interfaces among the environmental information systems of NOAA and other departments and agencies
- Develop and use nationally-accepted formats and standards
- Integrate and interpret data from different sources

**Notes:**
Data management has also been a top Federal priority for several years.  Integration of data is explicitly mentioned in Public Law 102-567, which is the authorization act for NOAA

## Today’s Challenges

**Incompatible syntax (formats) and semantics (terminology) among science disciplines within NOAA.  Thousands exist.  Several examples:  **
**Naming standards – Surface Air Temperature**
**Meteorology (WMO) named “Temperature/dry bulb temperature**
**Meteorology (air pollution) named “Boundary layer temperature”**
**Oceanography named “Air Temperature”**
**Location standards (latitude, longitude, elevation)**
**Lat/Lon can be degrees/minutes/seconds or degrees to tenths and hundredths  **
**Latitude E/W, 0-180 positive and negative, or 0-360 running east or west**
**Z used to designate elevation in both atmosphere and ocean but positive is up in the atmosphere and down in the ocean**
**Formats  (>50 formats used within NOAA; translators and standards needed)**
**GRIB, NetCDF, HDF and others used for gridded data**
**BUFR, NetCDF, and **_**many**_** others used for observations  **
**Potential for no answer or the “wrong” answer to important societal questions due to separate NOAA data management systems**

## NOAA’s GEO-IDE

**Scope **– NOAA-wide architecture development to integrate legacy systems and guide development of future NOAA environmental data management systems
**Vision **– NOAA’s GEO-IDE is envisioned as a “system of systems” – a framework that provides effective and efficient integration of NOAA’s many quasi-independent systems
**Foundation** – built upon agreed standards, principles and guidelines
**Approach **– evolution of existing systems into a service-oriented architecture ** **
**Result **– a single system of systems (user perspective) to access the data sets needed to address significant societal questions

**Notes:**
NOAA’s response to improve data integration, improve efficiency of NOAA operations and enhance services to customers is the GEO Integrated Data Environment.

## Goals

**Through GEO-IDE NOAA will**
**Identify and address integration gaps in data management systems **
**Create interoperability across existing data management systems**
**Develop and adopt data standards for formats and terminology**
**Integrate measurements, data, and products **
**and will achieve**
**Cost avoidances in NOAA business through improved efficiency and reduced duplication**
**Better integration of data and products across disciplines**
**Reduced risks for US IEOS and GEOSS**

## GEO-IDE - an essential component ofenvironmental information management for NOAA

- Integrated observing, data processing and information management systems
- Connected by NOAA’s Integrated Data Environment
- Contributes to
- U.S. Global Earth Observation System (USGEO) and
- International Global Earth Observing System of Systems (GEOSS).

**Notes:**
GEO-IDE is NOAA’s primary contribution to US-GEO and GEOSS data management activities.

## Why Now?

**Why Now?**

**Critical to USGEO**
**six near term opportunities**
**Uncoordinated development leads to inefficiencies, incompatibilities, and duplication of effort.**** **
**Integration of data among systems is needed to answer questions that address diverse societal benefits**
**Increased efficiency is needed to handle the expected exponential increase in data volumes that will occur over the next decade**

**Societal Benefits**
- Improve weather forecasting
- Reduce Loss of Life and      Property from Disasters
- Protect and Monitor our          Ocean Resources
- Understand, Assess, Predict, Mitigate and Adapt to Climate Variability and Change
- Support Sustainable          Agriculture and Combat Land Degradation
- Understand the Effect of Environmental Factors on      Human Health & Well Being
- Develop the Capacity to Make Ecological Forecasts
- Protect and Monitor Water Resources
- Monitor and Manage Energy Resources

## Scope

- Concerned with environmental and geospatial data and information obtained or generated from worldwide sources to support NOAA's mission (as defined in NOAA Administrative Order 212-15)
- Does not consider administrative support systems such as finance, personnel, acquisition or facilities management
- Includes all aspects of data management, including data acquisition, ingest, data processing, archival and access

## Vision

- “System of systems” – a framework to effectively and efficiently integrate NOAA’s many systems
- Minimize impact on legacy systems
- Utilize standards
- Work towards a service-oriented architecture

## Approach

- Each NOAA LO/program/project continues to manage its data independently
- Standards
- Adopt, adapt and only as a last resort, create
- Open, inclusive process for adoption
- Inclusive not exclusive use of standards
- Service Oriented Architecture

- Reference: Federal CIO Council, Jan ’06  "Services and Components Based Architectures: A Strategic Guide for Implementing Distributed and Reusable Components and Services in the Federal Government"

## Standards

- Standard names and terminology
- Metadata standards
- e.g. FGDC and ISO 19115 w/ remote sensing extensions
- Standard formats for delivery of data/products
- WMO, NetCDF, HDF, GeoTIF, JPEG, etc.
- Web Services Standards
- World Wide Web Consortium
- OGC (Features, Coverage, GML)
- OPeNDAP

## Service-Oriented Architecture

- Under an SOA, capabilities are built one at a time to create “Web Services”
- The fabric of the SOA is built upon standards for:
- discovery (e.g. CF, FGDC, ISO, SQL)
- transport (e.g. HTTP, FTP, OPeNDAP)
- use (e.g. netCDF, HTML, etc.)
- Can be tightly coupled (SOAP) or loosely coupled (REST)

## SOA Concept

## Key Development Strategies

- Maintain and minimize impact on legacy systems
- Evolutionary development through pilot projects
- Coordinate activities through Communities of Interest organized by “Data Types”
- Grids, time-series, moving-sensor multi-dimensional, profiles, trajectories, geospatial framework, point data and metadata

## Project Management

***Undersecretary for Atmosphere and Oceans***

*NOAA Goal Theme:***Weather & Water**

*NOAA Goal Theme:***Ecosystems**

*NOAA Goal Theme:***Commerce & Transportation**

*NOAA Goal Theme:***Climate**

## Project ManagementRoadmap to Implementation

## Project ManagementPerformance Measures

**Performance Measure Outcomes**
**Ability to address multi-disciplinary societal issues**
**Cost avoidance of future investments in Data Management**
**Metrics**
**Number of interoperable Data Systems as measured by: **
**Data Sets within science disciplines**
**Data Sets among science disciplines**_** **_
**Number of users accessing cross-discipline data to address societal issues**
**DOC Strategic Goal Supported**
**Observe and Manage the Earth's Environment to Promote Sustainable Growth**

## Future Direction - Priorities

**FY07 Work with scientists/data system managers to assess requirements and systems**
**Develop enterprise architecture and GEO IDE Implementation Plan**
**Implement standards process**
**Active out-reach activities**
**FY08/09 Incrementally execute work packages**
**Develop data standards and interoperability mechanisms, e.g., translators and directory services**
**Direct, test and evaluate changes being made to data management systems**
**FY10/11 Re-evaluate architecture related to  new data systems (across NOAA & with national /international partners)**

## Conclusions

- NOAA faces daunting challenges of vastly increasing data volumes and an increasing need for interdisciplinary use of data
- NOAA is committed to enhancing access and ensuring data and products of enduring value are preserved for future generations
- NOAA has initiated several activities to actively respond to these challenges.  GEO-IDE is a key component

## Backup Slides

- Risk Management
- Architectural compliance

## Risk Management

**NOAA’s Integrated Data Management System is not a traditional IT Investment Project**
**Integration as opposed to new development**
**No initial hardware investment and Modest software investment**
**Large up front effort to do Enterprise Architecture**
**Hardware/software from existing data management systems will be integrated**
**This Enterprise Architecture Development ****IS**** a risk reduction effort**

## Risk Management Architecture

| Area of IT Risk | Description | Strategies for Mitigation  |
| --- | --- | --- |
| Organizational Change Management  (Level of Risk: Medium) 
 | An integrated data management system of systems is a huge cultural change throughout NOAA | NOAA senior management and the oversight bodies put in place (councils, Goals, etc) will ensure the system of system goals are achieved |
| Business   (Level of Risk: Low) 
 | NOAA’s business mission(s) changes | Unlikely.  The system of systems is still needed. |
| Data/ Information (Level of Risk: Medium) 
 | Inability to develop data management functional capabilities and interoperability due to uniqueness of data/information | Develop scalable architecture and infrastructure plans to upgrade NOAA facilities. Use market driven technologies.   |
| Technology (Level of Risk: Medium)  | Incompatibility of architectural elements | Manage the coordination between different organizations' planned architectural changes |
| Strategic  (Level of Risk: Low) | Compatibility with National and Departmental Strategies | USGEO plans and Congressional Mandate to Commerce  |
| Security (Level of Risk: Low) 
 | Unauthorized users may corrupt the data, algorithms and processes throughout NOAA’s data management systems.  (These are all critical NOAA assets that support the fundamental mission of the agency.) | (1) Security implementation plan scheduled for FY05,  (2) Identify and position appropriate security barriers (firewalls, Owl devices, etc.) (3) Development of an backup infrastructure |
| Privacy (Level of Risk: Low) | No private information within system scope? | Periodic system-wide review for any added requirements. |
| Project Resources   (Level of Risk: Low) 
 | Resources for system integration for critical data management services required | Plan for adequate funds |

## Architectural Compliance and Standards

**The project creates and executes the integrated management component of the NOAA Enterprise Architecture**
**Consistency with NOAA Architecture Standards:**
**NOAA Planning, Programming, Budgeting, and Execution Process (PPBES)**
**Federal Geographic Data Committee (FGDC) metadata standards, **
**NOAA Observing System Architecture (NOSA) **
**Geospatial One-Stop**
** ****COTS Software and Commodity Hardware**
**COTS applications software and previously developed government software **
**Commodity hardware **
** **
**Information Quality Standard (Section 515).  NOAA Guidelines to ensure and maximize Quality, Objectivity, Utility, Integrity (in support of Reproducibility) **
**Accessibility Standard (Section 508) compliance**
**Government Paperwork Elimination Act (GPEA)**

## Architectural ComplianceObserving System Architecture Component -ERD

**Architectural Compliance*****Observing System Architecture Component -ERD***

- Observing System

- Data Handling System

- Human

- Environmental Parameter

- Sensing Element

- Sensor

- Platform / Station

- senses

- is a

- contains

- provides data to

- Mission Goal

- drives

- Location

- located at

- Mobile

- Fixed

- is

- Programs

- is type of

- Stake-holder/User

- Operator

- operated by

- situated on

- Support

- supported by

- Owner

- owned by

- Environmental Parameter
- Requirement

- Drives the need to measure

- Capability Requirements

- have

- provides info to

- drives

## Architectural ComplianceIntegrated Data Management Component -ERD

**Architectural Compliance*****Integrated Data Management Component -ERD***

- Data Handling System

- Network
- Services

- Archive Systems

- Applications

- Web Servers

- contain

- delivers to

- delivered via

- Data

- travels over

- Services

- yield

- Products

- produce

- Observing Systems & Other Data Sources

- provide

- Peripherals

- contains

- contains

- delivered via

- delivers to

- contain

- delivers to

- delivers to

- Peripherals

- Peripherals

- delivers to

- User

- delivers to

## Architectural Compliance

- Services to Citizens

- Support Delivery of Services

- Internal Operations/Infrastructure

**Inter-Agency**

**Intra-Agency**

| Legislative Management
Business Management of Information
IT Management
Planning and Resource Allocation
Regulatory Management | Controls and Oversight
Public Affairs
Internal Risk Management and Mitigation
Federal Financial Assistance |
| --- | --- |

| Human Resources
Admin | Financial Mgmt
Supply Chain Mgmt. | Human Resources
Admin | Financial Mgmt.
Supply Chain Mgmt. |
| --- | --- | --- | --- |

| Public Asset Management
Marketable Asset Management
Defense & Nat’l Security Ops
Diplomacy & Foreign Relations
Disaster Management
Domestic Economy
Education
Energy Management
Insurance
Public Health 
Recreation & National Resources
Social Services
R&D & Science | Regulated Activity Approval
Consumer Safety
Environmental Management
Law Enforcement
Legal
Revenue Collection
Trade (Import/Export)
Transportation
Workforce Management |
| --- | --- |

**Program Admin**

**Compliance**

**Architectural Compliance**

*OMB Business Model*