# Presentation: 183987

## ESIP Federation Insights on Technology

- Rob Raskin
- NASA/Jet Propulsion Laboratory
- California Institute of Technology

## Outline

- Part 1: Federation context for technology development
- Part 2: A tour of technologies developed by ESIP members
- Part 3: SEEDS-related activities within the Federation

## Federation Context 

## Historical Context of Federation 

- Federation created to be a self-governing entity
- Governed by constitution, bylaws, elected officers
- Define its own direction
- Provide an alternative to rigidness of ECS
- One of the first resolutions was:
- “No requirement shall be imposed on an ESIP without its consent”

## NASA-Imposed Requirements

- ESIPs were required to submit DIF entries to GCMD for all their products and services
- This satisfied the FGDC metadata requirement for recipients of gov’t funds
- A System-Wide Interoperability Layer (SWIL) was to be created by the Federation to enable the member holdings to appear as a unified whole
- Proposers were to describe in their proposals how this might be carried out

## Creation of the SWIL

- A Federation Interoperability Group (FIG) was formed – it ultimately selected a catalog interoperability system
- SWIL based on DIFs, ESIP Web pages, EDG references
- Mercury and GCMD were the developers
- No additional work required by ESIPs to comply – submittal of DIFs exposed data to SWIL
- Data interoperability was deemed too difficult at the time

## SWIL Development

- FIG became the Interoperability Standing Committee
- SWIL was renamed FIND
- Data interoperability subsequently explored via Clusters
- ESIPs added later were not required to comply

## Interoperability Standing Committee

- Only one of the 9 Federation Committees that has technology as its primary focus
- Organized Technical Workshop at Spring Federation meeting
- Will have another at Fall meeting (Pasadena)

## Interoperability Standing Committee (cont.)

- Exploration of becoming a Technology Committee, to serve larger community needs
- New efforts to look at service and semantic interoperability
- How can we answer the question: Why has data interoperability been so difficult?

## Technology-Oriented Clusters

- DODS Cluster
- Increased participation in DODS
- GIS Services Cluster (*was Digital Earth)*
- Assisted ESIPs in installing Web Mapping Servers (WMS)
- Developed “Guide to Making your Dataset WMS-compliant”
- Content-Based Search Cluster *(no longer active)*
- Shared expertise in data mining and content-based search
- NewDISS Cluster *(no longer active)*
- Evolved into Strategic Evolution Working Group

## 2. A Tour of Federation Technologies

*Based on the ESIP Federation Technical Workshop*
*May 14-15, 2002*
*http://oceanesip.jpl.nasa.gov/workshop.html*

## Federation Technologies:  Data Access 

## DODS

- Server-side read/subset for most data formats
- Client-side integration with most visualization/ analysis tools (IDL, MATLAB, VisAD, GrADS)
- About 300 datasets available
- Data Access Protocol (DAP) to be separately developed and distributed
- One of the few ESIPs with a specific mission to work with other ESIPs

## DODS (cont.)

- Advantages
- Integration with science visualization software
- Disadvantages:
- Catalog system remains weak
- Data must be converted to intermediate format for transfer
- User interacts with array row/column parameters rather than geographic parameters

## WMS/WCS

- Open standards developed by Open GIS Consortium (OGC)
- Web Mapping Server (WMS) for maps;
- Web Coverage Server (WCS) for data
- NASA plays major role in standards development processes for WMS/WCS
- Eight WMS or WCS servers in place
- Advanced by Digital Earth Cluster (*now GIS Services Cluster*)

## WMS/WCS (cont.)

- Advantages
- Part of larger suite of standards, e.g. Web Feature Server (WFS) for vector data
- Enables overlay of disparate datasets
- Standards developed in conjunction with broader communities
- Disadvantages
- WCS still in development
- Complex data types generally not supported

## MapServer

- Lightweight, public domain GIS
- alternative to ArcIMS
- Over 1000 downloads to date
- Runs on most Unix environments
- MapScript scripting language
- Developed at U. Minnesota (TerraSIP)
- Limited GIS functionality
- Does not enable users to seamlessly link with ESRI software

## Federation Technologies: Data Description

## Earth Science Markup Language (ESML) (UAH)

- Specification for XML descriptions of Earth science datasets
- Associated tools to generate XML descriptions
- Associated library to read the data
- Competes with XDF as a description language

## Federation Technologies: Data Management

## Earth System Science Workbench (ESSW)

- Based loosely on Sequoia 2000 Project
- Provides recording of parameters
- Provides client “notebook” view of end-to-end process
- Currently used only at UCSB

## BigSur (ScienceTools Corp.)

- Also has roots in Sequoia 2000
- Database-centric approach
- Put programs, parameters, and data in DBMS
- Pure Java
- Currently used at Langley DAAC
- Also tested at Ocean ESIP 1997-2000

## Federation Technologies: Data/Knowledge Discovery

## Federation Interactive Network for Discovery (FIND)

- GCMD developed portal to Federation holdings, accessible both from Federation and GCMD pages
- Mercury developed portal to:
- Federation GCMD listings
- Federation Web pages

## Slide 25

## Slide 26

## FIND Usage

- Mercury: 400 searches/month
- GCMD: 1000 searches/month (includes both Federation and GCMD entry pages)
- EOS article submitted to advertise this search service

## ADaM Data Mining (UAH)

- Suite of tools to carry out data mining in space and time
- Clustering, pattern recognition, image analysis, filtering, genetic algorithms, selection, texture operations, histograms
- Contributed scientific algorithms become part of system (e.g. cyclone detection)
- Extensive data readers, preprocessing, postprocessing
- Mature technology, but underutilized

## Peer-to-Peer (MODster)

- NAPSTER-like functionality for MODIS data
- Essentially a redirection service enabling users to find MODIS granules of interest
- Appropriate model for cases where multiple sites have similar data product

## THREDDS (Unidata)

- Thematic Realtime Environmental Data Distributed System (THREDDS)
- Unidata is newest Federation member
- Funded as a DLESE Collections Center
- Discovery through DLESE discovery tools
- Will provides common catalog and access to datasets accessible through DODS, ADDE, WMS, others
- Fills in catalog deficiencies of DODS and others
- Links with existing visulaization tools (e.g. Live Access Server, VisAD)

## WSDL/UDDI

- WSDL and UDDI provide *Web service* interoperability
- Standard way to access Web services
- Explored by IBM ESIP
- UDDIs currently for business services

## SWEET (JPL)

- Semantic Web for Earth and Environmental Terminology (SWEET)
- Semantic Web provides *semantic* interoperability
- Enables Web pages to contain XML tags that describe semantic meaning of terms

## 3. SEEDS-Related Activities in the Federation

## Federation-SEEDS Prototypes

- Last year (’01)
- 3 proposals funded
- $400K total ($300K from Federation)
- Funding not yet received
- Current year (’02) and (’03)
- Combined 2 years => approx. $600K available
- RFP released this month
- Projects required to be cross-ESIP

## Technologies in Last Year’s Winning Proposals

- Universal Interchange Technology for Earth Science Data (UNITE) (UAH, JPL, ORNL)
- Plug & play based on ESML descriptors
- ESML, WCS integration into FIND
- Standards Framework in Support of Dynamic Assembly of NewDISS Components (BASIC, IBM, JPL, ORNL, JHU)
- WSDL/UDDI, WMS/WCS, FIND integration
- MODster (UCSB, DODS)
- Peer-to-Peer

## Strategic Evolution Working Group

- Formed at Spring Federation Meeting
- Created jointly by Products & Services and Interoperability Committees
- Outgrowth of Federation NewDISS Cluster
- Primary focus is on evolution of Federation’s own vision
- With respect to technical issues
- Intent is to work in parallel with SEEDS and to be a point of contact

## Strategic Evolution Working Group Objectives

- Explore/implement promising technologies, useful in the natural evolution of Federation data systems
- Further develop the technology
- Engage customers:  Implement technologies that would further facilitate customers use of Federation products and services
- Identify technology gaps in Federation Services
- Determine methods and/or standards to facilitate evolving collaborations
- Document the processes and methods used to achieve evolution

## Conclusions

- Working together in the technology arena to support mutual goals has been challenging
- This has not been a high priority area for the Federation (relative to SWIL and sustainability)
- Federation-SEEDS prototypes are a notable exception
- Standards not always looked upon favorably – positive aspects often overlooked
- Reactions to ECS have made ESIPs weary of imposed standards

## Conclusions (cont.)

- Federation technologies need additional showcasing
- Data interoperability is our strong area
- Technology standards are important
- Federation members must demonstrate their value for other members to voluntarily take part