# Presentation: 806844

## Introduction

**Two components**
**Data Exchange Infrastructure (DEI)**
**Spatial Data Model (SDM)**
** **
**Together, they make up “NCDDC Middleware”**

## Introduction to NCDDC Middleware - what it does and why

- Richard Owens

## What Is DEI (High-level overview)

**DEI is a client/server framework by which applications and data repositories can interact. The goal of the framework is to solve the problems of Data Access and Data Representation.**
** **
**DEI is a CORBA-based set of interfaces which are used to locate and transfer geospatial data. It is a client/server architecture which uses CORBA's IIOP (uses TCP) as its transport protocol.**

## DEI Concepts 

**Gateway**
**Server component that adapts DEI interfaces to a specific **
**	****data source**** **
***	******	******Data source dependent***
**DEI Client**
**Any software that uses DEI CORBA interfaces to **
**	****communicate with gateways**

- Client

- Gateway

- CORBA

## DEI Concepts (continued)

** ****Data Flavor (formats) **
***	******	******Like a MIME type****** ***

## What DEI Is Not

**(graphic here)**

**What DEI Is Not**

**Complete****	**

**Web services**** **

** ****Lacks generic query processing**
** ****Poor support for very large datasets**** ****	**

**Data Archive/Storage**
**Data Format**

** ****Supports them, though****	**

## History

**History**
**	****Originally Navy NRL funded project**
**	****Now being developed by NCDDC**
**Current Status**
**	****Java based gateways via Java GDK**
**	****	*****Current focus***
**	****C++ gateways**
**	****	*****Original focus, now lagging behind Java***
**Future**
**	****Mature Java GDK, then C++ GDK**
**	****Add generic query processing**
**	****Add support for very large datasets**
**	****Open source**

## What is the SDM

**What is the SDM**
** ****Object-oriented library that is used to represent and manipulate spatial data objects.**
** ****3D**
**	****Some other dimensional capabilities, too**
**	****Structured Grids**** **
**	****	*****Regular, Rectilinear, Curvilinear, Projected***
**	****Unstructured Grids**
**	****Coordinate Systems**
**	****	*****Cartographic, Cartesian, Cylindrical, Spherical***** **
**	****GIS-like structures**
***	******	******Points, Lines, Polygons, etc.***** **

## What is the SDM (continued)

**What is the SDM ****(continued)**
** ****Java**
**	****~160 classes**
**	****~1050 Unit tests**

## Key Features

**Key Features**

**Called the “Field Spatial Data Model” by its developer   because it uses a ‘field’ concept, which relates spatial objects to data.**
**Fundamental storage is multi-dimensional array.**
**Decoupling of data storage from its geometry and topology.**
**Requires basic metadata (quantity, units, validity, accuracy, etc.) for all data.**
***	******	******Aids in interoperability***

## Key Features (continued)

**Small, consistent API for data access.**
**	****The SDM hides all implementation details of data  ****	****representation behind its abstract interfaces.**
** ****	****Decouples many issues regarding storage and internal ****	****representation of the data, while providing a consistent ****	****interface to it.**
** ****	****Data may be logically represented one way and physically ****	****stored another way.**
** ****	****Many manipulations may be done without actually ****	****having to have the data physically in memory.**

**Key Features *****(continued)***

## Key Features (continued)

**Externalized format for representing datasets**
**	****XML based (FDML)**
**	****Can be generated without the use of the SDM.**

**Key Features *****(continued)***

## Field Concept

**Field Concept**

## What SDM Is Not

**Transport mechanism**
**Storage/Archival**
**Magic. Won’t solve all problems.**
**Just a data format**
**Complete**

## History

**EPA project**
**Ported from Eiffel to Java in 2002**

## Future

**Documentation**
**FY03**
**Text data (done)**
**Generalize dimensional support beyond existing 3-D **
**	****(to 2-D, 1-D or N-D)**
**Investigate Time dimension support (complete in FY04)**
**Application Layer Services**
**	****	*****Common Query mechanism***
***	******	******Subsetting (parameters, spatial)***
***	******	******Format support (NetCDF, Shapefile)***
**FY04**
**	****Aggregation of data types**
**	****Time dimensions support - including subsetting in **
**	****	****Application Layer**
**Open Source**

** **

** **

## NCDDC Middleware Objective

**Develop a middleware architecture that can be used to tie data and applications together across a local or wide-area network in a way that is independent of the data and application platforms, operating systems, and formats.**

## NCDDC Middleware Defined

**	****“****NCDDC Middleware” is the fusion of DEI with SDM**
**	****DEI**
**	****	****Data Transport**
**	****SDM**
**	****	****Data Representation**
**	****DEI + SDM (“NCDDC Middleware”)**
** ****Spatial data representation distributed over a network**
** ****Common form for data, and the means for transporting it**

## DEI/SDM, HTTP/HTML Analogy

**	****Transport**
**	****	****HTTP, DEI**
**	****Content**
**	****	****HTML, SDM**

## Benefits

**Gateways can be created that can transport descriptions of the data with embedded links to the actual data – the actual transport of the data is done piecewise, “on demand”.**
** ****Can have datasets that are larger than memory.**
** ****Caching mechanism improves performance.**
**Data is transported in binary form**
** ****Sun’s XDR format**
***	******	******	******	******	******	******	******Foundation for netCDF***
** ****Fast and efficient**
**Since DEI is integrated within the SDM, clients can use SDM interfaces to access gateways without the need to code to DEI’s CORBA interfaces.**