# Presentation: 199019

## EPIC Toolsfor in-situ data collections

- Donald W. Denbo, NOAA/PMEL
- Willa H. Zhu, NOAA/PMEL/JISAO
- John Osborne, OceanAtlas Software
- Christopher Moore, NOAA/PMEL/JISAO

- NOAA Coastal Storms Initiative MeetingMay 6-8, 2003

## EPIC Components

- Data Archive and Retrieval
- Multiple file formats supported with “epslib”
- Metadata database supports subsetting of large data collections
- “Classic EPIC”
- Unix programs for desktop display, analysis
- Gateways to Matlab, PPLUS, Ferret, IDL, etc.
- EPIC Web Browser
- Search, graph, list, download from the Web
- Toolkit for building custom web applications
- Java applications
- Java OceanAtlas - Sophisticated browse and data exploration
- Climate Data Portal - distributed data access (CORBA/OPeNDAP)
- OceanShare - Network data collaboration tool
- ncBrowse - Generalized browse tool for data/attributes (local files, network files, OPeNDAP files)

## EPIC Archive and Retreivalfor Classic EPIC, Web, and Java Applications

- Data collections may include thousands or even millions of profiles or time series
- Typically the researcher will use EPIC tools to:
- Select data by lat, lon, time, keyword, etc.
- Refine the initial data selection
- Display and/or analyze the selected data

## EPIC Data Management Strategy

- Pacific

- NODC

- Alaska

**Meta Data**

**Data**File Collections

- Data LoaderUtility

- Selecteddatasets

*pointers*
*to data files*

**Data Search**Lat, long, timekeyword, etc

*creates*

- DataSearchandSelection

- “Classic
- EPIC”programs

- Matlab
- FerretIDLPPLUS

- EPIC Web

- Java OceanAtlas
- Climate Data PortalOceanSharencBrowse

- MySQLdatabase

*EPIC Data Management Strategy*

*EPIC data usage*

## EPIC Data File Input/Output

- Applications programs

- API

- File Independent Layer

- FDL

- Existing

- Planned

- Data file format
- Multiple data file formats are supported
- New formats are readily added
- Applications software
- Independent of file format
- API for most programming languages
- Gateways
- to Matlab and other graphics/analysis packages

- EPS Library

## Classic EPICSuite of Unix programs for the desktop

- Over 100 EPIC programs are available for:
- Data selection, metadata and file editing...
- Graphics
- Time series, multivariable plots, section plots...
- Parameter calculation
- Dynamic height, geostrophic velocity...
- Statistical, spectral, tidal harmonic analyses...
- EPIC’s modularity allows a researcher to easily add new analysis techniques or use a favorite graphics or analysis program.

## EPIC Web Browser

## EPIC Web BrowserData selection, graphics, download

## EPIC Web Browser Capabilities

- Plots
- User customized plots for profiles and time series
- (T, S, Sigma-t, Ox) vs Pressure/Depth
- Temperature vs Salinity
- One or two user-selected Properties vs Pressure/Depth
- Contoured section plots
- List/download data, metadata
- New plots and functions readily included

## EPIC tools for the Web for building customized web access to data

-       EPIC Web Toolkit
- Interface design
- Backend graphics
- Map data selector

-       Example applications
- EPIC Web Browser (general)
- TAO (El Nino buoys)
- Unaami (Arctic time series)
- OSCAR (Satellite ocean currents)
- Multibeam data (hydrography)
- http://www.epic.noaa.gov

## EPIC Java Tools and Applications Distributed data accessand  Advanced data browsing

## Java Tools

- EPIC Java Applications

- for locating and selecting from distributed data collections

## Challenge

- Locating and selecting individual observations from large, distributed collections of *in situ* data observations
- Examples: Individual oceanic or atmospheric profiles, time series, buoy tracks, etc.

- EPIC Java Applications

## Some solutions

- Tools and Applications
- LDAP Directory Services
- NdEdit for subsetting large in-situ collections
- Interactive Java graphics
- Intensive exploration of multiple data types
- OPeNDAP (DODS) clients and servers for in-situ data
- Access from Desktop and Web
- Assembled from OpenSource components
- http://www.epic.noaa.gov/epic/software/ep_java.htm

- EPIC Java Applications

## LDAP Directory ServiceFinding distributed data collections

- Search on geographic region, observed quantities or keyword
- Automatically populated from data servers
- Metadata instantly known to LDAP directories
- Access control for experimental data products
- Open Source
- Standards-based
- Platform independent
- Scalable, extensible
- Supports desktop or web client

- EPIC Java Applications

## NdEdit

- X-Y map view

- Y-T  time range

- Investigate space-time distribution
- Select data
- in any View

- Explore Data Locations and Subset Data Collections

- Interactive, linked windows
- View any 2D sliice
-  xy, xt, xz, yt, yz, yt

- Select time rangeSee data in range

- X-Y map view

- Y-T  time range

- EPIC Java Applications

## NdEdit

- EPIC Java Applications

## NdEdit

- Selected stations are in a narrow time band

- EPIC Java Applications

## NdEdit

- Polygon tool – select an oblique section line

- See time range of stations

- EPIC Java Applications

## NdEdit

- NdEdit is an interactive Java tool
- NdEdit is being re-engineered for the Web

- EPIC Java Applications

## SGT: A Scientific Graphics Toolkit

- Java toolkit for scientific graphics
- On-line documentation, examples, tutorials, streaming video
- JavaBeans in next release
- Features
- Contour, line and vector plots,
- Axes user settable or automatically scaled
- multiple Time Axis formats
- mouse-movable, customizable Labels, line keys, icons
- Legend generation automatic
- Widely used, freely available
- scientific and commercial applications use SGT
- 14,782 downloads by 5,151 unique sites from 70 countries
- http://www.epic.noaa.gov/java/sgt/

- EPIC Java Applications

## Java Applications

- for the Desktop and the Network

- EPIC Java Applications

## ncBrowseJava tool for data/metadata browse

- NetCDF and full OPeNDAP support
- Desktop Java application
- Axis mapping, parametric plots, flexible mapping of independent variables to axes
- Line, vector and contour plots
- Animations, 3D graphics via VisAD
- Popular, freely available
- 8,187 downloads by 3,415 unique sites in 59 countries
- OPeNDAP (DODS) client, Unidata netCDF client, WOCE data CD, etc.
- http://www.epic.noaa.gov/java/ncBrowse/

- EPIC Java Applications

## Java OceanAtlasfor browsing ocean data

- Intensive Java desktop browsing of ocean data
- Property-property plots with up to 7 x axes, waterfall diagrams, Contour plots/residual contour plots, Map plots with optional "bubble" isosurface plots, Linked browsing of all plot types, Built-in and custom calculations, NdEdit for data location/selection
- Reads WOCE, EPIC, OPeNDAP netCDF data, some ARGO formats
- Extended for wide range of data types
- Profile, time series, drifting buoy, gridded analyses, model outputs
- Widely used, freely available
- http://www.epic.noaa.gov/epic/software/JavaJOA.htm
- http://odf.ucsd.edu/joa/jsindex.html

- EPIC Java Applications

## Climate Data PortalDistributed ocean observations

- Major *in situ* data collections:
- TAO El Nino buoy data (realtime/PMEL), Global sea level data (realtime/U of Hi), GTSPP (NODC), PMEL hydrographic data and time series collection, WOCE ADCP (U of Hi), IPRC/APDRC *in situ* data, US Coastal Ocean Time Series database (NODC)

- OPeNDAP:  in-situ data server and client
- LDAP directory services
- NdEdit for data selection/subsetting
- Data Servers, Clients (Java and Web) – focus on *in situ* data
- http://www.epic.noaa.gov/cdp/cdpjava.htm

- EPIC Java Applications

## OceanShareNetwork collaboration tool

- Collaboration features
- Local, remote data file access and Java graphics
- Standard collaboration tool features (annotation, whiteboard, etc)
- Secure LDAP Document Repository
- Proposals, papers in progress, data, images
- Access control by document or by folder
- Accessible from web and desktop
- OPeNDAP compatible soon
- http://www.epic.noaa.gov/collab/

- EPIC Java Applications

## EPIC Java Tools and Applications 

- Modular Web and Open Source components
- Java interactivity
- Low cost development/maintenance
- OpenSource, standards-based
- Scalable, extensible
- OPeNDAP supported or planned
- http://www.epic.noaa.gov
- http://www.epic.noaa.gov/epic/software/ep_java.htm