# Presentation: 456191

## Bay/Delta & Tributaries Cooperative Data Management System

**Background**
**How The BDAT/CEDEN Distributed Systems Work**
**Local and Regional Data Management (Participation)**
**Strategic and Tactical Benefits**
**Improving Access to Data**
**Answer Questions**

**Notes:**
Background: Provide background on Bay/Delta and Tributaries Cooperative Data Management System.  Describe a distributed data management system

Local Client Databases: Discus local client database applications.  (Provide a demonstration of the SWAMP local client database)

Discuss enterprise applications: Identify applications developed to query data from BDAT

Wrap-up: Summary/Answer questions

## Participants

**Background/Distributed Data Management System**

**Notes:**
Participants: SWAMP, IEP, CVPIA/CAMP, SRWP, CALFED, DWR , Ag Waivers, Stakeholder groups, etc

## Data Types

**Background/Distributed Data Management System**

**Notes:**
Data types include fisheries, benthic, pesticides, nutrients, phyto-zooplankton, neomysis, water quality, meteorological, hydrodynamic, vegetation data, GIS coverages, etc. 

## Slide 4

## Cooperative Data Management System

**Local Client Databases**

- ODBC/FTP Link

- QA/QC

- Transaction
- Database (TDB)

- QA/QC

**Node**
** ****Database**

**Cooperative Data Management System**

**Notes:**
The local database is used to meet certain criteria that are needed to make this project successful – 	Emphasize Local Ownership and Control

Data providers need to have the tools necessary to participate in an enterprise system

Data entry needs to be dynamic, quick and easy 

Data needs to be made accessible to the system

The system needs to support a large diverse volume of data

Applications: 
	Reports
	QA/QC
	Data Sharing

## Background-BDAT

- Started with IEP
- Expanded and Used By Other Groups
- Became a Permanent Program in DWR as Part of the SWP in 2007
- Still waiting for staff allocation approval
- BDAT will now focus on SWP data user needs
- Providing necessary maintenance and as resources are available;
- Include additional data sets at required frequency
- Set up additional and revised data delivery systems

## Background-BDAT

- Forecasted simulations in support of O&M operating decisions
- Recent historical simulations (fingerprinting) and weekly water quality forecasts in support of O&M operating decisions and in support of the Municipal Water Quality Investigations Program’s weekly Real-Time Data and Forecasting reports sent out to the water contractors
- Modeling and data analysis in support of the recent SWRCB Cease and Desist Hearings
- Verify continued compliance with the objectives set forth in the State Water Resources Control Board’s Decision 1641 and Suisun Marsh agreements.
- Meet requirements for Water Level and Water Quality response plans for Joint Points of Diversion (JPOD) actions.
- Determine carriage water costs for completed water transfers and to estimate those costs for planned transfers
- Provide data to staff conducting analysis for DAT who help perform decision support for SWP operations

## BDAT Background SWP Needs

- Calibration and Validation of DSM2
- Hydrodynamic modeling of recent historical conditions in support of the South Delta Temporary Barriers program
- Forecasts of Water Stage in support of the South Delta Temporary Barriers program
- Particle Tracking Forecast Modeling in support of the Delta Smelt Workgroup
- Particle Tracking Modeling of historical conditions in support of the Pelagic Organism Decline research
- Modeling of hydrodynamics, EC, organic carbon, and taste and odor for the Jones Tract Levee Failure
- SWC analysis
- Aggregating and gaining access to data from many sources for analysis
- Other future modeling needs include data in the following areas:
- Delta Island volumes – to help in flood modeling
- Other water quality constituents in addition to EC including
- Organic Carbon
- Dissolved Oxygen
- Temperature

## BDAT Background SWP Support

- BDAT will continue to Support SWP funded Programs
- The Interagency Ecological Program (IEP),
- Bay-Delta Office, Suisun Marsh
- Compliance and Monitoring, and
- Operations Control Office

## CEDENCooperative Data Management System

**C**
**E**
**D**
**E**
**N**

**Notes:**
Infrastructure model is fully scaleable – Architecture can be replicated to expand as interest and funding are provided.

Data management is conducted locally
	Data are available locally
	Changes in the local database are reflected in the comprehensive database
	IT staff provide technical assistance that benefit both the data provider and comprehensive database project
	Historic data can be incorporated
	Requires maintenance

Relational Database 
Relates divers types of information, including GIS, from multiple agencies, stakeholder groups. 
Data are combined using common naming conventions, units, datum, etc
Saves data users time by providing these data in a readily accessible format
Data can be distributed to remote applications
The system keeps track of changes in data

System combines the best features of a distributed and comprehensive data management system
Data can be queried, with referential integrity, by scientists and applications
Data management is conducted by those who collect the data

## CEDENCooperative Data Management System

**CEDEN****Cooperative Data Management System**

**Notes:**
Many applications are needed to access and process data and information
Application development is dependent on user need specifications
Modelers (who will have specific needs), managers, WQ scientists, fisheries biologists, general public, etc.
The new BDAT web application provides users with another method to retrieve data

## CEDEN EPA/USGS Coordination Summary

- USGS and EPA have begun coordination on developing common web services to serve data out of STORET and NWIS
- Services will follow the WQX Schema
- EPA’s services are planned for release Summer ‘07
- USGS services are planned for release Fall ’07
- Services will include Summary level information, Monitoring Stations, and Results
- For current access to these data and information on WQX visit:
- _[www.epa.gov/storet](http://www.epa.gov/storet)_
- _[http://](http://waterdata.usgs.gov/nwis)__[waterdata.usgs.gov/nwis](http://waterdata.usgs.gov/nwis)_

## CEDENBackground

- Easy to implement by other partners
- Architecture-Federated is distributed
- Can support many types of institutional participation

## Saves on IT budget
Saves on monitoring budget
Greatly improves on the ability to perform key business functions
Integrates standards, metadata and architecture
Federated distributed approach

**Saves on monitoring budget**
**Greatly improves on the ability to perform key business functions**
**Integrates standards, metadata and architecture**
**Federated distributed approach**

**CEDEN****Working with CEDEN**

**Notes:**
I would zip through these and just describe the agency and what the use is in general terms 

## New Precedence For Interagency Data Systems
IT Architecture: CEDEN, BDAT, EPA Exchange Network ...
Metadata: CERES
Standards: SWAMP
Legislation-SB 1070, AB 1404...
DWR will work with the SWRCB on Interagency data systems
Requires Executive leadership and cooperation
Dedication to following the software engineering principals and making the process subservient to water resources planning in California
Executive sponsorship for the project

**IT Architecture: CEDEN, BDAT, EPA Exchange Network ...**
**Metadata: CERES**
**Standards: SWAMP**
**Legislation-SB 1070, AB 1404...**
**DWR will work with the SWRCB on Interagency data systems**
**Requires Executive leadership and cooperation**
**Dedication to following the software engineering principals and making the process subservient to water resources planning in California**
**Executive sponsorship for the project**

**Working with CEDEN**

**Notes:**
I would zip through these and just describe the agency and what the use is in general terms 

## Cooperative Data Management System

**Cooperative Data Management System**

## Cooperative Data Management System

## Cooperative Data Management System

**Cooperative Data Management System**

## Cooperative Data Management System

**Cooperative Data Management System**

## CEDENCooperative Data Management System

## CEDENCooperative Data Management System

## CEDENCooperative Data Management System

## Future Plans BDAT

- GET STAFFED
- Finish Web Services Implementation with the EPA (OWQX)
- EPA Data
- USGS Data
- NOAA Data?
- Complete List of Initial Tasks Provided by SWP Data Users
- Work with CEDEN

## BDAT Task List

- Remove IEP web site.
- Copy/merge IEP data into BDAT
- Copy/move IEP interface to BDAT (as-is or upgrade)
- Output options are DSS, ASCII and Excel
- Time series graphing capability
- Client/server? Who is going to do it?
- Use RKI standard naming convention
- Missing data uses same convention
- Both time series and grab sample data
- All water quality parameters
- Biologics?
- On-line mechanism to notify BDAT of:
- Incorrect data
- Missing data
- Missing data sources
- General comments/feedback
- Metadata:
- Clear indication of original sources of data
- Description of QA/QC applied to data, if any
- Standardized format (for time-series data)
- Summary
- Extra:
- Links to QAPP and other original information about the monitoring program.
- Photos, other ancillary and anecdotal data
- Data Searches:
- Map based
- Text filtered based
- Search by location, collector (agency name), parameter (flow, stage, ec, ...), methods, data category, temporal using multiple date ranges, time interval, source (model, fill, observed, etc), average type (mean max min, etc)
- Combine map and parameter-filter searches
-   GIS Data
-   Include BDAT in DSM2 Users Group; provide category in DSM2 Users Group Forum.

## Future Plans CEDEN

- Propagation of Data Standards
- SWRCBs requirement to be SWAMP comparable  for grant recipients
- Other Departments?
- Grants Data
- SB 1070 and other legislation
- TMDL Data Sets
- Metadata
- Catalog Data in CERES

## CEDEN/BDATComponents of Data & Information Sharing Systems

**Data Sharing**
** ****&**
** ****Management**

- Architecture

- MetaData

- Standards

## CEDEN/BDATComponents of Data & Information Sharing Systems

**Data Sharing**
** ****&**
** ****Management**

- Dissemination

- Feedback

- Monitoring

## Conclusion

- BDAT & CEDEN are distributed data systems that will continue to provide more access to ambient monitoring data.
- These systems subscribe to a distributed architecture, standards and metadata
- We look forward to your input as these systems continue to develop

## Conclusion

- Questions?