# Presentation: 577936

## Applying Submission Agreements to Legacy Data Flows

- A NOAA story.

**Provider Interactions**
**Science Archives **
**in the 21st Century**
**April 26, 2007**
**Dan.Kowal@noaa.gov**

## Presentation Outline:

-          Then and Now:  Historical perspective on data ingest
-  Case Study scenario:  One dataset, many data providers
-    Proposed strategy:  Phased-in process for submission agreements

## Data Ingest: Two Reports about the State of NGDC Data Management

- Reviews in Progress:
-  Analog data at Warehouse
-  Analog data at Federal Records
-   Center
-  Digital data migration from Tivoli
-   to ADIC (Quantum) SAN.

## Data Ingest: Two Reports about the State of NGDC Data Management

- Recommendations:
*NGDC can strengthen relationships with scientific data providers by working with them from the beginning of data collection projects to ensure that relevant data and metadata are obtained and archived properly.*
*...**Recruiting Scientists to work with the data...*
*...**make its own holdings more accessible...*

## Potential Problems without a Submission Agreement: 

**Potential Problems without a Submission Agreement:**
-  The digital objects received do not conform to what the
-   Archive expects.
-  What the Producer delivers has not been clearly defined.
-  The ingest schedule is not fulfilled by the Producer.
-  Errors in transfers are detected late by the Archive, or are
-   not detected prior to use.

**Results could be:**
-  Increased workload
-  Poor quality of archived information
-  Negative effect on the relationship between Archive and
-   Producer.

## Data Ingest: Applications of Submission Agreements.

-  Provider:  Take my data please...
-  Provider:  Backup my data please...
-   Archive:  We want your data!

## Case Study: Ionosonde History

**Case Study: Ionosonde History**

-  First vertical sounding made
-   in 1925, ionosonde machine
-   deployment in 1947.
-  Worldwide network establish
-   during IGY period (1957-59).
-  Since IGY, 100 – 200 stations

## Case Study: Ionosonde Data Types through the Years

## Case Study: Designated Community

- 1. Air Force Weather Agency (AFWA):
- 	Forecasting ionospheric impacts to DoD systems.
- 2. Space Environment Center (SEC):
- 	Forecasting ionospheric conditions for civilian users.
- 3. Air force Research Laboratory (AFRL):
- 	Development and transition to operations of observation, modeling and forecasting technologies for the benefit of the DoD.
- 4. NGDC: Obtaining, archiving and distribution to users a complete, high quality set of observations.

## Case Study: Desired Mirrion Data Flow

**Case Study: Desired Mirrion Data Flow**

## Slide 11

## Proposed Strategy: Provider Network and Community

**Provider Network**
-  AFWA DISS MOA is up for renewal
-  Requested Supplemental Section for SA.
-  MIRRION QC Process requires SIP Specifications
-  6 - 10 data types
-  Sell the QC process as a value-added data ingest
-   and archive function.
-  Influence data quality inclusion in evolving XML
-   data type.

**Scientific Community**
-  Petition International Space Environment Service
-  Standardized schema for all ionosondes
-  Establish frequency of data transmission close to real time.
-  Ensure long-term access to archive and redistribution of data.

## Proposed Strategy: OAIS Ingest Functions

- Quality
- Assurance

- Receive
- Submission

- Generate
- AIP

- Generate
- Descriptive
- Information

- Co-ordinate
- Updates

- A
- D
- M
- I
- N
- I
- S
- T
- R
- A
- T
- I
- O
- N

- PRODUCER

- SIP

- Receipt confirmation
- Resubmit request

- SIP

- QA Results

- Data Formatting and Doc Stds.

- SIP, AIP for Audit

- Audit Report

- DATA
- MANAGEMENT

- Archival Storage

- SIP

- AIP

- Descriptive
- Info

- Descriptive
- Info

- AIP

- Database
- Update
- Request

- Report
- Request

- Report

- Storage Request, AIP

- Storage Confirmation

- (Updated)
- SIP

- Receive
- Submission

- Quality
- Assurance

**Proposed Strategy: OAIS Ingest Functions**