# Presentation: 549008

## Validation Tasks

- PDS NASA-HQ Review

- PDS Systems Engineering Team
- September 8, 2004

*http://pds.jpl.nasa.gov/*

## PDS Pipeline by Project Phase

***OPERATIONS***

***PLANNING/DESIGN***

- Phases A-D				Phase E-F

**Trade Studies*****1***

**ICD / SIS’s per Instrument**

**Data Mgmt/Archive Plan**

**Validation*****2***

**Ingest Data (Receive/Store/Catalog)**

**Distribution/Access**

**Track**

**Archive/Transfer*****1***

**Instrument Team Data Production**

**Delivery to PDS**

**Science Users**

***1, 2****** - Areas of Risk to Schedule/Resources***

_**Core System:**_**  Within planned budget/staffing, with the exception of need for (1) Storage Technology ****	****  Study to determine MRO-era hardcopy archive media and (2) changes in Validation area**
_**Extended System:**_**  Requirements/planning to be done on deferred LOE basis**

**Development*****2***

**Pre-Brief / MOU**

## Background

- In order to fulfill its charter, the PDS requires that data it handles:
- be understandable,
- be in formats that future scientists will find easy to use, and
- follow standards for organization and content that facilitate machine-assisted correlative science across missions and science disciplines.
- The PDS requires that submitted data meet published standards regarding format, content, and documentation.
- akin to requirements of a refereed journal submission

## Policy Governing Quality Assurance

- Every archive quality data product must adhere to a level of quality assurance.
- There are three measures of quality assurance:
- Adherence to compliance with PDS Standards
- Adherence to compliance with the applicable Data Management Archive Plan / Interface Control Document (DMAP / ICD)
- Merit of scientific integrity

## Quality Assurance Adherence to compliance with PDS Standards

- CURRENTLY, ensuring data submitted to the PDS adheres to (is in compliance with) PDS Standards:
- Mix of MACHINE-ASSISTED & MANUAL processes

## Quality AssuranceAdherence to compliance with DMAP / ICD

- CURRENTLY, ensuring data submitted to the PDS  complies with the detailed design documents and that the detailed design documents accurately reflect the actual development of the submitted data:
- Mix of MACHINE-ASSISTED & MANUAL processes

## Quality AssuranceMerit of scientific integrity

- Quality AssuranceMerit of scientific integrity

- CURRENTLY, ensuring data submitted to the PDS contains information / documentation relevant to the mission, the instruments, and the calibrations that a scientist of a future generation can intelligently use the data, understand and interpret the data, and, if appropriate, even recalibrate the data:
- MANUAL process

## Quality AssuranceSummary of Current Processes

- Currently, ensuring adherence to measures of quality assurance is a largely manual effort using a mixed suite of tools.
- Approximately 85% of validation, ingestion, and tracking is “manual”
- The remaining 15% is “machine-assisted”
- There are no “automatic” processing / processes within the PDS
- The entire suite of validation tools can be run against an archive volume within 10 minutes - Machine-Assisted processing
- Interpreting the output from those tools can consume multiple days – Manual processing

## Quality AssuranceSummary of Workforce Estimates

- Quality AssuranceSummary of Workforce Estimates

- The Baseline production profile (2002 thru 2003) indicates average of:
- FTEs = 1 (required to process 82.5K data products / quarter)
- Validation, Ingestion, and Tracking is most “labor intensive” for the initial “release” of data.  The second “release” requires fewer workhours.  The third and subsequent “releases” are most workhour efficient
- Initial release estimate  = 1.5K data products / 8 hrs (validation)
- Second release = 2.5K data products / 8 hrs (validation)
- Subsequent release = 5K data products / 8 hrs (validation)

## Ingestion, Validation, & Tracking Services(IV&T)

- LOOKING FORWARD
- The problem
- Increase in production profiles necessitates increase in FTEs so as to not compromise quality assurance
- The solution
- Create a highly-automated pipeline -  the end result being an ingested, validated, and online accessible set of science data
- Mix of AUTOMATED & MACHINE-ASSISTED processes

## Ingestion, Validation, & Tracking Services(IV&T)

- PROPOSED SOLUTION
- Create a highly-automated pipeline -  the end result being an ingested, validated, and online accessible set of science data

## IV&T Services

**Ingest, to** (verb) – the process of registering, storing and cataloging a product in the repository
**Validate, to** (verb) – the process of affirming compliance with PDS Standards and design documents
**Track, to** (verb) – the process of tracking and reporting status of data sets

## Tech Infusion for PDS-IV&T

- OODT’s Catalog and Archive Service (CAS) Component
- Provides the framework for capturing and managing data products and the associated data processing (workflow)
- Delivered for SeaWinds operations beginning Dec 2002
- Basis for future Earth missions including
- Ocean Vector Winds
- InSAR (proposed start 2005)

- SeaWinds on ADEOS II (Launched
- Dec 2002)

**Notes:**
As I said early, OODT has really made a significant contribution to cancer research.  We’ve provided the data sharing infrastructure for NCI’s early detection research network and they’ve connected ten institutions together.  This the beginning of a very large vision for NCI which grids-together the data assets captured by various researchers at home institutions.  OODT was able to connect institutions together without changing their implementation.  

NCI is currently using OODT to share biospecimens so that researchers can have a unified view of biospecimen repositories across the nation.  Their use of OODT will grow not only in terms of the number of connected institutions, but also in terms of the bioinformatics tools and data that is being shared. They are currently working to designing a validation study for early detection and OODT will provide the core data management infrastructure to capture, process, and share the data across all the scientists.

NCI is also working to open up access to this system outside of NCI and make it available to all cancer research as a national asset. 

## Quality Assurance using IV&T Services

- There are three measures of quality assurance:
- Adherence to compliance with PDS Standards
- Adherence to compliance with the applicable Data Management Archive Plan / Interface Control Document (DMAP / ICD)
- Merit of scientific integrity
- IV&T Services will ensure:
- Adherence to compliance with PDS Standards
- Adherence to compliance with the applicable Data Management Archive Plan / Interface Control Document (DMAP / ICD)
- IV&T Services will NOT ensure:
- Merit of scientific integrity

## SummaryWorkforce Projections

- The  division of processing dictates the level of workforce projections:

**# of Data Products**

## SummaryFuture Production Profiles / WorkForce Estimates

- Using current manual processes, FTEs are predicted against “# of products” produced
- Using IV&T processes, FTEs are predicted against “# of standard_product_types” produced – groupings of products
- IV&T will handle predicted load without increase in FTEs

## IV&T Development Timeline

- IV&T Development Timeline

**Cassini Data Set Tracker**

**IV&T**
**Version 2.0**

**IV&T Prototype**

**IV&T **
**Version 1.0**

**Requirements**
**Review**

## IV&T Work Breakdown Structure

## Backup Material

## IV&T - Validation Service

**Validate, to** (verb) – the process of affirming compliance with PDS Standards and design documents

## Validation References / Tool Suite

- PDS references
- Standards Reference
- Data Processing Workbook (DPW)
- Online reference materials
- Validation Tools
- NasaView, LVTOOL, TBTOOL, etc
- Ingestion Tools
- Web based Catalog Loader, NRI2

## PDS-A Objective

- Provide a mechanism whereby:
- the data-supplier deposits one more files into the repository:
- the system detects the file(s), marks the files as “new”, and routes for processing
- the system validates the file(s) against a “manifest”
- the system compares the validation results against a “master”
- passed validation – flag file(s) for distribution, notify subscribers of availability
- failed validation – send operator & data supplier validation report

## Cassini Tracking System (CATS)

- Tracking
- Track deliveries by standard data product id:
- Acknowledge delivery
- Accept delivery
- Reject delivery
- Withdraw delivery
- status date, user id affecting status, total bytes, delivery location, media, and notes for each delivery.
- Notification
- Email users defined in the database for the following actions:
- new delivery
- acknowledge delivery
- accepted delivery
- rejected delivery
- withdrawn delivery

## Cassini Tracking System  (CATS) Users

- Instrument Teams
- Insert  / Withdraw deliveries.
- Notified when delivery records are inserted or have a change in status.
- Verify identified gaps (in data) as valid or late deliveries.
- Create reports.
- PDS Teams
- Acknowledge  / Accept / Reject deliveries
- Notified when delivery records are inserted or have a change in status.
- Create reports
- Project Archive Coordinator
- Notified when delivery records are inserted or have a change in status.
- Delete delivery records (in case of errors)
- Insert gap records.
- Create reports
-  Administrator
- Adds / deletes uses / rights and privileges /  instruments
- Adds standard data product ids and defined attributes..
- Guest
- Create certain reports.

## PDS Archive

**PDS Archive**

**MRO Era**

_**Assumptions**_
**51 Tbit MRO Mission**

## 4-Tier Architecture - Summary of Multi-Year Implementation

- Client Tier

- Web Tier

- Service Tier

- Storage Tier

- Repositories

- Product Catalogs

- Data Set Catalog

- Product
- Servers

- Profile
- Servers

- DS Cat
- Profile
- Server

- Product
- Servers

- Product
- Servers

- Profile
- Servers

- Profile
- Servers

- Ingest-Workflow MGR

- Validation Service

- Tracking Service

- Data Set View

- Data Set Browsers

- Query Server

- PDS Explorer

- Subscription MGR

- Volume Packer

- Authentication Server

- Web Browser

- Bulk Downloader

- Server Manager