# Presentation: 948627

## NPOESS Data Exploitation (NDE)Preliminary Design Review

- November 21, 2006
- NSOF Building 4231 Suitland Road Room 2001

## Welcome-J. Silva

## PDR Agenda

**   ****8:00 a.m.    Welcome****	**
**	****	**
**   ****8:15 a.m.    Introduction/Background**
**   ****9:00 a.m.    Project Overview**
** ****10:45 a.m.    NDE Context and Hardware****	**
** ****11:15 a.m.    Algorithm Development****	**
** ****12:00 p.m.    Lunch**
**  ****1:00 p.m.     Data Handling System**
**  ****2:30 p.m.     IT Security**
**  ****3:00 p.m.     Communications Study **
**  ****3:40 p.m.     Closing Remarks****	**
**  ****4:00 p.m.     PDR Board Review of RFAs**

## Agenda

-    8:00 a.m.    Welcome**	****	**
**   ****8:15 a.m.    Introduction/Background**
-    9:00 a.m.    Project Overview
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

** ****Introduction/Background****	****J. Silva**
** **Mission & Design Drivers	J. Silva
-  Architectural Overview	G. Goodrum
-  Data Products		T. Schott

## NDE Design Review Board

**	**
- Chairperson: Vanessa Griffin, (NESDIS/OSD) Ground Systems Division Chief
- Mike Mignogno (NESDIS/OSD) Polar Program Manager
- Mike Tanner (NESDIS/HQ) Senior Program Advisor
- Reginald  Lawrence (NESDIS/OSDPD), Environmental Satellite Processing Center Chief
- David Benner (NESDIS/OSDPD) Satellite Services Division Chief
- Mitch Goldberg (NESDIS/STAR) Satellite Meteorology and Climatology Division Chief
- Joseph Mulligan (NOAA/IPO) NPOESS Interface Data Processing Segment Lead
- Kevin Schrab, Ph.D. (NWS) Satellite Product Improvement Manager
- Brian Gockel (NWS/OST/SEC) Systems Engineer
- Lt. Amanda Bittinger (NESDIS/STAR) Satellite Oceanography and Climatology Division
- Charles MacFarland NESDIS/OCIO Representative

## Request for Action

**Briefing Title: ****	****	****   **
**Slide Number: **
**  ****_________________________________________**
** ****Comment/ Question:**
**	****	****	****	****	****	****	****	****	**
**	**** **
**  ****_________________________________________**
**  ****Background on Comment or Question:**
** ****_________________________________________**
** ****Team response:**
**  **
**___________________________________________**
**Submitted by:  _________**

- RFAs are available on tables at back of room
- RFAs to be submitted to a Review Board Member
- All RFAs are due today by the end of the meeting (4:00 p.m.)
- Please state your questions clearly
- Review Board will disposition RFAs
- Board Members are requested to convene to disposition RFAs

## The NDE IT Design Team

## Today’s Objective

**Objectives**: To ensure that NDE’s design concepts are consistent with NPOESS and NOAA architectures and that NDE plans are feasible
*NDE Design Review Board* roles and responsibilities:
- Review NDE system conceptual design
- Review project plans necessary to reach Critical Design Review (CDR) in August 2007
- Review strategies (acquisition, development, testing, implementation, transition)
- Assess risks
- Request actions that will improve NDE designs and plans

## Assessment Criteria

- Does the proposed preliminary design satisfy the contractual requirements?
- Has the overall system framework been clearly presented?
- Has the functionality been appropriately allocated to subsystems?
- Have the interfaces between subsystems and to external entities, including users, been identified?
- Have the security features and controls been defined?
- Have delivery strategies been defined for the subsystems (build, buy, reuse)?
- Are the plans and schedules leading to CDR feasible?

## Background Documents

**NDE Requirements & Performance Measures**
- Software Engineering
- Infrastructure
- Systems Management
- Data Retention & Archive
- Interfaces
- Operational Product Generation
- Communications
-       & Distribution

- See NDE Web site:
- _[http://projects.osd.noaa.gov/nde](http://projects.osd.noaa.gov/nde)__[ ](http://projects.osd.noaa.gov/nde)_

## NDE Design Review Board:Review Calendar

| Date | Deliverable | Tasked |
| --- | --- | --- |
| Nov 21 | Requests for Action (RFA) | Review Board |
| Dec 15 | Action Responses to Board and Chairperson | NDE 
Design Team |
| Jan 26 | Final Report | Design Board Chairperson  |

## Agenda

-    8:00 a.m.    Welcome**	****	**
**   ****8:15 a.m.    Introduction/Background**
-    9:00 a.m.    Project Overview
- 10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

-  Introduction/Background	J. Silva
** ****Mission & Design Drivers****	****J. Silva**
-  Architectural Overview	G. Goodrum
-  Data Products		T. Schott

## Mission

**The NPOESS Data Exploitation (NDE) will provide critical environmental products derived from NPP and NPOESS observations to NOAA’s operational community in near-real time **

## Capabilities Assessment:NDE’s Δs

| Current State |  | Target State |
| --- | --- | --- |
| DMSP + POES |  Δ | NPP + NPOESS |
| End-to-End Control |  Δ | Post-processing |
| Orbital Processing |  Δ | Granule Processing |
| Stove-pipe Applications Processing |  Δ | Centralized Data Processing |
| Bolted-on Security |  Δ | Security Compliant w/ Standards |
| 100 GB/day Ingest |  Δ |  ~2.5 (NPP) TB/day Ingest   |
| 300 GB/day Distribution |  Δ | ~6 TB/day Distribution |
| Heritage Instruments |  Δ | New Instrument Technologies |

**Notes:**
Main Idea: OPPORTUNITIES & CHALLENGES
All NDE design and planning is intended to narrow the gap between current and future states (the “deltas”) 

## NDE Project:Capabilities Assessment

-  Instrument specific
-  SafetyNet
-  “Continuous” ingest
-  Near-real time = <30 minutes
-  All data packaged as HDF5

- NDE Project:Capabilities Assessment

- _Current State_	                 _Target State_
**    ****Orbital Processing****	****     **Δ**	****Granule Processing**

-  Orbital Processing
-  Near-real time = 156 minutes from observation to product

- NDE systems must adapt to all new input formats, alter formats to user needs, take advantage of improved latency

**Notes:**
Current (orbital) systems have dictated:
 Structure and logic of programs  
 Rhythm of  NESDIS operations 

Future emphasis (consistent with NOAA Strategic Plan)
 Anytime processing 
 More timely delivery
 Work rhythms more attuned to customer needs

## NDE Project:Capabilities Assessment

-  Reusable objects
-  Database Technologies & Data Management
-  Common CM, testing, and transition processes
-  Common toolsets
-  Centralized management and control

- NDE Project:Capabilities Assessment

-          _Current State_	                                     _Target State_
**           ****Stove-pipe **
**Applications Processing             ****Δ****	****     **

-  Stove-pipe Applications Processing
-  Duplicated, redundant functionality
-  High cost of management & control

- NDE will establish latest proven technologies and manage them in 3 basic environments: Development, System Test, Operations

**Centralized IT Architecture**

**Notes:**
Current (product line) ways of working are part of NESDIS culture, therefore the challenges are not just technological. 

Future: NDE, which will be executed within OSDPD and STAR, expects to be a full participant in the changes that are anticipated with the NESDIS-wide reorganization currently being analyzed (SWOT by Short & Assoc.) 

“Latest proven”  is preferred over “cutting edge.”  We will use “proven in many real systems” as a selection criterion.  But it also says that we will select products that take advantage of current technologies provided by forward looking vendors.  These days, that forward look is very commonly summarized as “Service Oriented Architecture, “ or SOA.  This implies many “latest proven technologies:” object oriented and 4GL programming languages, web services, XML and other kinds of markup languages (and, for us, GIS).  

## NDE Project:Capabilities Assessment

-  Online subscriptions
-  GUIs
-  More Dissemination Options:
- Push, pull, network, fiber, internet, etc.

- NDE Project:Capabilities Assessment

- _Current State_	                                _Target State_
** ****300 GB/day Distribution          ****	****       ****Δ****	****    ****~6 (tbc) TB/day Distribution**

-  Product benefits realized by relatively small customer base
-  Limited number of dissemination options
-  “orders” require developer intervention
-  FTP (pull) in most cases

- NDE to establish a Communications Architecture in collaboration with NOAA’s Network Advisory Committee (NAC) and AWIPS

**Notes:**
This is NPP estimate.  Assuming a 10% annual increase, and introduction of additional products, this figure will expand to 38TB/day by 2020.
Will get a 2 sentence rationale from Barbara that answers question:  What logic applied to derive these numbers.

## Environmental satellite industry challenge
NESDIS history: ~ 2 1⁄2 years from data availability, through research,  to provision of products to operational users (R2O)

All new NOAA projects must have Transition Teams and submit Transition Plans for approval
NDE submitted plans (MIRS, CrIS/ATMS, Sea Surface Temperature) developed with SPSRB oversight

NDE’s Goals
Minimize IT obstacles as algorithms migrate through development, test, and operational environments
Work with end users to prepare operations for products

- NESDIS history: ~ 2 1⁄2 years from data availability, through research,  to provision of products to operational users (R2O)
- All new NOAA projects must have Transition Teams and submit Transition Plans for approval
- NDE submitted plans (MIRS, CrIS/ATMS, Sea Surface Temperature) developed with SPSRB oversight
- NDE’s Goals
- Minimize IT obstacles as algorithms migrate through development, test, and operational environments
- Work with end users to prepare operations for products

- The Transition Challenge

## NDE Milestone Plan

## Agenda

-  Introduction/Background	J. Silva
-  Mission & Design Drivers	J. Silva
** ****Architectural Overview****	****G. Goodrum**
-  Data Products		T. Schott

-    8:00 a.m.    Welcome
-    **8:15 a.m.    Introduction/Background**
-    9:00 a.m.    Project Overview
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

## NDE System  Objectives

- Disseminate NPOESS Data Records to NOAA Operational Community
- Generate and disseminate tailored NPOESS Data Records (versions of NPOESS Data Records in previously agreed alternative formats and views)
- Generate and disseminate NOAA-unique products (augmented environmental products constructed from NPOESS Data Records)
- Deliver NOAA-unique products, product processing elements, and associated metadata to CLASS for long-term archiving
- Provide services to customers, including NDE product training, product enhancement, and *implementation support across NOAA*
- Provide software for NPOESS Data Record format translation and other data manipulations

**Notes:**
Jim,
Suggest giving this slide to Geoff or moving it to end of your brief.   If you deliver it up front, it may force you to try to say too much before the design drivers have been explained.  

## NPOESS Data Exploitation (NDE) Mission Context Diagram

**NPOESS**
**Ground**
**System**

**NOAA Environmental Satellite Processing Center**
**(ESPC)**

**NOAA Unique**
**Products**

**Tailored**
**Products **

**Tailoring**
**Tools**

**Data Records**
**(xDRs)**

**NOAA Unique**
**Products**

- NPOESS will
- deliver more than
- 100 different data
- records (xDRs)

**Data Records (xDRs) **

**Data Records**
**(xDRs)**

**Near Real**

**-**

**time **

**Customers**

**Near Real**

**-**

**time **

**Customers**

**Near Real**

**-**

**time **

**Customers**

**Climate**

**Community**

**NOAA**

**-**

**Unique Products**

**Tailoring Tools**

**xDRs  **

**CLASS**

**NPOESS**
**Data**
**Exploitation**
**(NDE)**

**CLASS**

**Near Real-time**
**Customers**

**Climate**
**Community**

## Scope & Requirements

- Design, develop and implement
- Development Environments for NPP/NPOESS
- System Test Environment for NPP/NPOESS
- Operational Environment for NPOESS

## Vision:3 Environments, Common Standards

- Environmental Satellite Processing Center

- Shared Capabilities, Common Standards

**NDE**
**Operational**
**Environment**

**NDE**
**System**
**Test/Backup**
**Environment**

**NDE**
**Development**
**Environment**

## Design Considerations

- Algorithm modularity
- Reuse
- Centralization
- Scalability

## IDPS / NPP & NDE Integration Timeline

- ...

- Note: New NPP IPAC Verification events (Performance Verification) not yet Planned

**Concepts in work**

- NDE-NPP
- Integration

- NDE System Test
- Environment

- NDE CriticalDesign

- NDE PreliminaryDesign

## Risk Handling and MitigationAssessment of impact and project plans

- Risk Handling and Mitigation*Assessment of impact and project plans*

- Mitigating steps in NDE’s current plans
- Develop a risk management plan
- For IT work, schedule control via earned value management
- Develop lifecycle cost and benefit estimates
- Develop the project budget
- Plan for needed resources
- Design systems to support reliability requirements
- Monitor NOAA and DoC security requirements
- New risk mitigating activities
- Develop and implement a ***quality management plan*** to address system and data quality, quality assurance, and quality control.
- Develop and implement ***NDE continuity plans*** that are coordinated among participating organizations.
- Coordinate development of ***preliminary (pre-launch) data*** with the IPO and instrument designers

**Notes:**
The method that we used to develop the risk management plan was drawn from various sources including NASA, DoD, and the Project Management Body of Knowledge (PMBoK).  We started with last year’s risk analysis and augmented it with risks identified from a risk database and stakeholder interviews.  The database query drew from generic project and software project risks.  The stakeholders have experience in various aspects of polar satellite data and processing.  Approximately 380 risks were identified.

Similar risks were combined to create a comprehensive set of about 50 risks.  These risks were incorporated into a survey where respondents provided information about likelihood and impact for each risk.  Respondents also had the opportunity to state additional risks and to put value on NDE objectives.  This helped provide a complete risk picture.

High impact, very likely, and other significant risks were analyzed for their impact on NDE objectives.  Risk handling approaches and mitigation activities were developed for these risks.  These analyses and plans are documented in the Risk Management Plan.

The notional chart in the lower right shows how the significant risks were identified.  The arrows show how risks can be tracked over time, sometimes moving over category thresholds.

## Risk Management Summary

- Risks management plan will be revised as risks are handled, new risks are identified, or stakeholder views change
- Continued input from stakeholders and other sources
- Developers have a critical responsibility to identify issues, risks, and challenges and their potential impact on operations
- Preliminary design review will identify issues and challenges for key design elements

## Agenda

-  Introduction/Background	J. Silva
-  Mission & Design Drivers	J. Silva
-  Architectural Overview	G. Goodrum
** ****Data Products****	****	****T. Schott**

-    8:00 a.m.    Welcome
-    **8:15 a.m.    Introduction/Background**
-    9:00 a.m.    Project Overview
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

## Product Definitions

- Products delivered by NPOESS contractor
- xDRs
- Raw Data Records (RDR)
- Sensor Data Records (SDR)
- Temperature Data Records (TDR)
- Environmental Data Records (EDR)
- Intermediate Products (IP)
- Pre-planned Product Improvements (P3I)
- Products produced by NOAA using xDR information
- Tailored xDRs and IPs
- NOAA Unique Products (NUPs)

## NPOESS Products Requirements

**Integrated Program Office (IPO)**

**NPOESS Integrated Operational Requirements Document (IORD – II)**
**Dec 10, 2001**

**NPOESS Technical Requirements Document**

**Sensor, Temperature, Environmental Data Records (xDRs)**

**Requirements Document**

- Contract Document

- Deliverables

**NDE Product Matrix **

**(1) Tailored xDRs **
**(2) NOAA-unique products (NUPs)**

**NDE Product Development**

## Sample from Mission Goal Version of Product Matrix

**Master, W&W, Climate, Ecosystems, C&T spreadsheets**

**Products                           Line Office                                    Mission Goal Review**
**                                                                                                 ****(CORL items in gray)**

- Motivation:  Develop NPOESS products to meet all NOAA user requirements

## NPP Product Summary

|  | NPP
Totals |
| --- | --- |
| Data Records (xDRs/IPs/P3I)  | 39 |
| NOAA Unique Products | 59 |
| Totals | 98 |

## Acquisition StrategyEmphasis POES Mission Continuity

- Nunn-McCurdy process restructured NPOESS acquisition efforts
- POES mission continuity might rely on NPP data operationally

## Define NPP Products

- POES and EOS Mission Continuity
- Essential that we address these products
- Development products
- Under development and could go operational prior to NPP launch
- New products
- Never provided to user community and not under development within NESDIS

## NPP Mission Continuity Products

| NOAA-Unique Products |  | Tailored Products |
| --- | --- | --- |
| CrIS Thinned Radiances | SST Anomalies | ATMS Radiances |
| Total Precipitable Water (ATMS) | Coral Reef Degree Heating | CrIS Radiances |
| Snow Cover (ATMS) | Coral Reef Bleaching | VIIRS Radiances |
| Precipitation Type/Rate (ATMS) | Tropical Rainfall Potential | OMPS Radiances |
| Surface Emissivity (ATMS) | Vegetation Fraction | Cloud Mask |
| Cloud Liquid Water (ATMS) | Hazard Support (Tropical) | Sea Surface Temperature (SST) |
| Sea Ice Cover/Concentration  | Hazard Support (Volcano) | Ozone Profile |
| Snow Water Equivalent (ATMS) | Total Ozone (CrIS) | Ozone Total Column |
| Ice Water Path (ATMS) | Blended Ozone | Snow Cover |
| Land Surface Temperature | Outgoing Longwave Radiation  | Imagery |
| Temperature Profiles (ATMS) | Outgoing Longwave Radiation  | Ocean Color/Chlorophyll |
| Moisture Profiles (ATMS) | Absorbed Radiation | Vegetation Index |
| Blended Snow Cover | Rainfall Prediction | Active Fires |
| Harmful Algal Blooms | Tropical Cyclone Intensity | Atmospheric Temperature Profile |
| Regional Ocean Color | Tropical Rainfall Potential | Atmospheric Moisture Profile |
| Blended SST | Vegetation Fraction | Aerosol Optical Thickness |
|  |  | Surface Type & Vegetation Cover |
|  |  | Surface Albedo |
|  |  | Cloud Cover/Layers |

## NPP Development Products

| NOAA-Unique Products | Tailored Products |
| --- | --- |
| CrIS Cloud Cleared Radiances | Aerosol Particle Size |
| Clear Sky Radiances (VIIRS) | Cloud Top Temperature |
| Polar Winds (VIIRS) | Cloud Top Pressure |
| Vegetation Health | Land Surface Temperature (VIIRS) |
| Vegetation Moisture |  |
| Drought Indices |  |
| Vegetation Thermal Conditions |  |
| Leaf Area Index |  |
| Fire Potential |  |
| SST (AVHRR-like) |  |
| Aerosol (AVHRR-like) |  |
| Carbon products |  |

## NPP New Products

| NOAA-Unique Products | Tailored Products |
| --- | --- |
| Integrated xDRs at CrIS Resolution | Cloud Base Height |
| Cloud Liquid Water Path (VIIRS) | Cloud Effective Particle Size |
| Cloud Ice Water Path (VIIRS) | Cloud Optical Thickness |
| Cloud Top Temperature (VIIRS) | Cloud Top Height (VIIRS) |
| Inversion Strength and Height | Ice Surface Temperature |
|  | Net Heat Flux |
|  | Sea Ice Characterization (VIIRS) |
|  | Suspended Matter |
|  | Atmospheric Pressure Profile |

## Agenda

** ****Project Overview****	****	****G. Sloan**
-  Requirements Mgmt	L. Fenichel** **
-  Development Approach	E. Wilson
-  Tool Selection		E. Wilson

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
**   ****9:00 a.m.    Project Overview**
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

## A Reminder

- Parts of NPP, NPOESS, and NDE are subject to:
- ITARs (International Trafficking in Arms Restrictions)
- “NOFORN” Classification

## Topics

- Requirements Management
- Development Approach
- Algorithm Development
- Data Handling System
- IT Security
- Data Distribution
- The Way Forward

## Master Phasing Schedule

## NDE Baselined Documents

| Document | Status |
| --- | --- |
| NDE Concept of Operations  | Draft Delivered 8/24/2006
Draft Delivered 11/7/2006 (Incorporated RIDS from Peer Review) |
| NDE Federal Enterprise Architecture  | Draft Delivered 8/24/2006 |
| NDE Data Distribution Conceptual Design  | Draft Delivered 8/24/2006 |
| NDE Stake Holder Survey Summary Status 2006  | Draft Delivered 8/24/2006
Draft Re-Delivered 8/28/2006 |
| NDE Network Services Design  | Draft Delivered 8/24/2006 |
| NDE System/Subsystem Design Description  | Draft Delivered 8/24/2006
 |
| NDE Logical and Geographical Connectivity Design  | Draft Delivered 8/24/2006
 |
| Standards for NDE Algorithm Development, Delivery, Integration & Test  | Draft Delivered 8/24/2006
 |

## NDE Baseline Documents (Cont’d)

| Document | Status |
| --- | --- |
| Project Plan | Draft Delivered 8/2004
Draft Delivered 8/2005
Draft Delivered 8/2006
Draft Delivered 11/2006 |
| NDE Systems Requirement Specification | Draft Delivered 11/7/2006 |
| NDE Configuration Management Plan | Draft Delivered 5/18/2006
Draft Delivered 6/30/06
Draft Delivered 8/30/2006 |
| Risk Management Plan | Delivered 7/14/2006 |

## Management Approach

- Staff the NDE Design Team with experienced professionals
- Use design tools
- Prototype of Development Environment
- IBM p570 using AIX
- Use NPP as the NDE risk reduction mission
- Maximize use of COTS and GOTS (minimize custom code)
- Collaborate with similar, current projects
- Develop prototypes to validate requirements and design
- Sell-off requirements as early as possible

## Lessons Learned from NASA’s EOSDIS Core System

- Integrate Algorithms with the Data Handling System early in the life cycle
- Test the system under full operational load – data driven systems behave in strange ways
- Build in Operator Dashboard and System Metrics for reporting
- NetIQ and CA’s Unicenter are candidates
- Configuration Parameters
- Put in Database for change control, audit trail
- Document CPs, min/max values, default, suggested values
- Use “Rolling Wave” technology refresh approach
- Constant effort
- Keeps budgets flat – no large peaks every 5 years
- Just in Time Hardware buys

## NDE Design Assumptions

- All NDE users are *registered users*
- Users get data by entering an *NDE* *subscription*
*Subscriptions *have two types
- Standing Order (no expiration date)
- Ad-Hoc (covers a limited time period)
- Subscription to NOAA-unique products to be archived to CLASS will be determined by Archive Review Board
- NDE will subscribe to all IDPS xDRs (in lieu of implementing complex logic using IDPS API to order only xDRs needed)
- NDE will not distribute RDRs in near-real time to users (exception:  Telemetry)
- Users will need to request RDRs from CLASS
- Product latency is customer and product specific

## Data Access

- NDE will flag customers as authorized for data or not during data denial
- Use Initial Joint Polar System (IJPS) Data Denial Implementation Plan for initial authorization guidance
- NDE will maintain a customer database that will contain sufficient attributes in order to set flag
- Restricted data will be delayed for subscriptions from unauthorized customers
- Timeliness more an issue for NPOESS than NPP
- Only authorized users will be notified of data access status
- End User License Agreement with customer restricts redistribution

## Agenda

-  Project Overview		G. Sloan
** ****Requirements Mgmt****	****L. Fenichel**** **
-  Development Approach	E. Wilson
-  Tool Selection		E. Wilson

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
**   ****9:00 a.m.    Project Overview**
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

## Requirements Management Matrix

| ID | Section | Requirement | Contract |
| --- | --- | --- | --- |
| SRS203 | 3.14.10 | Development Lifecycle Tools - The Contractor will specify a suite of proven development lifecycle tools to enhance NESDIS capabilities in performing developmental and software maintenance tasks.  The documentation will demonstrate that the selected tools are widely supported in the remote sensing software industry and the most likely to be known by future NESDIS support staff.  Technologies in this category are: CASE tools, modeling tools, 4th Generation Languages, Testing tools, and Requirements Tracking tools. | SE3 |
| SRS94 | 3.14.6 | SEI Capability Maturity Model - The Contractor shall design the System so that its management capabilities can be evaluated in terms of the Software Engineering Institute's (SEI) Capability Maturity Model (CMM), with the goal of Level 2 certification during proposal evaluation and Level 3 certification three years after contract award. | SM1 |
| SRS201 | 3.14.8 | NOAA IT Best Practices - The System shall be designed and built with NOAA IT "Best Practices" guidance from the NESDIS Information Technology Architecture Team (ITAT). | SE8 |

## Requirements Management – The Process

-  Define Business Objectives
-  Contained in Contract Requirements Matrices under Desired Outcomes and Performance Standard columns
-  Additional Objectives detailed in the Concept of Operations
-  Automate Requirements Analysis, maintain Traceability (Use DOORS)
-  Reduce Ambiguity
-  Contractual Requirements reviewed, System Requirements Spec (SRS) created
-  Derive Requirements to further capture NDE functionality
-  Use Case Driven Process
-  Cause/Effect Tables
-  Automate using UML compliant tools
-  Domain Expert Reviews and Refinements
-  Several Iterations
-  Capture and Formalize Detailed Requirements
-  Test Cases generated
-  ‘Final Round’ of Domain Expert Reviews

## NDE Requirement Allocations

| Allocations | Number of SRS Requirements |
| --- | --- |
| Customer Services | 16 |
| Ingest | 24 |
| Production | 26 |
| Distribution | 25 |
| Product Management | 22 |
| Common Services (Infrastructure) | 50 |
| System Monitoring and Control | 22 |
| Security | 1 |
| Documentation | 12 |

## NDE Requirements Traceability

| ID | Section | Short Title | Contract | Allocation | Use
Case | Test
Case |
| --- | --- | --- | --- | --- | --- | --- |
| SRS62 | 3.2.1.3.2
 | SARSAT Telemetry from IDPS | XF1 | Ingest | UC101 | TC25 |
| SRS63 | 3.2.1.3.3	 | ADCS Data from IDPS | XF1 | Ingest | UC101 | TC26 |
| SRS58 | 3.2.1.3.4
 | Product Subscriptions to IDPS | XF1 | Ingest | UC102 | TC44 |
| SRS101 | 3.2.2.3.1
 | Available Data Product Projections | PG5 | Production | UC181 | TC124 |
| SRS102 | 3.2.2.3.2
 | Available Data Product Aggregations | PG6 | Production | UC182 | TC131 |

-  Maintained in DOORS Tool with Active Links to other NDE Documents
-  Telelogic Tool Suite (DOORS, TAU Developer)
-  Use Cases Mapped
-  Test Cases Mapped
-  Test Results Tracked

## Challenges Matrix

| Challenge | Related Requirement | Impact | Mitigation |
| --- | --- | --- | --- |
| Maintain Requirements Traceability throughout the Project Life Cycle | (SRS201) 3.14.8 NOAA IT Best Practices | Potential testing gaps and schedule delays as a result | Use of Tools (SRS201/Automated Tool Suite) |
| Full specification of NDE System Elements and Components | N/A | Cost, quality, schedule | Employ a methodology (SRS204) where domain experts are involved early in the process, delivery at CDR |

## Agenda

-  Project Overview		G. Sloan
-  Requirements Mgmt	L. Fenichel** **
** ****Development Approach****	****E. Wilson**
-  Tool Selection		E. Wilson

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
**   ****9:00 a.m.    Project Overview**
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-    1:00 p.m.     Data Handling System
-    2:30 p.m.     IT Security
-    3:00 p.m.     Communications Study
-    3:40 p.m.     Closing Remarks
-    4:00 p.m.     PDR Board Review of RFAs

## Development ApproachRequirements

| ID | Section | Requirement | Contract |
| --- | --- | --- | --- |
| SRS204 | 3.10.1 | The Contractor shall identify a widely accepted software engineering methodology to be used on the project. | SE12 |
| SRS94 | 3.14.6 | The Contractor shall design the System so that its management capabilities can be evaluated in terms of the Software Engineering Institute's (SEI) Capability Maturity Model (CMM), with the goal of Level 2 certification during proposal evaluation and Level 3 certification three years after contract award. | SM1 |
| SRS172 | 3.9.2 | The Contractor shall evaluate candidate System Elements for operational fitness using Unit, Integration, Regression, Stress (load), and End-to-End System testing. | SE4 |
| SRS173 | 3.9.2.1 | Each algorithm for creating NOAA-Unique products will be described in terms of explicit, expected test results prior to the installation of the NOAA-supplied algorithm on the operational product generation system.  The NOAA-Unique algorithms must satisfy these test requirements. | PG8 |
| SRS182 | 3.11.6 | The Contractor shall cooperate with algorithm developers to identify System Test procedures, standards, and the criteria to be applied in determining a system elements' fitness for operational status. | SE4 |

## Development Approach Requirements Based Testing 

*What is it?*
- Requirements Based Testing (RBT) is a System Implementation Methodology that
**Substantially Reduces the Risk of**
**Building the Right System to the “Wrong” Requirements !**
*Why are we Concerned about That ?*
- Over half all software project defects originate in the requirements phase
- 82% of application re-work is related to requirements errors
- Only 54% of initial project requirements are actually realized
- Requirements problems account for 44% of project cancellations

- Statistics by James Martin and others, taken from “Eliminate the Testing Bottleneck”, Borland White Paper, August, 2006, p. 4.

## The Methodology of Requirements Based Testing (RBT) 

*It’s about Testing as an Integral Step of Implementation -- not a Separate Phase*
**Design *****Tests before***** Designing the *****Software***** -- the Test Cases become the Ultimate Statement of the Develops’ Requirements**
***Iterate***** the Requirements-to-Test Case Loop *****with Users and Domain***** *****Experts***** until they concur that successful Test Execution will produce the result they want – not necessarily what they originally specified!**
*...**to insure that:*
**What we Build is What we Say**
**and What we Say is What we Mean **

## Traditional Implementation Hand-Off Phases

- Natural Language Requirements

- Design

- Code

- Test Plans

- Natural Language Requirements

- Design

- Test Plans

-  Analysts talk to Domain Experts to express *Natural Language      Requirements*
-  S/W Designers Interpret these into *Design*, which is then interpreted into *Code*
- Independent Test Team Interprets Requirements to develop *Test Plans*
**The Result:  Unresolved ambiguities and inconsistencies between what was developed, what is tested, and what is really needed!**

## RBT Focuses on Formalized Requirements & Test Cases

- Formal Req’ts
- and
- Test Case
- Repository

- _Requirements Analysts_

- _Users_
- _Domain Experts_

***Create Formal Req’ts from Natural Language***

***Create Logical Test Cases***

***Review/Fix Req’ts & Logical Test Cases***

***Review/Fix/Validate Req’ts & Logical Test Cases***

- _Developers_

- _Test Experts_

***Study Logical Test Cases Complete Physical Test Cases***

***     ******Study Logical Test Cases***
*** ******Review Design versus Validated Test Cases ***
***Review Code versus Validated Test Cases***

## Requirements-Based Testing (RBT) Workflow*

**Validate against **
**Business Objectives**

**Ambiguity Analysis**

**Req’ts Review by**
**Domain Experts**

**Map against**
**Use Cases**

**Structure/Formalize **
**Requirements**

**Define/Optimize**
**Test Cases**

**Test Case Review**
**By Req’s Authors**

**Test Case Review**
**By Domain Experts**

**Developers Study**
**Test Cases**

**Test Experts Study**
**Test Cases**

**Design & Code**
**Review**
**with Test Cases**

**Create**
**Test Procedures**

**Execute Tests**

_**Requirements Quality**_

_**Test Case Quality**_

_**Design and Code Quality**_

_**Test Execution**_

_**Logical Test Case Design**_

**Validated Requirements**

**Requirement ‘Fixes’**

**Code**
**Baseline**

**Logical**
**Test Cases**

- *Diagram based on  “Eliminate the Testing Bottleneck”, Borland White Paper, August, 2006, Fig. 2.

**Baseline of **
**Validated**
**Test Cases**

## Sample Technique for Formalized Requirements

| REQ’T ID | CAUSES | EFFECTS |
| --- | --- | --- |
| SRS80 | System is in Degraded Operations Mode
Customer meets Pre-Defined Criteria [TBD] 
Customer has Pending Data Request | Pending Data is distributed to Customer 
Customer’s Pending Data Subscriptions are Ignored while Degraded Operations Mode persists |

## Development ApproachChallenges

| Challenge | Related Requirement | Impact | Mitigation |
| --- | --- | --- | --- |
| Build the NDE Systems as we Say, and Say what we Mean them to Be | SRS94, SRS173, SRS182 | Unresolved ambiguities and inconsistencies between what gets developed, what gets tested, and what is really needed!
 |  Utilize Requirements Based Testing (RBT) Methodology |
| Find a Suite of Development Tools that Support the RBT Implementation Workflow | SRS203 | Worst case:  develop RBT artifacts manually (using only Excel, Visio, Word) and develop a homegrown database to track baselines, team member updates, and requirements to test traceability | Identify the Development Functionalities needed to Support RBT; establish Criteria to Evaluate Tools’ performance w.r.t the Functions, taking into account interoperability between Tools in  a Candidate Suite |

- Development ApproachChallenges

## Agenda

-  Project Overview		G. Sloan
-  Requirements Mgmt	L. Fenichel** **
-  Development Approach	E. Wilson
** ****Tool Selection****	****	****E. Wilson**

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
**   ****9:00 a.m.    Project Overview**
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

## Tool Selection Requirements

| ID | Section | Requirement | Contract |
| --- | --- | --- | --- |
| SRS93 | 3.8.3.2 | The NDE Processing System Design will use COTS and Open Source software where it is possible, practical, and approved by the Government. | SE10 |
| SRS203 | 3.14.10 | The Contractor will specify a suite of proven development lifecycle tools to enhance NESDIS capabilities in performing developmental and software maintenance tasks.  The documentation will demonstrate that the selected tools are widely supported in the remote sensing software industry and the most likely to be known by future NESDIS support staff.  Technologies in this category are: CASE tools, modeling tools, 4th Generation Languages, Testing tools, and Requirements Tracking tools. | SE3
 |
| SRS215 | 3.10.2 | The contractor shall develop the data processing elements of the future NDE system using the latest proven technologies (programming languages, CASE tools, object repositories, data base management systems, etc.) that are appropriate for remote sensing data processing. | SE13 |

## RBT Steps whereNDE Expects to Employ         *

- *Diagram based on  “Eliminate the Testing Bottleneck”, Borland White Paper, August, 2006, Fig. 2.

**Validate against **
**Business Objectives**

**Ambiguity Analysis**

**Developers Study**
**Test Cases**

**Test Experts Study**
**Test Cases**

**Baseline of **
**Validated**
**Test Cases**

**Code**
**Baseline**

**Tools**

## IT Development ToolsFunctional Categories

|          SUMMARY OF FUNCTIONAL CATEGORY SCORES |  |  |  |  |
| --- | --- | --- | --- | --- |
|   1.  Interoperability (with respect to a Suite, and with Common Interchange Tools) |  |  |  | 0 |
|   2.  Requirements Specification & Maintenance |  |   |   | 0 |
|   3.  Configuration Management |  |  |   | 0 |
|   4.  Requirements Verification |   |   |   | 0 |
|   5.  Engineering Management/Team Support |   |   |   | 0 |
|   6.  General Architectural Design and Modeling |   |   |   | 0 |
|   7.  Data Modeling |   |   |   | 0 |
|   8.  Business Process Modeling |   |   |   | 0 |
|   9.  Software Analysis, Design, and Construction |  |   |   | 0 |
|  10.  Extensibility |   |   |   | 0 |
|  11.  User Interface |   |   |   | 0 |
|  12.  OS/Network Support - Tool Server |   |   |   | 0 |
|  13.  Tool Administration & Vendor Support |   |   |   | 12 |
|  14.  Documentation, Training, and Ongoing Education |  |   |   | 0 |
|  15.  Vendor Comprehension & Licensing Offer |   |   |   | 0 |

## Tool Selection Methodology

- Goal:
- To Define a Process for Trade-Off Comparisons between IT Development Tools
- That:
-  Establishes a Consistent Approach regardless of the Individuals performing the Evaluation
-  Produces Quantitative Results for Comparison and Justification
-  is Conducted as Objectively as Possible
- . . . and can also be applied to:
-  Architecture Operational Components
-  Science Development Tools

## Sample SectionIT Development Tool Evaluation Worksheet

|   13. Tool Administration & Vendor Support  |  |  |  |  |  |  | 12 |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 13.1 | Ease of Installation (server/client combined) | 50 | <none> | 0 | 10 |   |   |
| 13.2 | Degree of Site/User Configurability | 50 | <none> | 0 | 10 |   |   |
| 13.3 | Vendor includes personal tool setup assistance in the purchase price | 90 | <none> | 0 | 10 |   |   |
| 13.4 | Number of Release Defects Reported by Vendor | 50 | <none> | 100 | 0 |   |   |
| 13.5 | Number Un-reported Defects uncovered during Evaluation of Release | 80 | <none> | 10 | 0 |   |   |
| 13.6 | Number of old versions of the tool currently supported | 65 | Enter the number: 1,2,3,4... | 0 | 5 |   |   |
| 13.7 | Vendor provides online help support tool | 50 | <none> | 0 | 10 |   |   |

- Sample SectionIT Development Tool Evaluation Worksheet

## We Attempt to Establish  an Evaluation Framework that is as Objective as Possible

- We evaluate each tool with respect to all functional categories, and develop our own profile of a tool’s capabilities – giving each tool a numeric score for each functionality
- For a suite of tools, we calculate functional category scores by combining scores for all tools in the suite for each category:
**	****Suite Functional Score = Min(Tool Scores) for a**
- 													**“****weakest link”**
**	****Suite Functional Score = Max(Tool Scores) for a**
- 		     												 “**strongest component”**
- Similar approach applied to:
- Science Development Tools	-	K. Tewari
- Architectural Components	-	A. Al-Jazrawi

**Common Function**

**Specialized  Function**

## IBM Rational:
	Requisite Pro, Clear Case, Clear Quest, Software Architect, (Test Manager)

Telelogic:
	DOORS, Synergy CM, Synergy Change, System Architect, Tau Developer, (Rhapsody), ... plus Mercury Test Director

Borland:
	Caliber DefineIT, Caliber RM, Silk Central, Together, 	StarTeam

**	**Requisite Pro, Clear Case, Clear Quest, Software Architect, (Test Manager)
**Telelogic**:
- 	DOORS, Synergy CM, Synergy Change, System Architect, Tau Developer, (Rhapsody), ... plus Mercury Test Director
**Borland:**
- 	Caliber DefineIT, Caliber RM, Silk Central, Together, 	StarTeam

- IT Development Tool Suites Under Consideration

## Tool Selection Challenges

| Challenge | Related Requirement | Impact | Mitigation |
| --- | --- | --- | --- |
| Find a Suite of Development Tools that Interoperate smoothly, allowing us to move from step to step or backwards without manual cut-and-paste  transfer of data | SRS203 | Vendor Tool Suites are a mix of in-house products and products from buy-outs, not as well integrated as heralded; members of some suites don’t even run on the same set of platforms | Our evaluation methodology makes special provisions to evaluate a Suite as a whole; Tool Interoperability  is scored w.r.t the Suite Context; Suite score for that and other common functions is taken  to be the “weakest link” of the Suite members |
| Find a Suite of Development Tools that can be expected to be supported several years into the future. | SRS203 | IT Tool development is currently a very dynamic area with many contenders, some of them with outstanding new products | Our evaluation criteria include vendor stability factors, as well as number years experience with the tool (e.g. after a buyout); our down-select list is composed of 3 of the most well-known, stable vendors in this field. |

## Tool Selection Challenges

| Challenge | Related Requirement | Impact | Mitigation |
| --- | --- | --- | --- |
| Business Process modeling tools are closely tied to their vendor’s middleware for collection of information to feed the model; they are not interchangeable to other vendors’  middleware | SRS215 | It could be that our first choice for modeling NDE business activities would force us toward middleware that might be inadequate for our near real-time requirements | We will evaluate the middleware architectural components first, as that component will have the broadest impact on overall system performance; however, before making a final decision we will evaluate the Business Process tools to check that we aren’t forced toward an unacceptable product; we may have to balance the gap in tools with the gap in architectural components. |

## Agenda

** ****Context****	****	****	****G. Roth**
** ****Environments****	****	****G. Roth****	**
** ****Hardware****	****	****G. Roth****	**

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
-    9:00 a.m.    Project Overview
** ****10:45 a.m.    NDE Context and Hardware**
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

## NPOESS Context Diagram

- FNMOC
- (Fleet Numerical
- Meteorology and
- Oceanography Center)

- AFWA
- (Air Force
- Weather Agency)

- NAVOCEANO
- (Naval Oceanographic
- Office)

- NPOESS
- (National Polar-orbiting Operational
- Environmental Satellite System)
- (IDPS)

- NESDIS
- (National Environmental
- Satellite Data and
- Information Service)

- (Supports NPP)

- (Supports NPP)

## IDPS Context Diagram

- NSIPS
- (NPOESS Science
- Investigator-led
- Processing System)
- (cal val)

- SDS
- (Science Data
- Segment)
- (NASA)

- CLASS
- (Comprehensive Large
- Array-data Stewardship
- System)
- (archive)

- IDPS
- (Interface Data
- Processing Segment)
- (at NOAA)

- SAN

- NESDIS

- NDE

## NDE Context Diagram

- IDPS
- (at NOAA)

- Users

- Algorithm
- Developers
- (STAR)
- (Center for Satellite
- Applications and Research)

- CLASS
- (LTA)
- (Long Term Archive)

- ESPC
- (Environmental Satellite Processing Center)

- NDE

**Notes:**
Comprehensive Large Array-data Stewardship System 

## NDE Environment Requirements

| ID | Section | Requirement | Contract |
| --- | --- | --- | --- |
| SRS83 | 3.14.3 | The System shall be designed to support Operational, System Test, and Development Environments. | SE2 |
| SRS237 | 3.14.3.1 | The Contractor shall provide an Operational Environment design that lowers the cost and risks of generating and distributing NPOESS-derived products to customers. | SE2 |
| SRS85 | 3.14.3.2 | The System shall provide the capability to support the development of the Data Handling System and the development and integration of Scientific Algorithms in a segregated Development Environment. | I3 |
| SRS84 | 3.14.3.3 | During the NPP mission, the System Test Environment will be segregated in a manner that supports "Quasi-Operational" product generation and distribution (Quasi-Operational is defined as 24 X 7 automated product generation and distribution with 8 X 5 Science and Operations support services). | I1 |
| SRS86 | 3.14.2.3 | - During the NPP mission, the System will have the capability to support product volumes of 4 TB/day.
- During the NPOESS C1 mission, the System will have the capability to support product volumes of  8 TB/day.
- During the NPOESS C2 mission, the System will have the capability to support product volumes of 12 TB/day. | I1 |

## NDE System

- IT Development Environment
- Where the Data Handling System (DHS) is developed
- Science Development & Integration Environment
- Science Algorithm Development Environment
- Science Algorithms may be delivered from STAR Collaborative Environment (CE)
- Algorithms integrated with Data Handling System (DHS)
- Algorithm Performance Tuning
- Functional, end-to-end testing of NDE System
- System Test Environment
- Stress Testing, Regression Testing and Operations Acceptance Testing
- System and Algorithm Performance Testing
- Serves as Backup for Operations Environment
- Augmented to make products during NPP (quasi-operational)
- Operations Environment
- DHS and Algorithms
- High Availability
- Low NDE Latencies
- High Product Volume

## NDE System

- Configuration
- Management

## DHS Promotion & Integration Process

## Algorithm Promotion &

-  Integration Process

## Hardware: Design & Development Environment (Phase 1)

- Design

- Automated Tool Servers
- Dell PE1850 Servers
- Dual 3.0GHz
- 4 GB RAM
- 2 Win2003 & 1 RHEL4
- 42U Rack

- CITS Administration LAN

- Storage
- ESPC SAN Expansion
- 146 GB SATA Drives
- 2.33 TB (usable)
- StorNext FS Server
- 42U Rack

- Processing
- IBM p570
- 16 x 2.2 GHz CPUs
- 32 GB RAM
- AIX 5.3
- 42U Rack

- ESPC Network

- StorNext

-             Development Environment & CM

## “Rolling Wave” Capacity & Upgrade Planning

- “Rolling Wave” Capacity & Upgrade Planning

- SAN Disk
- Storage: Ingest & Distribution

- CPUs:
- Processing
- & Database

- Data
- Direct
- Networks
- Disk
- Storage

- Data
- Direct
- Networks
- Disk
- Storage

- N + *n *< 2N

- additional capacity

- Operations

## IBM Virtualization

**Virtual I/O Server**
- Shared Ethernet
- Shared SCSI and Fibre Channel-attached disk subsystems
- Supports AIX 5L V5.3 and Linux* partitions
**Micro-Partitioning**
- Share processors across multiple partitions
- Minimum partition 1/10th processor
- AIX 5L V5.3, Linux*, or i5/OS**
**Partition Load Manager**
- Balances processor and memory request
**Managed via HMC or IVM*****

**AIX 5L**
**V5.2**

**Linux**

**Hypervisor**

**Dynamically resizable**

**2 **
**CPUs**

**4**
**CPUs**

** ****6 **
**CPUs**

**  **

**Linux**

**Linux**

**AIX 5L V5.3**

*** ******Virtual I/O paths ***

**AIX 5L V 5.3**

**AIX 5L V5.3**

**AIX 5L V5.3**

**AIX 5L V5.3**

**Micro-Partitioning**

**Manager**
**Server**

**LPAR 2**
**AIX 5L V5.3**

**LPAR 1**
**AIX 5L V5.2**

**LPAR 3**
**Linux**

**PLM  partitions**

**Unmanaged partitions**

**Hypervisor**

- PLM agent

- PLM agent

**AIX 5L**
**V5.3**

**6**
**CPUs**

**Ethernet**
**sharing**

**Virtual I/O server partition**

**Storage****sharing**

**1 **
**CPU**

**i5/OS**
**V5R3****

**1**
**CPU**

-   * SLES 9 or RHEL AS 4 and above
-  **Available on selected p5-570, p5-590 and p5-595 models
- ***IVM on p5-560Q and below

**IVM**

## NDE Design Challenges

| Challenge | Related Requirement | Impact | Mitigation |
| --- | --- | --- | --- |
| IDPS requires use of StorNext to write to ESPC SAN vs ESPC security zones | Imposed on NDE by IDPS and ESPC | Multiple NDE hosts also need to “see” the same file system on the ESPC SAN that IDPS uses; therefore NDE is also required to use StorNext as its metadata server | ESPC (Brian Callicott) is pursuing technical solution |
| Processing multiple read & writes to & from SAN (utilizing StorNext) may not meet product latency requirements | SRS 133
3.2.5.4
Data Product Latency Table
CD9 | File I/O to/from SAN may not meet latency requirements. | Early testing of StorNext from IDPS to the ESPC SAN and early interface testing of StorNext from the ESPC SAN to NDE. |
| Potential risk of not having a Product Technical Baseline and model may impact the ability to correctly size the number of CPUs, RAM, Disk, and internal networks required to meet latency requirements. | SRS 133
3.2.5.4
Data Product Latency Table
CD9 | Insufficient hardware sizing of future System Test and Operations Environments or increased spending to overcompensate for lack of information | Develop a Product Technical Baseline and model with Product Development Lead Tom Schott
 |

## Agenda

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
-    9:00 a.m.    Project Overview
-  10:45  a.m.    Building Blocks and Hardware**	**
** ****11:15 a.m.    Algorithm Development****	**
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     Communications Study
-   3:30 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

-  **Research to Operations            M. McHugh**
-  SADIE			       K. Tewari

## GOAL:  Faster, more efficient transition of research to operations.
HOW:  By facilitating interaction between STAR and NDE developers
Similar: 
Standards; 
Configuration management (CM) tools; 
CM procedures.
Common: 
Libraries, error handling, development and visualization tools etc.
Compatible
Architectures.

**GOAL**:  Faster, more efficient transition of research to operations.
**HOW**:  By facilitating interaction between STAR and NDE developers
**Similar**:
- Standards;
- Configuration management (CM) tools;
- CM procedures.
**Common**:
- Libraries, error handling, development and visualization tools etc.
**Compatible**
- Architectures.

- Research to Operations

## Similar Environments

## Research to Operations 

- Create the Delivered Algorithm Package (DAP) for an algorithm

- Send DAP to the SADIE

- Research to Operations

- STAR Collaborative Environment (SCE)

- Integrated Product Team

## Research to Operations 

- Create the Delivered Algorithm Package (DAP) for an algorithm

- Send DAP to the SADIE

- Research to Operations

- STAR Collaborative Environment (SCE)

- Integrated Product Team

## Science Algorithm Development and Integration Environment (SADIE)

- Receive DAP from
- outside of SADIE

- Place received DAP under
- configuration management
- (CM) control

- Compile algorithm

- Baseline the DAP

- Ensure DAP has
- correct format

- Integrated Product Team

- Integrate algorithm into
- Data Handling System

- Execute algorithm
- using its DAP

- NDE Development Team

- CM Manager

## Science Algorithm Development and Integration Environment (SADIE)

- Receive DAP from
- outside of SADIE

- Place received DAP under
- configuration management
- (CM) control

- Compile algorithm

- Baseline the DAP

- Ensure DAP has
- correct format

- Integrated Product Team

- Integrate algorithm into
- Data Handling System

- Execute algorithm
- using its DAP

- NDE Development Team

- CM Manager

## Science Algorithm Development and Integration Environment (SADIE)

- Receive DAP from
- outside of SADIE

- Place received DAP under
- configuration management
- (CM) control

- Compile algorithm

- Baseline the DAP

- Ensure DAP has
- correct format

- Integrated Product Team

- Integrate algorithm into
- Data Handling System

- Execute algorithm
- using its DAP

- NDE Development Team

- CM Manager

## Science Algorithm Development and Integration Environment (SADIE)

- Receive DAP from
- outside of SADIE

- Place received DAP under
- configuration management
- (CM) control

- Compile algorithm

- Baseline the DAP

- Ensure DAP has
- correct format

- Integrated Product Team

- Integrate algorithm into
- Data Handling System

- Execute algorithm
- using its DAP

- NDE Development Team

- CM Manager

## Science Algorithm Development and Integration Environment (SADIE)

- Receive DAP from
- outside of SADIE

- Place received DAP under
- configuration management
- (CM) control

- Compile algorithm

- Baseline the DAP

- Ensure DAP has
- correct format

- Integrated Product Team

- Integrate algorithm into
- Data Handling System

- Execute algorithm
- using its DAP

- NDE Development Team

- CM Manager

## SCIENCE ALGORITHM DEVELOPMENT AND INTEGRATION ENVIRONMENT (SADIE)

- Receive DAP from
- outside of SADIE

- Place received DAP under
- configuration management
- (CM) control

- Compile algorithm

- Baseline the DAP

- Ensure DAP has
- correct format

- Integrated Product Team

- Integrate algorithm into
- Data Handling System

- Execute algorithm
- using its DAP

- NDE Development Team

- CM Manager

## Science Algorithm Development and Integration Environment (SADIE)

- Receive DAP from
- outside of SADIE

- Place received DAP under
- configuration management
- (CM) control

- Compile algorithm

- Baseline the DAP

- Ensure DAP has
- correct format

- Integrated Product Team

- Integrate algorithm into
- Data Handling System

- Execute algorithm
- using its DAP

- NDE Development Team

- CM Manager

## System Test Environment

- Install Algorithm

- Test Algorithm

- Promote to Operations

- Install Algorithm

- Operations Environment

- Monitor Algorithm
- Performance

- CM Manager

- NDE Development Team

- Integrated Product Team

- ESPC CCB

## System Test Environment

- Install Algorithm

- Test Algorithm

- Promote to Operations

- Install Algorithm

- Operations Environment

- Monitor Algorithm
- Performance

- CM Manager

- NDE Development Team

- Integrated Product Team

- ESPC CCB

## System Test Environment

- Install Algorithm

- Test Algorithm

- Promote to Operations

- Install Algorithm

- Operations Environment

- Monitor Algorithm
- Performance

- CM Manager

- NDE Development Team

- Integrated Product Team

- ESPC CCB

## System Test Environment

- Install Algorithm

- Test Algorithm

- Promote to Operations

- Install Algorithm

- Operations Environment

- Monitor Algorithm
- Performance

- CM Manager

- NDE Development Team

- Integrated Product Team

- ESPC CCB

## System Test Environment

- Install Algorithm

- Test Algorithm

- Promote to Operations

- Install Algorithm

- Operations Environment

- Monitor Algorithm
- Performance

- CM Manager

- NDE Development Team

- Integrated Product Team

- ESPC CCB

## Agenda

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
-    9:00 a.m.    Project Overview
-  10:45  a.m.    Building Blocks and Hardware**	**
** ****11:15 a.m.    Algorithm Development****	**
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     Communications Study
-   3:30 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

-  Research to Operations            M. McHugh
** ****SADIE****	****	****	****    K. Tewari**

## Science Algorithm Development & Integration Environment (SADIE)

- Has similar components and configuration as the IT, System Test, and Operational environments
- Uses common CM and Defect Tracking tools as other environments
- Accessible locally and remotely to algorithm developers
- Supports development, integration with DHS & test of new algorithms and those delivered from STAR Collaborative Environment (SCE)
- Supports a suite of software development and science tools (COTS, GOTS, etc.)
- Receives Delivered Algorithm Package (DAP) from SCE

## SADIE

- SCE users receive
-    -  Access and training in use of SADIE
-    -  Defect Reports
- Supports development of additional science tools and applications (i.e., validation of products, determination of coefficients, etc.)
- Supports trouble-shooting of product generation failures in System Test and Operational Environments
- Supports algorithm functional and regression testing
- Supports fine tuning of algorithms for meeting latency goals in System Test and Operational Environment

## Candidate Science Development & Integration Tools

**ArcExplorer**
**ArcInfo**
**ArcView**
**CDAT**
**EDGEIS**
**Geomatica**
**GeoTIFF **
**GrADS**
**HDF4 Tools**
**HDF5 Tools**

**IDL/Envi**
**IMSL**
**MATLAB**
**McIDAS**
**netCDF**
**OPeNDAP**
**PV-WAVE**
**SAS**
**TeraScan**
**WXP**

## Science Algorithm Development, Delivery, Integration & Test Standards

**Algorithm Development Standards**** **
- Example: Use Similar Development and Test Environments
**Algorithm Delivery Standards**** **
- Example: Standardized Algorithm Delivery Package
**CM and Defect Tracking Standards**** **
- Example: Algorithm Developers and NDE use same CM and Defect Tracking Tools
**Algorithm Input Standards**** **
- Example: Production Rules Not Hard-coded in Science Algorithms
**Algorithm Processing Environment Standards**** **
- Example: Customized Toolkit Containing Utilities
**Algorithm Output Standards**** **
- Example: Common File Format for All NOAA-Unique** **Products

## Design Challenges

| Challenges | Related Requirement | Impact | Mitigations |
| --- | --- | --- | --- |
| Agreement on Standards...
Adherence to Standards... | SE14 | Delays in research to operations | Collaboration w/STAR |
|  Provide low NDE Latency times for products (minutes)
 NDE Latency largely determined by algorithm performance 
 Algorithms are developed by another part of NESDIS and sometimes in another environment
 | SRS 10 3.3.3 
 | Fail to achieve major goal of NDE |  Integrate algorithms and DHS as soon as possible
 Algorithm Development, Delivery, Integration and Test Standards being developed jointly with STAR
 Make STAR Collaborative Environment and SADIE as alike as possible to facilitate Delivered Algorithm Package (DAP)  |

## Agenda

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
-    9:00 a.m.    Project Overview
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
** ****12:00 p.m.    Lunch**
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

## Agenda

** ****External Interfaces****	****E. Wilson**
-  DHS Architecture		E. Wilson** **
-  Common Services		P. MacHarrie
-  Subsystem Design		A.Al-Jazrawi

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
-    9:00 a.m.    Project Overview
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
**  ****1:00 p.m.     Data Handling System**
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

## External Interface Requirements

| ID | Section | Requirement | Contract |
| --- | --- | --- | --- |
| SRS243 | 3.3.1 | IDPS Data Acquisition - The System shall provide the capability to acquire all xDRs, Intermediate Products, SARSAT Telemetry, A-DCS, ancillary, and auxiliary data and metadata from IDPS. | XF1 |
| SRS244 | 3.3.2 | External Ancillary Data Acquisition – The System will provide the capability to configure ancillary data acquisition streams from external sources. | PG9 |
| SRS10 | 3.3.3 | Data Product Retrieval - The NOAA-Unique and Tailored Products generated by NDE shall be made available to customers by placement in locations where data can be extracted within a time not to exceed the time specified in the Data Product Latency Table. | XF3 |

## External Interface Requirements

| ID | Section | Requirement | Contract |
| --- | --- | --- | --- |
| SRS137 | 3.3.4.1 | Archive Data Used for Functional Testing - Metadata and ancillary data, as well as Intermediate , NOAA-Unique, and Tailored Products used in documented Functional Tests, will be sent to CLASS. | DA5 |
| SRS138 | 3.3.4.2 | Retrieve Data From CLASS - The System will have the capability to retrieve data from CLASS. | DA9 |
| SRS140 | 3.3.5 | MMC Interface Through ESPC - ESPC Operations shall provide an interface between NDE and the NPOESS Mission Management Center (MMC) such that 100% of the NDE inquiries to the MMC and NDE replies to MMC reuests are received by the MMC in a time not to exceed that specified in the ICD, and that 100% of the notifications and inquiries from the MMC to NDE are received by NDE in a time not to exceed that specified by the ICD. | XF9 |

## External Interface Requirements

| ID | Section | Requirement | Contract |
| --- | --- | --- | --- |
| SRS141 | 3.3.6 | Service Requests to IPO - An interface for NDE service requets to the IPO shall be provided such that 100% of the NDE service requests intended for the IPO are delivered to IPO and 100% of the IPO responses to NDE service requests are received by NDE. | XF10 |
| SRS142 | 3.3.7 | Service Requests to IDPS - The System shall provide the capability to enable the ESPC/NDE Operations Staff to communicate with the IDPS Operations Staff. | XF11 |
| SRS235 | 3.3.8 | ESPC Trouble Tickets - The System shall provide the capability to interface with the ESPC Trouble Ticket system. | SM16 |

## NDE Context: Operational Environment (NPOESS)

**NDE**
**OPS**

**ESPC**

**METOP**

**NPOESS MMC**

**CLASS**

**IDPS**

**NDE**
**Customers**

- NPOESS Status

- MMC Messages

- Ancillary Data

- Delivery Consolidated Reports

- External Ancillary Data

- Subscription Requests/ Approvals

- xDRs, IPs, NUPs,    Tailored Products, ADCS Data,  SARSAT Tlm

- Applicant Registration  & Customer Support Dialogues

- Recovery Datasets Logs

- ARWG Approved NUPs

- Retrieved Data

**PALs**

- Ordering & Tracking Messages

- Customer Profiles

- ESPC Operator Dialogues

- Subscription/Status Requests

- Status Reports

## Sample InterfaceOperational IDPS 

## Summary of External Interface Documents (1/2)

_**Interface Control Documents (ICDs):**_
- Environmental Satellite Processing Center (ESPC)
- NPOESS Mission Management Center (MMC)
- Interface Data Processing Segment (IDPS)
- External Suppliers of Ancillary Data (METOP, NAVOCEANO,...)
- Program Area Lead (PAL) Handbook
- Comprehensive Large Array-data Stewardship System (CLASS)
- Continuity of Operations Plan (COOP) Repository

## Summary of External Interface Documents (2/2)

_**Service Level Agreements (SLAs):**_
- National Weather Service (NWS)
- National Ocean Service (NOS)
- United States Mission Control Center (USMCC)
- United States Global Processing Center (USGPC)
- Any Customer Requiring “Denied” Data
- Any Customer Requiring System Status Reports
_**Standard End User License Agreement (EULA)**_
- All Other Customers
_**Customers’ User Guide**_
- All Customers

## Design Challenges

| Challenge | Reqt | Impact | Mitigation |
| --- | --- | --- | --- |
| Supporting simultaneous Quasi-Operations as well as System Test modes | SRS84 | Customer satisfaction, system performance, operational quality  | Staffing plan, design trades |
| Defining the interface with the CIP  | J-I1  |  Incomplete design | Continuance of Operations Plan |

## Agenda

-  External Interfaces		E. Wilson
** ****DHS Architecture****	****	****E. Wilson **
-  Common Services		P. MacHarrie
-  Subsystem Design		A.Al-Jazrawi

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
-    9:00 a.m.    Project Overview
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
**  ****1:00 p.m.     Data Handling System**
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

## DHS Architecture Requirements

| ID | Section | Requirement | Contract |
| --- | --- | --- | --- |
| SRS93 | 3.8.3.2 | The NDE Processing System Design will use COTS and Open Source software where it is possible, practical, and approved by the Government. | SE10 |
| SRS203 | 3.14.10 | The Contractor will specify a suite of proven development lifecycle tools to enhance NESDIS capabilities in performing developmental and software maintenance tasks.  The documentation will demonstrate that the selected tools are widely supported in the remote sensing software industry and the most likely to be known by future NESDIS support staff.  Technologies in this category are: CASE tools, modeling tools, 4th Generation Languages, Testing tools, and Requirements Tracking tools. | SE3 |
| SRS215 | 3.10.2 | The contractor shall develop the data processing elements of the future NDE system using the latest proven technologies (programming languages, CASE tools, object repositories, data base management systems, etc.) that are appropriate for remote sensing data processing. | SE13 |

## Requirements Focus on the NDE Perimeter

- NDE Perimeter

**User Scenarios**

**Test Cases**

## Defining What’s Inside NDE(Functions, Data, Activities)

- NDE Perimeter

**IDEF0**

**E-R**

**Activity**

## Overview of the Data Handling Subsystems

| 1.0 Customer Services

 Customer Subscription Web Service

PAL Subscription Approval  Web Service

 Customer 
Status Reporting Web Service | 2.0 System Monitoring & Control

 MMC Message Handling

 Process & Component Monitoring & Control

Mode Switch & Failover Management

 System  Logs

 Recovery Datasets

 | 3.0 Product Management

 Algorithm Definition Web Service

 Product Definition Web Service | 4.0 Ingest

 Receive  & Store Data

 Receive IDPS Delivery Messages & Consolidated Reports | 5.0 Production

 Processing Manager

 Production Scheduler (Rule Based) | 6.0 Distribution

 User Data Transmission

Archive to CLASS

 Data Cleanup

 | 7.0 Common Services

 Enterprise Service Bus

Relational Data Base Management System

Rule Management |
| --- | --- | --- | --- | --- | --- | --- |

## Functional Categories for Architecture Operational Components

- Interoperability
- Customer Service Framework
- System Monitoring and Control
- Product Processing (Ingest, Production, Distribution) Service Framework
- Database Management
- Business Process Specification
- Security Policy Enforcement
- Network Availability  & Performance Monitoring
- Administrative User Interface
- OS/Network Support – Tool Server
- Component Administration & Vendor Support
- Documentation. Training, and Ongoing Education
- Vendor Comprehension & Licensing Offer

##  Service Framework Candidates

**BEA:**
- Tuxedo, Weblogic, Aqualogic
**IBM:**
- Websphere
**Oracle:**
- Fusion
***Note:  Business Process Modeling capabilities are spread between IT Tools (modeling) & Components (performance data accumulation)***

## DHS Architecture Challenges

| Challenge | Related Requirement | Impact | Mitigation |
| --- | --- | --- | --- |
| Integrating Middleware with CASE tools | SRS215 | Maintainability of design | Prototyping, early testing,  staffing |

**Notes:**
Our choice of  a middleware architectural tool will dictate our choice of a Business Process modeling tool.  Business Process modeling tools are closely tied to their vendor’s middleware for collection of information to feed the model; they are not interchangeable to other vendors’  middleware
Our choice of  a middleware architectural tool will dictate our choice of a Business Process modeling tool.  Business Process modeling tools are closely tied to their vendor’s middleware for collection of information to feed the model; they are not interchangeable to other vendors’  middleware

It could be that our choice for middleware would force us toward a Business process modeling tool that might be substantially inferior to the norm
We will evaluate the middleware architectural components first, as that component will have the broadest impact on overall system performance; however, before making a final decision we will evaluate the Business Process tools to check that we aren’t forced toward an unacceptable product; we may have to balance the gap in tools with the gap in architectural components.

## Agenda

-  External Interfaces		E. Wilson
-  DHS Architecture		E. Wilson** **
** ****Common Services****	****P. MacHarrie**
-  Subsystem Design		A.Al-Jazrawi

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
-    9:00 a.m.    Project Overview
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
**  ****1:00 p.m.     Data Handling System**
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

## Common Services Requirements 

| ID | Section | Requirement | Contract |
| --- | --- | --- | --- |
| SRS89 | 3.14.1.3 | Starting with the NPOESS-C1 mission, the NDE Operational Environment shall not be interrupted for more than 2 hours in any 24 hour period and no more than 4 hours in any 30 day period. | SE2 |
| SRS215 | 3.10.2 | Use Proven Technologies - The contractor shall develop the data processing elements of the future NDE system using the latest proven technologies (programming languages, CASE tools, object repositories, data base management systems, etc.) that are appropriate for remote sensing data processing. | SE2, SE13 |
| SRS159 | 3.6.2 | Scalability - The Operations, System Test, and Development Environments will be scalable--that is, additional capacity (throughput, latency, performance) can be added to it as needed without redesign of the system infrastructure.  It will be scalable from the initial system to support NPP up through a system to support the simultaneous operations of NPOESS-C1 and NPOESS-C2. | I1 |
| SRS87 | 3.2.5.2 | Provide Automatic Failover - The System shall provide an automatic failover capability that will re-create a fully functioning configuration from a failed configuration with no more than 1% of the tasks requiring manual intervention. | DD5 |

## DHS Architecture

- Database

- Event Service Bus

- Data Handling Subsystems

- NDE

- Ingest

**Production**

- DHS GUIs

**Distribution**

**Product/Customer/System Mgmt**

## Initialization

- Database

- Event Service Bus

- Data Handling Subsystems

- Ingest

**Production**

- DHS GUIs

**Distribution**

- Initialize:
- Algorithms
- Product-Generation
-  Rules
- Customers
- Subscriptions,
- System Mode (i.e.
-  Data Access)

- NDE

**Product/Customer/System Mgmt**

## Data Driven

- Database

- Event Service Bus

- Data Handling Subsystems

- Ingest

**Production**

- DHS GUIs

- Input Events

- Output Events

- Files Arrive at  Landing Zone

- Product Distribution and Notification

**Distribution**

- NDE

**Product/Customer/System Mgmt**

**Notes:**
Design strategies:
data driven processing
use enterprise tools to support availability and scalability and COOP requirements
Alternate processing systems
OPUS – schedule driven
S4PM – data driven, but doesn’t use an RDBMS

## RDBMS Evaluation

- Database (Oracle, DB2)
- Scalability, availability, disaster recover and spatial data types and functions narrows vendors to Oracle, DB2
- Highest ranked within industry standard transaction benchmarks (about 1,600 transactions per second / per CPU)
- Both provide clustering approaches for high availability
- Both provide disaster recovery using database replication
- Decision
- In-house developed spatial benchmark testing the performance and precision of spatial data types and functions

## Spatial Benchmark (1 week simulation)

- Database

- Event Service Bus

- Data Handling System (DHS)

- Ingest

- Production

- DHS GUIs

- Input Events

- Output Events

- (NPOESS-Like) MODIS,
- OMI Data
- MADIS Data

- Product Distribution and Notification

- Distribution

- NDE

- Product/Customer/System Mgmt

- 260,000 xDRs / Day
- / Satellite

- 173,000 NUPs & TPs
- / Day / Satellite

- Ingest * 3
- _Production * 5_
- 433,000 / Day / Sat.

**Notes:**
Week in the life of NDE
MODIS, OMI, and MADIS data input. 
Translation of data to simulate characteristics of NPOESS input data
Scripts simulating DHS processing under C2 load.
Ingest: 600 (179 * 3 (satellites)) products every 30 seconds
Production: 120 products (40 NUPs and tailored products * 3) every 30 seconds
Distribute every 30 seconds
Ingest * 3 - (NCEP, AWIPS, NWS)
Production * 5
Cleanup (72 hour)

## Benchmark: File Level Spatial Qualification

- Limit algorithms execution by the spatial area of the input file. (e.g. Coastal and CONUS products)
- Distribute product to customer only when the file’s geographic area intersects the subscription’s geographic area of interest

## Benchmark: Gazetteer

- “Names” a geographic location
- Sources: USGS and National Geospatial-Intelligence Agency (NGIA)
- Purpose
- Serves as an alternate method to specify a geographic filter on a product or subscription
- Provide labels for mapped images (i.e. label ports on products for C&T customers)

## Benchmark: Event Gazetteer

- Names an Event
- Geographic and temporal location of an event
- _Event Variable __	__	__Value  or Range_
- Present Weather		‘Fog’
- Present Weather    	‘Volcanic Ash’
- 24Hr Accum. Precip. 	0.175m - .300m
- Sources (EPA, MADIS)
- Purpose
- Provides a means of producing or distributing a product in response to an event that has not yet happened, but might in the future.
- Provide additional optional data layers overlapping the geographic a temporal area of the product

## Benchmark: Pixel Level Spatial Qualification

- Store geo-location data in the RDBMS
- Limited Addt’l data (i.e. Cloud Mask, Land/Sea Mask, Sensor Zenith)
- Purpose
- Method of determining whether to produce or distribute based on Cloud Free in an entire geographic area
- Determining whether to execute blending algorithms
- Simplifies the development of new tailored products (i.e. subsetting) by providing filenames and pixel locations to tailoring tools

## NDE Data Model

- Product Development View
- Supports development of data to define products and “plug” algorithm into NDE
- Operations View
- Supports Ingest, Storage, Archive, Production, Distribution, System Metrics
- Card Catalog View
- Supports ingest, quality control, order specification, archive.
- Science Data View
- Supports product tailoring, algorithms, SOA enabler
- Alternatives: Relational View, J-METOC, CDM
- Selecting: Relational

## Design Challenges

| Challenge | Related Requirement | Impact | Mitigation |
| --- | --- | --- | --- |
| COOP requirements for NPOESS need to be determined | Part of ESPC overall COOP requirements | Needed to identify the details of the database  replication implementation | G. Goodrum resolving this issue |

## Agenda

-  External Interfaces	 	E. Wilson
-  DHS Architecture		E. Wilson** **
-  Common Services		P. MacHarrie
-  **Subsystem Design****	****A.Al-Jazrawi**

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
-    9:00 a.m.    Project Overview
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
**  ****1:00 p.m.     Data Handling System**
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

## Related Requirements

| ID | Section | Requirement | Contract |
| --- | --- | --- | --- |
| SRS215 | 3.10.2 | Use Proven Technologies - The contractor shall develop the data processing elements of the future NDE system using the latest proven technologies (programming languages, CASE tools, object repositories, data base management systems, etc.) that are appropriate for remote sensing data processing. | SE2, SE13 |
| SRS169 | 3.8.3.4 | Modularity - The NDE System Elements shall be designed so that Scientific Algorithms are invoked as objects. | SE5 |
| SRS168 | 3.8.3.3 | Reusability – The NDE System Elements shall be designed to be reused in other Satellite Data Processing applications. | SE2 |
| SRS217 | 3.10.2.1 | Use a 4GL - The Contractor shall develop the data processing elements of the future NDE system using tools that will support the ability to alter executable components without altering source code. | SE14 |

## Service Oriented Architecture (SOA) for NDE

- NDE drivers for considering a Service-Oriented Architecture (SOA)
- High availability system, need to be able to update components without bringing system down
- New products and algorithms need to be easily added to the system
- Each launch requires scaling up of the system capabilities
- Future NDE functionality will be easily added as services

## Service Oriented Architecture (SOA)

- ”Black-box” design - In an SOA environment, resources on a network are made available as independent services with defined interfaces
- In a SOA, processes and services are flexible and can be easily added and rearranged
- The SOA architecture style is geared to provide enterprise business solutions that can change on demand
- The SOA software architecture is based on the key concepts of an application front-end, service, service repository, and service bus.
- Business functionality is provided by services used by application front-ends and other services or even servers

## SOA Specification

- OASIS (Organization for the Advancement of Structured Information Standards) is a not-for-profit, international consortium that drives the development, convergence, and adoption of e-business standards.
- The OASIS Reference Model for Service Oriented Architecture V 1.0 was approved as an OASIS Standard in 2006.
- Specification available at: _[http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=soa-rm](http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=soa-rm)__ _

## Elements of SOA

- Elements of SOA, by Dirk Krafzig, Karl Banke, and Dirk Slama. Enterprise SOA. Prentice Hall, 2005

## Developing Services

- The following are architectural principles for the design of services
**Service Encapsulation** – information hiding and breaking a program into distinct features
**Service Loose coupling** - Services maintain a relationship that minimizes dependencies
**Service contract** - Services adhere to a communications agreement
**Service abstraction** - Services hide logic from the outside world
**Service reusability** - Logic is divided into services with the intention of promoting reuse
**Service statelessness** – Services minimize retaining information specific to an activity
**Scalability** – ability to handle growing amounts of work or be easily enlarged

## Enterprise Service Bus

- Custom/Existing
- Applications

- Service
- Orchestration

- Presentation
- and Portals

- Data
- Services

- Web
- Services

- Enterprise Service Bus (Routing and Transformation Services)

- HTTP

- Internet

- DB

**Notes:**
An enterprise service bus (ESB) provides services via an event-driven and standards-based messaging engine (the bus).
An ESB provides a registry that allows clients to locate and use  services at runtime
Load balancing services enhance scalability and high availability.
Can have multiple copies of a service on different systems
Can increase system capacity by adding hardware until the network is saturated
Failure of one system will cause others to take up the load
An ESB supports queuing, holding messages if applications are temporarily unavailable. 
An ESB also helps support:
Change management (hot deployment, versioning, lifecycle management)
Quality of service (transactions, failover)
Qualify of protection (message encryption, security)

## NDE Data Handling System (DHS) Design Status

- Major System Subsystems/Services have been identified
- Basic use case diagrams have been modeled and used to define much of the problem domain
- Data Flow Diagrams have been created
- Entity Relationship Diagrams (ERDs) developed

## Customer View of NDE

- Customer works with PAL
- to define desired products

## Customer Request View

- Data products are defined by a Product Area Lead (PAL)
- Defined tailored and NOAA unique product parameters
- Format
- Input data sources
- Users log in and subscribe to Data Products (ad hoc or standing)
- User Request View provides access to a list of products available
- Optional parameters include
- Spatial coverage area
- Data quality thresholds
- Compression options
- Delivery options – FTP push or pull, etc.
- Notification options – web-based GUI planned, other possible options

## NDE Subsystems

**System**
**Monitoring**
**& Control**

**Customer**
**Services**

**Production**

**Product**
**Management**

**Common**
**Services**

**Distribution**

**Ingest**

## Customer Services Subsystem

- User Registration Web Service
- User accesses registration form via web browser, enters identification information, and submits request
- The  user registration request is stored in NDE DB waiting for operator approval
- User Subscription Web Service
- User logs in and accesses subscription web page, selects data for subscription or ad-hoc request and submits request
- Request is automatically approved unless it exceeds configurable NDE order limits
- User can view approved subscription/ad-hoc requests
- User Status Reporting Web Service
- The user logs and can receive information on all orders requested by the user in the system for the past 72 hours
- Authorized users can receive system status information
- Customer help requests/ESPC interfaces/roles to be defined in working groups planned in January

## Customer Services Subsystem Activity Workflow Diagram

## Customer Services Subsystem Activity Workflow Diagram

## Customer Services Subsystem

- Registration Approval Web Service
- The operator logs into the operator account using a web browser
- The operator receives user registration information
- Based on input from NDE Manager the operator grants or denies the registration request
- Subscription Request Approval Web Service
- The operator logs into the operator account using a web browser
- The operator receives user order information
- Based on information from the Product Area Lead (PAL) the operator grants, edits, or denies the Subscription request
- Archive to CLASS Web Service
- Archive algorithms
- Archive data – based on guidance from the Archive Requirements Working Group (ARWG)

## Customer Services Subsystem Workflow Activity Diagram

## Customer Services Subsystem Workflow Activity Diagram

## Product Management Subsystem

- Product Definition Web Service
- Web-based GUI allows the different data types handled by NDE to be defined or modified
- XML ingest of parameters currently considered in design
- Alternatively, an installation script can be used to enter initial set of data products
- Algorithm Definition Web Service
- Web-based GUI allows processing parameters for algorithms to be entered into the NDE database
- Parameters include
- Production Rules to be used in selecting input data
- Expected outputs
- Expected size of inputs and outputs
- Expected processing time
- XML skeleton generation for Algorithm Definition parameters currently considered in design
- XML ingest of Algorithm Definition currently considered in design

## Product Management Subsystem

- Day N

- Day N
- 8h

- GranuleA

- GranuleA1

- GranuleB

- GranuleB3

- AlgorithmA

- GranuleB1

- AlgorithmA

- GranuleC

- GranuleC

- GranuleC

- GranuleB2

- GranuleC

- AlgorithmA

- AlgorithmA

- 10h

- 14h

- 12h

- Example Production Rules

## Ingest Subsystem

- Receive and Store Data Service ingests data from IDPS and other sources
- A data delivery message is received from IDPS or other files received on SAN will have controls written to detect file appearance and initiate the Receive Data Service for processing
- Verifies the data is on the NDE SAN
- Performs and enables validation functions – checksum, quality flag checks as input to problem notifications, file size check
- Requests retransmit if file fails checksum/size validations
- Inserts granule metadata into NDE database
- Inserts spatial metadata into NDE Database
- Receive IDPS Consolidated/Data Delivery Report
- Consolidated Data Delivery Report is sent to NDE SAN
- ESB configured to generate an event when report is received
- Report is compared with granules received by NDE
- Missing files are requested from IDPS

## Ingest Subsystem Activity Diagram

## Ingest Subsystem Activity Diagram

## Production Subsystem

- Production Scheduler Service - selects algorithms to execute when data is available
- Executes Production Rules
- Retrieve needed data from CLASS LTA
- Schedule ready algorithms by sending a message to the Production Server subsystem
- Processing Manager Service – manages NDE algorithms’ execution when their inputs are ready
- Runs Tailoring
- Reports algorithm execution statistics
- Monitors algorithm execution
- An algorithm writes data to NDE Production Processing Landing Zone area to be ingested into the NDE

## Production Subsystem Activity Diagram

## Distribution Subsystem

- Archive to CLASS Service sends data to the CLASS
- Initiated when a message is received from another Subsystem
- Sends data indicated in message to CLASS
- Receives status back from CLASS
- User Data Transmission Service forwards data products to users
- Calculates checksum
- Compress (gzip, lossy, lossless, etc) data if required
- Pushes data to user if required
- Stages data to pull location if required
- Implements Data Denial
- Data Cleanup Service removes old data from the NDE SAN
- Reads NDE DB for products that exceed the maximum cleanup interval or exceed maximum user defined cleanup interval
- Verifies data has been archived to CLASS
- Verifies data has been sent to all requesting users
- Deletes data from SAN
- Updates NDE Database to remove metadata/spatial metadata

## Distribution Subsystem Activity Diagram

## Distribution Subsystem Activity Diagram

## Distribution Subsystem Activity Diagram

- Included in Cleanup Rules
- Exceeds maximum cleanup interval for NDE
- Exceeds maximum user window
- NOAA Unique Granules are removed when
- Archived to CLASS
- Distributed to Users
- Non-NOAA Unique Granules are removed when
- Distributed to Users
- SAN Full Check will cause removal earlier

## NDE DHS Challenges

| Challenge | Impact | Mitigation |
| --- | --- | --- |
| Validating system actors and their roles |  User acceptance | Working groups with PALs |
| Defining web services for use by the PALs, Users, NDE Engineer, and other roles we identify | Ineffective and inefficient web service displays interfacing with DHS | Prototyping |
| Defining interfaces | Performance | ICD Working Groups |
| Transition of legacy algorithms | Cost, schedule, performance | Standards |
| Defining user notification methods | User needs unmet | Working with PALs Customer feedback |

## Agenda

** ****IT Security****	****	****B. Callicott**

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
-    9:00 a.m.    Project Overview
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
**  ****2:30 p.m.     IT Security**
-   3:00 p.m.     Communications Study
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

## NDE Security Requirements

| ID | Section | Requirement | Contract |
| --- | --- | --- | --- |
| SRS160 | 3.7.1 | The Contractor shall follow ESPC (DoC/NOAA/NESDIS) procedures and policies for securing ESPC systems. | SA5 |

## Security Considerations

- Security is incorporated and addressed from the initial planning and design phase
- Multiple, overlapping protection approaches are used ensuring the failure or circumvention of any individual protection approach will not leave the system unprotected
- NIST 800-27A “Engineering Principles for Information Technology Security (A Baseline for Achieving Security), Revision A” has been followed
- Establishment of a Plan of Actions and Milestones (POA&M) that lays out the resources and milestones to achieve full compliance with NIST 800-53 controls
- Patch Management procedure in development and will be implemented as one of the POA&Ms
- ESPC CCB process has been modified to require ISSO approval of all configuration changes to ensure the system remains secure and documentation is up to date

## Stage 1 Network Diagram

## Stage 2 Network Diagram

## VPN Zone

- Associates roles with individuals, allowing individuals to only access certain systems in certain zones
- One subnet of public IPs for VPN on Outside Subnet
- Juniper Network Admission Control Appliance (NAC) for verifying compliance prior to network access, specified for future deployment
- Compliant with US Department of Commerce IT Security Program Policy and Minimum Implementation Standards Revised June 30, 2005
- Compliant with U.S. Department of Commerce Unclassified System Remote Access Security Policy and Minimum Implementation Standards

## Stage 2 – Development Zone

**ESPC security controls and configuration management processes will apply, one C&A for the ESPC/NDE system**
- Removes developers from the National Critical production environment as a risk reduction approach
- Improves collaboration with non-OSDPD developers
- Systems dedicated to development – will not be used operationally
- Data acquisition and distribution through drop boxes in Public Zone
- Development programmer access through VPN Server for Development Only

## NDE Security Challenges

| Challenge | Related Requirement | Impact | Mitigation |
| --- | --- | --- | --- |
| IDPS requires use of StorNext to write to ESPC SAN vs ESPC security zones | Imposed on NDE by IDPS and ESPC | Multiple NDE hosts also need to “see” the same file system on the ESPC SAN that IDPS uses; therefore NDE is also required to use StorNext as its metadata server | ESPC (Brian Callicott) is pursuing technical solution to allow StorNext FS to run securely in the stage 3 system for file sharing between zones |
| Moving from stage 2 to stage 3 requires implementing some legacy design into the NDE architecture | NDE architecture will serve as technical refresh for non-legacy ESPC systems | Legacy design ESPC systems will require modification to move into stage 3 | POA&Ms will be created and resources assigned to address these legacy subsystems |
| Brocade and Cisco Fiber-Channel Switch incompatibilities | ESPC must be able to properly interface with external organizations over fiber-channel for data delivery | Unless compatibility issues are resolved, ESPC may suffer reduced throughput | Initially, Cisco will attempt to resolve incompatibilities; firmware upgrades must be well-coordinated; long term a single FC-switch vendor should be selected |

## Agenda

-  **Infrastructure Design**	B. Hickman
-  NOAANet Efforts		B. Hickman
-  Distributed Volumes	B. Hickman

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
-    9:00 a.m.    Project Overview
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     IT Security
**  ****3:00 p.m.    Communications Study **
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

## NDE Telecommunications Task Introduction

- NDE telecommunications infrastructure design
- Updated status of NOAA migration toward a One-NOAA NOAANet
- NDE Distribution volumes to users 2007-2020 and Section Summary

| ID | Section | Short Title | Contract |
| --- | --- | --- | --- |
| SRS216 | 3.2.3.4.1
 | Trade Study Alternatives | CD5 |

## Risk Reduction Telecommunications Options

- Flexible Extensible Switch Based NDE Design Options scale multiple selectable layers of redundancy for COOP
- Each of the 4 Options offer progressively increasing:
- component redundancy (power supply, board, chassis, switch, and WAN) to seamlessly meet operational failover contingencies
- scaled firewall protection and security context organization, aggregation, and management
- Maintains optimum compatibility with ESPC Operations:
- ESPC Consolidation of IT Architecture
- IPV6  Migrations

## Slide 182

**Notes:**
Bypassing Option 1 (least redundant):
No chassis redundancy
Board redundancy (Supervisor, FWSM, Power Supply) within each chassis
No redundant WAN links
Only one level of firewall protection shown, however two security contexts could be implemented
No distinct layers of organization (untrusted, aggregation or trusted)
This option maximizes cost savings

## Slide 183

**Notes:**
Option 2 (NPP)
Distinct layers of organization (untrusted, aggregation and trusted) to support current and future shared services (firewall, load balancing, VPN, etc.)
Additional switch chassis redundancy required in aggregation layer
No switch chassis redundancy
Board redundancy (Supervisor, FWSM, Power Supply) within each chassis
No redundant WAN links
Agg1-FWSM operates in Layer 2 mode with multiple contexts

## Slide 184

**Notes:**
Option 3 (NPOESS-C1)
Chassis redundancy in aggregation and trusted tiers
Board redundancy in untrusted tier
No redundant WAN links
Distinct layers of organization (untrusted, aggregation and trusted) 
Agg1 and Agg2 FWSM operate in Layer 2 mode with multiple contexts

## Slide 185

**Notes:**
Option 4 (NPOESS-C2)
Redundant links and chassis added in untrusted tier

## Agenda

-  Infrastructure Design	B. Hickman
** ****NOAANet Efforts****	****	****B. Hickman**
-  Distributed User Volumes	B. Hickman

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
-    9:00 a.m.    Project Overview
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     IT Security
**  ****3:00 p.m.    Communications Study **
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

## Data Distribution Design Challenges

- Stakeholder Interviews, Model creation, & PAL product profiling
- Current ESPC not able to handle NDE order of magnitude increased loads
- DoC and NOAA security requirements limit design options, particularly the SAN
- Consolidated approach needed to develop infrastructure to handle NDE distribution to users

**Notes:**
Stakeholder interviews conducted to provide traffic loading estimates from NOAA NDE Central to distributed user sites
NDE Data and product volume modeling construction incorporating:
Start point as IDPS modeling I/O to NDE SAN
PDSI planning and NUP development profiling
Stakeholder responses & comparisons with legacy sizing where possible
NAC outreach and round table communication of next generation traffic estimates and delivery needs

## Data Distribution - WAN

- NOAA has 12 legacy hub-and-spoke networks
- Accelerate migration to a unified NOAANet to achieve increased capacity, MPLS any-to-any topology, cost-efficiency
- Washington DC MAN recently upgraded by NWS Qwest solution
- DWDM high speed ring with access to distributed sites via MPLS Cloud
- Leverage Internet2 possibilities
- NWS, NEXRAD, & CLASS provide precedents for NDE design consideration

## Overview of August 2006 Peer Review Slides

- Slides depicting Internet2 opportunities
- Lambda Rail MBone could support Multicast and other Real time protocol studies for NDE

**Notes:**
Note:  Abilene via College Park MAX  - - - and - - - NCAR Front Range GigaPop on Lambda Rail MBone (encrypted IP tunneling for Multicast connects

## Legacy Washington DC MAN

## NWS 2006 Upgrade to Washington DC MAN

- NSOF

**Notes:**
10TB/Day ring in 2006 with 1 TB/Day legs out to cloud and 100 GB to NWS NWP Centers

## NWS 2009 Projected Growth 

- NSOF

**Notes:**
By 2009 (NPP Launch) we see 4 times that in growth
40TB/Day ring in 2009 with 4 TB/Day legs out to cloud and 400 GB to NWS NWP Centers
Note that AFWA replaces the Gaithersburg CCS

## Agenda

-  Infrastructure Design	B. Hickman
-  NOAANet Efforts		B. Hickman
** ****Distributed Volumes****	****B. Hickman**

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
-    9:00 a.m.    Project Overview
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     IT Security
**  ****3:00 p.m.    Communications Study **
-   3:40 p.m.     Closing Remarks
-   4:00 p.m.     PDR Board Review of RFAs

## August 2006 Peer Review Update

- Large number of slides showing Legacy and Nextgen satellite comparison charts
- Projected volumes did not include:
- Latency
- Overhead
- Surge
- Firewall delay
- Encryption delay
- Compression delay
- Distributed Delivery slides to follow do reflect (all but firewall delay) scaling for throughput
- Nunn-McCurdy changes revising estimate basis and model refinements

## OverArching NDE Models Basis 

- xDR (RDR, SDR, EDR, & IP) sizing model establishes IDPS delivery to NOAA NDE SAN
- xDR model outputs drive NOAA Product model inputs
- NOAA Product model tracks legacy/next generation traffic growth and required distribution to identified End-users
- Orders of magnitude larger than legacy systems
- Large number of significant distributed users

**Notes:**
International Users provide their own links to NOAA NDE PoP

## Predicted Domestic Traffic

**Notes:**
InterNational Users listed in full Tables (Backup Slides)

## Trade Outcomes

- Commercial satellites are band-limited
- MPLS savings realized by distributed, low bandwidth networks (T-1 and below OC3) in range of 25%-40%
- SONET still found to be the most cost effective for distributed networks of OC3 rates and above
- Multicast or alternative ‘One-Many’ technologies must be established at customer edges and in the commercial clouds to reduce NDE redundant loads
- Dense Wave Division Multiplexing (DWDM) can run very large (~40 Gbps) traffic volumes simultaneously over the same (1Gbps) fiber land lines

**Notes:**
References: 
Network World 10/02/06 – Making the most of MPLS (Eye on the Carrier) Johna Till Johnson

## Bandwidth Reduction Recommendations 

- IP multicast
- Lossless compression
- Lossy compression
- IBM General Parallel File System (GPFS)
- Real-time protocols
- Threaded & other highly parallel processing methods

- Reduced resolution requests
- Reduced aerial coverage requests
- Reduced frequency of requests
- Updating (transmitting) only the ‘new data’ for composites
- Increasing product requests over data requests

- _Technologies_

- _Strategies_

## Summary

- NDE Telecommunications Design for NSOF flexible and extensible to scale appropriately and meet all throughput requirements
- Current NAC feedback:
- NDE & NWS can add capacity to new Qwest DWDM ring
- NDE to apply for HPCC prototyping of Internet2 (Abilene & Lambda Rail)
- Real time protocols, one-to-many (Multicast) solutions, and enabling technologies prior to C1 launch
- Work with NCEP (model assimilation), vendors (e.g., CISCO, IBM, Oracle, etc.), & COTS providers (e.g., ESRI, ITTVIS, SSEC, Mathworks, etc.)
- Specific NAC solution recommendations to NDE regarding connectivity to distributed National & International end-users expected by March 2, 2007 (NLT June 2007)

**Notes:**
Front Range GigaPop / University Corporation for Atmospheric Research  Internet2

## NDE Telecommunications Design Challenge Mitigation

| Challenge | Related Requirement | Impact | Mitigation |
| --- | --- | --- | --- |
| To define NOAA Enterprise solution  | 3.2.3.4.1 - Trade Study Alternatives
 | Higher cost, lower performance, schedule risk | Outreach - OSD, ITAT, & NAC 
NAC to recommend solution by 3/2/07 |
| High NDE Data/Product Volumes & Large Distributed customer base | 3.2.3.4.1 - Trade Study Alternatives
 | Unable to meet customer product requirements | HPCC prototype study 
IP multicast
Real-time protocols Threaded & other highly parallel processing methods |
| Location of CIP | 3.2.3.4.1 - Trade Study Alternatives
 | Single point of failure  | AFWA, CDA-W alternatives under consideration – TBD before C1 Launch |
| IPV6 Migration | 3.2.3.4.1 - Trade Study Alternatives
 | Inadequate performance | HPCC protocol & one-to-many prototype studies |

- NDE Telecommunications Design Challenge Mitigation

## Agenda

** ****What’s Next****	****	****G. Sloan**
** **Technical Challenges	G. Goodrum
-  Closing Remarks		J. Silva

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
-    9:00 a.m.    Project Overview
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
**  ****3:40 p.m.     Closing Remarks**
-   4:00 p.m.     PDR Board Review of RFAs

## The Road to CDR

- We have a resource-loaded schedule and an EAC
- Maintain current staffing through CDR
- NPP Build Cycle: Development Environment (PDR)
- IT Development Environment
- Science Algorithm Development & Integration Environment
- Configuration Management Control Processes
- Tools Analysis and Selection
- Product Technical Baseline
- NPP Build Cycle: System Test Environment (post PDR)
- Document updates and baselining
- CM Plan
- Concept of Operations
- QA Plan
- Project Plan
- System Requirements Specification
- System/Subsystem Design Specification for NPP
- FEA Mapping
- Prototyping
- Subscription
- Ingest
- Production

## Agenda

-  What’s Next		G. Sloan
** ****Technical Challenges****	****G. Goodrum**
-  Closing Remarks		J. Silva

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
-    9:00 a.m.    Project Overview
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
**  ****3:40 p.m.     Closing Remarks**
-   4:00 p.m.     PDR Board Review of RFAs

## Technical Management

|  | Challenges | Strategies |
| --- | --- | --- |
| Networks | Volume, widely distributed customers, low latency, diversity and inadequacy of current networks  | Develop ‘enterprise’ solutions in collaboration with AWIPS and NAC  |
| Testing  | - Different communities (IT, Users, Science Developers)  
- Design  (algorithms as objects supported by IT elements) | - ‘Requirements Based Testing’ for IT 
- PALs to work with science users to develop acceptance criteria and test specs  |
| System Design | Tendency to prototype  and implement too early in life cycle | Quality Management: Monthly government review of  design work products |
| Security Policy | Changing policy over the project life cycle |   Work with ESPC ISSO, IT Security Team, and ESPC management |
| External Interface  | - SAN switch interoperability
- Customer Readiness |  - Work with NPOESS External Interface Working Group
- Establish “Systems Integration Manager” to coordinate transition teams by product line  |

## Agenda

-  What’s Next		G. Sloan
-  Technical Challenges	G. Goodrum
** ****Closing Remarks****	****	****J. Silva**

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
-    9:00 a.m.    Project Overview
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
**  ****3:40 p.m.     Closing Remarks**
-   4:00 p.m.     PDR Board Review of RFAs

## Program Management

|  | Challenges | Strategies |
| --- | --- | --- |
| Transition to Operations |  Continuity of current operations
 Exploitation of new products | Establish “Systems Integration Manager” to coordinate transition teams by product line |
| Stakeholder
Comm. | NESDIS, Line Offices, Other Centrals, Standards Bodies, Goal Teams, Budget Examiners | Monthly Team Meetings with NESDIS, NPOESS & Centrals Working Groups, Product Workshops for Goal Teams & Line Offices, actively seek CIO & ITAT guidance, dedicate resources to budget exercises  |
| Responsibility Transfer | - IT developers to IT operators and maintainers
- Algorithm developers to algorithm maintainers | Develop new estimating and scheduling procedures with STAR & OSDPD  |
| Budget  | - FY08 deficit,  FY09 threatened
- Flat-lined thereafter | - Prioritize FY08/FY09 allocations & redirect resources accordingly
- Continue to seek plus-ups for product development |
| Schedule | NPP/NPOESS program milestones are NDE constraints | Plan for readiness ~ 6 months prior to NPP/NPOESS milestones |
| Resources |  HR: Contractors dedicated to NDE 
 Infrastructure: Potential underutilization (e.g. NPP/NPOESS delays) | - HR fold contractors into operational support
- Infrastructure (scalable, extensible architecture) |

## Review Calendar

| 2006 Date | Deliverable | Tasked |
| --- | --- | --- |
| Nov 21 | Requests for Action (RFA) | Review Board |
| Dec 6 | Action Responses to Board Chairperson | NDE 
Design Team |
| Dec 15 | Final Report | Design Board Chairperson  |

## Happy Thanksgiving!

**Notes:**
Jim Silva wished everyone a good holiday and requested attendance to the next meeting.

## Agenda

** ****RFA Review****	****	****V. Griffin**

-    8:00 a.m.    Welcome
-    8:15 a.m.    Introduction/Background
-    9:00 a.m.    Project Overview
-  10:45 a.m.    NDE Context and Hardware
-  11:15 a.m.    Algorithm Development
-  12:00 p.m.    Lunch
-   1:00 p.m.     Data Handling System
-   2:30 p.m.     IT Security
-   3:00 p.m.     Communications Study
-   3:40 p.m.     Closing Remarks
**  ****4:00 p.m.     PDR Board Review of RFAs**