# Presentation: 801463

## Methods and Procedures for the Independent Verification and Validation of Neural Networks19 July 2004

- NAG5-12069-OSMA-F001-UNCLASS-071904

## Project Description

- The IVVNN Project began in May 2002 in an effort to develop a standard for verifying and validating NN and adaptive systems for the NASA IV&V Facility
- As NN and adaptive systems are becoming more common in NASA and commercial applications, the need for such a standard became paramount
- The project is scheduled to continue until September 2005
- The IVVNN project is a collaborative effort among the ISR, the NASA IV&V Facility, and WVU
- The majority of the research under this initiative was conducted by the ISR with support from WVU
- Previous research efforts conducted by the NASA DFRC and the NASA ARC provided both insight and direction

- NAG5-12069-OSMA-F001-UNCLASS-071904

## Project Overview

- Goals:
- To develop a methodology for the IVVNN
- To test the IVVNN methodology on the NASA DFRC IFCS GEN1 system (and possibly the IFCS GEN2 system)
- To train IV&V personnel on the use of IVVNN methodology
- To integrate the IVVNN methodology as guidance augmentation to the IEEE 1012
- The IVVNN project is separated into 4 major tasks:
- Task 1: NN (I)V&V Literature Search
- Task 2: Methodology Research
- Task 3: Methodology Development
- Task 4: Methodology Training

- NAG5-12069-OSMA-F001-UNCLASS-071904

## Task 1: NN IV&V Literature Search

- Goal was to identify the current state-of-the-art for the V&V and IV&V of NN research
- Conducted an extensive literature search in which over 300 artifacts were collected
- Artifacts included conference papers, journals, technical reports, tools, applications, and presentations that were available in the public domain
- Artifacts contained methods and techniques that were claimed to be useful for the V&V, and potentially IV&V, of NN
- All collected artifacts were evaluated to determine which ones were more promising and applicable to the IV&V of NN.
- The results of the literature search and evaluation of collected artifacts were collected in a document titled *Toward Reliable Neural Network Software for the Development of Methodologies for the Independent Verification and Validation of Neural Networks*
- The conclusion was made that six methods were found to be most promising for the V&V of NN and consisted of:
- traditional and automated testing techniques
- run-time monitoring
- Lyapunov stability analysis
- rule extraction
- cross validation
- Visualization
- The ISR also concluded that there were very few tools that were directly applicable to the V&V of NN

- NAG5-12069-OSMA-F001-UNCLASS-071904

## Task 2: Methodology Research

- Goal was to:
- examine and evaluate a subset of the methods and techniques identified in Task 1
- conduct research in other areas that the ISR hypothesized would be beneficial to the (I)V&V of NN
- Task 2 was divided into twelve research elements:
- Human Factors Analysis Based on Pilot Certification
- Formal Methods
- Run-time Monitoring Methods (including Data Sniffing and Lyapanov Stability Analysis)
- Testing Methods
- Visualization Methods
- Automated Neural Network Selection
- Neural Network Design Verification
- Study of IV&V Current Practices
- Failure Modes and Effects Analysis / Risk Assessment
- Simulation Methods
- Safety Monitors
- The results of Task 2 were collected in a book entitled *Methods and Procedures for the Independent Verification and Validation of Neural Networks*

- NAG5-12069-OSMA-F001-UNCLASS-071904

## Task 3: Methodology Development

- The goal is to:
- Utilize work from ‘Standards Mapping’ in Task 2 which identifies neural network / adaptive systems gaps within IEEE 1012 directives
- Combine the experiences from the IFCS project with the techniques researched in Task 2
- Add additional directives as a complement to IEEE 1012
- Provide guidance in completing the directives, using the techniques, and highlight important areas of consideration

- NAG5-12069-OSMA-F001-UNCLASS-071904

## Task 3: Methodology Development

- Methodology Design Approach:
- Structured into 3 phases
- Methodology to cover all development lifecycles
- Concept to operations
- Methodology testing with IFCS GEN1 and possibly GEN2

- NAG5-12069-OSMA-F001-UNCLASS-071904

## Task 3: Methodology Development

- In addition to Task 2 and IFCS Gen 1 experience, several documents are being considered as we develop this methodology:
- DCP-S-007: Dryden Flight Research Center Software Assurance Document
- Verification & Validation of Adaptive Neural Networks for Aerospace Systems (Stacy Nelson et. al) [multiple versions]
- V&V of Advanced Systems at NASA (Stacy Nelson & Charles Pecheur)
- IEEE 1059: IEEE Guide for Software Verification and Validation Plans
- IEEE 828: IEEE Standard for Software Configuration Management Plans
- IEEE 730: IEEE Standard for Software Quality Assurance Plans
- IFCS Software Verification and Validation Plan
*Developing Software to Government Standards* (William H. Roetzheim)
- NISTIR 5589: A Study of Hazard Analysis in High Integrity Software Standard and Guidance (Laura M. Ippolito and Delores R. Wallace)
- Safety Lifecycle for Developing Safety Critical Artificial Neural Networks (Zeshan Kurd and Tim Kelly)
- Safety Criteria and Safety Lifecycle for Artificial Neural Networks (Zeshan Kurd, Tim Kelly and Jim Austin)

- NAG5-12069-OSMA-F001-UNCLASS-071904

## Task 3: Methodology Development

- Concept Phase
- Analysis of the concept tasks completed
- Rough draft of concept IVVNN guidance created
- To date, have written guidance for the following concept phase tasks for NN and adaptive systems
- Concept documentation evaluation
- Criticality analysis
- Hardware/software/user requirements allocation analysis
- Traceability analysis
- Hazard analysis
- Risk analysis
- Preliminary testing involved applying the concept phase guidance to the IFCS GEN1 documentation

- NAG5-12069-OSMA-F001-UNCLASS-071904

## Task 3: Methodology Development

- Concept Documents Considered:
- IFCS Objectives and Requirements Document
- IFCS Project Plan
- IFCS Systems Requirement Document
- Pre-Trained Neural Network Report
- Dynamic Cell Structure Neural Network Report
- Sigma-Pi Neural Network Report (GEN2)
- Single Hidden Layer Neural Network Report (GEN2)
- Neural Network Evaluator Design Document (GEN2)
- Software Verification and Validation Report - Traceability Analysis, Criticality Analysis
- IFCS System Safety Plan
- IFCS Preliminary Hazard Analysis

- NAG5-12069-OSMA-F001-UNCLASS-071904

## Task 3: Methodology Development

- Requirements Phase
- Currently working on analysis of requirements tasks
- Begun writing guidance for the requirements task of traceability analysis
- Requirements Documents Considered:
- IFCS Systems Requirement Document
- IFCS Software and Interface Requirements Specification (ISR)
- IFCS System Test Plan and Description Document (ISR)
- IFCS Preliminary Hazard Analysis

- NAG5-12069-OSMA-F001-UNCLASS-071904

## Task 3: Methodology Development

**Task 3: Methodology Development**

- Issue:
- Level of documentation detail will vary from project to project
- Lack of documentation
- Detailed requirements early
- Informal documentation by researchers
- Solution:
- Develop the guidance to be adjustable to the IV&V practitioner

- NAG5-12069-OSMA-F001-UNCLASS-071904

## Next Steps

- Remaining Phase I Methodology Design Tasks
- Requirements
- Design
- Implementation
- Testing
- Operations
- Management
- Acquisition
- Methodology Design Review (MDR)
- Phase II
- Phase III
- IFCS GEN2 Testing
- NASA Dryden Flight Research Center
- NASA IV&V Facility

- NAG5-12069-OSMA-F001-UNCLASS-071904

## Questions

- NAG5-12069-OSMA-F001-UNCLASS-071904