# Presentation: 334749

## Johann Schumann and Pramod Gupta 
NASA Ames Research Center
schumann@email.arc.nasa.gov
pgupta@email.arc.nasa.gov

- NASA Ames Research Center
- _[schumann@email.arc.nasa.gov](mailto:schumann@email.arc.nasa.gov)_
- _[pgupta@email.arc.nasa.gov](mailto:pgupta@email.arc.nasa.gov)_

**Bayesian Verification & **
**Validation tools for **
**adaptive systems**

## Motivation for NN V&V 

**Basis for Case Study I:**
- Neuro-adaptive control (IFCS Gen-II)
- Network “learns” to compensate for deviations between plant and model
**Previous work:**
- SW V&V process for NN-based control
- “Confidence tool” for dynamic monitoring

**The major obstacle to the deployment of adaptive and autonomous**
**systems is being able to verify their correct operation – *****In Flight***

- Fixed gain controllers cannot deal with *catastrophic changes* or *degradation* in plant
*Adaptive systems (e.g., NN)* can react to unexpected situations through learning
- Relevance and potential:
- IFCS NN controlled aircraft (F-15, C-17)
- UAV control
- Space exploration
- Any safety-critical application of NN control

## V&V Issues & our Approach

**V&V Issues & our Approach**

- Our approach combines mathematical analysis, intelligent validation, and dynamic monitoring and supports specific software V&V process,
- targets multiple aspects and phases of V&V of adaptive control systems, and
- uses a unique combination of research in
- Neural Networks
- Control Theory
- Numerical Methods
- Bayesian Statistics

- Verification: how to specify an unforseen event?
- Validation: not possible to test all configurations

**While traditional V&V methods will remain useful, these methods alone are insufficient to verify and certify adaptive control systems for use in safety-critical applications**

**Our Bayesian Approach**** **

*How good is the network performing at the moment?*

-  Traditional: NN as a Black Box
-  Here: Look at probability distribution of the NN output
-  Variance (confidence measure) depends on:
- How well is the network trained?
- How close are we to “well-known” areas

- Large variance = bad estimate; no reliable result, just a guess

- Small variance = good estimate

**Our approach, based on a Bayesian approach, provides a **
**measure of how well the  neural network is performing at the moment**

## Milestone I:  Envelope Tool

- Basis: Adaptive NN-based controller
-  Lyapunov error bound defines regions of *eventual* stability
-  Regions where confidence is small might cause instability
-  Informally: a *safe envelope* is a region where the confidence level is sufficiently high
-  Bayesian approach combined with sensitivity analysis
-  Challenge: methods for *efficient* determination of safe envelope

**Can help answer questions like**
**How large is the current safe envelope?**
**How far is the operational point from the edge?**

- Current status: mathematical background formulated, prototypical Matlab/Simulink implementation designed, first simulation experiments

## Confidence Envelope

- Confidence Surface

- Safety Envelope:
- area of good
- confidence

- airspeed

**The Envelope tool uses a Bayesian Approach to calculate the current safety envelope**

- 1/confidence

***good***

***bad***

- altitude

## Conclusions & next steps

-  Current work as scheduled toward deliverable (9/2004)
-  prototypical implementation in Matlab/Simulink
-  report on mathematical background and tool
-  Getting Case Study I ready: IFCS Gen-II simulink model
-  Next steps in research:
-  system identification (sysID): estimate confidence of parameters
-  other model representations (e.g., parameter tables with polynomial interpretation)
-  Preparation of Case Study II and III