Verification and Validation of Real-time Adaptive Neural Networks using ANCT Tools and Methodologies, an application to Intelligent Flight Control Systems F-15 Project

The Automated Neural Flight Controller Test (ANCT) tool i s an application that runs in the MATLAB environment, d esigned to h elp engineers perform computational t est experiments on a simulation model of a complex adaptive system for the purpose of verification and validation of system performance. Current research in Intelligent Flight Control Systems is yielding very promising results and this leads us to consider how we can apply this level of intelligence elsewhere in other vehicles. This paper describes a concept of applied intelligence in aerospace vehicles that reaches for new levels of autonomy, robustness, safety, mission success, survivability, and efficiency (1, 2, 3, 4, 14, &15) We have developed testing methods to evaluate the performance of online adaptive neural networks that are essential for the successful deployment of neural network-based adaptive controllers in safety-critical applications. Safety-critical applications require that t he system i s working correctly and safely for a variety of operating conditions. The focus of the Strategic Methodologies for Autonomous & Robust Technology Testing (SMART-T) project at NASA Dryden Flight Research Center is to develop an approach to ensure the safe application of complex system designs on aerospace vehicles. Current effort i s focused on the verification and validation of neural networks in flight control systems. The SMART-T project has developed several t ools and techniques to evaluate the performance a nd stability of online adaptive neural networks. Current tools include: The Confidence Tool that checks all neural network input and output values then determines if the outputs are reliable. The tool calculates a c onfidence interval around the outputs of the neural network based on a Bayesian statistical model. The Envelope Tool similarly uses a Bayesian statistical approach to calculate the envelope around the c urrent point of operation where the performance of the neural network would remain satisfactory. The Sensitivity Tool performs a sensitivity analysis of the adaptive neural network during training. It is helpful to consider how small changes in the weights affect the error function, and thus how fast and how well the network can be trained. The Automated Neural Controller Test (ANCT) Tool was designed to help test engineers validate neural flight controllers in various flight conditions, quantify performance, and determine regions of stability. Input parameters, minimum and maximum values, step increments, and success/failure requirements can be specified. The test tool also has a database to store input and output parameters and test case results. It also allows the test engineer to sort test cases and pick particular test cases of interest to be reevaluated. The Automated Neural Flight Controller Test (ANCT) tool was designed to h elp engineers analyze complex control systems and assist with the validation and v erification task by helping the e ngineer conduct, manage and analyze the outputs of test experiments in a simulation environment.