# Presentation: 532332

## M&C 2003Developments in Software Engineering for High Consequence Software

**Digital Lessons Learned**

**By Edward (Ted) L. Quinn  - tedquinn@cox.net**
**Past President, ANS**
**Triconex Corporation Industry Advisor**

## DIGITAL MISHAPS

- Therac – 25 – Software – controlled radiation treatment for cancer
- Ariane V – European space agency rocket with recyled software
- Airbus highly automated flight control
- Salem Annunciators
- Calvert Cliffs Feedwater Regulating Valve
- IT CAN BE DONE RIGHT OR WRONG

## SOFTWARE FAILURESTHERAC 25 FAILURE

- Computerized radiation therapy machine – Therac-25
- June, 1985 – Jan. 1987 – 6 known accidental overdoses – resultant deaths and serious injury – worst in the history of radiation therapy
- Used earlier version Therac-20 software – without adequate testing and hardware safeguards

## SOFTWARE FAILURESARIANE V FAILURE

- 4 June, 1996 – French site, Maiden flight of Ariane V
- Launch + 37 seconds – veered off flight path – destroyed
- Cause – complete loss of guidance and attitude
- Two redundant computers – inertial ref. System failed – same software as Ariane IV – with different mission.

- The Application of Digital Technologies in Nuclear Power Poses Unique ‘Issues’

- Unintended Functions/New Failure Modes
- Silent Failures
- Configuration Management
- HMI
- Training
- Software Quality/V&V
- Unanticipated Conditions/Events
- ‘Black Box Syndrome’

## SOFTWARE FAILURESNUCLEAR PLANT EXPERIENCE

- TYPES OF FAILURES – CATEGORIES
- - Software V&V
- - Configuration Control
- - Electromagnetic Interference
- - Human-Machine Interface
- - Random Component Failure

## LERs from 1990-1993 ShowDigital I&C System Failures

- LERs from 1990-1993 ShowDigital I&C System Failures

## LERs from 1990-1993 ShowDigital I&C System Failures (cont)

- LERs from 1990-1993 ShowDigital I&C System Failures (cont)

## Digital Radiation Monitoring System Benefits

- Multiplex Many Channels
- Common HMI Display
- Maintain Historical Data
- Alert Users to System FailuresThrough Watchdog Timers

## Watchdog Timer

## Digital Radiation MonitoringSilent Failure

- Watchdog Timer Reset At Wrong Location
- Surveillance Procedure Did NotDetect Failures
- Original Software Design Was Changed To Avoid Hardware Design Change
- Engineering Change Analysis Failed To Consider Potential For Infinite Loops

## The Moral of the Stories:

- Even a Watchdog Can Bite Its Owner
- Just Because They Installed It Before You Doesn’t Mean They Looked First
- Just Because Their System Hasn’t Crashed Doesn’t Mean Your System Won’t
- If You Look Before You Leap, You Can Save Yourself a World Of Hurt

## Invensys Nuclear Experience – Example NRC Qualification

- Supplier to the Nuclear Industry for over 40 years
- Fully certified Nuclear QA Program
- Provided over 125 Safety & Control Systems to over 25 NPP’s around the world
- Introduced digital control technology to the Nuclear Industry in 1987
- Recent Experience Includes:
- ABWR Distributed Control & Information System for Lungmen
- Feedwater Systems for Palo Verde NPP
- DAS Systems for FP&L - St. Lucie  1St Phase of a Multi-Phase/Year Project
- AULD System at Crystal River NPP - EPRI 2001 Technology Transfer Award
- INVAP / ANSTO Research Reactor - Control & 1E Reactor Protection System
- Kozloduy NPP - Bulgaria  1E COPS (LTOP) and Feed Water Control
- Duke Oconee Steam Reheat Turbine Retrofits ( 3 Units )
- Diablo Canyon Steam Reheat Turbine Retrofits ( 2 Units)

## Tricon TMR Controller Architecture 

**Tricon TMR Controller Architecture **

**Comparison Diagnostics (2oo3) and Reference Diagnostics**
**High Reliability and High Availability = 200 Million Runtime Hours**

**Zero Calibration and Diagnostics = Minimized Surveillance**

**COTS - TMR Architecture 1E and Important to Safety**

## Nuclear Qualification 

**SER received from NRC Dec. 2001**
- Qualification based on **EPRI TR-107330**
- Triconex compliant to 10CFR50 Appendix B & 10CFR21
- Major Milestones since qualification
** ****Crystal River AULD System - EPRI 2001 Technology Transfer Award**
** ****INVAP / ANSTO Research Reactor - Control & 1E Shut Down System**
** ****Kozloduy NPP - Bulgaria  1E COPS (LTOP) and Feed Water Control**
** ****Duke Oconee Steam Reheat Turbine Retrofits ( 3 Units )**
** ****Diablo Canyon Steam Reheat Turbine Retrofits ( 2 Units)**
**Required meeting all of IEEE-1012 Verification and Validation including all phases:**
**Detailed Design**
**Testing**
**Use of a confirmatory matrix**
**Legacy software validation**

## Why Invensys

- Only truly generic COT’s 1E qualified platform
- Easily applied and licensable in a wide variety of applications
- Only TMR approved 1E qualified platform
- Allows plant owners to reap the benefits of TMR architecture
- Minimize or eliminate Nuisance Trips and performance loss
- Applicable for 1E & Non-1E important to safety applications
- Minimize or eliminate LCO conditions for the plant
- Minimize O&M costs (Spares, training, surveillance, calibration)
- 10CFR50 Appendix B & 10CFR21 compliant organization
- True migration path for safety related products
- Allows for upgrades and performance enhancements which minimize obsolescence issues.

## Slide 17