ox7

NASA-CR-178058

19860015674

NASA Contractor Report 178058

AN EMPmICAL STUDYOF

FLIGHT CONTROL SOFTWARERELIABILITY

Janet R. Dunham

John L. Pierce

Software Research and Development

Center for Digital Systems Research

Research Triangle Institute _OP_ _P,EFEEENCE

Research Triangle Park, North Carolina 27709

rIOT TO BE IAKEtl FIIOI&TILLSP.O01-_

Contract NAS1-16489

Task Assignment No. 22

March 1986 _[B_/[[_ _]P_

APR1 6 1986

_A_A UNC_EREYSEARCH CEmZR

LIBRARYNA, SA

National Aeronautics and HA.,L./ONVI , I_II'_tIA

Space Administration

Langley Research Center

Hampton, VirginJa23665
NASA Contractor Report 178058

AN EMPIRICAL STUDY OF

FLIGHT CONTROL SOFTWARE RELIABILITY

Janet R. Dunham

John L. Pierce

Software Research and Development

Center for Digital Systems Research

Research THangle Institute

Research THangle Park, North Carolina 27709

Contract NAS1-16489

Task Assignment No. 22

March 1986
TABLE OF CONTENTS

Page No.

List of Tables .................................................................................... i

List of Plots...................................................................................... ii

Acknowledgement ............................................................................. iii

1. INTRODUCTION .......................................................................... 1

1.1 Background ............................................................................ 1

1.1.1 Reliability Goals for Flight-Critical Software ............................ 1

1.1.2 NASA-LaRC Software Reliability Research Goals ..................... 1

1.2 Study Objectives ...................................................................... 2

1.3 Related Work ......................................................................... 2

1.4 Summary of Study and Conclusions .............................................. 2

2. THE SYSTEMUNDERSTUDY

2.1 The PA28PitchAxisControlModule............................................ 3

2.2 The PA28AircraftPitchAxisResponseModel................................ 4

3. TESTPROCEDUREANDRF__ULTS................................................ 4

3.1 Configurationof the N-VERSIONCONTROLLER.......................... 4

3.2 Test Initialization..................................................................... 5

3.2.1 PitchCommandGeneration................................................. 5

3.2.2 Tuning the N-VERSION CONTROLLER ............................... 7

3.2.3 Tuning the NVS Control Law Coefficients ............................... 8

3.3 Stress Test Results .................................................................. 10

3.4 Case Analysis Results .............................................................. 12

4. CONCLUDING REMARKS ........................................................... 14

i

5. REFERENCES ............................................................................ 15

LIST OF TABLES

TABLE 1. NVS CONTROL LAW RUNS OF LENGTH 1,000

INPUT CASES ................................................................... 9

TABLE 2. STRESS TEST RUNS OF LENGTH 1,000,t300

INPUT CASES. ................................................................. 11

TABLE 3. APPLICATION TASK ELEVATOR COMMAND DIFFERENCES

AND PITCH RANGE ............................. _.......................... 12

TABLE 4. DESCRIPIION OF SOFTWARE FAULTS

PRESENT IN AT3 ............................................................. 13
Page No.

LIST OF PLOTS

PLOT 1. SQUARE (20,40,20,40,60¢30) ................................................. 7

PLOT 2. SINE (30,.16,0,0) .................................................................. 7

PLOT 3. TS(_ SYSTEM PERFORMANCE ............................................ 8

PLOT 4. TSN3 SYSTEM PERFORMANCE ............................................ 8

PLOT 5. SQUARE WAVE RESULTS .................................................. 10

PLOT 6. SINE WAVE RESOLTS ........................................................ 10

il
ACKNOWLEDGEMENT

The authors wish to acknowledge the technical direction given by G. Earle Migneault of NASA - Langley Research Center. We also thank Don Rich, Sam McBride,

and Jay Duma for their perserveranee in the development of the N-VERSION CONTROLLER. Finally, we thank Brian Lupton of the Flight Control Systems Division of

NASA-LaRC who was the contract monitor for this research project and our manager

J-;mClary for his continued support.

Ill
1. INTRODUCTION

1.1. Background

1.1.1. Reliability Goals for Flight.Critical SoRware

Examplesof digitalcomputersbeing used in increasinglycritical functions in flight

management systems are the navigation, guidance, and energy management system for

the Boeing 757/7671,2 commercial jet transport series, the slat and flap control system

for the Airbus Industries A310,3 and the flight control system for the Grumman X29A. 4

Since software failures can be equally as hazardous as hardwarc failures, the use of

software to perform critical functions of these flight management systems necessitates

demonstrating that the flight management software complies with Part 25 of the Federal

Aviation Regulations. Toward this end, DO-178, published by thc Radio and Technical

Commission for Aeronautics (RTCA), 5 provides guidance on techniques and methods

that may be used for building and certifying reliable flight control software. The FAA

Advisory Circular 25.1309-1 6 which references DO-178 def'mes the software reliability

requirements by specifying three overlapping quantitative ranges which arc to be interpreted as the allowable risk for an hour of flight time based on a flight of mean duration

for the type of airplane being certified. These ranges are associated with the frequency

with which failure events may be expected to occur several times during the operational

life of each airplane (i.e., probable events that occur with a probability on the order of

1 x 10-5or greater). The failure events may not be expected to occur during the total

operational lifc of a random single airplane of a particular type but may be expected to

occur during the total operational life of all airplanes of a particular type (i.e., improbable events that occur with a probability on the order of I x 10-5 or less ). These failure

events are so unlikely that they need not be considered to occur ever unless engineering

judgement would require their consideration (i.e., extremely improbable events that

occur with a probability on the order of 1 x 10-9or less).

1.1.2. NASA-LaRC Software Reliability Research Goals

The software reliability research sponsored by NASA-LaRC focuses on the development of a credible method for predicting operational reliability- that is, predicting the

improbability that the system will fail duc to residual faults remaining in the software. 7

It is these residual faults, which surface infrequently, that cause the rare event or

extremely improbable failures. As evidenced by the first well-publicized Space Shuttle

software bug, the failure of the initialization logic in J. Garman's words resulted from a

"very small, very improbable, very intricate, and a very old mistake.''8 This bug typifies

the rare and convoluted combination of events which cause carefully developed software

to fail.

Althoughconsideringall faults isimportantin reliabilityprediction,the most probable faults are often eliminatedusingthe softwarequalityassurancemethodsdescribed

in RTCADO-178.The researchthus concentrateson the developmentof a method

which yieldsquantitativeassurancethat the aggregateprobabilityof the remaining,less

probable faults, constitutes an acceptable risk. Accordingly, the System Validation
°-

Branch of NASA - Langley Research Center has used a probability value of 10-9 for a

ten hour flight as an informal standard in the construction of a credible reliability prediction method for validating critical software.9 To date no known software has been validated to that extent.

1.2. Study Objectives

The construction of a credible reliability prediction method for critical software is

hindered by our lack of knowledge about the underlying nature of the software failure

process. This lack of knowledge contributes to our inability to eradicate or tolerate faults

and to our lack of confidence in the extent to which we have al_[_roximatedthe goal of

minimal defects, i.e., achieving a reliability of less than I x 10-_ for a ten hour flight.

The study was undertaken to provide data for use in developing a model for predicting

the reliability of software in systems with feedback, namely flight control. In particular,

we were interested in the software error burst phenomenon 10 and in gauging the effect

of these bursts on the behavior of the system. A software error burst is a sequence of

observations of erroneous outputs which are repeated manifestations of latent software

faults that due to the memory of the system and the correlated nature of the inputs can

eventually lead to a system failure. The frequency and duration of these bursts may

severely effect the operational reliability of process control software, and consequently

may be a critical reliability prediction parameter.

1.3. Related Work

The flight control software tested as a part of this study was developed in a manner

which emulated a realistic software development effort. It was developed in a software

research and development laboratory at RTI by three moderate to advanced skill level

programmers using a remote link to the computational facilities of NASA's AIRLAB at

Langley Research Center. Each of the programmers was given two programming tasks

which involved implementing a launch interceptor condition module for a radar tracking

system and a pitch axis module for a PA28 flight control system. The specifications for

both modules had been written in English by a senior analyst who had also written and

extensively tested a fourth or comparison version of each application. The programming

activity was managed in a conventional fashion with the exception that the programmers

were not permitted to discuss their code with anyone other than their manager or the

senior analyst who was responsible for answering all specification questions. The programmers were instructed to optimize the reliability of their code. Additional information about the software dcvelol:rmcnt activity can be found in NASA CR-172553, An

Experiment in Software Reliability. 11

1.4. Summary of Study and Conclusions

The development and testing of the pitch axis control module for a flight control

system was undertaken to provide data for use in constructing a model for predicting the

reliability of software in systems with feedback. In particular, the study was undertaken
to investigate the software error burst phenomenon10 and to gauge the effect of these

bursts on system behavior. A software error burst is a sequence of erroneous outputs

due to the memory of the system and a correlated nature of the inputs. The frequency

and duration of these bursts may severely effect the operational reliability of process

control software and, consequently, may be a significantreliabilityprediction parameter.

To investigate the software error burst phenomenon, three pitch axis control

modules were implemented for a PA28 aircraft. Input to the control law modules are

the pitch command and the current pitch of the aircrrfft. Output from the control law is

the elevator deflection command. The specification,written in English and provided to

the programmers for coding, contained three operating modes of increasing complexity.

A one axis, single input, single output simulation of the aircraft comprised the PA28 aircraft response model.

The three implementations were tested in parallel with over 14 million pitch commands (about 78,000 flight hours) having varying levels of additive input and feedback

noise. This testing surfaced four software faults in an implementationof the pitch axis

control law. This small number of detected faults may be due to the overly simpleexample chosen for the control law software and the programmer's ability to exploit this simplicity in validating it. As software components of the flight control systems increase in

functionality and complexity, the full impUtation of software errors remains to be seen.

2. THE SYSTEM UNDERSTUDY

2.1. The PA28 Pitch Axis Control Module

A survey of flight control software systems 12,13,14,1,4,15,16,17,2,3which could be

used for this study culminated with the decision to select a system for which we had both

the control laws and a description of the aircraft response model. As a result, the PA28

aircraft was chosen because its aircraft response model was known and relatively simple

to code. The pitch axis control module was chosen as reliable control of the pitch axis is

safety-critical during landing. Loss of control of the elevator during final approach wiU

almost certainly result in a catastrophic accident. The pitch axis control law is typically

computed first and used as input to the control lawsfor the longitudinal,axes, although,

in a few flight control systems it is computed in parallel with the roll axis.

Appendix A contains the specification of the pitch axis control law provided to the

three programmers for independent development. The specification requires that the

pitch axis control law operate in nine modes. The modes correspond to the use of Proportional (P), Proportional/Derivative (PD), and Proportional/Integral/Derivative (PID)

control strategies and to different computations of the error term. A fourth module was

developed and extensively tested for use as a comparator. The pitch axis module specified constitutes a simplified pitch axis control system as it ignores voting to handle sensor failures and couplingwith an analog backup system.

3
2.2. ThePA28AircraftPitchAxisResponseModel

A single input-singleoutput aircraftresponsemodel def'mesthe PA28 aircraft's

responseabout the pitch axis. It wasdef'medin the frequencydomainand implemented

in the time domainusingTustin's approximation.In the frequencydomain,the aircraft

responsein pitchis givenby the followingtransferfunction

otlS 2 + ct2S + ot3

-_(s) _31s4+ 132s3+ t332s+ 134s+ a_

and the aircraft response in pitch rate is given by

8_r 50

An elevator transfer function of the form --_-(s) = s (s+50) ' is used.

where

OA is the aircraft response in pitch.

0A is the aircraft response in pitch.

8N is the elevator deflection for the N-version system after voting and prior to

the application of the elevator transfer function.

8_ is the elevator deflection for the N-version system after the application of

the elevator transfer function.

_1 , _2 , Or3

and

' _1 , _2., f13, _4, flS" are model coefficients

and

s is the Laplace domain variable.

3. TEST PROCEDURE AND RESULTS

3.1. Configuration of the N-VERSION CONTROLLER

The three independently developed software implementations of the pitch axis control law are executed in parallel using the N-VERSION CONTROLLER 11 shown in Figure 1. This CONTROLLER relies on the technique of n-version programming to detect

program errors. (N-version programming, first widely publicized by Avizienis 18 and

recently examined by Knight, Leveson, and St. Jean 19 involves n programmers
independently coding the problem from the same specification). All program outputs

are compared pairwise during this testing. Whenever an output inequality occurs which

is outside the range specified, the testing is halted, and the faulty implementation is

identified, analyzed, and corrected. Using n-version programming for error detection

avoids reliance on a comparisonversion to determine output correctness.

The N-VERSION CONTROI.L,ERsimulates the execution of the pitch axis control

system and provides both dynamic and static viewing of the system behavior through an

interface program. The CONTROLLER can be used to control the pitch axis simulation

for combinations of the followingconfigurations:

• Extensively Tested System(ETS) flying

• N-Version System(NVS')flying

• ETS monitoring NVS flying.

These configurations can have noise added to the pitch commands generated and to the

aircraftresponse in pitch.

3.2. Test Initialization

3.2.1. Pitch CommandGeneration

The pitch command waveform selected to test the control law software _ be trropogated as either a sine wave or a generalized square wave. The sine wave is of the

form

I](t) = Asin(Bt+_/) + k

where 13,A, B, ",/,and k are parameters specified by the experimenter. The generalized

square wave has six specifiable parameters which characterize the waveform; 1: lead

time, 2: rise time, 3: duration, 4: fall time, 5: height, and 6: vertical displacement. The

sine waveform is denoted by sine ( A,B,_/,k ) and the square waveform is denoted by

square (1,2,3,4,5,6).

For the purpose of this study, the coefficientsof the input square wave and the sine

wave are chosen to have a maximum rate of change in pitch of 30 degrees over 40

timesteps or 2 millisecs. These requirements yield a square waveform of square

(20,40,20,40,60,-30) and a sine waveform of sine (30,.16,0,0) as shay in Plots 1 and 2

respectively.

The noise which can be added to the pitch command and the pitch feedback is samg'_..qL:0 and a trlmcated

pied from a truncated skewed Normal (i,l,cr_) distribution with o'_

IJ,2

skewed Normal (l_2.cr_)distribution with _ _:0 respectively.
y

To,odvooiool, I i,ch

I+

b,

Pitch N-Version_,t Elevator Pitch

' Comm Generaatondr "_-+ _-- , (Flying)/ Deflection Response

FIGURE 1. CONFIGURATION OF TH_ N-VERSION CONTROI r _R

where

0c(n) - Pitch command at time step n for the E'rs aS(n) - Deflection of elevator at time step n for NVS

and NVS SYSTEMs prior to the addition of flying the aircraft after voting and prior to

random noise, appl(cation of the elevator transfer fmx-xion.

Rt(n) - Random noise added to 0c_.). Aaron(n) - Difference between a_(n) and aM(n)

_c(n) - Pitch command at time step n for the E'I'S Aam'(n) -Difference between a"c(n)and aF(n)

and NVS SYSTEMS after the addition of ran- a_(n) - Ddlect_on of elevator at time step n for ETS

don noise, flying the aircraft after application of the eleva-

_V(n) - Defle_'tlon of elevator at time step n for ETS tor transfer function.

flying the aircraft prior to application of the a_(n) . Deflection of elevator at time step n for NVS

elevator transfer function, after application of devator transfer function.

a_(n) . [x-n_ctionofdevatorattime,tepnfarver- ._'(n) -Differ='be_ c.e _(n) and_(n) sfon i of NVS prior to vote and application of

devator transfer function. O_(n) - Pitch of aircraft at time step n for the ETS

flying the aircraft prior to adding random

aM(n) . Deflo:tion of elevator at time step n for the noise.

ETS monitoring the NVS prior to application of

the elevator transfer function. 0_(n) - Pitch of aircraft at time step n for the NVS

flying the aircraft prior to adding random

Aai_eC(n)- Differerx:e between /_U(n) for all tmordered noise.

rqVS pairs (i.e. (a,_(n),a,_(n)) for all i, j

[0,2]). R2(n) - Rara:k_ noise added to 0_ and 0K at time

A_i"qF(n)- Difference between ai_(n) for all NVS pairs step n.

(i.e. (a_(n),aV(n)) for all i • [0,2]). _(n) - Pitch of aircraft at time step n for the ETS

A_'_(n) for all NVS pain (i.e. (a_C(n)t._n)) for all i • flying the aircraft after adding randona noise.

[0,2]). - Difference between a_'(n) _(n) - Pitch of aircraft at time step n for the NVS

flying the aircraft after adding ranckma noise.
PLOT 1. SQUARE (20,40,20,40,60,-30)

Time Step t

PLOT 2. SINE (30,.16,0,0)

Tune Step.t

3.2.2. Tuning the N-VERSION CONTROLLER

Tuning the N-VERSION CONTROIJ.ER involved (i) testing of the ETS module in

conjunction with the aircraft response model, (ii) testing the control law software written

by programmer 1 in conjunction with the aircraft response model, and Off) testing of the

NVS in conjunction with the aircraft response model. The first two test efforts were

conducted as the ETS and the independent implementations of the control law are implementations of different forms of the same control law algorithm in that the ETS control

law does not have the mode switching capability and it operates in ProportionalIntegral-Derivative mode only. The above testing resulted in near optimal performance

of the software to control the pitch axis by setting of the aircraft response model coeffidents as follows:

a I = 17.5; ot2 = 39.4; a 3 = 17.0;

131= 1.02; 132= 7.01; 133= 18.7; 134 = 8.6; 135= 2.84;
3.2.3. Tuning the NVS Control Law Coefficients

The control law coeffidents were tuned to minimize the amount of overshooting and

undershooting of the pitch response in the absence of input and feedback noise when the

NVS is flying the aircraft. Table 1 shows the results of executing the NVS system with

different values of the control law parameters for the square (20,40,20,40,60,-30) and

sine (30,.16,0,0) waveforms. The values shown are estimates derived from visual

inspection of the corresponding graphs. Plots 3 and 4 show the system performance for

the tuning runs using the square (20,40,20,40,60,-30) and sine (30,.16,0,0) waveforms

respectively. The ETS control law was tuned during the system tuning runs.

PLOT 3. TSO4 SYSTEM PERFORMANCE PLOT 4. TSN3 SYSTEM PERFORMANCE

Pitch Command vs NVSPitch Reslxx_e PitchCommandvs NVSPitch I_e

"q'0. "4'0

?.o. ?.0

_" Z<

-20 -20.

Pitch Command vs NVS Pitch Resfxxase PitchCommand vs NVS Pitch Response

q'0. "4"0.

v '0. _ O. "

Z< z<

-4 (] -4 0

-'to 0 _0 -40 0 40

0c(n) Oc(n)

8
TABLE 1. NVS CONTROL LAW RUNS OF LENGTH 1,000 INPUT CASES

high freq. _dUation in ON,

tsql .1300 .167 .084 .042 (-180,180) (-300,300) (-200200) (-30,30) overshoot_8_

changessire every time step

high freq. €_dIlationin 8_

tsq2 .0CA) .167 '.042 .021 (-30,30) (-200,200) (-50,50) (-20,20) changes sign alter, time steps

low freq. esdllation in O_

tsq3 .000 .167 .021 .042 (-200,200) (-500,500) (-200,200) (-110,110) changessign everytime step

low freq. osdllation in O_

uq4" .000 ,167 .021 .042 (.7,7) (-125,125) (-32,35) (-7,10) at peak and low amp/itude

J

low freq. c_dllation in ON

tsq5 .000 .167 .010 .021 (-i0,10) (-200,200) (-29,29) (-6,8) at peak and low amptitude

low freq. oxci'!ationin ON

tsq6 .000 .167 .1205 .021 (-10,10) (-200,2130) (-29,29) (-6,8) at peakand low art_itude

low freq. o_cillationin oN

tsq7 .000 .200 .013 .025 (-10,10) (-200,200) (-35,33) (-10,10) at _ and low amt:fitude

[ _[ low freq. osdllation in 8'_

tsq8 .000 .200 .006 ._ (-10,10) (-200,200) (-32,33) [ (-8,10) at peak and low an_irude

A I 1] I

unl .000 .167 .025 .042 (-12,12) (-75,75) (-27,30) (-6,7) I_ freq. osdllafion

('undershot-is8H !

( .

tsn2 .000 .200 .013 j .0_ , (-12,12) (-75,75) (-25.28) (-6,,) i)noosdllation i .: [

I il over ho0p, i

tsn3 " .000 .167 I .042 .0_ i] (-10,10) (-50,50) (-32,M) (-7,9) lc_v freq. _cillation )r

where

t

tsql through tsq8 are the square wave tuning runs

tsnl through tsrd are the sine wave tuning runs

a_, a!, a2, a3 are the control law coeffidents

_ is the current error in 0 at time step n for NVS flying the aircraft.

F_6"_ is the cummulative error in pitch of NVS at time step n.

0/_' is the pitch of aircraft at time step n for the NVS flying the aircraft prior to

addingrandom noise.

E_ is the deflection of elevator at time step n for ETS flying the aircraft prior to

application of the elevator transfor.

9
Based on the results of the tuning runs, the control law coefficients for the square

wave stress test runs were set at a0=O.O00; a1=0.167; a2=0.032; a4=0.084; (based on

tuning run tsq4) and for the sine wave stress test runs a0=O.O00; a1=0.167; a2=0.021;

a3=0.042; (based on tuning run tsn3).

3.3. Stress Test Results

Stress testing entailed executing the system in Pr0portional-Integral-Derivative

(PID) mode 7 (see Appendix A) with the goal of detectingsoftware failures under worst

case input conditions. The noise distributions used during this testing were selected to

simulate both continuous turbulence and discrete incremental gust loads. The stress test

runs were made with the square waveform and the sine waveform and with and without

input noise on the pitch command and the pitch response as depicted in Table 2. Plots 5

and 6 show the signal and noise to signal ratio for the square (20,40,20,40,60,-30) and

sine (30,. 16,0,0) stress test runs respectively.

During the stress tests if the absolute range of the pairwise comparisons of all 8_

exceeded the limit specified, the N-VERSION CONTROLLER logs the system state for

the next two hundred test eases inclusive of the failure causing case. The control law

implementations are then scrutinized for the presence Ofa software fault and the impact

of that fault on the system performance is analyzed.

PLOT 5. SQUARE WAVE RESULTS PLOT 6. SINE WAVE RESULTS

(Signal -_ .No!_),3ignal for _% Additive l.npu: No;.._e (Signal + No[se)/Signnl for 25% Additive L'lput Noise

1

.

_, I '_ _'_

._1 .;_01 . . . , ,

0 _0 :00 DO 200 0 _) 100 l_ 2C0

"l"tmeStep t Time Step t

(Signal . NoL_e),'Signalfor 10% Additive Input Noi._ (Signal + Noi._)/Sigmd for 10% Additive Input NoL_

-2 " ,,, _ +._ ,

0 .'-3 :."2 I_ 200 _ 100 I_0 2'30

"['tmeStep t Time Step t

10
TABLE 2. STRESS TEST RUNS OF LENGTH I,CCY0,000INPUT CASES

IRma R1(n) rlL(n) r1_n) _(n) r2L()n r2y._nl) C_

ssqI N(O.O,O.O) 0.0 0.0 N(O.O,O.O) 0.0 0.0 0% inl_ & _tputnoise

i

ssq 2 [ N(3.0,1.0) 2.0 i 4.0 N'(0.0,0.0) 0.0 0.0 10%intxtr noLse

i

ssq3 N(0.0,0.0)0.0 0.0 N(3.0,1.0) 2.0 4.0 10% _rputndse

i I

ssq4 N(3.0,I.0) 2.0 4.0 N(3.0,1.0)2.0 4.0 10%input& outputnci.,e

ssq5 N(7.52..5) 5.0 I0.0 N(0.0,0.0)0.0 0.0 _% inputncise

ssq6 II N(0.0,0.0)0.0 0.0 N(7.5,2.5)5.0 10.0 25% outputnoise

'.. ssq7 N(7.5_.5) 5.0 10.0 N(7.5,2.5)5.0 I i0.0 _%input & outputnoise

i I

ssnI [ N(0.0,0.0)0.0 0.0 N(0.0,0.0)0.0 0.0 0% inputnoisoutput e noise

l

, [ J

ssn 2 : N(3.0,1.0) [ 2.0 4.0 N(0.0,0.0) 0.0 0.0 10%input noise

i p r

i i

tssn 3 N(0.0,0.0) 0.0 0.0 N(3.0,1.0) 2.0 4.0 10% out-_t noise i

I

I +i,+, 1

ssn 5 N(7.5,2.5) 5.0 10.0 N(0.0,0.0) 0.0 0.0 25% input noise

ssn6 iN(0.0,0.0) 0.0 0.0 N(7.5,2.5)5.0 I0.0 _% out-pnoutise

ssn7 i N(7.5,2.5)5.0 I0.0 N(7.5,2.5)5.0 i0.0 _% input& outputndse

i, I. I

I!
Table 3 shows the maximum range in the elevator deflection angles computed by

each implementation of the control law software for the various stress test runs.

Although the actual values of the elevator deflections varied among the independent

implementations they remained in a range of 2.5, as shown in Table 3, which was specified as an acceptable performance parameter during the system tuning runs. The range

specification is somewhat arbitrary in that a tight bound on the range increases the rate

with which we observe false alarms, and a lenient bound increases the probability of

non-detection.

TABLE 3.

APPLICATION TASK ELEVATOR

COMMAND DIFFERENCES AND PITCH RANGE

RUN ID MAX DIFFERENCE PITCH RANGE

ssq 1 0.6 [-31.7,34.3]

ssq 2 0.6 [-31.7,37.2]

ssq 3 0.6 [-35.0,34.8]

ssq 4 0.6 [-34.7,40.4]

sscl 5 0.7 [-34.0,48.8]

ssq6 0.5 [-45.1,44.1]

ssq 7 0.6 [-39.9r55.7]

ssn 1 0.7 [-32.1,34.2]

ssn 2 0.7 [-33.6,39.6]

ssn 3 0.7 [-36.4,36.6]

ssn 4 0.8 [-37.4,43.0]

ssn5 1.3 [-39.2,56.2]

ssn 6 1.3 [-49.5,48.8]

ssn7 2.0 [-51.7,66.1]

3.4. Case Analysis Results

Since we were uncertainifthe range in elevator deflection commands were a function of mismodeling or if a software error was occurring, (This al_aears to be a common

problem with performance testing of process control software.) zu we investigated the

elevator deflections for 1000 time steps by executing the NVS system using the ssql run

parameters, setting the permissible range of the elevator deflection commands computed

by the three implementations for each time step ( i.e., ASh(n) ) to 0, and analyzing the

test case when non-zero ranges occurred. The analysis identified four faults in an implementation of the control law, as shown in Table 4. The fault made in computing the

derivative term in Modes 4 and 7 was corrected. A re-execution of the ssql run resulted

in numerical agreement (to the least significant bit) of the elevator deflection command

output from the control laws.

12
TABLE 4. DESCRIPTION OF SOFTWARE FAULTS

PRESENT IN AT3

Modes Correct Term

Effected Term Used

Oc(n)-OA(n) 6c(n)-6A(n) 4,7 t(n)-t(n- 1)

6c(n)-- 0A(n ) 6c(n)-- 6C(Ia-- 1)-- 0A(n)-- 6A(n-- 1)

5,8

t(n)- t(n- 1) t(n)- t(n- 1)

3,6,9 0c(n)--0A(n ) 0c(n)-6A(n) t(n)-t(n-1)

0c(n)- 6A(n) 6c(n)-- 0A(n)-- 6c(n-- 1) + 6A(n-- 1)

6,9

t(n)-t(n-- 1) t(n)-- t(n-- 1)

where

n is the time s:ep index

t(n) is the clcxk time at step n

0c(n) is the pitch command at step n

0A(n ) Is the actual pitch of the aircraft at step n

0c(n) is the change in pitch command, i.e. (0c(n)-0c(n-1))

0A(n) Is the change in actual pitch, i.e. (0A(n)--0A(n--1))

AT3 Is the third implementation of the control law

13
4. CONCLUDING REMARKS

N-version testing of three implementations of a PA28 pitch axis control law with

over 14 million pitch commands (about 78,000 flight hours) having varying levels of

additive input and feedback noise surfaced only four software faults in an implementation, thus precluding the analysis of software error bursts. This small number of

detected faults may be due to the overly simple example chosen for the control law

software and the programmers' ability to exploit this simplicity in validating it. As

software components of the flight control systems increase in functionality and complexity, the full implication of software errors remains to be seen.

14
5. REFERENCES

References

1. R.E. Spradlin, Boeing 757 and 767 Flight Management System, Boeing Commercial Airplane Company (November 20-21, 1980). Presented at the RTCA 1980

Technical Symposium and Annual Assembly Meeting, Washington, D.C.

2. K.M. Homback, "Development Tools - Case Study for Large Systems," Proceedings of the AIAA/IEEE 6th Digital Avionics Systems Conference, American Institute

of Aeronautics andAstronautics, (December 1984).

3. R. Troy and C. Baluteau, "Assessment of Software Quality for the AIRBUS

A310 Automatic Pilot," Proceedings of the Fifteenth Annual International Symposium on Fault-Tolerant Compua'ng, IEEE Computer Society Press, (June 1985).

4. J.T. Murray et al., "DS-28703-01 Machine Language Program, FSW,

NH3500AN," Honeywell Avionics Division, (June 1982).

5. "RTCA/DO178 Software Considerations in Airborne Systems and Equipment

Certification," Radio Technical Commission for Aeronautics Secretariat, (November

1981).

6. Advisory Circular 25.1309-1 System Design Analysis, U.S. Department of Transportation Federal Aviation Administration, Washington, D.C. (September 1982).

7. J.R. Dunham and G. E. Migneault, "An Experiment in Software Reliability,"

Talk presented at Seventh Minnowbrook Workshop on Software Performance

Evaluation, Blue Mountain Lake, New York (July 84).

8. John R. Garman, "The "Bug" Heard Round the World," Software Engineering

Notes 6 pp. 3-10 ACM Sigsoft, (October 1981).

9. J.R. Dunham and J. C. Knight, eds., "production of Reliable Flight Crucial

Software: Validation Method Research for Fault-Tolerant Avionics and Control

Systems Sub-Working-Group Meeting," NASA Conference Publication 2222,

NASA (1982).

10. G.E. Migneauit, "Emulation Applied to Reliability Analysis of Reconfigurable,

Highly Reliablc, Fault-Tolerant Computing Systems," AGARD Conference

Proceeding No. 261(1980).

11. J.R. Dunham and J. L. Pierce, "An Experiment In Software Reliability," NASA

CR 172553, NASA, Langley Research Center (March 1985).

12. K.J. Szalai, et al., Digital Fly-By-Wire Flight Control Validation Experience,

NASA Technical Memorandum 72860 (December 1978).

13. Alfred Spector and David Gifford, "Case Study: Thc Space Shuttle Primary Computer System," Co,n,nunicatior_ of the ACM 27(9)(September 84).

14. J.M. Corney, "The Development of Multiple Redundant FLight Control Systems

for High Integrity Applications," Aeronautical Journal, Marconi Avionics Limited,

(October 1980).

15
15. David J. Martin, Dissimilar Software in High Integrity Applications in Flight Controis, Marconi Avionics Limited, Rochester, England (1983).

16. R.B. Smith, "Flight Clearance of the Jaguar Fly-By-Wire Aircraft (part I0," The

Royal Aeronautical Society, (April 1982).

17. K.G. Wilkinson and M. L. Shooman, "Probabilistie Models for Software Reliability Prediction," The Royal Aeronautical Society, pp. 485-502 Academic Press,

(1972).

18. A. Avizienis, "Fault Tolerance: The Survival .Attribute of Digital Systems,"

Proceedings of the IEEE 66(10)(October 1978).

19. J.C. Knight, N. G. Leveson, and L. D. St. Jean, "A Large Scale Experiment in

N-Version Programming," The 15th International Conference on Fault-Tolerant

Computing Digest of Papers FFCS 15 pp. 135-139 IEEE Computer Society Press,

(June 19-21, 1985).

20. Ronald R. Luman, "Practical Kalman F'dter Software Performance Testing &

Validation," IEEE Transactions on Reliability R-33 pp. 219-226 (August 1984).

16
Appendix A. Pitch Axis Control Law Specification

17

.L i. • ,. r
1. BACKGROUND

a. Aircraft Control Surfaces And Axes Of An Aircraft In Flight

Aircraft control surfaces are divided into two groups; primary and secondary. The

primary control surfaces are the ailerons which are located at the trailing edge of the

wings, the elevators which are located at the rear portion of the horizontal tail assembly

(in some aircraft the entire horizontal tail is movable), and the ruder which is the rear

portion of the vertical tail assembly. The secondary control surfaces are the trim tabs,

balance tabs, and servo tabs. They are used to reduce the force required to activate the

primary control surfaces, and for trimming and balancing the airplane in flight. These

tabs are ia actuality small airfoils attached to,; or recessed into the trailing edge of the

primary control surfaces.

There are three axes about which an aircraft rotates about whenever it changes its

altitude with respect to the earth, or inertial, or moving axis coordinate system. These

axes are the longitudinal, lateral, and vertical axes. Roll is motion about the longitudinal axis. This motion is controlled by the ailerons. Pitch is motion about the lateral

axis. This motion is controlled by the elevators. Yaw is motion about the vertical axis.

Yaw is controlled by the rudder.

18
The above diagram illustrates the axes of an aircraft in flight. Tile set of rectangular axes is oxyz where o is the center of gravity of the aircraft.

Ox is the longitudinal axis

Oy is the lateral axis

CYzis the vertical axis

U, V, W are the velocity components of the center of gravity along (_x, "O-y,and (_z

respectively.

P, q, r are the components of angular velocity of the axis frame Oxyz about (:Yx,(_y,

andOz respectively. Hence, p is the aircraft's angular velocity in roll, q is the

aircraft's angular velocity in pitch, and r is the aircraft's angular velocity in yaw.

b. Flight Dynamics

1. Eqns. of Motion

Flight dynamics deals with the motion of an aircraft under the influence of forces.

These forces are of six types.

(i) Inertia forces, arising from the mass distribution and linear and angular

acceleration of the aircraft.

(ii) Aerodynamic forces and moments, depending on angular velocities of the aircraft (sometimes called rotary forces and moments).

(iii) Aerodynamic forces and moments depending on the linear velocities of the aircraft (sometimes called static forces and moments, since they depend on tile altitude of the aircraft relative to the airstream and not on its angular velocities).

(iv) Aerodynamic forces and moments due to the application of controls (usually

only the forces and moments due to control deflection are of importance; these are

sometimes called static forces and moments due to controls).

(v) Gravitational forces, and

(vi) Propulsive forces.

The equations of motion of an aircraft can be completely determined by considering

these forces. These equations constitute a set of non-linear differential equations, which

are separated with respect to the lateral and longitudinal axes ann linearized. These

equations can be found in texts on aircraft aerodynamics.

19
2. STABILITY

A dynamical system is said to be stable or possess stability, if, when slightly disturbed from a state of equilibrium or steady motion, it tends to return and remain in

that state, the disturbance acting for only a finite time. In general, an aircraft must be

both statically and dynamically stable. An aircraft has static stability if, immediately

on being disturbed, it has a tendency to return to equilibrium conditions. An aircraft

has dynamic stability if its motion subsequent to the initial disturbance is characterized

by its decreasing amplitude as equilibrium is restored. Static stability is prerequisite to

dynamic stability. The design of an aircraft must make it both statically and dynamically stable with respect to the three axes. Because symmetry keeps translation and

rotation in the vertical plane from producing forces in the other planes, longitudinal and

lateral-directional stability are considered separately.

To illustrate, assume that we are interested in the stability in pitch for three aircrafts. Stability in pitch represents longitudinal stability, i.e., stability about the airplanes' lateral axis.

CL I AirplaneA

Coeffi ci ent)

L__! ' kJ

[ I I Airplane Airplane axis

I Nose-up C B

.I Gust applied

I to aircraft

01

Timet

NOTES: Airplane A is unstable and continually diverges.

Airplane B is both statically and dynamically stable, but the return to equilibrium takes too long.

Airplane C is acceptable because it returns to equilibrium in a relatively short

time.

2O
c. Artificial Stability and Automatic Control

The stability and response characteristics of an aircraft without automatic control,

at a given speed and altitude, are determined completely by the design and distribution

of mass of the aircraft. There exists, however, various mechanisms to optimize the

aircraft's stability without changing the aircraft design. One way is by shifting the

ma.ss distribution, or center of gravity during various stages of flight (e.g.,shifting from

subsonic to supersonic speeds). Another way is by automatic control of the control surfaces in a manner which compensates for one or anore components of the flight disturbance. The goal for both these methods is to artificially improve the stability and

response of the aircraft.

The automatic control systems utilized may be an open-loop systems in which

input data, say 0 i are fed into a servo-system which produces an output 0 0. e i could

be a signal from a gyroscope or an accelerometer. The output 0 0 might represent an

elevator deflection• The automatic control system may be a closed-loop system or a system with feedback control. Closed-loop systems are applied in cases where the output

of the open-loop system depends upon the deviation of the aircraft from the datum

state. The essential requirement of a closed-loop system is that the error between the

desired state and the existing state is constantly monitored.

This describes the detailed computer program specifications for the Pitch Axis Control Problem. It is anticipated that you will code at least three successive control strategies of increasing complexity. In order, these are:

STRATEGY FORM

(I) Proportional: y--a 0 + al x

(2) Proportional-Derivative: Y--_a0 + alx + _ dx/dt

t

(3) Proportional-Integral-Derivative: Y=% + alx + a2.dx/dt + a3fx dt

0

where Y and the integral are to be computed, and the terms in x and the ai are

inputs. Specifications generally pertinent to all 3 control strategies and details of the

first are contained herein; details for strategies 2 and 3 will be forthcoming. The overhead (parameter lists, common blocks, clock calls, initialization, etc.,) developed in the

coding of strategy 1 will be directly applicable to strategies 2 and 3. Therefore,

appropriate editing techniques should expedite development of 2 and 3.

21
3. SPECIFICATIONS

Language: FORTRAN

Variables: Standard FORTRAN Rcal*4 and Integer*4 for variables passed via

common. Variables may be named as desired.

Module: A subroutine named PROB2 with no parameter list (as in PROB1,

Launch Interceptor Problem).

I/O: Variables will be passed through labeled common blocks; PROB2 will

generate no read/write operations except as necessary during developmeat. There should be no I/O operations in the delivered module.

Files: The source code for PROB2 and any supporting subprograms should

be in a file named prob2.for; any other subprograms or main "driving"

programs not delivered should be maintained on separate files.

Acceptance: There will be no formal acceptance test as there was in the Launch

Interceptor Problem; you may generate your own test data as you

desire.

Data Validity: Assume all input data are valid; no validity tests are necessary.

#

22
INPUTS: COMMON/INPUTS/

POSITION NAME TYPE GOMMENTS

1 Initialize 1.4 IF -- 1, zero-out all output

variables and return.

2 Mode 1,4 IF --- 0, automatic control is not in

effect; return without computing;

otherwise, determines the control

strategy and the measurement

variable.

3 t(n) 1,4 The current value of dimensionless

time.

4 Oc(n) R*4 Desired value of pitch at current

time-step, n.

5 CA(n) R,4 Actual pitch of aircraft at current

time-step, n.

6 (_c(n) R*4 Desired pitch rate at current timestep, n.

7 CA(n) R*4 Actual pitch rate of aircraft at

current time-step, n.

8 ai R*4 Coefficients; i -- 1,2,...,5

OUTPUTS:

COM:MON/OUTPUT/

1 5(n) R*,t Elevator deflection angle, computed.

2 E(n) R*4 Error in measurement quantity (as

determined by mode).

3 AOA/At R*4 Pitch rate, computed.

23
Saved variables: COMMON/SAVED/

Note: The saved variable name should differ from the I/O variable name.

POSITION NAME TYPE COMMENTS

1 O¢(n) R*4 Current O¢ ; to become

0¢(n-1) upon next entry of subroutine.

2 5(n) R*4 Current computed elevator deflection; to

become 5(n-I) upon next entry of subroutine.

3 Oh(n ) R*4 Current Oh; to become 0 h (n-l)

upon next entry of subroutine.

4 t(n) 1.41 Current value of dimensionless time; to become

t(n-1) upon next entry of subroutine.

5 Eo(n ) R*4 Current error in O; to become

Eo(n-I ) upon next entry of

subroutine.

6 E6(n ) R*4 Current error in E) ; to become

E6(n-1 ) upon next entry of

subroutine.

7 (_¢(n) R*4 Current 6 c ;to become O¢(n-1)

upon next entry of subroutine.

8 (_h(n) R*4 Current (_h; to become (_h(n-1)

upon next entry of subroutine.

9 Ec o R*4 Cumulative sum of EoAt since

t---0.

i0 EE6 R*4 Cumulative sum of E§At since

t---0.

I The type specification for t(n) was omitted from the specification provided to the programmem.

° .

24
4. ALGORITHMS

In any of the forms, the quantity x represents the measured error, _ , in the quantity being controlled which can be either pitch, O , or pitch rate, O----de/dr. The

quantity y is the output from the controller and represents the aircraft elevator

deflection 5.

The controller will compute x, the measured error. Whether it computes x(O) or

x((_) will depend on the value of MODE. In addition, whether O is furnished as an

input or is to be computed as (O(n) - O(n-1)/At) by the controller, will also be determined by MODE.

Table 1 shows the control strategy and measured error relationship as governed by

values of MODE.

Table A - Control Law Mode

Strat%_' Error Term

Oc - O.A 6c - e.A AOc AOA

P: Proportional 1 2 3

PD: Proportiona/Derivative 4 5 6

PID: Propor tional/Inte_ral/Derivative 7 8 9

TABLE 1: Control Strategy to be employed and Measui'ed Error to be Computed for

Different Values of MODE

THE FOLLOWING STEPS SHOULD BE CODED:

Strategy 1, Proportional, y -- ao + alx

1. Test Initialize (see INPUTS comments)

2. Test Mode

3. COMPUTE X per mode. If MODE----3, use values saved (in common block) from

previous call of your module.

4. COMPUTE Y using inputs a0 and a l

5. RETURN

Strategy 2, Derivative, y -- _ + alx + a2dx/dt

(Same steps as for Strateg-y 1)

t

Strategy 3, Integral, y ---- a0 + nix + a.2d×/dt + a3fxdt

0

(The same steps should be used as for 1 and 2, but in addition provision should be

made for initialization and computation of the integral as a cumulative sum.)

25

-t
Appendix B. Results of Stress Test Runs

26
STRESS TEST RUN SSQ1

TOTAL CASES EXECUTED 1,000,000

TEST CASE GENERATION

Pitch Command Waveforrn SQ(20,40,20,40,60,-30)

Pitch Command NoLeo, Pitch ResponseNoise

Distribution N(0.0,0.0) Distribution N(0.0,0.0)

Truncation Points [0.0,0.0] TruncationPoints [0.0,0.0]

Seed 1234567808 Seed 9876542464

CONTROL LAW PARAMETERS

ao=.000 al =-167 a2 =.021 a3=.042

AIRCRAFT RESPONSE MODEL PARAMETERS

a 1 = 17.5 a 2 = 39.4 a 3 = 17.0

131 1.02 132= 7.01 133= 18.70 134= 8.60 135= 2.84

SYSTEM PERFORMANCE REQUIREMENTS

Requirement Enabled Extreme Values

/_F No [-6.2,11.91

8_ No [-6.8,9.3]

0hF No [-30.8,49.2]

_N No [-31.7,34.3]

Ir_l No 63.0

Ir_NI No 121.o

INEQUALITY CONSTRAINTS

Requirement Enabled Extreme Values

IA_ii_'_[ Yes 0.6

It_5_'l No 7.0

Ins_a:l No 3.1

In_l No 5.0

27
STRESS TEST RUN SSQ2

TOTAL CASES EXECUTED 1,000,000

TEST CASE GENERATION

Pitch Command Waveform SQ(20,40,20,40,60,-30)

Pitch Command Noise Pitch Response Noise

Distribution N(3.0,1.0) Distribution N(0.0,0.0)

Truncation Points [2.014.0] Truncation Points [0.0,0.0]

Sccd 1234567808 Sccd 9876542464

CONTROL LAW PARAMETERS

a0 = .000 a1 = .167 a2 = .021 a3 = .042

AIRCRAFT RESPONSE MODEL PARAMETERS

o:1 = 17.5 a 2 = 39.4 a 3 = 17.0

13x = 1.02 132= 7.01 133= 18.70 [34= 8.60 135= 2.84

SYSTEM PERFORMANCE REQUIREMENTS

Requirement Enabled Extreme Values

No [-6.6:1.8]

8_' No [-7.4,9.7]

0AFF No [-31.5,51.0]

()_ No [-31.7,37.2]

[F_.oF[ No 67.0

IE_NI No 138.0

INEQUALITY CONSTRAINTS

Requirement Enabled Extreme Values

IAa_l y_s 0.6

IAa_'l No 8.o

IAap'rl No 3.2

IAS_I No 5.1

28
STRESS TEST RUN SSQ3

TOTAL CASES EXECUTED 1,000,000

TEST CASE GENERATION

Pitch Command Waveform SQ(20,40,20,40,60,-30)

Pitch Command Noise Pitch Response Noise

Distribution N(0.0,0.0) Distribution N(3.0,1.0)

Truncation Points [0.0,0.0] Truncation Points [2.0,4.0]

Seed 1234567808 Seed 9876542464

CONTROL LAW PARAMETERS

ao = .000 a I = .167 a2 = .021 A3 = .042

AIRCRAFT RESPONSE MODEL PARAMETERS

a 1 = 17.5 a 2 = 39.4 a 3 = 17.0

131 1.02 [32 = 7.01 133= 18.70 134 = 8.60 135= 2.84

SYSTEM PERFORMANCE REQUIREMENTS

Requireme. nt Enabled Extreme Values

_i_ No [-7.4,11.2]

a N No [-8.4,7.9]

_F No [-34.7,47.4]

6_ No [-35.0,34.8]

I_FI No 57.0

IE_NI No 113.0

INEQUALITY CONSTRAINTS

Requirement Enabled Extreme Values

IAai_l Yes 0.6

lAa_'l No 8.0

laa_a:l No 3.1

I_aE_l No 4.3

29
STRESS TEST RUN SSQ4

TOTAL CASES EXECUTED 1,000,000

TEST CASE GENERATION

Pitch Command Waveforrn $Q(20,40,20,40,60,-30)

Pitch CommandNoLle PitchResponseNoise

Distribution N(3.0,1.0) Distribution N(3.0,1.0)

Truncation Points [2.0,4.0] TruncationPoints [2.0,4.0]

Seed 1234567808 Seed 9876542464

CONTROL LAW PARAMETERS

a0 = .000 aI = .167 a2 = .021 a3 = .042

AIRCRAFTRESPONSEMODELPARAMETERS

a I = 17.5 c_2 = 39.4 a3 = 17.0

131 1.02 132= 7.01 133= 18.70 134= 8.60 135= 2.84

SYSTEM PERFORMANCEREQUIREMENTS

Requirement Enabled Extreme Values

St No [-8.2,11.4]

/_' No [-9.4,9.5]

0AF No [-34.4,49.3]

_N No [-34.7,40.4]

I_Vl No 6X.0

Iz0NI No 129.0

INEQUALITYCONSTRAINTS

Requirement Enabled Extreme Values

1_5_'_1 Yes 0.6

Izx_'l No 7.5

labial No 3.1

I_1 No 5.3

30
STRESS TEST RUN SSQ5

TOTAL CASES EXECUTED 1,000,000

TEST CASE GENERATION

Pitch Command Waveform SQ(20,40,20,40,60,-30)

Pitch Command Noise Pitch Response Noise

Distribution N(7.5,2.5) Distribution N(0.0,0.0)

Truncation Points [5.0,10.0] Truncation Points [0.0,0.0]

Seed 1234567808 Seed 9876542464

CONTROL LAW PARAMETERS

a0=.000 al =-167 a2 =.021 a3=.042

AIRCRAFT RESPONSE MODEL PARAMETERS

a 1 = 17.5 a 2 = 39.4 a 3 = 17.0

B1 = 1.02 132= 7.01 133= 18.70 B4 = 8.60 t35 = 2.84

SYSTEM PERFORMANCE REQUIREMENTS

Requirement Enabled Extreme Values

8EF No [-8.5,12.31

8_; No [-10.9,13.4]

0_ No [-32.3,53.7]

0AN No [-34.0,48.8]

I_1 No 77.0

IF-.oNI No 1_.0

INEQUALITY CONSTRAINTS

Requirement Enabled Extreme Values

It_i_l Y_ 0.7

I_1 No 9.5

IzX_l No 3.3

la_l No 6.6

31
STRESS TEST RUN SSQ6

TOTAL CASES EXECUTED 1,000,000

TEST CASE GENERATION

Pitch Command Waveform SQ(20,40,20,40,60,-30)

Pitch Command Not_ Pitch Response Noise

Distribution N(0.0,0.0) Distribution N(7.5,2.5)

Truncation Points [0.0,0.0] Truncation Points N[5.0,10.0]

Seed 1234567808 Seed 9876542464

CONTROL LAW PARAMETERS

a0 = .000 a 1 = .167 a2 = .021 a3 = .042

AIRCRAFT RESPONSE MODEL PARAMETERS

oq = 17.5 a 2 = 39.4 a 3 = 17.0

131= 1.02 132= 7.01 133= 18.70 134= 8.60 135= 2.84

SYSTEM PERFORMANCE REQUIREMENTS

Requirement Enabled Extreme Values

a_: No [-11.o,Io.5]

aN No [-14.0,II.2]

0AF No [-41.9,44.8]

0_' No [-45.1,44.1]

IE_I No 51.0

IEoNI No 102.0

INEQUALITY CONSTRAINTS

Requirement Enabled Extreme Values

IAai_'l V_s 0.5

l_a_'l No 9.5

It_a_:l No 4.8

lAaz_l No 10.5

32
STRESS TEST RUN SSQ7

• TOTAL CASES EXECUTED 1,000,000

TEST CASE GENERATION

Pitch Command Waveform SQ(20,40,20,40,60,-30)

Pitch Command NoLse Pitch Response Noise

Distribution N(7.5,2.5) Distribution N(7.5,2.5)

Truncation Points [5.0,10.0] Truncation Points [5.0,10.0]

Seed 1234567808 Seed 9876542464

CONTROL LAW PARAMETERS

a0 = .000 ai = .167 a2 = .021 a3 = .042

AIRCRAFT RESPONSE MODEL PARAMETERS

aI= 17.5 a2= 39.4 cx3= 17.0

131 1.02 132= 7.01 133= 18.70 134= 8.60 135= 2.84

SYSTEM PERFORMANCE REQUIREMENTS

Requirement Enabled Extreme Values

a_ No [-11.9,12.0]

B_" No [-15.4,16.1]

0mF No [-39.1,54.6]

_N No [-39.9,55.7]

IZ_l No 70.0

IZ0r"l No 140.0

INEQUALITY CONSTRAINTS

Requirement Enabled Extreme Values

I_ai_'_'l Yes 0.6

IAair'-_'l No 8.0

I_a_l:l No 5.3

IAa_'FI No 11.9

33
STRESS TEST RUN SSN1

TOTAL CASES EXECUTED 1,000,000

TEST CASE GENERATION

Pitch Command Waveform SINE (30,.16,0,0)

Pitch Command Noise Pitch Response Noise

Distribution N(0.0,0.0) Distribution N(0.0,0.0)

Truncation Points [0.0,0.0] Truncation Points [0.0,0.0]

Seed 1234567808 Seed 9876542464

CONTROL LAW PARAMETERS

a0 = .000 a1 = .167 a2 = .042 a3 = .084

AIRCRAFT RESPONSE MODEL PARAMETERS

a I = 17.5 a 2 = 39.4 a 3 = 17.0

131 1.02 132= 7.01 133= 18.70 134= 8.60 135= 2.84

SYSTEM PERFORMANCE REQUIREMENTS

Requirement Enabled Extreme Values

5EF No [-4.7,8.8]

/_' No [-6.9,8.1]

0mF No [-29.5,36.8]

0_x' No [-32.1,34.2]

I_1 No 47.0

IE_"I No 54.0

INEQUALITY CONSTRAINTS

Requirement Enabled Extreme Values

IAsi_'_'l Yes 0.7

IA_I No 1.1

IA_I No 1.3

NT lABEl No 2.7
STRESS TEST RUN SSN2

TOTAL CASES EXECUTED 1,000,000

TEST CASE GENERATION

Pitch Command Waveform SINE (30,.16,0)

Pitch Command NoL_ Pitch Response Noise

Distribution N(3.0,1.0) Distribution N(0.0,0.0)

Truncation Points [2.0,4.0] Truncation Points [0.0,0.0]

Seed 1234567808 Seed 9876542464

CONTROL LAW PARAMETERS

ao = .000 aI = .167 a2 = .042 a3 = .084

AIRCRAFT RESPONSE MODEL PARAMETERS

a I = 17.5 c,2 = 39.4 a 3 = 17.0

fll = 1.02 132= 7.01 133= 18.70 134= 8.60 135= 2.84

SYSTEM PERFORMANCE REQUIREMENTS

Requirement Enabled Extreme Values

S[ No [-5.8,8.7]

_" No [-8.8,9.8]

0AF • No i-30.0,38.6]

O_ No [-33.6,39.6]

Ir_l No 54.0

IZ0NI No 64.0

INEQUALITY CONSTRAINTS

Requirement Enabled Extreme Values

IA_i'_l Yes 0.7

la_?_'l No 1.4

It_Yl No 1.9

I_Xti_Vl No 4.5

35
STRESS TEST RUN SSN3

TOTAL CASES EXECUTED 1,000,000

TEST CASE GENERATION

Pitch Command Waveforrn SINE (30,.16,0,0)

Pitch Command Noise Pitch Response Noise

Distribution N(0.0,0.0) Distribution N(3.0,1.0)

Truncation Points [0.0,0.0] Truncation Points [2.0,4.0]

Seed 1234567808 Seed 9876542464

CONTROL LAW PARAMETERS

a0 = .000 a I = .167 a2 = .042 a3 = .084

AIRCRAFT RESPONSE MODEL PARAMETERS

a 1 = 17.5 a 2 = 39.4 ot3 = 17.0

131= 1.02 132= 7.01 133= 18.70 134= 8.60 135= 2.84

SYSTEM PERFORMANCE REQUIREMENTS

Requirement Enabled Extreme Values

_E No [-6.8,8.2]

5_' No [-9.8,8.3]

_F No [-33.5,35.0]

"N

0X No [-36.4,36.6]

lEVI No 44.0

IE_'_I No 51.0

INEQUALITY CONSTRAINTS

Requirement Enabled Extreme Values

IASi_tl Yes 0.7

la_p_'l No 1.3

IAS_a:I No 2.5

IA_E_I No 5.7
STRESS TEST RUN SSN4

TOTAL CASES EXECUTED 1,000,000

"I;ESTCASE GENERATION

Pitch Command Waveform SINE (30,. 16,0,0)

Pitch Command Noise Pitch Respon.,mNoise

Distribution N(3.0,1.0) Distribution N(3.0,1.0)

Truncation Points [2.0,4.0] Truncation Points [2.0,4.0]

Sccd 1234567808 Seed 9876542464

CONTROL LAW PARAMETERS

a0 = .000 al = -167 a2 = .042 a3 = .084

AIRCRAFT RESPONSE MODEL PARAMETERS

oq = 17.5 cx2 = 39.4 et3 = 17.0

131 1.02 132= 7.01 133= 18.70 134 = 8.60 135= 2.84

SYSTEM PERFORMANCE REQUIREMENTS

Requirement Enablcd Extreme Values

/iF No [-7.1,8.1]

5_" No [-11.3,11.4]

0AF No [-32.3,38.41

0aN No [-37.4,43.0]

131 No 52

IE_'I No 61

INEQUALITY CONSTRAINTS

Requirement Enabled Extreme Values

I_i_l Y_s 0.8

IA_tl No 1.5

IA_I No 3.1

NT [A/SE [ No 7.0

37
STRESS TEST RUN SSN5

TOTAL CASES EXECUTED 1,000,000

TEST CASE GENERATION

Pitch Command Waveform SINE (30,.16,0,0)

Pitch Command NoL*,e Pitch Response NoL*,e

Distribution N(7.5,2.5) Distribution N(0.0,0.0)

Truncation Points [5.0,10.0] Truncation Points [0.0,0.0]

Seed 1234567808 Seed 9876542464

CONTROL LAW PARAMETERS

a0 = .000 a1 = .167 a2 = .042 a3 = .084

AIRCRAFT RESPONSE MODEL PARAMETERS

a 1 = 17.5 ct2 = 39.4 ct3 = 17.0

131= 1.02 132= 7.01 133= 18.70 134= 8.60 135= 2.84

SYSTEM PERFORMANCE REQUIREMENTS

Requirement Enabled Extreme Values

_iFE No [-7.7,10.1]

/iN No [-14.6,17.7]

0AF No [-31.2,45.0]

6N No [-39.2,56.2]

IEoF{ No 68

Ir_NI No 84

INEQUALITY CONSTRAINTS

Requirement Enabled Extreme Values

1,__ii_'?"l Yes 1.3

Izx_'l No 2.2

It_8_a:l No 4.1

la_l No 9.9
STRESS TEST RUN SSN6

TOTAL CASES EXECUTED 1,000,000

TEST CASE GENERATION

Pitch Command Waveform SINE (30,.16,0,0)

Pitch Command NoLse Pitch Response Noise

Distribution N(0.0,0.0) Distribution N(7.5,2.5)

Truncation Points [0.0,0.0] Truncation Points [5.0,10.0]

Seed 1234567808 Seed 9876542464

CONTROL LAW PARAMETERS

a0 = .000 al = -167 a2 = .042 a3 = .084

AIRCRAFT RESPONSE MODEL PARAMETERS

aI= 17.5 _2= 39.4 _3= 17.0

131= 1.02 132= 7.01 133= 18.70 134= 8.60 135= 2.84

SYSTEM PERFORMANCE REQUIREMENTS

Requirement Enabled" ExtremeValues

/5_ No [-10.2,8.1]

6_" No [-18.0,14.4]

6_ No [-40.1,40.7]

0_ No [-49.5,48.8]

IEoFI No 18.5

IZ4_l No 29.5

INEQUALITY CONSTRAINTS

Requirement Enabled ExtremeValues

NN

[Abij [ Yes 1.3

It_6_'l No 2.1

[A6['TI No 6.0

,NT

IAsgl No 14.2

39
STRESS TEST RUN SSN7

TOTAL CASES EXECUTED 1,000,000

TEST CASE GENERATION

Pitch Command Waveform SLNE(30,16,0,0)

Pitch Command Noise Pitch Response Noise

Distribution N(7.5,2.5) Distribution N(7.5,2.5)

Truncation Points [5.0,10.0] Truncation Points [5.0,10.0]

Seed 1234567808 Seed 9876542464

CONTROL LAW PARAMETERS

a0 = .000 a1 = .167 a2 = .042 a3 = .084

AIRCRAFT RESPONSE MODEL PARAMETERS

ct1 = 17.5 a 2 = 39.4 a 3 = 17.0

131= 1.02 132= 7.01 133= I8.70 134= 8.60 135= 2.84

SYSTEM PERFORMANCE REQUIREMENTS

Requirement Enabled Extreme Values

_E No [-11.o,11.21

5EN No [-22.3,21.9]

(_FA No [-37.1,52.2]

0AN No [-51.7,66.1]

IZorl No 63

IEoNI No 77

INEQUALITY CONSTRAINTS

Requirement Enabled Extreme Values

laai_rl Yes 2.0

laa_'l No 2.5

laa_a:l No 7.7

It_aE_l No 17.4

4O
Standard Bibliographic ['age

I. Report No. I 2. Government Accession No. 3. Recipient's Cat;dog No.

NASA CR- 178058 f

4. Title and Subtitle 5. Report Date

March 1986

An EmpiricalStudyof FlightControlSoftware

Reliability 6. Performing 4120 Organization Code -2094-22

7. Author(s) 8. Performing Organization Report No.

Janet R. Dunhamand John L. Pierce

i9. Performing Organization Name and Address 10. Work Unit No.

ResearchTriangleInstitute

P.O. BOX 12194 11.ContracortGrantNo.

ResearchTrianglePark,N.C. 27709 NASI-17964

13. Type of Report and Period Covered

12. Sponsoring Agency Name and Address

ContractorReport

National_ronautics & SpaceAdministration

LangleyResearchCenter 14.SponsorinAgen g cCyode

N_mp_nn,VA 23665

15. Supplementary Notes

LangleyTechnicalMonitor:G. EarleMigneault

Final Report

16. Abstract

The reportdocumentsthe resultsof a small-scalelaboratoryexperiment

in flightcontrolsoftwarereliability.The experimenttestsa smallsample

of implementationsof a pitch axiscontrollaw for a PA28 aircraftwith over

14 millionpitchccmaandshavingvaryinglevelsof additiveinputand feedback

noise. The testingwhich uses themethodof n-versionprogramaingfor error

detectionsurfacedfour softwarefaultsin one implementationof the control

law. This shallnumberof detectedfaultsprecludedthe conductof the error

burst analysesand may be due to the overlysimpleexamplechosenfor the

controllaw softwareand the progranmer'sabilityto exploitthis simplicity

in validatingit.

The pitch axisproblemwas chosento providedata for use in constructing

a model for predictingthe reliabilityof softwarein systemswith feedback.

The experimentis one in a seriesof experimentsbeingpursuedby the Systems

ValidationBranchof NASA - LangleyResearchCenterto finda means of credibly

performingreliabilityevaluationsof flightcontrolsoftware.

17. Key Words (Suggested by Authors(s)) 18. Distribution Statement

SoftwareReliability Unlimited

N-VersionTesting

Life-CriticalSoftware

SoftwareEngineeringExperiments

19. Security reporCl_sif.(of t)unclassified this 20. Secu page)unclassified rity Classif.(of this ]21.No. 4 0ofPages[22.Price

For sale by the National Technical Information Service, Springfield, Virginia 22161

N.-kS,*_ l.atlgley Form 63 (JU|II' 19851