May 2011 

NASA/CR–2011-217147

Vehicle Integrated Prognostic Reasoner (VIPR) 

2010 Annual Final Report 

George D. Hadden, Dinkar Mylaraswamy, and Craig Schimmel 

Honeywell International Inc., Golden Valley, Minnesota 

Gautam Biswas, Xenofon Koutsoukos, and Daniel Mack 

Vanderbilt University, Nashiville, Tennessee 
NASA STI Program . . . in Profile

Since its founding, NASA has been dedicated to 

the advancement of aeronautics and space science. 

The NASA scientific and technical information (STI) 

program plays a key part in helping NASA maintain 

this important role. 

 The NASA STI program operates under the 

auspices of the Agency Chief Information Officer. It 

collects, organizes, provides for archiving, and 

disseminates NASA’s STI. The NASA STI program 

provides access to the NASA Aeronautics and Space 

Database and its public interface, the NASA Technical 

Report Server, thus providing one of the largest 

collections of aeronautical and space science STI in 

the world. Results are published in both non-NASA 

channels and by NASA in the NASA STI Report 

Series, which includes the following report types: 

TECHNICAL PUBLICATION. Reports of 

completed research or a major significant phase 

of research that present the results of NASA 

programs and include extensive data or 

theoretical analysis. Includes compilations of 

significant scientific and technical data and 

information deemed to be of continuing 

reference value. NASA counterpart of peerreviewed formal professional papers, but having 

less stringent limitations on manuscript length 

and extent of graphic presentations. 

TECHNICAL MEMORANDUM. Scientific 

and technical findings that are preliminary or of 

specialized interest, e.g., quick release reports, 

working papers, and bibliographies that contain 

minimal annotation. Does not contain extensive 

analysis. 

CONTRACTOR REPORT. Scientific and 

technical findings by NASA-sponsored 

contractors and grantees. 

CONFERENCE PUBLICATION. Collected 

papers from scientific and technical 

conferences, symposia, seminars, or other 

meetings sponsored or co-sponsored by NASA. 

SPECIAL PUBLICATION. Scientific, 

technical, or historical information from NASA 

programs, projects, and missions, often 

concerned with subjects having substantial 

public interest. 

TECHNICAL TRANSLATION. Englishlanguage translations of foreign scientific and 

technical material pertinent to NASA’s mission. 

 Specialized services also include creating custom 

thesauri, building customized databases, and 

organizing and publishing research results. 

 For more information about the NASA STI 

program, see the following: 

Access the NASA STI program home page at 

http://www.sti.nasa.gov

E-mail your question via the Internet to 

help@sti.nasa.gov

Fax your question to the NASA STI Help Desk 

at 443-757-5803 

Phone the NASA STI Help Desk at 

443-757-5802 

Write to: 

 NASA STI Help Desk 

 NASA Center for AeroSpace Information 

 7115 Standard Drive 

 Hanover, MD 21076-1320
National Aeronautics and 

Space Administration 

Langley Research Center Prepared for Langley Research Center 

Hampton, Virginia 23681-2199 under Contract NNL09AD44T 

 

May 2011 

NASA/CR–2011-217147

Vehicle Integrated Prognostic Reasoner (VIPR) 

2010 Annual Final Report 

George D. Hadden, Dinkar Mylaraswamy, and Craig Schimmel 

Honeywell International Inc., Golden Valley, Minnesota 

Gautam Biswas, Xenofon Koutsoukos, and Daniel Mack 

Vanderbilt University, Nashville, Tennessee 
Available from: 

NASA Center for AeroSpace Information 

7115 Standard Drive 

Hanover, MD 21076-1320 

443-757-5802 

Acknowledgements

We would like to thank the Honeywell and Vanderbilt teams for conducting the work 

under this program. In particular, we wish to acknowledge Howie Weibold for his many 

contributions, Darryl Busch, who kept us technically on track during the proposal and 

many reviews, and Dave Kolbet, Randy Magnuson, and Ginger Shao for their technical 

contributions to diagnostic, prognostic, and fusion algorithms. We would also like to 

thanks Hal Voges for his technical feedback and support, Craig Goodrich for helping us 

with all contract related issues, and Savit Boyd for her patience and her lessons on 

program management. 

The use of trademarks or names of manufacturers in this report is for accurate reporting and does not constitute an 

official endorsement, either expressed or implied, of such products or manufacturers by the National Aeronautics 

and Space Administration. 
Page5

TableofContents

1ExecutiveSummary...............................................................................................................................8

2IntroductionandBackground...............................................................................................................9

3Objectives/Approach..........................................................................................................................11

4VIPROverview.....................................................................................................................................12

5ProgressSummary..............................................................................................................................13

6YearOneDeliverables.........................................................................................................................15

6.1UserRequirements.....................................................................................................................15

6.2Honeywell7StepSMARTProcess..............................................................................................17

6.3Architecture................................................................................................................................18

6.4ReasoningMechanisms–ToolsandTechnologyConceptofOperations..................................20

6.5MessageProtocols......................................................................................................................21

6.6ConOpsandScenarios.................................................................................................................23

6.7VIPRDemonstration....................................................................................................................25

6.7.1VIPRDemonstrationInternals............................................................................................27

6.8MetricsforBenchmarkingVIPR..................................................................................................29

7RecommendationsforFutureWork...................................................................................................30

8PublishedDocuments.........................................................................................................................30

9ReferencedDocuments......................................................................................................................31

Page6

TableofFigures

Figure1.SystemBoundaryDiagram...........................................................................................................15

Figure2.AdverseEventsCubeforgatheringflightcrewrequirements.....................................................16

Figure3.Honeywell7StepSMARTProcess...............................................................................................17

Figure4.TheVIPRThreeLayerHierarchy..................................................................................................18

Figure5.AFaultCondition..........................................................................................................................19

Figure6.VIPRReferenceModelEntities....................................................................................................20

Figure7.FunctionalviewofVIPR...............................................................................................................21

Figure8.LayoutofaVIPRmessage............................................................................................................22

Figure9.VIPRConOpsforaslow,progressiveevent.................................................................................23

Figure10.VIPRConopsforawidespreadimpactevent...........................................................................24

Figure11.VIPRConOpsforfaultysensorinducedintermittentevent......................................................25

Figure12.VIPRDemonstration...................................................................................................................26

Figure13.CONOPSSimulationArchitecture..............................................................................................27

Figure14.ArchitectureSimulation.............................................................................................................28

Page7

TableofTables

Table1.AccomplishmentSummaryandCoursecorrectionswithrespecttoproposalconcepts.............14

Table2.FlightCrewRequirementsSummary.............................................................................................16

Table3.VIPRInstallerRequirementsSummary.........................................................................................16

Table4.MessageTypes..............................................................................................................................21

Table5.DetectionandDiagnosisMetrics..................................................................................................29

Table6.PrognosisMetrics..........................................................................................................................29

Table7.ComputationalMetrics.................................................................................................................29Page8

1 ExecutiveSummary

Honeywell’sCentralMaintenanceComputerFunction(CMCF)andAircraftConditionMonitoring

Function(ACMF)representthestateoftheartinintegratedvehiclehealthmanagement(IVHM).These

HoneywellproductsandtechnologiesarepurchasedbyairframemanufacturerssuchasBoeing,

Bombardier,andDassaultanddeployedontheiraircraft.Underlyingthesetechnologiesisafault

propagationmodelingsystemthatprovidesnosetotailcoverageandrootcausediagnostics.The

VehicleIntegratedPrognosticReasoner(VIPR)extendsthistechnologytointerpretevidencegenerated

byadvanceddiagnosticandprognosticmonitorsprovidedbycomponentsupplierstodetect,isolate,

andpredictadverseeventsthataffectflightsafety.

VIPRbringstogetheradvancesinsubsystemhealthmonitoringfromseveralongoingNASA,U.S.Army,

andAFRLprograms.Thisexperiencehasgivenourteamauniqueinsighttocharacterizeheterogeneous

anduncertainformsofevidencegeneratedfromthesesubsystemmonitorsandareasoningsystemto

correctlyinterpretthem.Wedefinedthedatastructuresandalgebraicoperatorsforthisinterpretation.

InyearoneofVIPR,welaidthetechnicalfoundationsforthisnextgenerationvehiclelevelreasoner

thatcanbeadaptedtoavarietyofuserrequirementsanddeployedwithinaircraftcomputational

constraints.

SignificantaccomplishmentsduringthefirstyearoftheVIPRprograminclude:

1. AbasicthreetieredframeworkhasbeendesignedandillustratedthroughananimatedConOps

demonstration(simulator).

2. Technicalrisksweremitigatedthroughacomprehensivesimulationofuserrequirements,

animatedConOps,andarchitectureflow.

3. Deploymentrisksweremitigatedthroughtheextensionofexistingstateoftheartdiagnostic

systemsextendedtohandleheterogeneousevidenceandprovideprognosticconclusions.

Webelievetherearetwoimportantelementsformovingtheunderlyingtechnologyintoproducts

expeditiously.First,itmustaddresssomeofthesafetygapsthatexisttodayoruserneeds.Second,

theremustexistapathwayforrealizingVIPRascosteffectiveextensionstoexistingaircrafthardware

andsoftware.UsingtheASIASdatabase,weidentifiedfoureventsasourdemonstrationscenarios.By

itsverydefinition,thecurrentvehiclelevelreasonerwasnotabletodetecttheunderlyingfaulteventin

allfoursituationsandhenceresultedassafetyincidents.Aspartofourdemonstration,wescriptedhow

VIPRwithitsprognosticabilityandadvancedreasoningcapabilitycannotonlydetecttheseevents

accurately,butalsoallowsufficienttimefortheflightandmaintenancecrewtoreactandavoidthe

safetyescalation.AnanimatedconceptofoperationsallowedvarioususerstovisualizehowtheVIPR

systemcanaddresstheiruniqueneeds.

Further,usinganexpandedsetofARINC624encodedmessages,wealsodemonstratedhowtheVIPR

canberealizedasextensionstotheexistingAircraftConditionMonitoringFunctionandtheonboard

DiagnosticReasoner.Ourprognosticreasoningformulationreusesexistingdiagnostic(faultpropagation

models)toaverylargeextent.Abstractionofevidencegeneration(monitors)providesaclearandPage9

practicalwayfor3rdpartiestoembedtheirknowledgeandthusprovideVIPRenrichedinformationfor

vehiclelevelinterpretationandreasoning.Withinthesoftwarebasedemulatorenvironmentwewere

alsoabletodemonstratehowvariousadvancedreasoningfunctionscanbedistributedtoaccommodate

availableaircraftcomputationresources.Thisisafirststepforcommunityacceptanceandhelpsusto

movetheVIPRtechnologyintoproductsexpeditiously.

2 IntroductionandBackground

Animportantchallengefacingaviationsafetytodayissafeguardingagainstsystemandcomponent

failuresandmalfunctions.Faultscanariseinoneormoreaircraftsubsystem;theireffectsinonesystem

maypropagatetoothersubsystems,andfaultsmayinteract.Theprimaryfunctionofavehiclelevel

reasoneristodetectfaultsandfailuresattheaircraftlevel,enableisolationofthesefaults,andestimate

remainingusefullife.Allthesefunctionsareaimedatmeetingthegoalofautomatedmitigationand

increasingaviationsafety.

Considercharacteristicsofsometypicalfaultsarisinginsomesubsystemswithinanaircraft:

1. [propulsion]Turbinebladeerosion.Thiserosionisanaturalpartofturbineagingandwearingof

theprotectivecoatingduetomicroscopiccarbonparticlesexitingthecombustionchamber.As

theerosionprogressesovertime,itstartstoaffecttheabilityoftheturbinetoextract

mechanicalenergyfromthehotexpandinggases.Eventuallythisfaultmanifestsitselfas

increaseinfuelflowandgradualdegradationofengineperformance.

2. [avionics/software]Loosewireharnessconnectors.Asconnectorpinscorrode,theymake

intermediatecontact.Thecorrespondingsoftwaremodulethatreceivesthissignalregistersa

seriesofintermittentopencircuitfaults.Eventuallythiscorrosionprogressestoapointwhich

resultsinanopencircuitfailure.Baddatafromthischannelcorruptsthenavigationsoftware

andcausesamemoryoverflowinstantaneously.

3. [airframe]Actuatorstiction.Astickingactuatorchangesthedynamicresponseofacontrolloop.

Thefeedbackactionprovidessomedegreeofresiliencemakingthisproblemdifficulttodetect.

Butitdoessteadilydecreasingthecontrolloop’sabilitytomeetsetpointcommands.Eventually,

thestictionprogressestoapointwheretheactuatorwillbecomenonresponsive.

4. [software]Thisscenariodescribesafastprogressionfaultinwhichtheincomingnavigationdata

corruptstheguidancesoftware(seeATSBInvestigationreport200503722),whichthenleadsto

anincorrectsolution.Theautopilotintervenesandovercompensatesusingtheenginethrust.

Thiscauseshightemperatureandhighspeedeventsintheengine,leadingtocascading

problemsinthegeneratorsandsecondarypowerdistributionsystem.Severalauxiliary

electronicsmodulesreacttothepowerglitch.

BroadlyspeakingtheVLRSneedstoaddressscenarioswherein(1)theunderlyingfaultprogressesboth

intimeandseverityand(2)theeffectsofafaultarefeltthroughouttheaircraftanditsoperations.

Morespecifically:

1. Faultswhoseseverityincreaseswithtime.Thesecanbefurthercategorizedbasedonthetime

constantofthisevolutionsuchasincipient,slowprogressionorfastprogression.

2. Binaryrepeatingfaultswhoserepetitionincreaseswithtime.Thesecanbefurthercategorized

basedonthetimeintervalbetweenrepeatssuchasconstantorincreasing.Page10

3. Faultswhoseeffectsspreadthroughouttheaircraftwithtime.Thesecanbefurthercategorized

basedonthesizeofthisinfluencesuchaslocalized(selfcontained)orwidespread.

Honeywell’sAircraftDiagnosticandMaintenanceSystem(ADMS)thatreasonsusingafaultpropagation

systemmodelisastateofartinvehicleleveldiagnosticreasoning.Ontheotherhand,theJointStrike

Fighter(JSF)PrognosticsHealthManagementSystemrepresentsastateoftheartingenerating

prognosticindicatorsatthesubsystemlevel.Interpretationoftheseprognosticindicatorsisimportant

tomeetthegoalofautomatedmitigationandincreasingaviationsafety.Ourprimaryresearchisto

extendvehiclelevelreasoningbyincorporatingtheseprognosticindicatorsanddesign,implementand

demonstrateavehiclelevelintegratedprognosticreasoner.WecallthisVIPR(VehicleIntegrated

PrognosticReasoner).

Evidenceprovidedbyaprognosticindicatorneedstobeinterpreteddifferentlyfromanevidence

providedbyabinaryon/offindicator.Mathematicalcharacterizationoftheseheterogeneousformsof

evidenceisanimportantpartofVIPRdesign.ThereasoningwithinVIPRneedstoaddressamultitudeof

timescalesinvolvedintheevidenceaswellthecoverageofaircraftsubsystemsandtheirinteractions.

Decomposingthisreasoningintosmallinferencingstepsisnecessarytomanagecomplexity.Withthe

introductionofnewaircraftandretrofitofcurrentplatforms,acleararticulationofthearchitecture

optionsismoreimportantthantheunderlyingreasoningtechnologies.

Dataminingandmachinelearningtechniquesprovidetheprimarymechanismforcharacterizing

interactionsbetweencomponents,subsystems,andpotentialcausalchainsofadverseeventsthat

impactsafety.TheunderlyingalgorithmssupportVIPRprogramgoalsby(1)establishingtheparametric

relationship(probabilities,coefficients,etc.)associatedwithvariousentitiesinVIPRfaultpropagation

systemmodeland(2)discoveringnewrelationshipsfromoperationaldata.Often,thelimitingfactoris

availabilityofrealisticdata.Thedatanecessaryforthisactivityneedtoretainstatisticalrichnesswhile

maintainingprivacyandproprietaryrestrictions.

WhiledesigningVIPRpresentsuniqueresearchchallenges,thesafetybenefitsfromavehiclelevel

reasonercanonlyberealizedfromitsacceptancewithintheaviationcommunity.Therearetwo

importantelementshere:(1)articulationofuserrequirementsand(2)demonstratinghowVIPRdetects

andpredictsfaultsandfailuresbeforetheyescalatetoflightsafetyincidents.

Developinganextgenerationvehiclelevelreasonerembodiesseveralrisks.

1. Inferencingoperatorsanddatamodeldesignforprognosticreasoningpresenttechnicalrisk;

2. Nonavailabilityofrealdatapresentcredibilityrisk;

3. Impropercaptureofuserrequirementspresentspracticalrealizationrisk;

4. Lackofclarityinthe“endstate”andsafetyimpactconstitutesadoptionrisk.

Itisimportanttoaddressesalltheserisksbefore“buildingtheVIPRsolution”.Thisriskreductionstep

wastheprimaryobjectiveofoureffortandthisreportsummarizesourprocessanddeliveredartifacts

thatculminateinasetofrecommendationsandfuturetasksforrealizingapracticalVIPRwithhigh

degreeofsuccess.WebeginwithanoverviewofVIPRinsection3.ProgressanddeviationsmadefromPage11

theVIPRconceptsdescribedintheoriginalproposalisdescribedinsection5.Documents,artifactsand

demonstrationsthatcontributetowardsriskreductionaredescribedinsection6.Weconcludethis

documentbysummarizingfuturestepsfordesigning,implementinganddemonstratingasuccessful

VIPR.

3 Objectives/Approach

ObjectivesfortheVIPRprogramflowedfromtheoverarchingNASAobjectiveofachievingahigherlevel

ofaircraftsafetythroughanembeddedVehicleLevelReasoningSystem.Ouryearoneworkincluded

definingthearchitectureandcommunicationprotocolsandestablishinguserrequirements.Basedon

theseandasetofscenariosdefinedinourConOpsdocument,wedesignedandimplementeda

demonstrationusingHoneywell’sSMARTlabsimulationfacilities.Inadditiontodemonstratingthe

communicationpathwaysandthethreetieredhealthmanagementarchitecture,scriptedscenarios

showVIPR’sabilitytodetectadverseeventsbeforetheyescalateassafetyincidents.Thisdemonstration

testbedisdesignedsothatfutureworkcanaddreasoningsoftwareforprognosticsanddiagnosticsand

later,actualaircrafthardware.

TheyearoneobjectivesincludedthosementionedaboveaswellasmakingavailabletotheIVHM

communityalargesetofdataacquiredfromtheMesabaBAeRJfleetandwererealizedthroughthe

performanceofthefollowingmajortasks:

ArchitectureRecommendations.ProduceanddocumentrecommendationsfortheVLRS

architectureaddressingareassuchasdatatransferprotocols,speeds,andcommunications

requirementsforairframe,propulsion,aircraft,andsoftwaresubsystems.The

recommendations,requirements,andassociatedmetricsshouldbebasedontheneedsofthe

usercommunity.

InformationProtocol.Developahealthmanagementinformationprotocolthatincludes

requirementsfortheinformationandformatsneededtobepassedthroughalllevelsofthe

VLRS.

ConceptofOperation.ProvideaconceptofoperationsoftheVLRSincludingastudyofthe

tradespacebetweencomplexity,accuracy,cost,andimpactonaviationsafety.Thetradespace

betweenthenumerous(andsometimesconflicting)userrequirementsandthecustomer’s

desiretominimizecostshouldbeclearlydocumented.

UserRequirements.DevelopacomprehensivesetofuserrequirementsforConditionBased

MaintenanceandtheapplicationoftheVLRStoenableappropriatepredictivemaintenance

basedonafleetmanagementperspective.DocumentinaNASATechnicalManuscriptorother

peerreviewedpublication.

MetricsRecommendations.ProviderecommendationsregardingappropriatemetricsforCBM

inthecontextofallofthesubsystemsmentionedaboveanddiscusshowtheproposedVLRSPage12

addressesthosemetrics.DocumentinaNASATechnicalManuscriptorotherpeerreviewed

publication.

ToolsandTechnologyConceptofOperations.ProvideaconceptofoperationsoftheVLRStools

andtechnology,describingthepotentialcostbenefittradeoffsintermsofCBMforarealworld

aircraftthatcanbeenabledbytheVLRS.Requirementsandcostbenefitanalysisshouldbe

documentedwithrespecttouserrequirementsthattheyaresupportingortradingoff(logistics,

maintenance,flight,fleetmanagement,training,etc.)DocumentinaNASATechnical

Manuscriptorotherpeerreviewedpublication.

Demonstration.Demonstratetheproposedconceptofoperationinasoftwaresimulationfora

subsetofseededfaults(selectedfromtheTable2AdverseEventsTableIVHMTechPlan)ina

vehicleconfigurationconsistingofatleastthreedifferentsubsystems.

Ourrecommendationforfutureworkincludesthemigrationofthedemonstrationsystemtotheuseof

diagnosticandprognosticreasoningsoftware.Werecommendthissoftwarebeaugmentedandtuned

usingdataminedfromtheMesabadataandothersources.Eventually,hardwareshouldbeinserted

intotheVIPRsysteminordertodemonstrateitscapabilitiesonrealworldproblems.Metricsshouldbe

definedandappliedtotheVIPRsystemtodiscovertowhatextent(ifany)itissuperiortoexisting

systems.

4 VIPROverview

SimilartoaCMC,VIPRhasseveralusers.Inyearone,wefocusedontheflightcrewasprimary

consumersofVIPRoutputs.Thesecondsetofusersarelinemaintainersandrepairdepotmaintainers.

ThethirdsetofusersincludesthesystemsintegratorsresponsibleforinstallingandmaintainingVIPR.

Flightcrewrequirementsincluderecognitionofconditionsthatmaycauseanadverseevent,mappingit

tofunctionaleffects,andverifyingthatthedesignedcontingency(ifany)isworkingproperly.Systems

integratorrequirementsincludeclearseparationofevidencegeneration(calledmonitorsandsupplied

bycomponentmanufacturers),aircraftconfiguration,andacommoncodebaseforminimizing

certificationcostsaswellasahierarchicalarchitecturethatcanbedeployedwithinaircraft

communicationandcomputationconstraints.

Itisnotsurprisingthattheseuserrequirementsimplyaneedfordifferentviewsofthesituation.VIPR

solvestheseproblemsbystartingwithawelldefinedseparationbetweenevidencegeneration,a

referencemodelthatencodesaircraftspecificconfigurationdata,andagenericplatformagnosticDP

(diagnostic/prognostic)reasonertoprovideacommoncodebasethatallowsforonetimecertification.

RecommendationsthatcenteronallowingLRUstointerfacewithVIPRwhileallowingtheir

manufacturerstomaintaincontroloftheirintellectualpropertyweredocumentedasadeliverablein

yearone.

Toaddressthespectrumofeventsthatadverselyaffectaviationsafety,theunderlyingreasoning

algorithmmustworkonenrichedevidencegeneratedbyproprietarymonitorproviders.WhileVIPR

doesnotcareabouttheinternalproprietaryknowledge,anabstractionintosimple,multivariate,Page13

multiclass,andprognosticmonitorsallowsVIPRtoformalizetheuncertaintyandheterogeneity

associatedwiththecollectedevidence.MotivatedbyourworkontheArmy’sFutureCombatSystems

(FCS)PlatformSoldierMissionReadinessSystem(PSMRS)program,wedefinedfaultconditionasa

fundamentaldatastructurewithinthereasoningprocess.Thepersistentsetofthesedatastructures

maintainedwithintheVIPRsoftwarealongwiththeirattributesestablishestheprevailingfault

hypotheses(foramaintainer),functionaleffects(forflightcrew),andmonitorsofinterest(foractive

faultisolationanddatacapture).Thisdefinitionisaccompaniedwithasetofoperatorsforcreating,

merging,splitting,resolving,andclosingthesefaultconditionsasnewevidencearrives.

Thediagnosticandprognosticprocessesreducetoasetofconfigurablemetarulesthatappliesthese

operatorswheneverapieceofevidenceisgenerated.Applyingtheseoperatorsrequirescomputational

resources.However,unliketheCMC,whereallthesecomputationsaredoneatacentrallocation,VIPR

includesahierarchicaltieredanddistributedarchitecture.Thisenablessubsetsoftheseoperationsto

beappliedatthecomputationallymostsuitablelocationwithintheaircrafttomeetthetimelinessneed

ofdetectingfastadverseevents.Theneedforinformationanddatapassingismetbydefiningmessage

passingprotocolsbasedonARINC624encoding.

VIPRembodiesnewconceptsandnewtechnologies.Validatingthesedefinitionsearlyonisnotonly

importanttoincreasingthelikelihoodoftechnicalsuccess,butalsoimportantforearlyadoptionwithin

thecommunity.TheSMART(SimulationandModelingforAcquisition,Requirements,andTraining)

process(seeSection6.2)emphasizesintuitivevisualization,ensuringthatcustomers“see”theVIPR

architecturedesignelementsearlyandoften.Weconcludedyearonewithaseriesofanimated

conceptsofoperationsthatclearlyhighlightedvariousdesignconceptswithinVIPR.Eventspertainingto

theMesabaairlinefleetrecordedinAviationSafetyInformationAnalysisandSharing(ASIAS)database

providedusscenariosforvisualizingtheVIPRdesignelementsandbenefitswithrespecttosafetyand

maintenance.

AnalyzingactualflightdataprovidestherightlevelofvalidationformeasuringtheaccuracyofVIPR.Our

developmentofananonymizertoremoveproprietaryencodingallowsustodistributetheMesabadata

foranalysisanddatamining.FutureworkinvolvesquantificationofthereasoneraccuracyandtheVIPR

designtradespaceusingthisdataandmetrics.Section6.4presentsourrecommendationsforhowdata

miningcanbeusedtoenhancetheVIPRreferencemodel.

5 ProgressSummary

TheVIPRproposalwasbuiltonsevenkeyconcepts(Table1).Threeoftheseconcepts(faultcondition

constructandoperations,systemreferencemodel,monitorandevidenceabstraction)wererelatedto

thereasoningalgorithm.TheSMARTprocessconceptallowsvisualizationoftheVIPRdesignelementsas

theyevolvenotonlyfortradestudiesbutalsoforearlyadoptionwithinthecommunity.Mesabadata

anddataminingconceptsprovidedthenecessarytoolstocontinuallyrefinethereasoneranddiscover

newknowledge.Activitiesinyearonefleshedoutthedesigndefinitionsandrequirementsand

shortcomings.TheanimatedConOpsdemo(step2intheSMARTprocess)notonlyhelpedusto

communicatethedesignthroughvisualization,butalsoallowedustozeroinongapsandmakecoursePage14

corrections.TheVIPRdesignnowstandsonasolidgroundandwecansaythatwewillbeenteringthe

implementationphase(yeartwo)withhighdegreeofconfidencetomeetallourproposedgoals.Table1

summarizesthekeyconcepts,progresstodataandcoursecorrectionsfromyearone.

Table1.AccomplishmentSummaryandCoursecorrectionswithrespecttoproposalconcepts

Proposal

Elements 

Progress to date Course Correction/Lessons 

Fault condition 

construct and 

operations 

Definitions complete. Salient features 

captured in the VLRS Concept of 

Operations document, CDRL 4.1.04.

On track to implement them within the 

SMARTlab simulator 

We expanded our original evidence set to 

include human provided evidence. 

Correspondingly we expanded the 

operations on fault conditions to interpret 

these TWO forms of evidence. 

System 

Reference Model 

Definitions and requirements capture 

complete. Described in User 

Requirements, CDRL 4.1.05. On track 

to instantiate a reference model for the 

propulsion, bleed, avionics and aircraft 

actuator subsystems. 

We discovered the need to extend the 

System reference model to include "data of 

interest" elements. During configuration 

time, this allows the VIPR installer to 

specify the sensors used by monitors. This 

information allows the reasoner to capture 

data with the onset of a primary evidence, 

and plays an important role in sensor fault 

isolation. 

Monitor and 

evidence 

abstraction 

We defined six forms of evidence 

heterogeneity. We also defined an 

abstraction for capturing uncertainty 

associated with these monitors without 

exposing proprietary knowledge to 

vehicle level reasoning complete. 

Captured in Architecture 

Recommendations, CDRL 4.1.02.

The monitor abstraction proposed originally 

could not handle evidence provided by 

humans. We extended the abstraction to 

include TWO forms of human monitors: 

loss of function and loss of asset.

Tiered & 

distributed 

architecture 

We defined messaging protocols to 

support distributed reasoning. 

Implementation of these message 

passing protocols within the simulator 

is complete. The protocol is defined in 

CDRL 4.1.03.

We discovered that ARINC 624 has proven 

precedence on commercial aircrafts to 

support diagnostic messages. We decided 

to adopt this protocol and expanded it to 

include information content requirements 

derived from the AFRL program ISHMAD 

[Jambor]. 

SMART Process Demonstrated the following steps of 

the process--Animated ConOps, 

architecture flow-- for four scenarios 

spanning five aircraft subsystems 

(propulsion, bleed, avionics, actuators, 

and software). Summary of these 

scenarios are described in CDRL 

No significant course corrections. 
Page15

4.1.04. User requirements were 

captured in a document, CDRL 4.1.06.

Mesaba Fleet 

data

We identified ten incidents recorded in 

the ASIAS database and relevant 

ACMF data from the Mesaba fleet. The 

ACMF data provides 1—16Hz aircraft 

parameters spanning at least 40 flights 

before and after the events. 

We completed the data anonymizer 

that allows us to make this data 

available to the VIPR program and 

NASA.

The Mesaba archive does not include data 

specifically captured for supporting 

software health monitoring. Our 

recommendation is to simulate these faults 

based on historical scenarios. 

6 YearOneDeliverables

6.1 UserRequirements

Figure1showsthesystemboundarydiagramforVIPR.Thefigureidentifiesvarioususersthatwill

interactwithavehiclelevelreasonersuchasVIPR.Theusersincludebothconsumersofinformationas

wellasprovidersofinformation.

PrimaryusersofVIPRinformationconsideredin

thisreportareshownusingsolidcircles.These

include:(1)theflightcrewthatisoperatingthe

aircraftandtheirrequirementstodetect

adverseeventsandmitigateeffectsofsuch

eventstoincreaseaviationsafety,(2)theVIPR

installerwhoisresponsibleforassemblingand

installingtheVIPRsystemfortheaircraft,and

(3)theVIPRmaintainerwhoisresponsiblefor

performanceevaluationandcontinualupgrades

toreflectchangingaircraftconfigurations. Figure1.SystemBoundaryDiagram

Secondaryusersinclude(1)providersofdiagnosticandprognosticmonitors(e.g.LRUmanufacturers),

(2)thegroundmaintainerresponsibleforperforminginspectionsandrepairactions,and(3)theaircraft

controlsystemsforsemiautomaticandautomaticmitigationinresponsetodetectionofadverse

events.

Onthistask,wedevelopedanovelmechanismfordescribingadverseeventsinthevehicle.Page16

Figure2.AdverseEventsCubeforgatheringflightcrewrequirements

SeeFigure2.Thismechanismconsistsofa

cubewiththreemutuallyorthogonal

axeslabeledtimeevolution(with

extremeslabeledfastandslow),impact

propagation(withextremeslabeled

localizedandwidespread),andsymptom

persistence(withaxeslabeled

intermittentandconstant).Webelieve

pointsinthisspacecorrespondwellto

eventsthatVIPRneedstoaddressto

increaseaviationsafety.

Table2summarizeskeyflightcrewrequirements.

Table2.FlightCrewRequirementsSummary

EventTypeTopLevelrequirements

(Flightcrew)

TimeEvolution

Slow1. Lessimportant.

2. Important,ifandonlyifitwillaffectthecurrentflight.

Fast1. Veryimportant.Earlydetectionofincipientconditions.

2. Quicklyidentifymitigation(couldbeautomaticcontrol)actions

Impact

Propagation

Localized1. Lessimportant.

2. Confirmandmonitorifredundancyisworkingasdesigned

Widespread1. Minimizeinformationoverloadtoavoidconfusion.

2. Suppressinformationpresentation,donotremovetheevidence.

Symptom

Persistence

Constant1. Reducefalsealarms.

2. MinimizesizeofAmbiguitygroupandrankorder.

Intermittent1. Accuratedetectionandestablishthatintermittencyistrue.

2. Identifyingarootcausemaynotbeimportant

KeyrequirementsforaVIPRinstalleraresummarizedinTable3.

Table3.VIPRInstallerRequirementsSummary

TopLevelrequirements

(VIPRInstaller)

Scalability

1. Separatethereasoningalgorithmsfromaircraftspecificconfigurations.

2. Acommoncodebaseiseasytovalidateandmakesiseasiertocertify.

3. Finitesetofoperations,eachofwhichisboundedcomputationally.Page17

Deployment

1. Reasoningfunctionneedstofitonavailableonboardhardware.

2. SupportLRU’sthatdonothavecomputationalresourcesforgeneratingmonitors.

3. VIPRshouldworkwithintheintellectualpropertyboundariesofamonitor

provider.

4. Unambiguousdefinitionofmonitortypestoavoidmisinterpretation.

Accuracy

1. Abilitytohandlemultipletimescales.Timestampofevidenceisimportant.

2. Mustinclude‘states’(necessaryandsufficientdescription)thatcanbearchived

andusedasinitialconditionsforanalysisacrosssuccessiveflights.

3. Statesaretrackedusingprobabilitiesandwelldefined‘update’operations

4. Capableofproposingandworkingwithmultiplefaulthypotheses.

AversionofthisreportdeliverablewasalsopublishedaspartofaninvitedpaperintheAIAA

Infotech@Aerospace2010Conferenceentitled“ArchitecturesforIntegratedVehicleHealth

Management”byTimFelke,GeorgeD.Hadden,DaveMiller,andDinkarMylaraswamy.

6.2 Honeywell7StepSMARTProcess

VIPRembodiesnewconceptsandtechnologiesthatintegrateandreasonaboutdatacapturedfrom

multiplesubsystemsinordertodetectapotentialadverseevent,diagnoseitscause,andpredictthe

effectofthateventontheremainingusefullifeofthevehicle.Validatingtheseconceptsearlyonisnot

onlyimportanttoincreasethelikelihoodoftechnicalsuccess,butalsoimportantforearlyadoption

withinthecommunity.TheSMART(SimulationandModelingforAcquisition,Requirements,and

Training)processdevelopedbytheUSArmyisasystemslifecyclemodelingenvironmentthat

Figure3.Honeywell7StepSMARTProcess
Page18

emphasizesintuitivevisualization,ensuringthatcustomer“sees”thearchitecturedesignelementsearly

andoften.

The 7 steps are, as shown inFigure3:

1. System Benefits Model: An early lifecycle, low fidelity model that demonstrates the utility of a 

system or process. This step often explores the cost to benefits tradeoffs. 

2. Animated Concept of Operations (CONOPS) Model: A visual, animated model or prototype that 

illustrates the system operation. This model is used to confirm the project approach and customer 

expectations and acts as a concrete representation of the system requirements. 

3. Architecture Flow Model: An interactive model that defines the interactions and information flow 

among system components. This model defines the roles and interfaces between subsystems. 

4. Detailed Design Emulation: A high fidelity model that represents the final system. It provides 

validation of algorithms and system operation prior to major purchases or development activities. 

5. Integration Testbed: A simulated environment capable of interfacing with real and simulated 

subsystems for integration of real assets as development matures. 

6. System Test Simulation: A combination of real and simulated components that provides a 

realistic test environment without the risk or cost of a live test. 

7. Training Systems: High fidelity models that may include real or simulated components, used to 

train operators in a controlled environment. (Note: Step 7 is not currently within VIPR’s scope.) 

The result of each step flows into the next, so that each step expands on and refines the models of 

previous steps. Each step of the process can be employed iteratively and recursively throughout the 

development cycle. For example, if the animated CONOPS model exposes requirements issues, the 

model would be iteratively refined until the issues are resolved. Once the model accurately reflects the 

system, the requirements are updated to match the model and the design process continues. 

6.3 Architecture

Architecturerecommendationsfellintofourcategories:ModularSolution,SystemIntegration,

ReasoningAlgorithms,andEvaluationMetrics.

AnimportantrecommendationintheModularSolutioncategoryisto

usetheISO13374(OSACBMOpenSystemsArchitecturefor

ConditionBasedMaintenance)functionaldecompositionasabaseline

fordefiningtheVIPRprocessingblocks.Anadditional

recommendationfromthissectionistobaseVIPR’sinternal

communicationprotocolonthatdevelopedfortheAFRLISHMAD(Air

ForceResearchLabIntegratedSystemHealthManagement

ArchitectureDesign)updatedtobeconsistentwiththeARINC624

standard(seeMessageProtocolssection).

ThemostimportantoftheSystemIntegrationrecommendationsis

thattheVIPRarchitecturebebuiltonathreelayerhierarchy(see

Figure4).TheselayerscomprisetheLRUHealthManageratthe

lowestlevel,theAreaHealthManager(concernedwithinteractions

Figure4.TheVIPRThreeLayer

HierarchyPage19

withinsubsystems,e.g.theengine),andtheVehicleHealthManager(whichallowsVIPRtodraw

conclusionsbasedoneventsinseparatepartsofthevehicle).OffvehicleservicesarebeyondVIPR’s

currentscope,howeverwerecognizetheimportanceoftheseservicesandwillavoiddesigndecisions

thatmakethiscapabilitydifficulttoadd.

OtherSystemIntegrationrecommendationscenteronallowingLRUstointerfacewithVIPRwhile

allowingtheirmanufacturerstomaintaincontroloftheirintellectualproperty.Thisisdonebydefining

aninterfacetoVIPRusingmonitorstocarrydiagnosticandprognosticinformationfromtheLRUs.LRU

internaloperationsneednotbevisibletoVIPR.Inthissection,wealsorecommendthatthe

communicationprotocolscalledforintheModularSolutioncategorybedistributedasanopensource

library.

Figure5.AFaultCondition

AkeyAlgorithmrecommendationistousethefaultconditionasafundamentaldatastructure.Fault

conditionscontainasetoffailuremodes(calledtheambiguityset),exactlyoneofwhichisassumedto

beoccurring.FaultConditionsalsocontaintheset(calledthe“MonitorsofInterest”ofallmonitorsthat

mightfireifanyofthefailuremodesintheambiguitysetweretooccur.Figure5showsthisgraphically.

Multiplesimultaneousfaultscanbediagnosed–andprognosed–usingmultiplefaultconditionseachof

whichmaintainsitsownambiguitysetandasetofevidencetolookfor.

TheEvaluationMetricssectiondefinessixmetrics:timetodetection,detectionaccuracy,timetoisolate,

sizeoftheambiguityset,falsealarms,andmisseddetections.Ourrecommendationfromthissectionis

toleveragepreviousAirForcework(asdescribedabove)aswellasdiagnosticsandprognosticsefforts

ontheArmy’sFutureCombatSystemsPlatformSoldierMissionReadinessSystem(FCSPSMRS).

Fault 

Condition

Primary 

Monitor

Ambiguity Set

Failure Mode 1

Failure Mode 2

Failure Mode 3

Monitors of

Interest
Page20

ThesubsystemsthatVIPRaddressesincludePropulsion,Avionics,Airframe,andSoftware.

6.4 ReasoningMechanisms–ToolsandTechnologyConceptofOperations

TheprimaryfunctionalunitsoftheVIPRarchitectureare:(1)thereferencemodelsthatcontainthe

informationthatthereasoningalgorithmsusetoderivediagnosticandprognosticconclusions,(2)the

messagepassingprotocols(describedmorefullyinthereportdeliverableforTask4.1.03),and(3)the

layered(LRU,Area,andVehiclelevel)diagnosticandprognosticreasoners.Detailsofthealgorithmsat

thearealevelaredescribed,aswellasfusionalgorithmsthatunifytheresultsatthevehiclelevel.The

approachtothetwowayinteractionsbetweentheproprietaryLRUmonitorsandthereasoners,i.e.,

bottomupinformationpassingandtopdownqueryingtorefinediagnostichypothesesispresented.In

addition,thereportdiscussesappropriateevidencecombinationschemesforrepresentingand

reasoningwithuncertaindata.Thisreportthenoutlineshowthenewlayeredreasonersimpactthe

ConceptofOperationsofaircrafthealthmanagementsystemsandprovidesanexample.

Figure6.VIPRReferenceModelEntities

Figure6IllustratesanumberofentitiesandrelationshipscapturedintheVIPRReferenceModel.Figure

7showshowthevariousfunctionalelementsofVIPRmapintothethreetieredarchitecturedescribed

above.

Schemesforcontinuallyimprovingthereasoningalgorithmswithoperationalfielddataareoutlinedin

[Biswas,6/2010]section7,andrecommendationsaremadeforsomeoftheconsiderationsin

preparationforafutureVIPRdataminingtask.

Withmorepreciseknowledgeofthefaultconditionstructuresandreferencemodel,werecommend

usingTreeAugmentedNaïveBayesianNetwork(TAN)structurestolearnnewrelationsratherthan

generalcausaldiscoveryalgorithms,suchasTETRAD.TANstructuresareinterpretable,modifiable,and

moreeasilyderived.CombiningTANstructureswithlocalcausaldiscoveryalgorithmsprovidesthe

frameworkforcontinuallyimprovingthereasoningalgorithmswithoperationalfielddata.Page21

Figure7.FunctionalviewofVIPR

6.5 MessageProtocols

VIPRoperateslargelybypassingmessagesthroughoutitssubsystems.Wehavebasedthesemessages

onthosedevelopedfortheAFRLISHMADprogramandontheARINC624standard.VIPRcontainsseven

basictypesofmessage(Broadcast,Command,Event,Query,CommandResponse,EventResponse,and

QueryResponse)aswellasasimulationspecificmessagetypeusedfordemonstratingVIPR

functionality.ThesemessagesarelistedinTable4.

Table4.MessageTypes

Message

Type

ARINC624

equivalent

Description

BroadcastPeriodic

Report

Broadcastmessagesareofinteresttomultipleelementsandcontainsuch

informationasflightphaseandtime.

CommandCommand

ACTION

CommandmessagestooperatethevehicleareissuedfromVHMand

maintenancecrew.Acknowledgmentissentfromreceiverandoften

containsdataresponse.

EventEvent

REPORT

Anomaliesaredetectedandsenttohigherlevelhealthmanagersas

events.Messagescontainoriginator,eventtype,time,location,analysisPage22

andsupportingdata.IncludesStatus,Capability,Maintenance,andEvent

Observe/Orient/Decidemessages.

QueryParameter

GET

Querymessagescanrequestadditionaldata.

Command

Response

Command

RESPONSE

Acknowledgesthereceiptofacommand.Canincludedataconfirming

theresultsofthecommand.

Event

Response

EventAckAcknowledgesthereceiptofaneventmessage.

Query

Response

Parameter

STATUS

ProvidesthedatarequestedbyaQuerymessage.



SimExec SimulationspecificmessagesfordemonstratingtheVIPRfunctions.

Eachmessagehasastandard(common)header,aspecificmessagesubheader,datapayloadanda

signature.Thecommonheadercontainstoplevelinformationsuchasthesender,destination,time,

uniquenumber,andmessagetype.Puttingthisinformationinacommonmessageheaderensuresthat

themessagescanbedeliveredtotheirintendeddestinationandinterpretedcorrectlynomatterwhat

protocolisusedtosendthem.Thetimestampandmessagenumberfieldsalsopromotetraceabilityof

themessages.Mostmessagesarefurtherdefinedwithasubheaderthatprovidesadditional

information.TheheaderandsubheaderareencodedusingARINC624protocols.Themaximumsizefor

asingledatapayloadis64KB.However,multiplemessagescanbe“chainedtogether”toaccommodate

datagreaterthan64KB.Figure8showsthelayoutofaVIPRmessage.

Figure8.LayoutofaVIPRmessage

Thechronologyofeventscanbeveryimportanttothereasoningfunction.Whileatimestampis

includedineverymessageheader,webelieveadditionaltimestampdatamaybeneededforreasoning

aboutfastprogressionfaults.Therefore,anadditionalsamplingtimeisincludedineacheventsub

headertopreciselydefinewhenthatparticulareventoccurred.However,VIPRdoesassumethat

timekeepingiswellsynchronizedacrossallsubsystems.Thebroadcastmessages(definedinthis

document)andthetemporalfusionblock(Figure6)areintendedtobeastartingpointtoaccomplish

timesynchronization.ItislikelythattheVIPRprognosticreasonermayberobusttohandlesmallerrors

inthissynchronizationstep.Ifnot,theadditionalsamplingtimeintheeventmessageprotocolcanbe

usedtoexperimentwithmorecomplextemporalfusionlogic.ThiscombinationofanextensiblePage23

protocolandaflexiblearchitecturewillallowustosynchronizetimeacrosssubsystemseventmessages

downtowhateverresolutionwillberequiredtodetectfastprogressioneventssuchassoftwarefailures.

ThereportdeliverableCDRL4.1.03containsdetaileddescriptionsofthemessageprotocolaswellasa

descriptionifARINC624encodingasitappliestoVIPR.

6.6 ConOpsandScenarios

TheConceptofOperations(ConOps)illustratestheVIPRarchitecturethroughasetofscenarios.Inthe

reportdeliverableforthistaskwedescribeaseparatescenarioforeachofthefollowingaircraft

systems:engines,flightactuators,andsoftware.ThesescenariosarealsoillustratedintheVIPR

Demonstration(Task4.1.07).Thescenariosdescribetheinitialconditionspriortotheoccurrenceof

eachfault,thenusesequencediagramstofollowthefaultthroughthediagnosticandprognostic

functionalityofVIPRandinsomecasescalculatetheimpactofthefaultonaircraftfunctional

availability.TheyarebasedonideasstemmingfromreportsintheAviationSafetyInformationAnalysis

andSharing(ASIAS)database,NationalTransportationSafetyBoard(NTSB)reports,etc.,aswellas

observationswehavedocumentedthroughourflightdatarecorders.Thescenarioscoveraspectrumof

AdverseEventTypeslistedinTable2oftheNASAIVHMTechnicalPlan.

Thehighlightsofthefourscenariosincludedinthedemonstrationarepresentedbelow.

1. Slow,progressivefaulteventwiththefuelmeteringcomponentofanengine.Thisscenariowas

basedontheMesabaairlineincidentwhicheventuallyledtoaninflight engineonfirealarm.

Figure9showsthekeyinformationusedbyVIPRtodiagnosethisevent.

Thescenariostartswithrelativelybenignobservationmadebyamonitor—theleftenginehasa

slowstart.ThisscenarioillustratesVIPRabilitytocreateafaultconditionwithseveralpossible

rootcausesfromarelativelynoncriticalsymptom.Thefaultconditionisdisambiguatedas

evidenceemergesfromsuccessiveflights,andactivequerymechanismwithinVIPR.Thecorrect

rootcauseisidentifiedfivetosixflightsbeforetheinflightengineonfireevent.

Figure9.VIPRConOpsforaslow,progressiveevent

AsshowninSeeFigure9,atthearealevel,VIPRusesthefactthattherearetwoenginesand

usesthisinformationtocomparethestarttimesandeliminatecommoncausesuchascoldoilorPage24

fuelpumpproblems.Atthevehiclelevel,VIPRusesthephysicalconnectionbetweentheengine

andthebleedsystemtoeliminateproblemswiththeenginecompressorandfocusonthe

engineelectricalcomponents.AttheLRUlevelsymptomsgeneratedbyvariousmonitorsare

trackedovermultipleflightsandaggregatedtoincreasetheconfidencelevelinthefuel

meteringunit.

2. WidespreadsymptomcascadefaulteventassociatedwiththelossofanAirDataInertial

ReferenceUnit(ADIRU).ThisscenariowasbasedbytheSingaporeairline(A330,10/7/2008)

incident,whereinafailedADIRUcausedmultiplefaultnotificationsandleduncommanded

pitchdownevents.

Thescenariostartswithflightcontrollerswitchingfromaprimarychanneltoasecondary

channel.TherootcauseisabadADIRUsignal.SoonthefailedARIDUcascadesassymptoms

fromthenavigationandthegroundProximitysubsystems.VIPRfollowsthecascadeschainsto

identifytherootcause.Multiplesymptomsgeneratedfromseveralconnectedsubsystemsare

consolidatedandexplainedawaybyrootcauseanalysis.Oncethefaultislocalized,VIPRuses

symptomcascaderelationshiptoidentifyacommonrootcause,namelytheADIRUbadvalues.

Figure10.VIPRConopsforawidespreadimpactevent

Throughputtheevolution,VIPRmanagesthecascadeandexplainsawayvariousfaultcodes

generatedbysubsystemthatisconnectedtothisADIRU.AdditionalmonitorsfromtheGround

Proxsystemexoneratetheinertialreference(IR)subsystem,whileindictingAirDataUnit(ADR).

VIPRthenproceedstocalculatethefunctionaleffectsofthisfaultandinformstheflightcrew

aboutalternativecontrollawsthatcanpreventsecondaryeffectssuchasuncommandedpitch

downevents.

3. Sensorinducedfaultevents,whereinafaultysensorfeedingtriggersintermittentevidence.This

scenariowasbasedonaMesabaairlineincidentwhereinainrangesensorfaultcaused

intermittentlossofengineperformanceandeventuallytheflightcrewreturnedbacktothe

baseafterbeingairbornefor1520minutes.Page25

Aninrangesensorbiasisextremelydifficulttodetect.Aninrangebiasdoesnotcauseahigh

limitorlowlimitexceedanceandhencedoesnotgetdetectedbystandardrangecheck

algorithms.However,thenumericaloffsetspoofedseveraldiagnosticmonitors.Theresultwas

thecreationofseveralfaultconditions–thatindicatedinletfouling,compressorbladeerosion,

turbinedistress,rupturedbleedvalve.

Figure11.VIPRConOpsforfaultysensorinducedintermittentevent

VIPRdoessupportmultiplesimultaneousfaults.However,metaruleswithinVIPRcalculatethe

likelihoodofsuchevents.Inthisscenario,thesefailuremodesarepossible,butveryunlikely.

VIPRhypothesizespotentialsensorfault.Usingthereferencemodel,VIPRidentifiesasetof

sensorsthatiscommontoprimarymonitorsassociatedwitheachfaultcondition.Through

activequery,itcomparestheengineinstalledtemperature(T2)sensorwiththeaircraftinstalled

temperaturesensorandisolatestheproblemtoaninrangebiasoftheengineT2sensor.

4. Softwareprognosticstriggeredbyarelativelybenignfault.Thisbenignfault,inthisscenario,

leadstoamuchmoreseriousfaultduetoasoftwaredesignflawasreportedbytheAustralian

TransportSafetyBureauofaBoeing777onAugust1,2005incident.VIPRkeepstrackofcurrent

contingencystate(backupsensor,activeI/Ochannel)tocollectdatasurroundingthese

relativelyroutineevents.ThearchivedevidencecanbeusedtorerunaportionofaV&Vmodel

withappropriateboundaryconditions,andcalculatelikelihoodofsystemlevelfailuresifand

whenthebackupsensoralsofails.Weareviewingthisscenariointhecontextofprognostics

andpreventionschema,sothatVIPRhelpsensurethatthemuchmoreseriouseventwhich

happenedinreallifenevercomestopass.

6.7 VIPRDemonstration

ThedemonstrationofVIPRinyearoneillustratesVIPR’s“plumbing”.Noreasonersareincludedinthe

firstyear,althoughascriptedillustrationofhowthesereasonerswillworkisincluded.Thedemoscreen

isshownbelowinFigure12.Page26

ThedemoiswhatwecallananimatedConceptofOperationsorConOps.Usingthesimulatorandthe

scenarioscripts,thedemonstrationallowedusto“see”theVIPRdesign,highlightsalientfeatures,

discoverlimitationsandmakecoursecorrections.Weusedfourscenariostovisualizethediagnosticand

prognosticsteps,messagepassingforactivedataqueryandhenceillustratehowVIPRconceptswork

togethertoachievetheoverallNASAIVHMgoals.

VisualizationoftheVIPRarchitectureandinternalworkingweredoneusing(RefertoFigure12):

1. Adynamicsequencediagram(upperleft)thatvisualizedthemessagepassingwithinVIPRtiers

followingtheinformationprotocols.Thesendersandreceiversareshownatthetopofthe

window.

2. Thewindowinthelowerleftillustratestheprogressionofthediagnosticcomputationusingthe

“Walgorithm”(see[Biswas6/2010]section5.2.1).

3. Theleftmiddlewindowshowsthedetailsforallactivefaultconditionsthatdescribethe

prevailingfaulthypothesis.

4. Thelowerrightisthewindowcontainingthecontrolsforthedemonstration.

Figure12.VIPRDemonstration

Otherwindowscanbedisplayedduringtheprogressofthedemo.OneoftheseistheEICASdisplay

whereanycrewmessagesfromVIPRcanbedisplayed.Anotherisamultiflightconfidenceplot

associatedwitheachfaultcondition.Finally,insupportofnextyear’smetricsevaluation,awindow

displayingstatisticsrelatedtocomputationalandnetworkingresourceusageforthescenariocanbe

selectivelydisplayed.Currently,thiswindowdisplaysthenumberofbytestransmitted,average

messagedelay,messagedistributionbytype,etc.foreachoftheVIPR’sHealthManagernodes.

Detailed 

State of the 

selected HM

Details for all 

active and 

waiting Fault 

Conditions

Simulation 

controls

Dynamic display 

of the ‘W’ 

algorithm 

processing

Dynamic 

sequence 

diagram 

illustrating the 

events and 

messages in 

VIPR
Page27

6.7.1 VIPRDemonstrationInternals

FollowingtheHoneywell7StepSMARTprocess(seeFigure3)fortheconceptdefinitionphase,theVIPR

simulationfocusesonconceptofoperations(CONOPS)simulation.TheVIPRCONOPSsimulationuses

predefinedscenariostoillustratetheexpectedoperationoftheVIPRsystemandisaPythonapplication

designedusingaModelViewControllerapproach.TheModelrepresentsthecurrentstateoftheVIPR

systemincludingallmessages,monitors,andtheprocessingstateofeachhealthmanager.

TheControllerprovidesalinkagebetweenthemodelandtheviewandseparatesthemodeldatafrom

thevisualrepresentation.Asthemodelstatechangesduetomessagetrafficormonitors,thecontroller

directstheviewtoupdateitspresentationofthemodel.Likewise,thecontrollerreflectsuser

interactions,suchaschangingthestateofamonitor,backtothemodel.Change Model

Figure13.CONOPSSimulationArchitecture

TheViewprovidesavisualrepresentationofthemodeltotheuser.TheCONOPSsimulationdividesthe

viewintoseveraldisplaypanels,eachpresentingadifferentperspectiveofthemodel(seeFigure12).

Thetwoprimaryviewsarethesequenceviewandthealgorithmview.Thesequenceviewillustratesthe

timesequencedmessagetrafficandeventsasadynamicsequencediagram,whilethealgorithmview

showsthedynamicstateofthe‘W’algorithmforahealthmanager.Other,secondaryviewsinclude

performancemetrics,pilotalertpanel,andafaultconditionplot.Page28

Simulationexecutionisdrivenbyascenarioscript.Thescenariocontainsatimelineofmessages,fault

conditions,monitorevents,andprocesseventsrepresentingtheactivitythatisexpectedinaVIPR

system.Duringplaybackofthescenario,eventsareappliedtothemodelintimesequenceaccordingto

thesimulationclocktime.Theusercancontrolthesimulationplaybackspeedorstepthroughthe

simulationeventsusingthesimulationcontroller.

ThenextsimulationphasewillevolvetheCONOPSsimulationintoanarchitecturalmodelbyreplacing

thescriptedscenarioeventswithasetofsimulatedhealthmanagersandmonitors.Themodelswill

communicateviatheVIPRInformationProtocolmessageformatoverasimulatedARINC624bus.

Figure14.ArchitectureSimulation

Thearchitecturalmodelprovidestheopportunitytoevaluateperformancemetricssuchasdata

throughputandprocessorloading,alongwithvalidationofthesystemcommunicationarchitectureand

messageformats.Page29

6.8 MetricsforBenchmarkingVIPR

Anumberofdiagnosticandprognosticmetricsexist,butthesestandardsaredefinedforwell

circumscribedalgorithmsthatapplytosmallsubsystems.Forlayeredreasoners,suchasVIPR,the

overallperformancecannotbeevaluatedbymetricssolelydirectedtowardtimelydetectionand

accuracyofestimationofthefaultsinindividualcomponents.Amongotherfactors,theoverallvehicle

reasonerperformanceisgovernedbytheeffectivenessofthecommunicationschemesbetweenthe

differentmonitorsandhierarchicalreasonersinthearchitecture,andtheabilitytopropagateandfuse

relevantinformationtomakeaccurate,consistent,andtimelypredictionsatdifferentlevelsofthe

reasonerhierarchy.Anaddedfunctionalityofthisarchitectureistheabilityofthevehicleandarealevel

reasonerstogeneratespecificqueriesforthecomponentmonitors.Toaddresstheseissues,wehave

developedanextendedsetofdiagnosticandprognosticsmetricsthatcanbeusedtoevaluatethe

performanceofthelayeredarchitecture.Themetricsaresummarizedinthefollowingtables.

Table5.DetectionandDiagnosisMetrics

Diagnosticcoverage

Identifytestscenarioswithfaultsthatcouldnotbedetectedand/or

isolatedwithexistingapproachesanddemonstrateVIPR’seffectiveness

forthesescenarios

Accuracy

Detection:falsepositiverate

Detection:falsenegativerate

Isolation:misclassificationrate

LatencyTimetodetect

Timetoisolate

SensitivityEvaluatethemetricsaboveinthepresenceofsystemuncertainty

Table6.PrognosisMetrics

Prognosticcoverage

Identifytestscenarioswithfaultsthatcouldnotbepredictedwith

existingapproachesanddemonstrateVIPR’seffectivenessforthese

scenarios

Accuracy

Error=predictedRUL–actualRUL

Averagebias

Timeliness

PrecisionEstimatethesizeoftheconfidenceintervalassociatedwiththeRUL

prediction

SensitivityEvaluatethemetricsaboveinthepresenceofsystemuncertainty

Table7.ComputationalMetrics

Offlinecomplexityanalysis

Worstoraveragecaseestimatesofrunningtime,memory,and

communicationbandwidthasafunctionofthesizeoftheinput

Numberofsoftwarecomponents

Numberoflinksbetweensoftwarecomponents

Numberofinputsandoutputscommunicated

SizeofcodePage30

Onlineprofiling

CPUexecutiontimes

CPUutilization

Networkdelays

BandwidthUtilization

Amountofmemory

AversionofthisreportdeliverablewasalsoacceptedatthePHMConferencetobeheldinPortland,OR

inOctober2010.

7 RecommendationsforFutureWork

WerecommendthatthenextstepsintheVIPRprogramincludetwoparallelactivities:DataMiningand

ReasonerCoding.TheDataMiningtaskincludesthediscoveryofnewrelationshipsbetweensymptoms

andfaultsaswellasmorerefinedvaluesoftheparametersgoverningtheserelationships.Thecoding

taskwillincludedetaileddesignandsoftwareimplementationoftheVIPRsystemreasonersfor

prognostics,diagnostics,andfusion.

Overall,thereisaclearsequenceforconstructingthefullaircraftreferencemodel.Giventheextensive

understandingofpropulsionandbleedsubsystemhealthmanagement,werecommendconstructing

theirreferencemodelsfirst.Theconstructionofreferencemodelsforothersubsystems(software,

actuators,andavionics)needstobeprecededbyadataminingtask.

Thenextstepswouldincludeintegrationofthereferencemodelsandthereasonercodewithina

simulationenvironmentsuchasSMARTlab,demonstrationofVIPRcapabilitiesthroughasetof

scenarios(section6.6),collectionofmetrics(section6.8),andtradespacedocumentation.

Followingasuccessfulsoftwaredemonstration,werecommendthatselecthardwarebeincorporated

intothesimulationenvironmenttodemonstrateVIPR’shealthmanagementonrealworldequipment.

8 PublishedDocuments

Thefollowingdocumentswerepublishedunderthiscontract:

TimFelke,GeorgeD.Hadden,DaveMiller,DinkarA.Mylaraswamy"ArchitecturesforIntegratedVehicle

HealthManagement",ProceedingsoftheAIAAInfotech@Aerospace2010Conference,April2010,

Atlanta,Georgia.

XenofonKoutsoukos,GautamBiswas,DinkarA.Mylaraswamy,GeorgeD.Hadden,DanielMack,Doug

Hamilton,“BenchmarkingtheVehicleIntegratedPrognosticReasoner”,PHM2010,Portland,Oregon(to

appearpostersession).

GautamBiswas,XenofonKoutsoukos,DanielMack,DinkarA.MylaraswamyandGeorgeD.Hadden,

“VehicleIntegratedPrognosticReasoner(VIPR):AHierarchicalReasonerforAircraftSystems”,

Proceedingsofthe2011IEEEAerospaceConference(toappear).Page31

9 ReferencedDocuments

Thefollowingdocumentswerereferencedintheresearchandanalysisconductedforthiscontract:

S.AbdelwahedandG.Karsai,“NotionsofDiagnosabilityforTimedFailurePropagationGraphs”,IEEE

SystemsReadinessTechnologyConference,pp.643648,2006.

C.F.Aliferis,A.Statnikov,I.Tsamardinos,S.Mani,andX.D.Koutsoukos,“LocalCausalandMarkov

BlanketInductionAlgorithmsforCausalDiscoveryandFeatureSelectionforClassification,PartI:

AlgorithmsandEmpiricalEvaluation”,JournalofMachineLearningResearch11(Jan),pp.171234,2010.

C.F.Aliferis,A.Statnikov,I.Tsamardinos,S.Mani,andX.D.Koutsoukos.“LocalCausalandMarkov

BlanketInductionAlgorithmsforCausalDiscoveryandFeatureSelectionforClassification,PartII:

AnalysisandExtensions”,JournalofMachineLearningResearch.11(Jan),pp.235284,2010.

D.Badenius,“FailureRate/MTBF”,IEEETransactionsonReliability,vol.R19(2),pp.6667,1970.

R.Bharadwaj,C.Kulkarni,G.Biswas,andK.Kim,“ModelBasedAvionicsSystemsFaultSimulationand

Detection”,ProceedingsoftheAIAAInfotech@Aerospace2010Conference,April2010,Atlanta,

Georgia.

G.BiswasandT.S.Anand,“AnExpertSystemShellforMixedInitiativeReasoning”,JournaloftheIndian

InstituteofScience,Bangalore,India,67,pp.465490,1987.

G.Biswas,R.Kapadia,andX.W.Yu,“CombinedQualitative–QuantitativeSteadyStateDiagnosisof

ContinuousValuedSystems”,IEEETransactionsonSystems,Man,andCybernetics—PartA:Systemsand

Humans,vol.27,no.2,pp.167185,March1997.

G.Biswas,X.Koutsoukos,D.Mylaraswamy,G.D.Hadden“ToolsandTechnologyConceptofOperations”,

TechnicalReport,submittedbyHoneywelltoNASA(CDRLSequenceNumber:4.1.07),June2010.

G.BiswasandT.S.Anand,“UsingtheDempsterShaferSchemeinaMixedInitiativeExpertSystem

Shell”,UncertaintyinArtificialIntelligence3,L.N.Kanal,T.S.Levitt,andJ.F.Lemmer,eds.,Elsevier

SciencePublishers(NorthHolland),Amsterdam,pp.223239,1989.

R.M.ButtonandA.Chicatelli,“ElectricalPowerSystemHealthManagement”,Proc.1stInternational

ForumonIntegratedSystemHealthEngineeringandManagementinAerospace,Napa,CA,November

2005.

C.S.Byington,M.Watson,P.Kalgren,andR.SafaBakhsh,“MetricsEvaluationandToolDevelopmentfor

HealthandUsageMonitoringSystemTechnology”,ThirdInternationalConferenceonHealthandUsage

MonitoringHUMS2003,Melbourne,Australia,G.FForsyth(editor),pp.2735,2003.

C.S.Chao,D.L.Yang,andA.C.Liu,“AnAutomatedFaultDiagnosisSystemUsingHierarchicalReasoning

andAlarmCorrelation”,JournalofNetworkandSystemsManagement,9(2),pp.183202,2001.Page32

L.ChittaroandR.Ranon,“HierarchicalModelBasedDiagnosisBasedOnStructuralAbstraction”,

ArtificialIntelligence,155,pp.147182,2004.

I.Cohen,M.Goldszmidt,T.Kelly,J.Symons,andJ.S.Chase,“CorrelatingInstrumentationDatatoSystem

States:ABuildingBlockforAutomatedDiagnosisandControl”,SixthSymposiumonOperatingSystems

DesignandImplementation,SanFrancisco,CA,Dec.68.Chen,K.(2006)TANTreesforfMRIBrainImage

Classification,CarnegieMellonUniversity,2004.

J.Colclough,“ThePSMRSReferenceModel”,U.S.ArmyContractW56HZV05C0724.

M.Daigle,X.Koutsoukos,andG.Biswas,“DistributedDiagnosisinFormationsofMobileRobots”,IEEE

TransactionsonRobotics,vol.23,no.2,pp.353369,2007.

R.DavisandW.Hamscher,“ModelBasedReasoning:Troubleshooting”,SanFrancisco,CA,USA,Morgan

KaufmannPublishersInc.,1988.

J.deKleerandB.C.Williams,“DiagnosingMultipleFaults”,ArtificialIntelligence,vol.32,pp.97130,

1987.

R.Debouk,S.Lafortune,andD.Teneketzis,“CoordinatedDecentralizedProtocolsforFailureDiagnosis

ofDiscreteEventSystems”,DiscreteEventDynamicSystem:TheoryandApplications,10(1/2),pp.33–

86,January2000.

T.Felke,“ApplicationofModelBasedDiagnosticsTechnologyontheBoeing777Airplane”,IEEEAero

Conf.,BigSky,MT.1994.

T.Felke,“KnowledgeManagementforMaintenanceofComplexSystems”,

https://dashlink.arc.nasa.gov/topic/integrateddiagnostics,BriefingtoNASAARC,April21,2009.

P.Frank,“FaultDiagnosisinDynamicSystemsUsingAnalyticalandKnowledgeBasedRedundancyA

SurveyandSomeNewResults”,Automatica,vol.26,no.3,pp.459474,1990.

N.Freidman,D.Geiger,andM.Goldszmidt(1997),"BayesianNetworkClassifiers”,MachineLearning,

29:131163.

J.J.Gertler,“FaultDetectionandDiagnosisinEngineeringSystems”,NewYork,NY,MarcelDekker,Inc.,

1998.

J.Goericke,K.Moeckly,andA.Stramiello,“AnEmbeddable,PhysicsBasedApproachforContinuous

PerformanceDeteriorationDetectionandPowerAssuranceforTurboshaftEngines”,AHS65thAnnual

Forum,Grapevine,TX,2009.

J.GordonandE.H.Shortliffe,“TheDempsterShaferTheoryofEvidence”,Chapter13,RuleBasedExpert

Systems:TheMYCINExperimentsoftheHeuristicProgrammingProject,AddisonWesley,Reading,MA,

1984.Page33

V.Guralnik,D.Mylaraswamy,andH.Voges,“OnHandlingDependentEvidenceandMultipleFaultsin

KnowledgeFusionforEngineHealthManagement”,2006IEEEAerospaceConference,2006.

V.GuralnikandD.Mylaraswamy,“AStochasticTheoryforEvidenceAggregation”,FirstPHM

Conference,October2008.

G.Hadden,R.Magnuson,B.J.Barton,G.Shao,“AMethodtoPerformPrognosticsThrough

PredictionofDiagnosticIndicators”,USPatentApplication,2009.

ISO13374Standardhttp://www.iso.org/iso/catalogue_detail?csnumber=21832

B.Jambor(ProgramManager,LockheedMartinSpaceSystemsCompany),“IntegratedSystem

VehicleHealthManagementArchitectureDesign”,AFRLContractFA86500555030001.

X.Koutsoukos,G.Biswas,D.Mylaraswamy,G.Hadden,D.Mack,D.Hamilton,“Benchmarkingthe

VehicleIntegratedPrognosticReasoner(VIPR)”,TechnicalReport,submittedbyHoneywelltoNASA

(CDRLSequenceNumber:4.1.06),April2010.

T.Kurtoglu,O.Mengshoel,andS.Pol,“AFrameworkforSystematicBenchmarkingofMonitoringand

DiagnosticSystems”,InternationalConferenceonPrognosticsandHealthManagementConference,

Denver,CO,2008.

P.J.MostermanandG.Biswas,“DiagnosisofContinuousValuedSystemsinTransientOperating

Regions”,IEEETransactionsonSystems,ManandCybernetics,PartA,vol.29,no.6,pp.554565,1999.

D.MylaraswamyandG.Hadden,“ArchitectureRecommendations:NASAVIPRProgram”,Technical

Report,submittedbyHoneywelltoNASA(CDRLSequenceNumber:4.1.02),December2009.

D.Mylaraswamy,D.Hamilton,andG.Hadden,“ConceptofOperations:NASAVIPRProgram”,Technical

Report,submittedbyHoneywelltoNASA(CDRLSequenceNumber:4.1.04),January2010.

D.Mylaraswamy,L.Olson,andE.Nwadiogbu,“EnginePerformanceTrending:PracticalInsights”,Trans.

OftheInstituteofMeasurementandControl,31,pp.341354,2009.

D.Mylaraswamy,P.Bullemer,andK.Emigholz,“FieldingaMultipleStateEstimatorPlatform”,National

PetroleumandRefinersAssociationComputingConference,October2000.

D.Mylaraswamy,D.Hamilton,andG.Hadden,“InformationProtocols:NASAVIPRProgram”,Technical

Report,submittedbyHoneywelltoNASA(CDRLSequenceNumber:4.1.03),December2009.

J.Pearl,“ProbabilisticReasoninginIntelligentSystems:NetworksofPlausibleInference”,Morgan

Kaufmann,SanMateo,CA,1988.

R.Reiter,“ATheoryofDiagnosisfromFirstPrinciples”,ReadingsinModelbasedDiagnosis,pp.2948,

1992.Page34

A.Saxena,J.Celaya,E.Balaban,K.Goebel,B.Saha,S.Saha,andM.Schwabacher,“MetricsforEvaluating

PerformanceofPrognosticTechniques”,inInternationalConferenceonPrognosticsandHealth

Management(PHM08),DenverCO,2008.

A.Saxena,J.Celaya,B.Saha,andK.Goebel,“PrognosticsPerformanceEvaluation”,NASA2009Aviation

SafetyConference.

G.Schwarz,“EstimatingtheDimensionofaModel”,TheAnnalsofStatistics,6(2),pp.461464,1978.

G.Shafer,“AMathematicalTheoryofEvidence”,PrincetonUniv.Press,1976.REPORT DOCUMENTATION PAGE Form Approved

OMB No. 0704-0188

2. REPORT TYPE 

Contractor Report

 4. TITLE AND SUBTITLE

Vehicle Integrated Prognostic Reasoner (VIPR) 2010 Annual Final Report 

5a. CONTRACT NUMBER

NNL09AD44T

 6. AUTHOR(S)

Hadden, George D.; Dinkar, Mylaraswamy; Schimmel, Craig; Biswas,

Gautam; Koutsoukos, Xenofon; Mack, Daniel

 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

NASA Langley Research Center 

Hampton, VA 23681-2199 

 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

National Aeronautics and Space Administration

Washington, DC 20546-0001

 8. PERFORMING ORGANIZATION

 REPORT NUMBER

10. SPONSOR/MONITOR'S ACRONYM(S)

NASA

13. SUPPLEMENTARY NOTES

Langley Technical Monitor: Eric G. Cooper

12. DISTRIBUTION/AVAILABILITY STATEMENT

Unclassified - Unlimited

Subject Category 06

Availability: NASA CASI (443) 757-5802

19a. NAME OF RESPONSIBLE PERSON

STI Help Desk (email: help@sti.nasa.gov)

14. ABSTRACT

Honeywell's Central Maintenance Computer Function (CMCF) and Aircraft Condition Monitoring Function (ACMF) represent the state-of-the art in integrated 

vehicle health management (IVHM). Underlying these technologies is a fault propagation modeling system that provides nose-to-tail coverage and root cause 

diagnostics. The Vehicle Integrated Prognostic Reasoner (VIPR) extends this technology to interpret evidence generated by advanced diagnostic and 

prognostic monitors provided by component suppliers to detect, isolate, and predict adverse events that affect flight safety. This report describes year one 

work that included defining the architecture and communication protocols and establishing the user requirements for such a system. Based on these and a set 

of ConOps scenarios, we designed and implemented a demonstration of communication pathways and associated three-tiered health management architecture. 

A series of scripted scenarios showed how VIPR would detect adverse events before they escalate as safety incidents through a combination of advanced 

reasoning and additional aircraft data collected from an aircraft condition monitoring system. Demonstrating VIPR capability for cases recorded in the ASIAS 

database and cross linking them with historical aircraft data is planned for year two.

15. SUBJECT TERMS

Advanced Diagnostics, Condition Monitoring, Prognostics, Vehicle Health Management, Vehicle-Level Reasoning

18. NUMBER

 OF 

 PAGES

35

19b. TELEPHONE NUMBER (Include area code)

(443) 757-5802

a. REPORT

U

c. THIS PAGE

U

b. ABSTRACT

U

17. LIMITATION OF 

 ABSTRACT

UU

Prescribed by ANSI Std. Z39.18

Standard Form 298 (Rev. 8-98)

3. DATES COVERED (From - To)

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

645846.02.07.07

11. SPONSOR/MONITOR'S REPORT

 NUMBER(S)

NASA/CR-2011-217147

16. SECURITY CLASSIFICATION OF:

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, 

gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this 

collection of information, including suggestions for reducing this burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and 

Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person 

shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

1. REPORT DATE (DD-MM-YYYY)

01- 05 - 2011