# Certification Authorities Software Team (CAST)

Position Paper
CAST-9

Considerations for Evaluating Safety Engineering Approaches to Software Assurance

Completed January, 2002

NOTE: This position paper has been coordinated among the software specialists of certification authorities from the United States, Europe, and Canada. However, it does not constitute official policy or guidance from any of the authorities

. This document is provided for educational and informational purposes only and should be discussed with the appropriate certification authority when considering for actual projects.

## Considerations for Evaluating Safety Engineering Approaches to Software Assurance

1. Introduction

.

Safety engineering approaches for software assurance (SEASA) can be applied to software with the purpose of satisfying the requirements that are mandated by aviation regulations for equipment and systems instal led on aircraft. However, policy to guide certification authority engineer s in their assessment of SEASA proposals as a means of compliance for the software has been limited to issue papers applied to specific projects. The purpose of this position paper is to propose guidelines for certification authorities to consider when evaluating th e acceptability of SEASA and for industry organizations considering such an approach as a method for providing safety, system and software compliance assurance. It should be noted that safety engineering approaches in general are not a replacement in total for the guide lines of DO-178B/ED-12B (reference a.). Rather, safety engineering approaches can be used in conjunction with DO-178B, supplementing and/or replacing some crit eria of DO-178B for some software components, using all criteria of DO-178B for some other software components, and perhaps validating software assurance criter ia for yet other components. The following sections of this paper will discuss some safe ty engineering approach es. It also indicates which assurance objectives are always common between SEASA and DO-178B, and which DO-178B objectives can potentially be supplemented, reduced or replaced by using a SEASA.

Note: Safety engineering appr oaches are variously called “safety directed development (SDD),” “systems safety engineering tec hniques (SSET),” and “safety engineering techniques for software (SETS).” For this pa per, all will be referred to by the acronym “SEASA” – safety engineering appr oaches for software assurance.

2. Reference Documents

.

a. RTCA, Inc. document DO-178B and EU ROCAE document ED-12B, “Software Considerations in Airborne Systems and Equipment Certification”, dated December 1, 1992.

b. FAA Advisory Circular (AC) 20-115B , “RTCA, Inc., Document RTCA/DO178B”, dated January 11, 1993.

c. FAA AC 23.1309-1C, “Equipment, Systems, and Installations in Part 23 Airplanes”, dated March 12, 1999.

d. FAA AC 25.1309-1A, “System Design a nd Analysis”, dated June 21, 1988.

NOTE: This position paper has been coordinated am ong the software specia lists of certification authorities from the United States, Europe, and Canada. However, it does not constitute official policy or guidance from any of the authorities . This document is provided for educational and informational purposes only and should be discussed with the appropriate certification authority when considering for actual projects. 2e. JAA Advisory Material-Joint (AMJ ) 25.1309, “System Design and Analysis”, dated May 11, 1990.

f. JAA Temporary Guidance Leaflet (TGL ) #4, “Software Guidance”, dated 1

October 1996

g. Society of Automotive Engineers (S AE) Aerospace Recommended Practice (ARP) 4754 and EUROCAE document ED-79, “Certification Considerations for Highly-Integrated or Complex Aircra ft Systems,” dated November 1996.

h. SAE ARP 4761, “Guidelines and Me thods for Conducting the Safety

Assessment Process on Civil Airborne Systems and Equipment”, dated

December 1996.

i. RTCA, Inc. document DO-254 and EUROCAE document ED-80, “Design

Assurance Guidance for Airborne Elect ronic Hardware,” dated 19 April 2000. j. RTCA, Inc. document DO-248B and EUROCAE document ED-94B, “Final Report for Clarification of DO-178B(/ED -12B) ‘Software Considerations In Airborne Systems and Equipment Certification’”, 2001

k. CAST Position Paper CAST-5, “Alternate Means”

l. “Capability Maturity Model for Soft ware, Version 1.1”, dated February, 1993.

m. IEEE/EIA 12207, “Software Life Cycle Processes”, dated March, 1998.

3. Background

.

Reference a. is recognized by the certificati on authorities directly by references b. and f., and indirectly by references c. through e. as providing an acceptable means of compliance for the software aspects of certification to aviation regulations when software is installed as part of an airborne system. Software as pects are usually associated primarily with FAR and JAR XX.1301, XX .1309 (XX = 23, 25, etc.) and 33.28 (engine control systems). In fact, FAR 33.28 is the only regulation that specifi cally uses the term

“software.”

Reference a. provides guidance for a struct ured, rigorous development and verification process. This rigor is based on the failure c ondition classification of the system functions

in which the software is used. A software level is assigned, based on the worst-case failure condition category to which anomalous behavior of the software could cause or contribute. It provides software process assurance objectiv es and criteria for software planning, development, verifi cation, configuration manageme nt, quality assurance and

certification liaison. Verification in reference a. consists of reviews, analyses and testing conducted throughout the development process. Reviews and analyses are performed on the software data throughout the development to ensure that errors and deficiencies are not propagated from one development process to a subsequent process. Testing consists of requirements-based testing for normal ra nge and abnormal range (robustness) at the software module level, integrated software module level and integrated hardware and software level, combined with requirements -based test coverage analysis and code

NOTE: This position paper has been coordinated am ong the software specia lists of certification authorities from the United States, Europe, and Canada. However, it does not constitute official policy or guidance from any of the authorities . This document is provided for educational and informational purposes only and should be discussed with the appropriate certification authority when considering for actual projects. 3structural coverage analysis. The reference a. software verification objectives and criteria were developed to ensure that the software functions as specified and to reduce the

likelihood that the software will contain errors or exhibit an omalous behavior that could effect safety. The amount of structure and ri gor of the software verification process is dependent on the software level as specified in reference a. for software levels A through E. The software level for a specific softwa re application is based upon a system safety assessment process, such as that described in reference h. This determines the worst-case failure conditions of the system that coul d result from software anomalous behaviors caused by errors. Reference g. also provides guidance for types of safety and system engineering analyses, methods and techniques that could be performed to ensure the safety of the system, satisfy the “fail safe design techniques” of re ferences c. through e., and comply with the regulations.

Because of the complex nature of many so ftware applications, there are inherent difficulties in implementing and showing independence between those software components that could contribute to a failure condition affecting safety and those that could not. This is particularly the case when the software application is large, and if the code is not structured primarily for the comp artmentalization of safety related elements. It may also be inherently difficult in such a pplications to identify a strict correspondence (traceability) between system safety requireme nts and the discrete components of the software application. With increasing size and complexity, safety analysis of the software becomes increasingly problematic for applications that are mainly constructed around performance and functional requirement s rather than safety requirements. Therefore, aircraft certification programs have traditionally relied on software process assurance combined with a specified level of software verification to reduce the potential

errors and anomalous behavior of the software . This approach has been accepted as the practical minimum requirement for showing compliance of the software components of a system, without directly addressing those safe ty elements inherently difficult to address in software. Industry and regulat ory authorities have jointly formalized specific criteria for implementing this appr oach in reference a.

Note: The reader should recognize that softwa re process assurance and verification is only one aspect, and perhaps a small part, of the overall development, verification and validation of the system and of the system installed on the airc raft. Other significant activities include: system safety assessment , system design, hardware design, hardware environmental qualification testing, system validation and verification, aircraft architecture design, laboratory bench testi ng, aircraft installation testing, simulation, aircraft ground testing, and aircraft flight testing.

The process assurance and verification criter ia of reference a. do not provide safety assurance in and of themselves. They provi de assurance that the software application adheres to the software requirements, includ ing safety-related, functional, performance, etc. requirements. They also provide a rela tive measure of how mu ch the code and its

NOTE: This position paper has been coordinated am ong the software specia lists of certification authorities from the United States, Europe, and Canada. However, it does not constitute official policy or guidance from any of the authorities . This document is provided for educational and informational purposes only and should be discussed with the appropriate certification authority when considering for actual projects. 4structure have been exercised, considering the assigned software level of the application. However, software requirements may or may not be focused on safety or system failure

conditions that can lead to ai rcraft hazards, and code can be exercised with no knowledge of the system failure conditions and aircraft ha zards. Rather, aircraft hazard and system failure elimination, minimization, and contro l have typically been addressed at the aircraft level and the system level, respec tively. Thus, error reduction of the software through process assurance and verification must be combined with analysis and validation at the aircraft and system levels and recognized as minimum requirements for showing that the safety objectives for the syst em (and the aircraft) have been met. While this approach is recognized as good practice, it is an indirect approach to the safety aspects of the software.

Other software standards and guidance have proposed “qualification” of software developers’ process or organi zation as ensuring that the process or organization will produce “high quality” products. Examples of these are the Carnegie Mellon University (CMU) Software Engineering Institute’s (S EI) Capability Maturity Model (CMM), the International Standards Organi zation’s (ISO) 9001, part 3; a nd Institute of Electrical and Electronics Engineers (IEEE), Inc.’s various software standards and guidance material (e.g., IEEE/EIA 12207). While all of these sta ndards and guidance materials have merit and are useful for many applications, refere nce a. has become the de facto standard guidance for process assurance of airborne systems and equipment using digital computers and software intended for application on commercial aircraft.

Some applicants have asserted that they prefer a more direct approach of tracing potential aircraft hazards and system failures to discrete software components, and then eliminating, minimizing, or controlling specifi c hazards and failure conditions to which the software could contribute, or provid ing protective mechanisms around susceptible software components, and thus directly assuring the safety of the software.

Applicants have proposed that the development of a “software safety program” integrated with the system safety program offers a direct assurance of total system safety. Such an integrated safety program would use analys is techniques derived from system safety assessment methods, combined with complete safety related software requirements, and use safety-focused software design, coding, and verification. App licants have also proposed that this approach of using an integrated software and system safety program may justifiably allow some relief from the criter ia of reference a., the structural coverage criteria and, in some cases, perhaps a loweri ng or reduction of other assurance criteria or

software levels from current reference a. criteria.

When evaluating an applicant’s or software de veloper’s proposal for a safety program for software or other alternative methods, other “i ndustry” standards and certainly other “best engineering practices” could be considered as criteria for de termining the acceptability of the proposal. However, for practical purposes, su ch as certification authority’s familiarity

NOTE: This position paper has been coordinated am ong the software specia lists of certification authorities from the United States, Europe, and Canada. However, it does not constitute official policy or guidance from any of the authorities . This document is provided for educational and informational purposes only and should be discussed with the appropriate certification authority when considering for actual projects. 5with reference a.; and to expedite the certi fication process and to provide standardized

and harmonized criteria and thus a level playing field for applicants, reference a. is most often used as the benchmark for determining the acceptability of alternative proposals for

software assurance. See also referen ce k., the CAST paper on “Alternate Means.”

Reference g. recognizes the use of a “Safet y Directed Development Concept” as “an alternative to the system de velopment assurance methods” of that document, without further specifying what a safety directed develo pment concept is, what its criteria are, or why it is an accepta ble alternative.

Recently published hardware development assurance guidance (reference i.) also recognizes a “Safety-Specific Analysis” as an “advanced verification method,” which can

be used in combination with a “functional fa ilure path analysis” (FFPA), for providing electronic hardware design assurance for Leve ls A and B hardware functions in addition to the normative process and hardware data guidance of the document. According to reference i., safety-specific analysis “f ocuses on exposing and correcting the design errors that could adversely affect the hardware outputs from a system safety perspective.” Appendix B of reference i. provi des some clarification of th is method and mentions some hardware process and data needed to use this analysis method, in addition to the normative hardware process and data guideli nes provided in reference i., sections 2 through 11 and Appendix A.

In the past, a specific safety approach of an airborne system developer was accepted for showing software compliance on certain system s installed during the course of specific aircraft system certification programs. Th e original approach was approved as a supplement to DO-178[ ], not as an alternate. However, these specific criteria have been confined to issue papers applicable only to these specific systems on specific programs and reviewed on a project-by-project basis. Cr edit granted by the certification authority

allowed some reduction in the software struct ural coverage analysis criteria for some

systems.

Some applicants and developers would like to see their proposed alternative safety approach accepted on a general basis, usable at their discretion and “pre-approved” for use on any of their projects and products . This has not yet been allowed by the certification authorities, excep t on a case-by-case basis.

NOTE: This position paper has been coordinated am ong the software specia lists of certification authorities from the United States, Europe, and Canada. However, it does not constitute official policy or guidance from any of the authorities . This document is provided for educational and informational purposes only and should be discussed with the appropriate certification authority when considering for actual projects. 64. Discussion

.

The application of a safety engineering a pproach for software assurance (SEASA) is intended to directly address system failure c onditions and aircraft h azards related to the system’s software. In its essentials, SEAS A is the practice of applying common systems’ safety assessment, analysis and design tec hniques to software. It is a practice of continuing the system safety assessment into the software components, and assuring that the software meets the regulations (i.e., §§ 23.1309, 25.1309, 27.1309, 29.1309, 33.28(e)). Some commonly used system sa fety assessment and analysis methods are documented in references g. and h. Fail-sa fe design techniques are documented in references c. through e. There is much commonality and consistency between these references for methods and techniques.

Safety design techniques propose to identify all safety requirements for the aircraft, the system and its components, including hard ware and software. SEASA proposes to identify all safety requirements for the softwa re and all safety effects of the software. Then, by a method of iterative analysis and design, combined with safety-focused verification, SEASA proposes to show that those unacceptable hazards, failure conditions and anomalous behavior are prevented, mitigated or isolated and controlled, using various architectural constraints, mon itors and design techniques (e.g., multiple, independent, dissimilar components performing th e same function). Aircraft hazards and system failure conditions are traced thr ough analysis to the software. Software requirements (“safety objectives” and “safety- related requirements” in the reference a. vernacular) addressing the hazards and failure conditions are specified and verified for safety related completeness crite ria. Code is analyzed, with hazards and failure conditions

traced to the code itself. Code is designed for risk reduc tion, including the prevention, mitigation, isolation and/or control of specific failure conditions. Verification is focused on the safety related code, “cri tical code.” Like any other software development process, this SEASA process is iterative in nature with attendant feedback and refinement.

It should be noted that the specific SEASA eval uated to-date is not a complete alternative for the criteria of reference a. In fact, the ma jority of reference a. criteria remain criteria for this or any SEASA as well. The process assurance objectives and criteria of reference a. are simply “best industry practice,” with a dditions for specific guidance for the aircraft certification and airborne syst ems domains, for assuring that the developed software is the specified software through the use of di sciplined, rigorous processes and complete, correct data. This is needed for a SEASA appr oach in the same manne r that it is needed for other software development approaches. Plans, procedures, standards, as well as processes for requirements, design, coding, integration, configuration management, quality assurance, and verification are all needed for a SEASA. Requirements-based testing and test coverage analysis are obvi ously critical to SEASA. Requirements completeness and assurance of safety related requirements are key elements of the

NOTE: This position paper has been coordinated am ong the software specia lists of certification authorities from the United States, Europe, and Canada. However, it does not constitute official policy or guidance from any of the authorities . This document is provided for educational and informational purposes only and should be discussed with the appropriate certification authority when considering for actual projects. 7SEASA proposal. In fact, even by the admissi on of a specific airborne system developer, their specific SEASA requires more resource s and effort than a program that just followed the guidance of reference a. for soft ware assurance. Again, the main area that the specific SEASA was given “credit” and allowed to reduce the reference a. criteria was in the area of structural coverage. Th e rationale for allowi ng that reduction is, however, not documented, nor is the actua l amount of “credit” granted known.

Structural coverage analysis is a verification activity designed to assure that the software code has been adequately exercised in order to detect otherwise hidden software errors not revealed by requirements-based (functional) testing, and to reveal unreachable code in the software application. However, structur al coverage can be ach ieved by testing in a technically correct, but perhaps inappropriate , manner: “structural testing.” Structural testing is testing the software, mostly at the software component level, using test procedures and cases developed on the basi s of the code component itself, without necessarily being conscious of safety issues, system issues, integra tion considerations, or even requirements issues. Unless the softwa re requirements, design and code have been developed very rigorously, this can lead to very non-value-added and inappropriate testing. See also related discussion paper 4.3 and frequently asked question (FAQ) 3.44 in

reference i.

The focus of reference a. is to conduct re quirements-based testing of the software, including safety-related requirements, high-le vel requirements, low-level requirements, and derived requirements, at the highest leve l possible, measure the structural coverage achieved, and then supplement that testing by conducting lower leve l requirements-based

tests until the structural c overage appropriate for the so ftware level is achieved.

Testing under SEASA is focused on safety re lated code, while code shown to be benign (i.e., non-safety related code) may not be exer cised to the appropriate software level to satisfy the structural coverage criteria of reference a.

It is unclear what other re laxation of reference a. assu rance criteria may have been granted in the past or what may be possibl e using a SEASA. As a SEASA is used to identify and logically separate safety rela ted code components from non-safety related code components (i.e., partitioning), it may be possible to reduce the reference a. criteria for the benign code components, as long as failures and anomalous behavior of those components cannot affect safety-related code components nor system availability or integrity, and it can be assured that any reduc tion will not lead to an undesirable safety affect. A SEASA proposal will likely incl ude a partitioning strategy for using protective mechanisms for specific components and/or software operations.

NOTE: This position paper has been coordinated am ong the software specia lists of certification authorities from the United States, Europe, and Canada. However, it does not constitute official policy or guidance from any of the authorities . This document is provided for educational and informational purposes only and should be discussed with the appropriate certification authority when considering for actual projects. 85. Guidelines for evaluating a safety en gineering approach for software assurance (SEASA)

.

The following guidelines are intended to help de termine the acceptability of an aircraft system applicant’s or developer’s proposal to use a SEASA for software assurance, software approvals and complia nce to aircraft regulations:

a. Integrated system, hardware and SEASA safety program

. The applicant (or airborne system manufacturer or software developer) should develop and implement a safety program which is specific to the airb orne product, and which combines and closely integrates the safety program and devel opment processes for the product system, its hardware components and its software component s. As any safety program for software relies on the identification and tracing of a ll aircraft hazards and all system failure conditions through the system requirements and architecture, hardware components, software requirements, software design and code, and the elimination or control of these hazards and failure conditions, it is imperativ e that the safety program be developed and implemented jointly by system developers, hard ware developers and software developers. Safety assessment and elimination, minimi zation, and control of hazards and failure conditions should be a seamless process thr ough the system development, hardware development and software development processes. The software developers must be informed of the system safety aspects of hardware failures and hardware and software behaviors, and the system developers must be informed of the potential hardware and software contributions to system failure condi tions and aircraft hazards. Safety related system components should be identified to th e software developers. Successful use of a SEASA relies on an approach where software developers are familiar with, and kept current on, safety aspects of the system. Fo r example, software developers should be knowledgeable about system variables, such as input and output signals, which have potential safety consequences for the aircra ft. This knowledge should then influence the manner in which software developers design th e processing of these signals as well as the verification procedures for ensuring they are processed correctly, including handling signals and inputs that may be outside the normal “tested” ranges. Similarly, the hardware developers should be informed a nd knowledgeable about sy stem variables that are processed by hardware. System developers should be knowledgeable about the safety aspects of the software and the hardware. This knowledge should influence how system requirements, architecture, design, constraints, and syst em level verification and validation procedures are specifi ed. The integrated safety program should be designed to assure that safety-related information (i.e ., requirements, design decisions, assumptions, constraints, problem reports, change imp act analysis, etc.) is provided and used effectively between the system developers, ha rdware developers and software developers, and those responsible for validating and ve rifying the system, its hardware and its

software.

NOTE: This position paper has been coordinated am ong the software specia lists of certification authorities from the United States, Europe, and Canada. However, it does not constitute official policy or guidance from any of the authorities . This document is provided for educational and informational purposes only and should be discussed with the appropriate certification authority when considering for actual projects. 9b. Plans, procedures, and standards

. All reference a. software planning and procedural data needed for compliance are al so needed for compliance using a SEASA. In addition, system, hardware and software plans and procedures should define the integrated safety program, with safety rela ted activities and data clearly specified.

Standards should also be written to reflect sa fety-related requiremen ts and constraints on development formats, methods, and practices . For example, software development standards should contain require ments and constraints on archite ctural elements related to isolation, partitioning, separation, monitori ng, cohesion, and coupling of safety-related software components and parameters. Standa rds should also drive system, hardware and software designs towards simplification a nd verifiability. Design simplification and verifiability will provide better assurance that all safety related software aspects have been correctly implemented.

c. Safety related software requirements, design, and code

. All aircraft hazards, system failure conditions and safety-relat ed requirements should be traced to the software-hardware interface, and software re quirements specific to the hazards and

failure conditions should be developed. Constraints on soft ware behavior and software related system behavior should be specified. System states and state transitions should be completely specified. Each state should have a transition to a safe state specified. All

states should be reachable and default transitio ns should be fully specified. Transitions to unsafe states should be identified by the sa fety assessment process and eliminated by specification of a replacement transition to a safe state. The system and software should always start in a safe state. All variables s hould be initialized correctly on system startup or initialization. All software components that perform safe ty related operations should be identified. These safety-related soft ware components and the resources (i.e., processor, memory, clock, input/output device s, buses, etc.) should be isolated and protected from other non-safety-related components. The so ftware architecture should be designed so that safety related components are highly cohesive and loosely coupled. Safety-related signal and data handling should be minimize d. Hazard reduction design strategies and fail safe design techniques shoul d be used to: eliminate hazards and failure conditions, minimize hazards and failure condit ions, control the effects of hazards and failure conditions, or minimize the effects of hazards and failure conditions (listed by most desirable to least). Hazard reduction st rategies for software-based systems include, for example:

(1) Isolation and containment (partitioni ng) of safety-related software and hardware components, (2) Addition of protective features, such as safety monitoring software or hardware, (3) Recovery schemes from hardware fa ilures and hardware or software anomalous behavior, (4) Addition of barriers, such as in terlocks, lockouts, and lock-ins, (5) Functional cohesion, isolation, and decoupling (partitioning),

NOTE: This position paper has been coordinated am ong the software specia lists of certification authorities from the United States, Europe, and Canada. However, it does not constitute official policy or guidance from any of the authorities . This document is provided for educational and informational purposes only and should be discussed with the appropriate certification authority when considering for actual projects. 10(6) Exposure reduction, (7) Simplification of design for increased verification and va lidation certainty.

d. Traceability and verifiability

. Aircraft hazards and system failure conditions

to which software can cont ribute, or which are contro lled by software, should be traceable to related software requirem ents, architecture, design components and constraints. These, in turn, should be trac eable to the code, an d to the verification procedures and cases that verify them under normal and abnormal conditions. Traceability should be shown from the hazards and failure conditions to the source code,

and verification should be conduc ted to the same software level criteria as stated by

reference a. criteria for the same software leve l. It is preferred that all safety-related software requirements-based testing be perf ormed on the target ex ecutable code loaded on the target computer in th e target environment. Testi ng on a high fidelity simulated environment can be acceptable if eviden ce exists to support the validity of the environment for the specific tests. A high fi delity simulated environment is one that provides identical or similar computing res ources, peripherals, interfaces and inputs and outputs such that tests conducted on the simula ted environment would result in identical,

valid results to tests conducted in the actua l operational environment. Qualification of

the simulated environment as a verification tool may be needed. For a SEASA program, it is preferred that all safety-related requi rements be verified on the executable object code integrated into the target computer and environment, or high fidelity simulated environment if applicable. If identical equipment cannot be used for technical reasons, deviations from the identical equipment must be presented to and accepted by the certification authority.

e. Verification

. A combination of both static and dynamic analyses should be specified by the applicant/developer and applied to the software. As with reference a., the emphasis of software verification shoul d be focused on safety-related components and potential failure conditions of the soft ware. Software reviews of requirements, design, architecture, code, integration, veri fication cases and procedures, and coverage analysis that would be expected under refere nce a., are applicable to SEASA as well. In addition, system safety analysis methods and fail-safe design techniques should be applied to the software to determine safety-r elated components and the safety effects of the software logic. Verification should be performed at the same level of rigor as specified by reference a. verifica tion criteria for the same soft ware level. Therefore, for a SEASA application, safety related requireme nts should be verified on the executable object code integrated into the target com puter and environment. If safety related software components are properly isolated, partitioned, protected and decoupled from other software components, then structural coverage criteria of reference a. may be reduced for the non-safety related components, as is allowed by reference a. Software testing should concentrate on showing that the safety-related components perform according to the software safety requirements and that safety-related components are not

NOTE: This position paper has been coordinated am ong the software specia lists of certification authorities from the United States, Europe, and Canada. However, it does not constitute official policy or guidance from any of the authorities . This document is provided for educational and informational purposes only and should be discussed with the appropriate certification authority when considering for actual projects. 11adversely affected by other, non-safety-related components (verifying the integrity of the partitioning).

f. Software configuration management and software quality assurance

. The configuration management and quality assu rance criteria for a SEASA program are

identical to those of reference a. and s hould include QA evaluation and oversight of SEASA processes as well as configuration management of all SEASA specific data

items.

g. Data items

. All software life cycle data items of reference a. should be produced for a SEASA program, with the ex ception of items that the certification authority has allowed a reduction or replacemen t to the reference a. criteria. SEASA will likely result in additional SEASA-specific data items which replace reference a.-specified data items. Examples could be: system safety assessment results for software, justification of SEASA criteria as acceptable for satisfying one or more reference a. objectives, reference a. compliance matrix, st ructural coverage an alysis of non-safety related components. In addition, all soft ware data items needed to support the applicant's/developer’s SEASA process are need ed. This includes those needed to support the integrated safety for system, hardware and software program. Additions to the content of reference a. software data items will be needed to support a SEASA application.

h. Software levels

. Software levels are determined by the severity of the aircraft hazards and system failure conditions to which either the software could contribute, or is

intended to control. Softwa re levels should be determined consistent with system assurance level guidance in reference a. firs t and then reference g. If there is any

difference in the determination of the software level as determined by references a. and g., the software level determination should be discussed and agreed-to by the specific project’s certification authority. Any area of software to which SEASA will not be applied should comply with reference a. for th e appropriate software level, as determined by the system safety assessment (SSA).

i. Protective mechanisms and protected components

. Where protective mechanisms are applied, the assignment of so ftware level for a prot ected, safety-related software component should be determined by the worst case failure condition of that component, assuming the protective mechanism( s) are operable. The software level assigned to each protective mechanism shoul d be based on the worst-case failure

conditions that would result from a failure of that protective mechanism in combination with the failure of the protected software component. This assignment should consider:

(1) The failure condition which could result from the failure or anomalous behavior of the protective mechan ism to properly deploy and the failure or

anomalous behavior of th e protected component, and

NOTE: This position paper has been coordinated am ong the software specia lists of certification authorities from the United States, Europe, and Canada. However, it does not constitute official policy or guidance from any of the authorities . This document is provided for educational and informational purposes only and should be discussed with the appropriate certification authority when considering for actual projects. 12(2) The worst case failure condition of an inadvertent deployment of the protective mechanism.

j. Software level exceptions

. Exceptions to i(2) may occur where application of SEASA results in mitigation of the effect of the inadvertent deployment.

k. Reference a. criteria exceptions

. Once the software level is assigned, all objectives of reference a. should be satisfi ed for the assigned software level; except where the applicant or deve loper has negotiated and obtai ned concurrence from the certification authority that their SEASA criteria, activity or data repr esents an equivalent means of satisfying the objective. Exceptio ns and equivalent means to reference a. criteria should be justified, documented, a nd agreed-to early in the product development with the certification authority. Analysis fr om the actual applicati on of their SEASA may, in some cases, determine that additional verification is needed to augment the SEASA method. In these cases, the applicant or developer should provide appropriate

verification results.

l. Software Reviews

. Software reviews should be conducted by the certification authority (and designees, if delegated) in th e same manner and to the same criteria as with a reference a. review. The approva l of SEASA processes and data should not require any extraordinary compliance monito ring beyond that required for a “standard” reference a. program. However, the reviewer s should be familiar with the application of system safety assessment methods and fail-safe design techniques.

m. SEASA proposals

. An applicant or develope r should present their SEASA proposal to the certification authority and gain their approval of the proposal early in the aircraft program or system life cycle to reduce the risk to th e aircraft program or system project associated with non-concurrence by the certification authority. The proposal should include a reference a. compliance matrix showing which objectives will be satisfied by their SEASA criteria, activities and data, including independence and data control categories, which will be partially sa tisfied by their SEASA criteria, activities and data, and which will not be satisfied by their SEASA criteria, activities and data. The proposal should include a justification for any objectives that will not be fully satisfied by the SEASA proposal. Failure of the applican t or developer to submit the compliance matrix and justification would be grounds to disallow the SEASA proposal immediately.

n. SEASA evaluations

. The certification authority should make every reasonable effort to evaluate the applicant’ s or developer’s SEASA proposal in a timely manner, and accept the proposal, accept the pr oposal with conditions, or reject the proposal with rationale for its unacceptability. It is in the best intere sts of the certification authority, the aircraft manufactur er, the applicant, the system developer and the software

NOTE: This position paper has been coordinated am ong the software specia lists of certification authorities from the United States, Europe, and Canada. However, it does not constitute official policy or guidance from any of the authorities . This document is provided for educational and informational purposes only and should be discussed with the appropriate certification authority when considering for actual projects. 13developer to resolve and come to agr eement on SEASA proposals early in the certification program.

6. Conclusion

.

This paper presents some considerations and guidelines for certification authorities to use when evaluating applicants’ proposals to us e system safety assessment and engineering methods and techniques to providing software assurance for airborne systems and equipment. Although alternate approaches were considered, they must still be carefully evaluated on a case-by-case basis. The use of these approaches may be acceptable, but

should be combined with a “t otal” system safety program and current “best-practice” software guidelines. Applicants should prepar e and provide descripti ons of their proposed approach and rationale for its acceptability fo r software assurance, which can then be evaluated by the certification authorities to determine the acceptabili ty of their proposed alternative approach.

NOTE: This position paper has been coordinated am ong the software specia lists of certification authorities from the United States, Europe, and Canada. However, it does not constitute official policy or guidance from any of the authorities . This document is provided for educational and informational purposes only and should be discussed with the appropriate certification authority when considering for actual projects. 14