Certification Authorities Software Team (CAST)

Position Paper

CAST-13

Automatic Code Generation Tools Development Assurance

Completed June 2002

Automatic Code Generation Tools Development Assurance

**Abstract:**

This paper clarifies DO-178B/ED-12B section 12.2.1.b, as it applies to
automatic code generation (ACG) tools. DO-178B/ED-12B proposes that the
software level of a development tool should be considered at least the
same as the software level of the airborne system's software application
the tool is being used to develop, **unless** the applicant can justify
a reduction in software level of the tool to the certification
authority. According to DO-178B/ED-12B:

"*A reduction in a tool's software level can be based upon the
significance of the software verification process activity to be
eliminated, reduced or automated, with respect to the entire suite of
verification activities. This significance is a function of:*

*-the type of software verification process activity to be eliminated,
reduced or automated ...*

*-the likelihood that other verification activities would have detected
the same errors*."

This paper proposes a list of candidate objectives in DO-178B/ED-12B
that could potentially be alleviated, when applicants are qualifying
Automatic Code Generation Tools, provided that the applicants supply
relevant rationale and justification for each objective's alleviation.
The paper also provides a road map to potentially reduce the ACG tool's
software level relative to the level of the airborne software.

The approach proposed in this paper is only one of many potential
approaches. Other approaches would need to be coordinated with the
appropriate certification authorities.

**Key words:**

Tool qualification, autocode generators, development tools, code,
verification, software level, objectives, tool software level

# **1 INTRODUCTION**

This paper clarifies DO-178B/ED-12B section 12.2.1.b, as it applies to
automatic code generation (ACG) tools. DO-178B/ED-12B proposes that the
software level of a development tool should be considered at least the
same as the software level of the airborne system's software application
the tool is being used to develop, **unless** the applicant can justify
a reduction in software level of the tool to the certification
authority. According to DO-178B/ED-12B:

"*A reduction in a tool's software level can be based upon the
significance of the software verification process activity to be
eliminated, reduced or automated, with respect to the entire suite of
verification activities. This significance is a function of:*

> *-the type of software verification process activity to be eliminated,
> reduced or automated ...*
>
> *-the likelihood that other verification activities would have
> detected the same errors*."

This paper proposes a list of candidate objectives in DO-178B/ED-12B
that could potentially be alleviated, when applicants are qualifying
Automatic Code Generation Tools, provided that the applicants supply
relevant rationale and justification for each objective's alleviation.
The paper also provides a road map to potentially reduce the ACG tool's
software level relative to the level of the airborne software.

The approach proposed in this paper is only one of many potential
approaches. Other approaches would need to be coordinated with the
appropriate certification authorities.

# **2 BACKGROUND**

According to DO-178B/ED-12B chapter 12.2.1.b, the software level of a
development tool should be at least the same as that of the airborne
system\'s software being generated by the tool, if the code generated by
that tool is not verified as defined in DO-178B/ED-12B Section 6. Since
an ACG is a development tool, this implies that ACG tools need at least
the same level of assurance as the airborne software they produce.

DO-178B/ED-12B chapter 12.2.1.d recognizes that the objectives of the
tool's software verification are different from those of the airborne
software. It recognizes implicitly, therefore, that not all verification
objectives of DO-178B/ED-12B may be fully applicable to an ACG tool. It
provides a list of verification activities for software development
tools that is a subset of those suggested for airborne software.

Additionally, according to DO-178B/ED-12B chapter 12.2.1.b, the
applicant can justify a reduction in software level of the tool to the
certification authority. A reduction in a tool's software level can be
based upon the significance of the software verification process
activity to be eliminated, reduced or automated, with respect to the
entire suite of verification activities. This significance is a function
of:

-   *The type of software verification process activity to be
    eliminated, reduced or automated*

-   *The likelihood that other verification activities would have
    detected the same errors*."

Therefore, the accurate way to determine the necessary qualification
process of the ACG tools (the applicable verification objectives) is
somewhat debatable and requires clarification. There is a need to
identify the verification objectives that may not be applicable to the
ACG tools and rationale for why they may not be applicable, and to
justify why other applicable objectives can be satisfied without
reducing the level of assurance for the airborne software application.

# **3 DISCUSSION**

## 

## ***3.1 ACG context presentation***

ACG tools are applications that execute on ground computers using
commercial off-the-shelf (COTS) operating systems.

ACG tools may directly translate the software specification (low-level
and/or high-level requirements) into source code, thereby potentially
eliminating or automating several software development and verification
activities. ACG tools are generally composed of two entities:

-   Entity 1: The library of elementary symbols (e.g., code primitives
    or basic functions) that contain basic symbols that contain the
    source code associated with implementing the function of each
    elementary symbol.

-   Entity 2: The architect (e.g., tool logic and program constructor)
    that reads the software specification and selects the association to
    the elementary symbol, and may insert the corresponding source code
    for each selected symbol.

The qualification approach for each of these entities may vary.

When tools do not eliminate, reduce, or automate software processes
necessary to be performed on the airborne software, those tools do not
need to be qualified.

When tools eliminate, reduce, replace, or automate life cycle process
activities on the airborne software, without their output being verified
as specified in Section 6 of DO-178B/ED-12B, the tools should be
qualified.

Once it is determined that a development tool needs to be qualified, the
software level of the tool to be qualified needs to be determined.

In the case where there is no justification to reduce the software level
of the tool, the tool should be qualified to the same software level and
objectives as that of the airborne software.

In the case where there is a justification to reduce the tool's software
level, the verification objectives and activities for the reduced
software level should be satisfied. Justification for reducing the
software tool's level should be evaluated and analyzed addressing the
following issues:

-   What are the possible consequences of a tool's software level
    reduction on the airborne software product itself (for example,
    possible undetected errors in the airborne software and implications
    on the system)?

-   What are additional, complementary verification activities proposed
    for the tool's output or on the airborne software to detect these
    errors, or mitigate software effects of these errors on the system?

-   What analysis supports the proposed reduction of the tool
    qualification objectives?

## ***3.2 ACG tool qualification level and objectives clarification for potential alleviation***

According to Section 12.2.1.b of DO-178B/ED-12B, a development tool
should be qualified to the level of the embedded software. The intention
of this statement is explained in section 12.2.1.a: the software
development processes for the tool should meet the same objectives as
the software development processes of the airborne software.

The technical objective is that ACG tools should not introduce
unintended or erroneous code in the embedded software. The ACG tool
behavior should be deterministic; i.e., the produced code should not
differ when the input data does not.

More precisely, the primary issue for an ACG tool is the production of
source code that does not comply with its requirements, but can still be
compiled without any error detected and is executable. For airborne
software, the same event can occur, but it's not the only one. Halts
during execution, overflows, variations in time response, hardware and
software incompatibilities, hardware failures, unbounded recursive
algorithms, bad stack usage, resource contention, tasks conflicts, bad
interaction with others systems, etc. are examples of issues which may
jeopardize flight safety, if they appear in aviation software. However,
these types of errors may not have any influence on the flight safety,
if they occurred in the ACG tool that generated the aviation software.

Therefore, the following list better describes how to demonstrate
compliance to DO-178B/ED-12B for a typical ACG tool (i.e., one that uses
the two entities described above):

-   The operating system on which the ACG tool executes should be
    considered. The applicant should demonstrate that the tool is
    designed and developed in such a way that erroneous functioning of
    the operating system cannot produce unintended or erroneous code
    (e.g., showing tool operational requirements in abnormal
    conditions). Neither can it jeopardize determinism properties of the
    tool (i.e., the produced code should not differ when the input data
    does not, as previously mentioned).

-   The ACG tool entity 1 (symbol libraries) should be shown to comply
    with all the objectives of DO-178B/ED-12B.

-   The ACG tool entity 2 (program architect), should satisfy all of the
    objectives of DO-178B/ED-12B (for the applicable level), except for
    a list of candidate objectives, proposed according to DO178B/ED-12B
    chapter 12.2.1.d, and any additional field experience. These
    candidate objectives are objectives that applicants could consider
    for alleviation or removal. The candidate objectives for alleviation
    or removal are: A-3 (3), A-4 (3, 10, 11), A-5 (6), A-6 (partial 2,
    partial 4, 5), and A-7 (5). All other objectives should be applied,
    unless further justification is provided. The table below summarizes
    the DO-178B/ED-12B reference, alleviation rationale, and suggested
    demonstration approach for each of the candidate objectives. Each
    applicant should provide rationale and justification for alleviation
    or removal of each objective, but this provides a starting point:

  ---------- ----------- ------------------------------ -------------------------
  **Obj**    **Ref.**    **Rationale**                  **Demonstration
                                                        requirements**

  A-3 (3)    6.3.1.c     This objective could be        Applicant justification
                         alleviated. ACG tool           is needed. It should be
                         incompatibility with the       demonstrated that the
                         target computer will likely be objective alleviation can
                         obvious as the ACG tool will   only cause ACG tool
                         likely not execute nor would   availability problems and
                         the ACG tool likely satisfy    cannot lead to ACG tool
                         its Tool Operational           integrity problems (i.e.,
                         Requirements. The hardware and production of unintended
                         software incompatibilities are or erroneous code).
                         managed by the operating       
                         system using hardware          
                         resources (e.g., CPU, memory,  
                         I/O devices, etc. installed in 
                         a PC or a Unix Workstation).   
                         Therefore, incompatibility     
                         problems may be obvious and    
                         readily detected.              

  A-4 (3)    6.3.2.c     This objective could be        Applicant justification
                         alleviated. ACG tool           is needed. It should be
                         incompatibility with the       demonstrated that the
                         target computer will likely be objective alleviation can
                         obvious as the ACG tool will   only cause ACG tool
                         likely not execute nor would   availability problems and
                         the ACG tool likely satisfy    cannot lead to ACG tool
                         its Tool Operational           integrity problems (i.e.,
                         Requirements. The hardware and production of unintended
                         software incompatibilities are or erroneous code).
                         managed by the operating       
                         system using hardware          
                         resources (e.g., CPU, memory,  
                         I/O devices, etc. installed in 
                         a PC or a Unix Workstation).   
                         Therefore, incompatibility     
                         problems may be obvious and    
                         readily detected.              

  A-4 (10)   6.3.3.c     This objective could be        Applicant justification
                         alleviated. ACG tool software  is needed. It should be
                         architecture may not be        demonstrated that the
                         defined at the ACG tool level. objective alleviation can
                         An ACG tool is application     only cause ACG tool
                         software running on a          availability problems and
                         workstation operating system   cannot lead to ACG tool
                         with its own task management,  integrity problems (i.e.
                         memory management and          production of unintended
                         scheduling. Incompatibility    or erroneous code).
                         problems between the ACG       
                         tool's architecture and the    
                         computer may be obvious and    
                         readily detected, and the ACG  
                         tool would likely not satisfy  
                         its Tool Operational           
                         Requirements if                
                         incompatibilities existed.     

  A-4 (11)   6.3.3.d     This objective could be        Applicant justification
                         alleviated. The recursive      is needed. It should be
                         algorithm is allowed if the    demonstrated that the
                         unbounded recursive algorithm  objective alleviation can
                         is detected and cannot lead to only cause ACG tool
                         production of unintended or    availability problems and
                         erroneous code. An infinite    cannot lead to ACG tool
                         loop has different             integrity problems (i.e.
                         implications for a tool's      production of unintended
                         execution than it does for     or erroneous code).
                         airborne software, and would   
                         likely be detected during tool 
                         execution; also, the operator  
                         could stop it. Nevertheless,   
                         the software architecture is   
                         partially verified during Tool 
                         Operational Requirements       
                         testing.                       

  A-5 (6)    6.3.4.f     This objective remains         Applicant justification
                         applicable except for the      is needed. It should be
                         worst-case execution timing,   demonstrated that the
                         stack usage, resource          objective alleviation can
                         contention, task or interrupt  only cause ACG tool
                         conflict, which could all be   availability problems and
                         possibly alleviated.           cannot lead to ACG tool
                         Worst-case execution is not an integrity problems (i.e.
                         issue for the ACG tool         production of unintended
                         execution as it only impacts   or erroneous code).
                         tool availability. Stack usage 
                         is not an issue, if the data   
                         used to produce source code    
                         are not corrupted. Resource    
                         contention is not an issue if  
                         data used to produce source    
                         code are not corrupted. Task   
                         or interrupt conflict may not  
                         be an issue, if it only        
                         impacts ACG tool real-time     
                         availability and cannot        
                         corrupt data used to produce   
                         source code.                   

  A-6        6.4.2.2 b   This objective could be        Applicant justification
  (partial 2             potentially alleviated. Any    is needed. It should be
  and                    system initialization problems demonstrated that the
  partial 4)             will likely be obvious and     objective alleviation can
                         result in temporary or         only cause ACG tool
                         permanent ACG tool             availability problems and
                         unavailability or the need to  cannot lead to ACG tool
                         restart the tool or operating  integrity problems (i.e.
                         system. Also, the abnormal     production of unintended
                         conditions will likely be      or erroneous code).
                         obvious.                       

  A-6        6.4.2.2 c   This objective could be        Applicant justification
  (partial 2             alleviated. There is no data   is needed. It should be
  and                    coming from external systems.  demonstrated that the
  partial 4)             Input data may be formalized   objective alleviation can
                         requirements specifications    only cause ACG tool
                         and the output data is source  availability problems and
                         code. An ACG tool is not a     cannot lead to ACG tool
                         system, but an application     integrity problems (i.e.,
                         running on an operating        production of unintended
                         system. If the formalized      or erroneous code). The
                         specification contains errors  applicant should also
                         or defects, the tool should    demonstrate that any
                         detect them.                   error or defect in
                                                        formalized specification
                                                        would be detected by the
                                                        tool.

  A-6        6.4.2.2 e   This objective could be        Applicant justification
  (partial 2             alleviated. The operating      is needed. It should be
  and                    system is managing real-time   demonstrated that the
  partial 4)             resources, and time frame      objective alleviation can
                         exceedances should only lead   only cause ACG tool
                         to temporary or permanent      availability problems and
                         unavailability of the ACG      cannot lead to ACG tool
                         tool.                          integrity problems (i.e.,
                                                        production of unintended
                                                        or erroneous code).

  A-6        6.4.2.2 f   This objective could be        Applicant justification
  (partial 2             alleviated. An ACG tool        is needed. It should be
  and                    generally does not have        demonstrated that the
  partial 4)             real-time constraints, as do   objective alleviation can
                         some airborne software         only cause ACG tool
                         applications. It is a          availability problems and
                         ground-based application       cannot lead to ACG tool
                         running on an operating        integrity problems (i.e.,
                         system, which has its own time production of unintended
                         and task management schemes.   or erroneous code).
                         Problems in this area should   
                         only lead to temporary or      
                         permanent unavailability of    
                         the ACG tool.                  

  A-6 (5)    6.4.3a      This objective is applicable.  Applicant justification
                         Nevertheless, activities that  is needed. It should be
                         check for real-time properties demonstrated that the
                         (such as, memory overflow,     objective alleviation can
                         failure to detect execution    only cause ACG tool
                         time requirement's anomalies,  availability problems and
                         inability of built-in test to  cannot lead to ACG tool
                         detect failures, and stack     integrity problems (i.e.,
                         overflows) may not be          production of unintended
                         applicable for ground-based    or erroneous code).
                         software applications.         

  A-7 (5)    6.4.4.2.b   This objective remains         Applicant justification
                         applicable except for          is needed. It should be
                         additional verification on the demonstrated that the
                         object code of the tool (for   objective alleviation can
                         Level A tools). Problems in    only cause ACG tool
                         object code not directly       availability problems and
                         traceable (due to usage of an  cannot lead to ACG tool
                         OS for example) to the ACG     integrity problems (i.e.,
                         tool's source code are not an  production of unintended
                         issue as they occur on the     or erroneous code).
                         ground, may affect only        
                         real-time availability of the  
                         ACG tool, cannot corrupt data  
                         used to produce source code,   
                         and cannot lead to producing   
                         unintended or erroneous code.  
  ---------- ----------- ------------------------------ -------------------------

Potential errors caused by the objectives alleviated should be
demonstrated as not being a potential source of errors in the airborne
software. The potential errors should be shown as detectable by other
means and should not lead to production of erroneous code. Remaining
objectives should be applied at the same level as the airborne software
level, unless the applicant can justify that an objective or activity
alleviation cannot be the cause of errors in the airborne application.

Each time an objective is alleviated, the alleviation of the objective
should be described and justified.

## ***3.3 ACG software level reduction and road map proposal***

According to the previous section, DO-178B/ED-12B objectives that remain
applicable for an ACG tool should be applied at the same software level
as the embedded airborne software's level, unless the applicant can
justify reduction of the tool\'s software level or proposes other means,
such as additional verification activities to satisfy an objective.
Therefore, applicants could propose their reduction, justification and
additional means for alleviating further objectives than the ones
proposed in Section 3.2 above for ACG tool entity 2.

If it can be shown that software functions included in ACG tool entity 2
cannot contribute to the production of non-deterministic, unintended,
and erroneous code, the tool's software level for these items could
potentially be reduced to the associated failure condition
classification to which the tool's functions could contribute, if the
tool's functions can be partitioned, and independently verified.

If it cannot be shown that a software function included in ACG tool
entity 2 cannot contribute to the production of non-deterministic,
unintended and erroneous code, all objectives applicants propose to
satisfy for the tool should be satisfied to the same level as the
airborne software application which the ACG tool generates. Any possible
consequences of further reduction of the ACG tool software level or
objectives' alleviation (further to what is proposed in section 3.2)
should be evaluated for their potential impact on the airborne software
product, to justify the final ACG tool\'s software level and
qualification claimed.

-   A: Potential errors resulting from the ACG tool software level
    reduction can be detected by other means (including additional
    verification performed on the code produced). The equivalency of
    other means should be assessed on the ability of the method to
    detect exactly the same kind of errors as the verification objective
    that is being alleviated would have detected.

-   B: Potential errors and failure conditions in the airborne software
    as a result of the ACG tool\'s software level reduction or
    objectives\' alleviation cannot contribute to the failures of the
    airborne software. In this case, the ACG tool software level could
    perhaps be lower than the software level of the airborne software.
    The applicant can propose a reduction based on a safety assessment,
    software architecture, tool partitioning, additional verification,
    and/or by other means, for consideration by the certification
    authority on a case-by-case basis.

The flowchart in Appendix A illustrates the roadmap described in this
section.

# **4 CAST POSITION**

This paper proposes a list of candidate objectives in DO-178B/ED-12B
that could potentially be alleviated, when applicants are qualifying ACG
tools, provided that the applicants supply relevant rationale and
justification for each objective's alleviation.

The paper also provides a roadmap to potentially reduce the ACG tool's
software level relative to the software level of the airborne software.

CAST considers the proposed approach, if properly applied by applicants,
to be an acceptable way to alleviate DO-178B/ED-12B objectives for ACG
tool qualification, as well as to potentially reduce the ACG tool's
software level relative to the software level of the airborne software.

The approach proposed in this paper is only one of many potential
approaches. Other approaches would need to be coordinated with the
appropriate certification authorities.

**APPENDIX A**

![](media/image1.wmf)