**Assurance Process Mapping**

**J. Steven Newman, D.Sc.** ^(1)^**, Stephen M. Wander, M.E.A.**
^(2)^**, Don Vecellio, B.S.M.E.** ^(3)^

*^(1)^ NASA Headquarters, 300 E Street SW, Washington, D.C. 20546 USA,
Email: j.s.newman@nasa.gov*

*^(2)^ NASA Headquarters, 300 E Street SW, Washington, D.C. 20546 USA,
Email: stephen.m.wander@nasa.gov*

*^(3)^ ARES Corporation, 201 12^th^ Street South, Ste. 601, Arlington,
VA 22202 USA, Email: dvecellio@arescorporation.com*

**ABSTRACT**

Assurance Process (AP) Mapping is a means to graphically depict the
multi-dimensional assurance processes implemented on organizationally
complex NASA aerospace programs. AP Mapping helps to better understand
and manage program/project complexity, particularly as it relates to
managing key assurance activities and functions. Ultimately, the AP Map
assists decision makers by providing knowledge of the governing
assurance requirements, a baseline understanding of the assurance
processes implemented on the program, and visibility into their
implementation.

NASA and international space programs often involve multiple:
development and implementation phases; project elements; flight
elements; contractors; subcontractors; governments; and governmental
agencies. Assurance processes must be implemented throughout the program
lifecycle and across all levels of the program organization. Supporting
this lifecycle framework are a myriad of "assurance agents," individuals
accountable for implementation of governing assurance requirements and
the organizations that review them, each with some element of the
overall assurance picture.

This paper provides a high-level tutorial and examples of the AP-Mapping
protocol and graphical techniques that have been most recently applied
to the Space Shuttle Program in preparation for Return to Flight.

1.  **INTRODUCTION**

Assurance Process (AP) Mapping is a methodology developed in the mid
1990's by J.S. Newman and S.M. Wander as a means to graphically depict
the multi-dimensional assurance processes implemented on
organizationally complex NASA programs and projects. AP Mapping is a
means to better understand and manage complexity in two key ways:

First, as a management tool to identify gaps in assurance coverage and
accountability and provide a basis for resource requirements definition
and justification. Second, as a communication tool to provide a quick
and effective means of communicating the overall safety and mission
assurance (SMA) function to program management, NASA Center management,
the Mission Directorate, and, ultimately, the Chief Safety and Mission
Assurance Officer.

***\"A wonderful harmony arises from joining together the seemingly
unconnected.**\"*

-- *Heraclitus c.500 BC*

Early analyses were performed for the X-33, X-34, X-37, Quiksat, FUSE,
and NASA ELV programs. Over the past five years the methodology has
evolved from an art toward a science. Most recently, AP Mapping has
played a key role in NASA Review and Assessment Division (RAD)
Programmatic Audit and Review activity, and it has been further refined
by the authors in partnership with RAD support contractors from the ARES
Corporation.

This paper provides a high-level tutorial and examples of the AP Mapping
protocol and graphical techniques that have been recently applied to the
Space Shuttle Program in preparation for Return-to-Flight. The paper
also explores the advantages of an enhanced electronic AP Map compared
to the inherent limitations of a two-dimensional or paper copy AP Map.
Electronic or e-Map features can include hyper-links and pull-down
menus. The concept of building layers, each representing a unique
dimension or aspect of the overall assurance picture, is also explored
along with the challenges of implementing a composite AP Map.

2.  **THE ASSURANCE IMPERATIVE**

The AP Map was originally developed as a tool to assist and enable the
effective management of assurance responsibilities, i.e., "corporate due
diligence," that compel managers (by law and by good practice) to
acquire knowledge of the governing assurance requirements, to develop a
baseline understanding of the assurance processes implemented on the
program, and to gain visibility into their implementation. Principle
beneficiaries of AP Mapping and analysis include safety and mission
assurance managers, independent assessment bodies such as the US
Government Accounting Office (GAO), the Agency Inspector General, and
special "blue ribbon" investigation teams. The most important customer,
however, is the program or project manager who can employ the AP Mapping
process as a program life-cycle management tool to assist in ensuring
the integration and effectiveness of the overall mission assurance
network. Ideally, the program manager and the entire program management
team would collaboratively develop the map and matrix establishing
shared ownership.

3.  **BENEFITS OF ASSURANCE PROCESS (AP) MAPPING**

AP Mapping establishes the goal of assembling the integrated or holistic
view of the totality of assurance activities within a program/project.
An immediate benefit of the AP Mapping process is the ability to see and
understand key SMA roles, responsibilities, and interrelationships. In
summary it:

-   Provides management a *systems view* of the assurance net,

-   Provides a formalized structure of program SMA complexities and
    *actively tracks changing participants and responsibilities*,

-   Facilitates implementation of mission assurance processes,

-   Identifies *potential gaps* in assurance coverage and
    accountability,

-   Provides program management with the basis for *resource
    adjustments*,

-   Provides critical and reusable background information for
    *independent assessment and review* *teams* (e.g., Corporate, ASAP,
    IG, GAO),

-   Facilitates *communication* between levels of program management,

-   Provides *rapid insight* to the most crucial assurance activities,
    and

-   Provides basis for management to *support major operations and
    launch decisions*.

4.  **THE COMPLEX, MULTI-VARIATE ENGINEERING MANAGEMENT ENVIRONMENT**

Every flight or operational element and every program phase has a set of
governing assurance requirements with specific organizations and
individuals accountable for implementation. Organizational complexity
within a given program (in-line reporting, matrixed reporting
relationships), multiple government and/or commercial entities, multiple
communication interfaces, and geographical dispersion are all factors
that complicate understanding and management of the overall assurance
effort.

NASA and international space programs are typically comprised of
multiple flight elements including the booster, spacecraft, and payload
of scientific instruments, or human occupants. Space programs and
missions are also multi-phase, with the need to implement life-cycle
assurance processes beginning in conceptual design, advancing through
critical design, into design verification testing and analysis,
manufacturing and manufacturing verification, leading to integrated
systems test and operation. System elements are typically integrated at
the launch site where hazardous processing and final test & check-out
occur. Once on-orbit, another set of organizations and processes engage
in the tasks of safely conducting and managing on-orbit operations.
Finally, end-of-life mission operations require other assurance
organ-izations to minimize orbital debris and ensure public safety
during any planned or unplanned re-entry phase.

Within this life-cycle framework there exists a myriad of "assurance
agents," individuals, and reviewing organizations responsible for some
element(s) of the overall assurance picture. Examples include quality
assurance inspectors embedded with the prime contractor; government
inspectors conducting safety critical inspections; assurance process
oversight by the program manager; installation or NASA Center-based
assurance oversight of the program; independent agency-level assurance
oversight of NASA Centers and installations; engineering oversight of
program safety-related decisions; Agency, Center, and program level
reviews; and outside independent assessment teams such as NASA's
Aerospace Safety Assurance Panel.

5.  **ASSURANCE PROCESS (AP) MAPPING DIMENSIONS AND VARIABLES**

The AP Map is typically supported by an associated AP Matrix and AP
Directory. The Matrix allows presentation of detailed assurance process
descriptions while the Directory provides specific assurance agent
affiliation and contact information. The collective of the map, matrix,
and directory capture information and knowledge that can be indexed to
the variables described below:

1.  **Program Phase**

Recent AP Mapping experience has demonstrated the need for establishing
program phase-dependent AP Maps. Programs typically span three to six
years and evolve through the classic systems engineering life-cycle. The
network of assurance agents will vary as a program progresses from
requirements definition to design, to manufacturing, test, integration,
operations, sustaining engineering and finally retirement. Thus the AP
Mapping process provides an ideal tool for managing the dynamic nature
of the SMA function as it evolves over the various program/project
development stages.

2.  **Design / Implementation Activity**

Differentiation of design and implementation activities for both
hardware and software is important. NASA programs typically employ
different assurance agents (and/or assurance organizations) operating in
these two arenas. A rigorous and complete AP-Map and matrix will ensure
inclusion of both hardware and software design, test, and integration
activities.

3.  **Governing SMA Policies, Procedures and Requirements**

The governing assurance process requirements for a program (usually
identified in the AP Matrix) are linked with key assurance processes
shown on the AP Map. The set may include Agency policies, procedures, or
standards, program assurance requirements documents, and prime
contractor process documentation, all of which govern the implementation
of program assurance requirements. The AP Map provides a visual
affirmation assisting in flow-down analyses undertaken to ensure that
assurance process requirements are being implemented. The AP Map also
assists programmatic audit and review teams in scoping their reviews and
understanding the linkage between policies, require-ments, processes,
and practices.

4.  **Program Organizational Structure**

Most of NASA's aerospace programs are inherently complex. This is
particularly true for those that involve multiple NASA Centers, large
numbers of contractors, suppliers, and international partners. The
baseline AP Map provides the starting point for portraying the
interactions of assurance agents with program elements over the program
life-cycle for these organizationally complex programs.

5.  **Agency Headquarters and Center Management Organization Structure**

Because NASA programs all exist within Headquarters Mission Directorates
and are implemented (typically) at NASA Centers, it is important to also
show the appropriate accountability linkages among the Mission
Directorate, program management, and Center manage-ment. Likewise, it is
important to show linkage between Center SMA organizations and the
Headquarters-based Office of Safety and Mission Assurance.

An alternative and/or companion approach to AP-Mapping for depicting
organizational accountability was examined in the case of the Space
Shuttle Program. This approach involved building an "Assurance
Accountability Work Breakdown Structure." This disciplined outlining
process, while potentially useful, quickly became cumbersome requiring
seven or more levels of indenture (e.g., 2.1.3.4.5.6.1) to identify the
accountability of an individual "assurance agent."

6.  **Assurance-Organizations**

Multiple assurance organizations and agents are required to interact
during a typical program/project life-cycle. AP Mapping for complex
programs will invariably require a multi-layer approach in which
different independent assurance networks can be represented relative to
a program's basic organizational structure. Examples include the program
assurance infrastructure, the prime contractor assurance organizations,
the Agency SMA organization, the Center or facility SMA organization,
other Federal agency assurance organizations such as the FAA and
Department of Defense (supporting range safety), the Department of
Energy (supporting nuclear payload safety), other Agency independent
assurance groups such as the Software Independent Verification and
Validation Facility, the Independent Technical Authority, the NASA
Engineering and Safety Center, and the Independent Program Assessment
Office (IPAO). Other key assurance agents include the Defense Contract
Management Agency (DCMA) and assurance support contractors such as the
Aerospace Corporation.

[]{.mark}

One particularly difficult challenge is to capture or depict
"dotted-lined" or matrixed reporting relationships wherein an individual
assurance agent "wears two hats," reporting to an independent or
functional organization but accountable to the program manager.

7.  **In-Line vs. Independent Assurance Activities**

Assurance activities are typically characterized as "in-line," that is
belonging to and accountable to the program manager or, more generally,
that which exists within the program management chain or organizational
hierarchy, versus an "independent" reporting and accountability chain
that does not flow through the program manager. In-line assurance
activities include the program, project, prime, and sub-tier contractor
assurance activities including: assurance planning, risk management,
quality assurance, system safety and reliability analyses, verification,
inspection, surveillance, and review. Descriptions are also included
addressing scope, frequency, and depth of penetration for a given
activity. In the case of independent assurance activities, the
performing organization's level or degree of independence can be defined
on the basis of funding source, accountability, and/or reporting
responsibility. Independent analyses and process review activity are
identified with appropriate identification of scope, frequency, and
depth of penetration.

8.  **Assurance Process & Review Forums**

Key in-line and independent assurance review forums and boards are also
shown on the AP Map. Depending upon which development phase the map
represents, one will find a varying combination of boards and review
panels. These typically include preliminary requirements review panels,
program configuration control boards, system safety review panels,
material control board, NASA Center director or program manager reviews,
safety and mission assurance reviews, pre-launch operational review
forums, or mission management team meetings.

9.  **Information Flow**

An important dimension in the AP Map is the flow of information between
assurance agents or organizations and the myriad of assurance forums.
Information channels are shown between all elements on the AP Map, both
in-line and independent.

6.  **CREATING AN AP MAP**

The AP Map is intended to be used as a one-page, graphical "roadmap" to
help understand the SMA "what," "how," and "who" of the program/project.

1.  **Key AP Map Elements**

Key elements of the map include the "documents," the "boxes," and the
"arrows" (see Table 1).

Table 1. Key AP Map Elements

+-------------------+--------------------------------------------------+
| The               | -   Top-level or Governing Documentation         |
| "***Documents***" |                                                  |
|                   | ```{=html}                                       |
|                   | <!-- -->                                         |
|                   | ```                                              |
|                   | -   Policies                                     |
|                   |                                                  |
|                   | -   Requirements                                 |
+-------------------+--------------------------------------------------+
| The\              | -   Basic Organizational Structures              |
| "***Boxes***"     |                                                  |
|                   | ```{=html}                                       |
|                   | <!-- -->                                         |
|                   | ```                                              |
|                   | -   Program                                      |
|                   |                                                  |
|                   | -   SMA                                          |
+-------------------+--------------------------------------------------+
| The\              | -   Key Interrelationships                       |
| "**A*rrows***"    |                                                  |
|                   | ```{=html}                                       |
|                   | <!-- -->                                         |
|                   | ```                                              |
|                   | -   Flow of information                          |
|                   |                                                  |
|                   | -   Reporting and accountability                 |
|                   |                                                  |
|                   | -   Activity                                     |
+-------------------+--------------------------------------------------+

1.  **Constructing an AP Map**

Steps to follow in constructing map:

1\. Lay out a top-down map structure depicting management organizations,
followed by implementing organizations, and then flight or operational
elements (e.g., orbiter, external tank, solid rocket booster, etc.)

2\. List contracts and applicable assurance requirements, and identify
management hierarchies, reporting accountabilities (in-line and
matrixed), and flow of information.

3\. Identify assurance agents on the map.

4\. Enter numerically-indexed assurance "vectors" or activity arrows
flowing from assurance agents to flight/operational elements or other
recipients of assurance processes.

5\. Enter detailed descriptions of assurance vectors into the AP Matrix
to define the breadth and frequency of the activity, the depth of
penetration, and the degree of independence.

![](media/image1.wmf){width="3.1493055555555554in"
height="2.439583333333333in"}

Figure 1. AP Map Checklist

Ultimately, the design layout and content of the map is driven by the
life-cycle phased information contained in the AP Matrix. The overall AP
analysis package should contain the map, matrix, and a directory (i.e.,
an assurance agent "Yellow Pages" with names and contact information).

![](media/image2.wmf){width="3.1506944444444445in"
height="2.4458333333333333in"}

Figure 2. AP Matrix Checklist

2.  **Mapping Protocol**

AP Maps draw heavily on several general graphical principles advocated
by Tufte[^1], including presentation of information adjacent in space as
opposed to stacked in time. As a result, AP Maps are designed to be
content-dense and contextually rich while striving to provide a
40,000-foot overview on a single chart.

This represents a significant challenge if the goal is to construct a
two-dimensional map on paper in black and white. However, the addition
of color to the map to indicate function (e.g., use of blue shading for
in-line assurance activities/agents, use of yellow shading for
independent assurance activities/agents, etc.), and the use of symbol
and lines/arrows to indicate basic organizational structure and various
types of interrelationships, can significantly increase the amount of
information conveyed on a single page. In addition, the judicious use of
numbering schemes can further alleviate clutter on the map. The
following guidelines are proposed to achieve these goals:

-   Use solid lines without arrowheads to indicate direct reporting
    (performance appraisal test) relationships.

-   Use dashed lines without arrowheads to indicate
    organizationally-matrixed reporting relationships

-   Use dotted lines with small arrowheads to represent information flow
    (Information flow from assurance players to the assurance forums
    creating KUV).

-   Use block arrows to represent assurance vectors.

-   Establish a color scheme (both line color and shading) to represent
    the different players on the map.

-   Show reporting and information flow lines using the same color of
    the source of information.

-   Use muted, pastel hues for shading whenever possible to reduce
    "graphical glare." Consider using 5% gray or 5% color shading area
    fills.

-   Use solid, single lined boxes for organizational entities.

-   Use solid, double-lined boxes for operational elements (e.g.,
    orbiter, external tank, solid rocket booster, etc.)

-   Consider selectively using box outlines to show emphasis.

-   Where possible, use numbers to link boxes or activities on the map
    and show relationships, thereby eliminating lines and reducing
    clutter. For example, the box for an assurance agent managed by *2.3
    Review and Assessment Division* box would be indexed 2.3.1.

-   When possible, link number schemes on the map to the AP Matrix.

Figure 3. Proposed Legend for AP Maps

Perhaps the most important guideline: follow these guidelines when at
all possible, but remain flexible enough to adapt your AP Map and Matrix
for the situation at hand.

7.  **EXAMPLE AP MAP**

Although a complete description of AP Mapping and a thorough
understanding of the NASA assurance roles and responsibilities are
beyond the scope of this paper (a short course could be dedicated to
these topics alone), a representative AP Map for the Space Shuttle
Program (SSP) is shown in Fig 4.

As an example of the benefit of such a map, it can be seen in the lower
right hand corner of Fig. 4 that for the SSP External Tank (ET) element
at Marshall Space Flight Center (MFSC), the ET Change Control Board
(CCB) provides information (see green dotted line to left) to MSFC
Engineering Risk Review Panel (MERRP), which in turns provides
information to the MSFC CCB. The MSFC CCB provides information to the
Program Management Council (PMC), which flows information for
non-integrated hazards (i.e., hazards that do not affect more than one
Shuttle element) to the Program Requirements Change Board. This flow of
information, for example, provides critical insight to all waivers
associated with safety and technical requirements associated safe and
reliable operation of the Shuttle ET element to the Shuttle Program
Manager, along with the Chief Engineer's Independent Technical Authority
(ITA) Technical Warrant Holders (TWHs) for the Shuttle, who participate
in the PRCB. Based on this activity, the Shuttle Program Manager is an
informed participant in the Shuttle Flight Readiness Review and is
ultimately one of the individual signatories for the Certificate of
Flight Readiness (CoFR).

Similarly, beginning in the lower right hand corner of Fig. 1, it can be
seen that the MSFC Project SMA personnel for the ET (Ciancola) reports
to the MSFC SSP SMA Manager (Dumbacher), who in turn reports to the MSFC
Center SMA Director (Davis). While each of the Center SMA Directors is a
direct report to their respective Center Directors, they are also
matrixed to the program SMA Manager (Wilcutt). Therefore, the MSFC
Center SMA Director (along with the Center SMA Directors for Kennedy
Space Center (KSC), Johnson Space Center (JSC), and Stennis Space Center
(SSC)), provides information (see blue dotted line to right) on the
health of NASA Center SSP SMA activities directly to the SSP SMA
Manager, who in turn provides information to the Chief Safety and
Mission Assurance Officer (O'Connor) through participation in the SMA
Readiness Review (SMARR). Based on this activity, the Chief SMA Officer
is an informed participant in the Shuttle Flight Readiness Review and is
ultimately one of the individual signatories for the Certificate of
Flight Readiness (CoFR).

These are only two examples of the overall Shuttle Assurance Process
providing independent and in-line SMA activities and reports in support
of the SSP FRR and CoFR. Further details (e.g., description, key
point-of-contact information, etc.) for all critical flows of SMA
information and activities are provided in a SSP AP matrix, sample rows
of which are provided as an example in Table 2.

![](media/image3.wmf){width="6.486805555555556in"
height="4.845833333333333in"}

Figure 4. Space Shuttle Program AP Map (Notional)

Table 2. SSP Assurance Process Matrix (Notional)

+---+--------------------------------------------+---------------------+
| * | **Description**                            | **Notes**           |
| * |                                            |                     |
| I |                                            |                     |
| D |                                            |                     |
| * |                                            |                     |
| * |                                            |                     |
+===+============================================+=====================+
| 1 | The **SMARR** is designed to: 1) affirm    | Chief Safety and    |
|   | that assurance processes have been         | Mission Assurance   |
|   | implemented over the life of the program   | Officer, OSMA,      |
|   | and to verify compliance with the          | Bryan O'Connor      |
|   | applicable baseline requirements set, 2)   |                     |
|   | provide adequate knowledge and visibility  | Chairman            |
|   | for senior SMA managers to understand the  |                     |
|   | residual risks associated with the safety  | Review & Assessment |
|   | and mission success of program launches,   | Division (RAD),     |
|   | operations stages, or selected critical    | Steve Newman        |
|   | tests, 3) examine mission preparation      |                     |
|   | status, open-work issues and concerns, and | SMARR Coordinator,  |
|   | assess overall systems readiness, and 4)   | Len Sirota          |
|   | provide an SMA position on whether to      |                     |
|   | concur or non-concur in proceeding with    |                     |
|   | the event/operation in question.           |                     |
|   |                                            |                     |
|   | *Independent Assurance Process*            |                     |
+---+--------------------------------------------+---------------------+
| 2 | **Flight Readiness Review (FRR)** -- Held  | Deputy Associate    |
|   | approximately two weeks prior to launch,   | Administrator,      |
|   | FRR is comprehensive review of all         | Michael Kostelik    |
|   | activities/elements necessary for the safe |                     |
|   | and successful conduct of all operations.  |                     |
|   | The readiness of the Space Shuttle         |                     |
|   | vehicle, flight crew, and payloads is      |                     |
|   | determined at this review. A signed        |                     |
|   | Certification of Flight Readiness          |                     |
|   | endorsement certifies that all have        |                     |
|   | successfully completed their Flight        |                     |
|   | Preparation Processes.                     |                     |
|   |                                            |                     |
|   | *Program Assurance Process*                |                     |
+---+--------------------------------------------+---------------------+

8.  **EVOLUTION -- e-MAPS**

The computer digital environment affords a wide range of presentation
techniques that go far beyond the limitations of standard paper. The
first innovation under consideration is a check-box selection
functionality allowing the user to view selected layers in any desired
combination (e.g., in-line assurance agents/activities, independent
assurance agents/activities, or both layers at once). For very complex
programs such as the Space Shuttle, a layered approach provides a series
of less complex maps for each unique dimension of the assurance process
that build to a single "composite" map (Fig. 4), which can aid in
communicating the overall assurance picture one piece at a time.
However, the advantage of an e-Map with the capability to turn on and
off different combinations of assurance layers eliminates the need to
flip between paper pages and has the potential to further enhance
communication and comprehension.

Additional e-Map functionality will include pop-up "mouse-over"
descriptions of assurance activities or assurance agents. Right-clicking
can generate pull-down menus with options to view assurance agents
contact information, hyperlinks to the latest assurance records,
analyses, briefing slides, or audio/video clips. Expert rules-based
"wizards" can be deployed to detect gaps and prompt the program manager
to reexamine assurance process definition and implementation plans.

Electronic AP-Mapping and AP-Matrix functionality will evolve in the
very near future to complement and work synergistically with electronic
program management and systems engineering knowledge management tool
suites

9.  **CONCLUSION**

Communication of complex ideas and relationships through the linear and
relentlessly sequential format of PowerPoint is ineffective, prone to
misinterpretation, and fails to adequately communicate complex
relationships.

***"To make connections would take hours using words. Your subconscious
has to use pictures.\"***

-- *William JJ Gordon*

AP-Mapping can provide a clear and concise picture or map of any complex
undertaking. Color, shape, form, images, structure, relative position,
juxtaposition, visual stimulation all combine to communicate and help
plan work, accomplish work and to manage work. AP-Mapping allows one to
visualize and easily communicate the big picture and at the same time
have the option to drill-down to details. AP-Mapping is recommended for
all aerospace programs as an essential management process.

[^1]: Edward R. Tufte in "The Visual Display of Quantitative
    Information" (Graphics Press, Cheshire Connecticut., 1983) defines
    graphical excellence as "... complex ideas communicated with
    clarity, precision, and efficiency ... is that which gives to the
    viewer the greatest number of ideas in the shortest time with the
    least ink in the smallest space ... is nearly always multi-variate"