The System Life Cycle and ISHEM

Charles D. Mott[^1]

University of North Dakota, Grand Forks, ND 58201

> **The creation of dependable systems involves the application of
> methods from a variety of subject areas and disciplines. Integrated
> System Health Engineering and Management (ISHEM) draws on these
> methods throughout the system life cycle to achieve dependability. An
> approach using the system life cycle is presented and developed as a
> framework for understanding the relationships and contributions of the
> various methods used to create dependable systems.**
>
> **The system life cycle consists of six phases: research, planning,
> design, testing, manufacture, and operations. The interconnectedness
> of these phases is explored to promote awareness of how the actions
> taken in one phase can influence the choices of action available in
> other phases. The need to consider this interconnectivity and the
> effects it can have on system dependability, costs, and efficiency is
> examined.**
>
> **Each of the phases of the system life cycle can be associated with
> several different subject areas or disciplines that affect system
> dependability. Categorizing the subject areas according to the phase
> of the system life cycle where they have the greatest influence
> facilitates increased understanding of the critical factors that
> create dependability during each phase. It also reveals opportunities
> for increased information sharing, better decision making, and reduced
> redundancy leading to more efficient and consistent system creation.**

# Introduction {#introduction .Style-Heading-1-+-12-pt}

The system life cycle is used to define a framework for analyzing the
process of creating systems. The framework that is described in this
paper uses six distinct phases to describe the actions performed during
the system life cycle. The six phases are; research, planning, design,
manufacture, testing, and operations. Figure 1 illustrates the relation
between these phases. Integrated System Health Engineering and
Management (ISHEM) methods are used in each of these phases to increase
the dependability of the system being created.

In the planning phase, systems are envisioned and conceived as an ideal
solution to meeting a particular need. However, during the design,
manufacture, testing, and operation phases the capabilities of the
system will be degraded by errors, omissions, and random events that
prevent the perfect system from being created (Landers, 1963, pg. 25).
ISHEM methods can reduce the system degradation in each of these phases
and improve overall dependability. For example, configuration control
and interface requirements reduce errors in design, quality assurance
techniques reduce errors in manufacturing, verification and validation
finds errors by testing, and fault protection methods reduce system
degradation during operations.

**Figure 1: The System Life Cycle**

![](media/image1.wmf){width="6.498611111111111in"
height="5.935416666666667in"}

System Health Management (SHM) is generally perceived as an operational
consideration. This paper attempts to increase the awareness of SHM
issues throughout the system life cycle. Creating dependable systems
requires a life-cycle engineering approach. As described by Blanchard
and Fabrycky, "classical engineering has focused mainly on product
performance as the main objective rather than on development of the
overall system (1998, pg. 19)." But the system requires additional
elements to perform its intended purpose such as logistical support,
maintenance capabilities, and operator training. "Accordingly, it is
essential that engineers be sensitive to utilization outcomes during the
early stages of systems design and development (1998, pg 19.)." More
generally, the engineer needs to understand and consider all the phases
of the system life cycle and the interactions between them. This is
especially important for integrating ISHEM subsystems into the final
product.

In Section II of this paper, each phase is described in terms of primary
activities and elements are identified that may affect dependability.
This provides a basis for the analysis of the methods used in ISHEM, and
their effects on system dependability during that phase.

Section III describes some of the methods used in ISHEM. Section IV
introduces a matrix of these methods and the life cycle phases that is
used to demonstrate the interconnectivity of the life cycle phases.
Finally in Section V, this matrix is used to discover relations between
the various methods and identify opportunities for improvements.

# The System Life Cycle {#the-system-life-cycle .Style-Heading-1-+-12-pt}

This section describes the primary activities that are performed during
each phase of the system life cycle. This provides a definition of the
phases used in the generic model and describes how the activities in
each phase may affect overall system dependability.

## Research Phase

The primary activity in the research phase is the identification of
customer needs. Based on these needs research is conducted to define the
goals and objectives of the system, identify constraints, and document
these items to define the project scope. Once the scope of the project
is determined, technologies and architectures can be developed for
evaluation. Basic and applied scientific research is included in this
phase as well as the research performed to mature technologies for use
in complex systems. Basic and applied scientific research creates new
knowledge and technologies that can be applied to system designs. This
research is often directed toward the development of specific
technologies and may be part of a larger system development effort. A
number of pure research centers, academic institutions, and non-profit
organizations are engaged in research activities.

The initial selection of technologies and processes can have a
significant effect on overall system dependability. "Numerous
retrospective studies indicate that uncertainties often constitute a
central consideration in \[the\] performance of engineering systems
(Neufville et al, 2004, pg. 1)." Uncertainty is managed by the selection
of proven, well developed technologies that are understood and have
historical performance data available that allows the system creator to
assess their dependability. While it is not always the case that
experimental or new technologies have greater chances of failure, they
are less understood and inject a greater degree of uncertainty into the
system creation process. The system goals and objectives will determine
if the development or use of new technologies is required. They may also
place limits on selection of existing technologies.

## Planning Phase

The development of requirements is the primary activity in the planning
phase. These requirements define what the system must do to meet
customer needs. But the planners must also consider requirements for how
the testing, building, and operation of the system will be conducted.
This is also the time to integrate ISHEM methods into the total system
design. Including consideration of ISHEM methods in all phases of the
system life cycle and defining requirements for their implementation
results in increased system dependability. Too often, the planning
focuses on the hardware needed to meet the system requirements and
ignores other factors that can have significant effects on system
dependability.

Planning involves identifying system requirements, but it also involves
planning how the system will be manufactured, tested, and operated.
Planning for testing is often overlooked. "As system-level requirements
are identified \[in the planning phase\] . . . a method must be
established for evaluation and subsequent determination of whether these
requirements will be met (Blanchard and Fabrycky, 1998, pg. 121)."

The planning phase also has activities that relate to the management and
administration of the system creation process. Budgets and schedules are
developed. The management team is created, usually consisting of a
project manager, system engineers, technical experts, and administrative
personnel. Legal and reporting requirements are determined. These types
of activities also have influences on the dependability of the system
being created. Shenhar and Bonen state "both project management and
systems engineering practice differ with each specific system \[being
created\] and that management attitudes must be adapted to the proper
system type (1997, pg. 137)." This statement is indicative of the
differences in space, aviation, and other types of projects; each has
different priorities and concerns for system dependability and each must
be managed using different approaches.

## Design Phase

The design phase translates requirements and architectures developed
during the planning phase into specific and tangible elements that will
execute the system functions. The design phase includes a number of
processes and activities that generally fall under the discipline of
engineering. Systems engineering is a specific collection of methods
used to manage the design process. It is applied to different projects
in different ways. For example, space systems are usually one of a kind,
high-tech systems that require a large up-front investment in design
because there will be no opportunity for modification during the
operation of the system. On the other hand, an automobile is a mass
produced item that uses well understood technologies and can be
modified, repaired, and maintained throughout its life cycle. The
management processes used by the system engineer will be different in
each case. The common factor is that the end product has been designed
to meet the system goals and objectives.

Designers transform requirements into specifications, drawing, and
processes that facilitate the creation of the system. "To design is to
project and evaluate ideas for bringing a needed system or product into
being (Blanchard and Fabrycky, 1998, pg. 86)." Creating a dependable
system requires designing for reliability, maintainability, usability,
serviceability, and affordability. Each of these 'ilities' are important
considerations that drive the need for health management tools and
techniques when building dependable systems.

## Manufacturing Phase

The manufacturing phase is focused on manifesting the objects described
by the design process. Most systems have tangible elements that must be
created. The manufacturing phase could also include the creation of
software code or detailed procedures and methods for implementing a
process. The manufacturing phase for a system may involve the creation
of one-of-a-kind objects or mass production of identical objects.

Within the manufacturing phase, dependability is created by attention to
detail, quality control, and training of the workforce. The assembly of
complex systems requires systems integration methods to ensure that the
materials and components used are of the highest quality, that they have
been thoroughly tested, and that they are installed and integrated into
the overall system correctly. Complex systems

> are characterized by two persistent trends, namely (a) incorporation
> of an increasing number of functionalities that increase the
> integration of the number of parts and components (multi-component) as
> well as services, and (b) incorporation of an increasing number of new
> and sometimes distant scientific and technical disciplines
> (multi-technology) (Dosi, Hobday, Marengo, and Prencipe, 2003,
> pg.101).

The coordination of this effort is difficult because of the number of
specialized firms that are part of the aerospace industry. Large
'systems integrators' use specialized methods to ensure the
dependability of the systems they create.

## Testing Phase

The testing phase consists of a number of activities that are usually
associated with creating dependability. Verification is the process of
ensuring the system has met all of the developed requirements.
Validation is the process of ensuring the correct system to meet
customer needs has been constructed.

There is a strong connection between verification activities and the
planning, design, and testing phases. During the planning phase, system
objectives and goals are translated into requirements. The design phase
further allocates these requirements to greater levels of detail in the
system. These requirements determine system specifications and impact
the selection of materials and components. From these requirements and
specifications, criteria for the verification of system performance and
dependability are developed.

Validation methods use measures of reliability, availability, and
dependability to determine if the system is capable of achieving its
goals. Validation requires understanding the customer needs, goals,
objectives, and constraints on the system to evaluate whether or not the
system meets these criteria. Verification determines if the system has
been built right. Validation determines if the right system has been
built.

Testing takes place as each component or subsystem is integrated into
the final product. Therefore testing overlaps the manufacturing phase,
and in fact takes place throughout the life cycle. Plans are tested to
make sure they can be executed, designs are tested to ensure they meet
requirements, parts are tested as assembled, and the system continues to
be tested during operations.

## Operations Phase

The operations phase of the system lifecycle is where the system
actually performs its intended functions. Operations typically involve
human-machine interactions that can have a significant effect on system
dependability. The organizational structure, details of process
definitions, extent of operator training, and the functionality of
human-machine interfaces are all critical factors that affect the system
dependability.

Operating complex systems requires the development of a 'cognitive map'
of the dynamic interactions taking place in the actual system. The
design of the human-machine interfaces can have a direct impact on this
process.

> For the most part, neither their equipment nor their task environments
> have been optimally, or even coherently, designed. But the operators
> are intimately familiar with both and have mastered them. However
> designed or laid out, the controls, dials, and other instruments of
> their working environment become part of the structuring of control
> room processes and procedures, dynamically interpreting and
> translating the technical environment into action and social processes
> (Rochlin, 1997, pg 129).

Understanding these processes and their effect on the system
dependability will allow designers to improve system operations and
eliminate the sources of some human-machine interface faults.

# ISHEM Methods  {#ishem-methods .Style-Heading-1-+-12-pt}

The methods used in ISHEM can be grouped by the actor (customer,
analyst, developer) who is most likely to apply that technique to an
ISHEM problem. For example, the customer or user is interested in
safety, human factors, and organizational methods that will allow for
the safe and efficient operation of the system. The analyst is concerned
with scientific methods such as failure assessment, reliability theory,
failure statistics, and probabilistic risk assessment. Developers focus
on subsystems (propulsion, power, command and communications, guidance,
life support, etc.), integration, and quality assurance techniques. All
of the actors must account for knowledge management, communication,
verification and validation, risk management, and safety. The grouping
of these disciplines along functional lines provides a first look at the
variety of skills needed to provide an integrated health management
system.

**Customer (operator) Methods**

ISHEM is not only built into a system, but also must be concerned with
how the system will perform its function, the human-machine interfaces,
and the larger social and organizational environment in which the system
operates. A number of disciplines focus on these areas and the results
of research in these areas can contribute to overall system
dependability. Some of these disciplines are; safety and hazard
analysis, high reliability organization theory, human factors
engineering, maintainability theory and practice, risk assessment, and
risk management. Each of these disciplines provides information on the
operation of complex systems and how to safely, reliably, and
consistently achieve high performance.

**Analyst Methods**

Because of the strong link between planning and operations, the analyst
will use a number of the same disciplines as the customers and user in
trying to obtain dependability. In addition the more scientific
disciplines become important to the analyst as they research and plan
for dependability throughout the entire life cycle. Areas such as
failure assessment, the physics of failure, reliability theory, fault
tolerant computing, vibration analysis, probabilistic risk management,
communications, and knowledge management all become important in
assessing and planning the implementation of health management systems.

**Developer Methods**

The developers have the responsibility of actually creating and building
the system. Because of this, they must integrate the ISHEM systems from
each subsystem into the overall system-wide health management system.
The aerospace developer uses methods specific to each subsystem to make
each subsystem or component dependable. These components and subsystems
then need to be integrated into the system's health management system.
Areas such as propulsion, electrical power, guidance, command and
communications, life support, structures, thermal protection, armaments,
and sensors each have developed specific methods to provide health
management. The integration is performed by applying information from
the analyst and customer that describes the overall system, as well as
the communication, verification and validation, knowledge management,
and risk management methods that are shared by all of the actors
creating ISHEM systems.

A representative subset of the methods used in integrated system health
engineering and management is summarized in Table 1. The effects that
these methods have on system dependability are included as a reference
for use in the next section. The methods have been drawn from a number
of different disciplines and subject areas to demonstrate the variety of
approaches to obtaining system dependability. These methods include both
specific techniques - such as a time to criticality analysis - and broad
disciplines -- like high reliability organizations -- that illustrate
the diversity of areas that must be considered when thinking about
system health management and the design of dependable systems. The
comparison of these methods on a side-by-side basis later in the paper
does not indicate that they are of equal content, only that they each
have something to contribute to the overall dependability of the system
being created.

**Table 1: ISHEM Methods and Their Effects on Dependability**

+-------------------------+--------------------------------------------+
| **ISHEM METHOD OR       | **EFFECT ON SYSTEM DEPENDABILITY**         |
| SUBJECT AREA**          |                                            |
+-------------------------+--------------------------------------------+
| Configuration           | -   Control changes by freezing design     |
| Management              |                                            |
|                         | -   Changes to the design must be          |
|                         |     justified and then approved by         |
|                         |     management                             |
|                         |                                            |
|                         | -   Ensures the system is built to the     |
|                         |     design                                 |
+-------------------------+--------------------------------------------+
| Data Analysis and       | -   Monitor and assess the system health   |
| Mining                  |     status                                 |
|                         |                                            |
|                         | -   Use large databases to assist in       |
|                         |     predictive capabilities                |
|                         |                                            |
|                         | -   Assists automated control of critical  |
|                         |     systems                                |
+-------------------------+--------------------------------------------+
| Design Methods and      | -   Include ISHEM in design of             |
| Fault Management        |     systems/subsystems                     |
|                         |                                            |
|                         | -   Identifies potential failures early in |
|                         |     the design process                     |
|                         |                                            |
|                         | -   Economical and efficient to remove     |
|                         |     faults early instead of                |
|                         |     rework/redesign later                  |
+-------------------------+--------------------------------------------+
| Diagnostics             | -   Monitor and report the health status   |
|                         |     of the system                          |
|                         |                                            |
|                         | -   Identify problems using rules, models, |
|                         |     and other reasoning methods            |
|                         |                                            |
|                         | -   Allows for detection of anomalous      |
|                         |     conditions                             |
+-------------------------+--------------------------------------------+
| Economics of System     | -   Contracting and sub-contracting        |
| Integration             |     practices                              |
|                         |                                            |
|                         | -   Manages system-building complexity     |
|                         |     (many organizations instead of one)    |
|                         |                                            |
|                         | -   Requires controls and procedures to    |
|                         |     coordinate activities                  |
+-------------------------+--------------------------------------------+
| Failure Assessment      | -   Determine how a system can fail        |
|                         |                                            |
|                         | -   Estimate the effects of failures       |
|                         |                                            |
|                         | -   Estimate the number of faults          |
|                         |                                            |
|                         | -   Identified faults can be eliminated or |
|                         |     effects mitigated                      |
+-------------------------+--------------------------------------------+
| Failure Statistics,     | -   Collect data and analyze the cause of  |
| Data Collection and     |     failure                                |
| Dissemination           |                                            |
|                         | -   Provide information on root causes     |
|                         |                                            |
|                         | -   Improve later designs in               |
|                         |     statistically-vulnerable areas         |
+-------------------------+--------------------------------------------+
| Fault Modes & Effects   | -   Identify possible faults and their     |
| Analysis                |     effects                                |
|                         |                                            |
|                         | -   Allows elimination of fault sources in |
|                         |     the system design                      |
|                         |                                            |
|                         | -   Required to create mitigation methods  |
+-------------------------+--------------------------------------------+
| Fault Tolerant          | -   Defines computer hardware and software |
| Computing and           |     configuration                          |
| Architectures           |                                            |
|                         | -   Provides dependable computing systems  |
|                         |                                            |
|                         | -   Mitigate hardware and software design  |
|                         |     flaws                                  |
+-------------------------+--------------------------------------------+
| High Reliability        | -   Investigates organizational structures |
| Organizations           |     and processes that improve             |
|                         |     dependability                          |
|                         |                                            |
|                         | -   Experience base of best and worst      |
|                         |     practices                              |
+-------------------------+--------------------------------------------+
| Human Factors           | -   Examines human-machine interfaces      |
|                         |                                            |
|                         | -   Identifies sources of "human error"    |
|                         |                                            |
|                         | -   Allows for better designs that support |
|                         |     dependable operation                   |
+-------------------------+--------------------------------------------+
| Interface Control       | -   Documents interface requirements for   |
|                         |     subsystems                             |
|                         |                                            |
|                         | -   Describes interactions between         |
|                         |     different functional components        |
|                         |                                            |
|                         | -   Allows identification of emergent      |
|                         |     faults                                 |
+-------------------------+--------------------------------------------+
| Knowledge Management    | -   Fosters creation, transfer, retention, |
|                         |     and utilization of knowledge           |
|                         |                                            |
|                         | -   Eliminates known faults                |
|                         |                                            |
|                         | -   Prevents miscommunications, reducing   |
|                         |     numbers of design faults               |
|                         |                                            |
|                         | -   Provides a repository of lessons       |
|                         |     learned                                |
+-------------------------+--------------------------------------------+

**Table 1: ISHEM Methods and Their Effects on Dependability (cont.)**

+-------------------------+--------------------------------------------+
| Maintainability Theory  | -   Provides structured maintenance and    |
| and Practice            |     repair procedures                      |
|                         |                                            |
|                         | -   Prevent failure by repairing before    |
|                         |     breakdown                              |
+-------------------------+--------------------------------------------+
| Management Structure    | -   How to organize and manage the project |
|                         |                                            |
|                         | -   Degree of control based on project     |
|                         |     size/complexity                        |
|                         |                                            |
|                         | -   Correct management structure results   |
|                         |     in better dependability                |
+-------------------------+--------------------------------------------+
| Physics of Failure      | -   Understand materials degradation and   |
|                         |     failure mechanisms                     |
|                         |                                            |
|                         | -   Describe the dynamics of material      |
|                         |     failures                               |
|                         |                                            |
|                         | -   Minimize inspection costs              |
|                         |                                            |
|                         | -   Improves maintainability, diagnostics  |
|                         |     and prognostics models and processes   |
+-------------------------+--------------------------------------------+
| Probabilistic Risk      | -   Create statistical estimates of the    |
| Assessment              |     chances of failures                    |
|                         |                                            |
|                         | -   Allows comparison of risk factors      |
|                         |                                            |
|                         | -   Provides management with decision      |
|                         |     information for resource allocation    |
+-------------------------+--------------------------------------------+
| Prognostics             | -   Uses predictive algorithms to assess a |
|                         |     systems future health status           |
|                         |                                            |
|                         | -   Provides advance notification of       |
|                         |     predicted failure                      |
|                         |                                            |
|                         | -   Allows corrective action to be taken   |
|                         |     before failure occurs                  |
+-------------------------+--------------------------------------------+
| Quality Assurance       | -   Methods to identify defects in         |
|                         |     materials, assemblies, or processes    |
|                         |                                            |
|                         | -   Looks for improvements in processes    |
|                         |                                            |
|                         | -   Reduces failures due to defects        |
|                         |                                            |
|                         | -   Parts tracking to ensure highly        |
|                         |     reliable components                    |
+-------------------------+--------------------------------------------+
| Reliability Theory and  | -   Provide probabilities of system        |
| Practice                |     failure                                |
|                         |                                            |
|                         | -   Evaluate effect of redundant           |
|                         |     components                             |
|                         |                                            |
|                         | -   Increase system dependability by       |
|                         |     identifying high probability of        |
|                         |     failure areas                          |
+-------------------------+--------------------------------------------+
| Risk Management         | -   Identify, analyze, and document risks  |
|                         |                                            |
|                         | -   Communicate and control risk factors   |
|                         |                                            |
|                         | -   Develop mitigation plans for           |
|                         |     identified risks                       |
+-------------------------+--------------------------------------------+
| Safety and Hazard       | -   Create safe environments by monitoring |
| Analysis                |     hardware, software, human-machine      |
|                         |     interfaces, management, and            |
|                         |     organizational design                  |
|                         |                                            |
|                         | -   Provide better awareness of the        |
|                         |     integration of different factors       |
|                         |                                            |
|                         | -   Identify and eliminate or compensate   |
|                         |     for hazards                            |
+-------------------------+--------------------------------------------+
| Simulation/Modeling     | -   Replicate the system in a controlled   |
|                         |     environment                            |
|                         |                                            |
|                         | -   Provides testing and analysis          |
|                         |     capabilities (what if scenarios)       |
|                         |                                            |
|                         | -   Improves final design by identifying   |
|                         |     emergent faults                        |
+-------------------------+--------------------------------------------+
| Time to Criticality     | -   Identify timing issues during failures |
| Analysis                |                                            |
|                         | -   Determine reaction times for health    |
|                         |     management systems                     |
+-------------------------+--------------------------------------------+
| Verification and        | -   Independent assessment of the system   |
| Validation              |                                            |
|                         | -   Confirm that the system meets          |
|                         |     specifications                         |
|                         |                                            |
|                         | -   Confirm that the system meets customer |
|                         |     needs                                  |
|                         |                                            |
|                         | -   Fault injection used for V&V feeds     |
|                         |     operational contingency planning and   |
|                         |     operator training                      |
+-------------------------+--------------------------------------------+

# The Interconnectivity of the System Life Cycle Phases {#the-interconnectivity-of-the-system-life-cycle-phases .Style-Heading-1-+-12-pt}

The system life cycle involves interactions between three key actors;
customers (or operators), analysts, and developers. The customer
supplies mission, objectives, goals, and constraints to the analyst. The
analyst works with the customer in an iterative process to identify the
customer's need, and create a plan for transforming that need into a
system that will meet that need. The developer uses the plan provided by
the analyst to design, manufacture, and test the system before
delivering it to the customer. The interaction between the customer,
analyst, and developer illustrates a first level of interconnection in
the system life cycle.

A matrix of the ISHEM methods and the system life cycle phases has been
developed to uncover relations between the phases of the system life
cycle and methods used in ISHEM. Each method is evaluated to determine
in which phase of the system life cycle it has the most influence on
dependability. At this time the rating is subjective, based on the
authors research and understanding of these methods. Future work will
include determining a more qualitative method of performing this
assessment. A simple scale (low, moderate, high) is used to rate the
influence a method has on dependability in each phase. Methods are often
used in more than one phase, and their influence on dependability in
these different phases will be different. The matrix is built using
spreadsheet software to facilitate the analysis activities, which
consist primarily of sorting the matrix based on the influence results
for each phase of the system life cycle. The matrix is presented in
Figure 2.

**Figure 2: ISHEM Methods Affect on Dependability in Phases of the
System Life Cycle**

![](media/image2.wmf){width="6.495833333333334in" height="3.375in"}

Connections between life cycle phases were determined by analyzing the
matrix. For each method, the phase where high influence on dependability
was found was connected to each phase where medium or high influence was
found. This assumes that if a method has a significant influence in more
then one phase it is because data or information is being shared in the
different phases. A count of the connections after all methods were
analyzed was used to make high, medium, and low interconnectivity
determinations. For example, human factors produced a connection between
Operations/Planning, Operations/Design, Operations/Manufacturing, and
Operations/Testing. The results of this analysis are presented
graphically in Figure 3.

**Figure 3: The Interconnectivity of the System Life Cycle**

![](media/image3.wmf){width="5.8590277777777775in"
height="5.5472222222222225in"}

One of the important connections uncovered is the operations/planning
link. System requirements are the drivers that guide the designer and
ensure that the system does what it is intended to do. If the analysts
who are writing requirements in the planning phase do not understand
what the intended operations are, then they cannot write good
requirements. An example from the NASA Space Shuttle program is given by
Ted Kenny and Jim Carr of the Mission Operations Directorate, NASA
Johnson Space Center.

> NASA has contemporary experience developing and operating two major
> manned programs (shuttle and ISS) that began with goals to
> reduce/control the long-term operations phase costs and risks of the
> vehicle. However, during the early design phases of the vehicle there
> was no tool or methodology to assess and document the long term
> impacts of design choices to enable fully educated trades during the
> design review process. In the end, many of these goals were not
> realized and the vehicle operations phase costs and risks were greater
> than were planned in phase A of the project (2004).

The number of high interconnections between the operations phase and
other phases illustrates the importance of considering operational
activities in the creation of dependable systems. The need to modify
systems is suggested by the interconnections between operations, design,
and testing. Note that the connections suggested in this diagram are
biased by the selected set of methods and the evaluation of their
influence on system dependability. What is illustrated however is the
richness of connections between the phases of the system life cycle, in
terms of trying to achieve dependability.

Peter Checkland from the University of Lancaster was describing the
systems approach when he wrote,

> . . . although the methodology is most easily described as a sequence
> of phases, it is not necessary to move from phase to phase: what is
> important is the content of the individual phases and the
> relationships between them. With that pattern established, the good
> systems thinker will use them in any order, will iterate frequently,
> and may well work simultaneously on more than one phase. This 'out of
> sequence' usage is particularly important in tackling broad problems
> which are not owned within a single organization (1981, pg. 17).

The creation of aerospace systems is certainly a broad problem that
involves multiple organizations. The application of a systematic
approach to ensure system dependability requires an understanding of the
interconnections between the phases of the system life cycle and the
methods used during those phases. These interconnections are further
explored in the next section.

# Classifications of ISHEM Methods Using the System Life Cycle {#classifications-of-ishem-methods-using-the-system-life-cycle .Style-Heading-1-+-12-pt}

The matrix presented in Section IV is now used to analyze the various
methods used in ISHEM by sorting the methods based on their influence on
dependability during each phase of the system life cycle. This provides
a grouping of the methods that have high influence on dependability
during a particular phase and allows for analysis of improvements in
information sharing and efficiency in the application of these methods
to increasing dependability.

## Research Phase

Figure 4 presents a matrix of the ISHEM methods described in the
previous section sorted by their influence on dependability in the
research phase. It is not surprising to find most of the
statistics-based methods in this list. As more data is collected, these
methods allow the designers and builders of systems to have a greater
degree of certainty that the system will perform as expected. These
methods are also useful to identify root causes of failures that have
occurred in the past, and provide lessons learned that can eliminate
defects and sources of failure in future systems.

**Figure 4: Classification of ISHEM Methods: Research Phase**

![](media/image4.wmf){width="6.498611111111111in"
height="3.379861111111111in"}

The matrix also reveals a possible source of differences in these
methods: the system life cycle phase where the results of each method
are commonly used is different. For example, both [failure
assessment]{.underline} and [fault tolerant computing]{.underline} have
more influence on the design of systems while [probabilistic risk
assessment]{.underline}, [high reliability organization
theory]{.underline}, and [management structure]{.underline} have more
influence on the planning and operations phases.

Opportunities for improvements in the research phase primarily involve
greater information sharing between the different methods, as well as
sharing techniques, tools, and procedures used to analyze existing data.
The goal of the research activities is to provide system creators with
information on the available technologies and methods that they can use
to meet the customer need. These methods increase the overall system
dependability by offering up-to-date, relevant, and integrated
information on the causes of failure and possible mitigation techniques.

## Planning Phase

The matrix sorted by the influence on dependability during the planning
phase is presented in Figure 5. There is considerable variety in the
methods and the disciplines that influence dependability in the planning
phase. This is due to the interconnectivity of the planning function
with activities performed during other phases. Because planners need to
consider how these activities will affect the system's dependability
they need a variety of tools.

**Figure 5: Classification of ISHEM Methods: Planning Phase**

![](media/image5.wmf){width="6.498611111111111in"
height="3.379861111111111in"}

The matrix also shows the strong interconnectivity between the planning
and operations phases. Over half of the methods that are influential in
planning are also important in the operations phase. A number of these
methods deal with the organizational and social issues that influence
system dependability. These influences are a direct result of the way
technologies are developed.

> Today, technological development tends to take place inside large,
> hierarchical organizations. This is particularly true for complex,
> large-scale technologies, since they demand large investments and
> extensive, coordinated development efforts. But large organizations
> inject into the development process a host of considerations that have
> little or nothing to do with engineering. Any institution has its own
> goals and concerns, its own set of capabilities and weaknesses, and
> its own biases about the best ways to do things. Inevitably, the
> scientist and engineers inside an institution are influenced -- often
> quite unconsciously -- by its culture (Pool, 1997, pp. 7, 8).

This illustrates the importance of planning and managing the
institutional environment in which systems are built and operated.

Opportunities for reduced costs and greater efficiency materialize when
operations are carefully planned and analyzed early in the system life
cycle. Better human-machine interfaces, training, and maintenance
practices can be developed that reduce operations costs. Preparations to
deploy and support the system can be made in parallel with the design
and manufacturing functions which can reduce the time needed to bring
the system from testing to full operational capabilities.

## Design Phase

Figure 6 shows the matrix sorted on the influence the methods have in
the design phase. During the design phase, a number of the methods used
come directly from systems engineering or systems management. For
example, [interface control]{.underline}, [configuration
management]{.underline}, and [quality assurance]{.underline} can be
found in almost every text on system engineering. This follows from the
fact that system engineering focuses on design activities. The matrix
indicates that design has connections with every other phase and is
central to the creation of dependable systems. This supports the
assertion that health management systems should be considered early in
the design process, as these connections will reveal opportunities to
eliminate defects in the system.

**Figure 6: Classification of ISHEM Methods: Design Phase**

![](media/image6.wmf){width="6.498611111111111in"
height="3.379861111111111in"}

Opportunities for information sharing in the design phase involve the
early consideration and integration of ISHEM systems into the overall
system design. It is much easier and cost efficient to incorporate
health management technologies during early design than to wait and add
them on later when many specifications, drawings, and schematics have
already been produced.

## Manufacturing Phase

Figure 7 shows the matrix sorted by influence during the manufacturing
phase. System dependability is greatly influenced in the manufacturing
phase by [quality assurance]{.underline}, [configuration
management]{.underline}, and the [economics of system
integration]{.underline}. These methods ensure that the system-as-built
is the same as the system-as-designed. They also provide controls
between the various contractors, sub-contractors, and suppliers to
ensure that all materials and components meet the design specifications.
The matrix also shows that manufacturing has strong links with the
design and planning phases.

Opportunities for improvement in manufacturing include increased
verification and validation activities, better risk management, and
organizing the contractor system to provide better communication and
cooperation among the builders of complex systems.

**Figure 7: Classification of ISHEM Methods: Manufacture Phase**

![](media/image7.wmf){width="6.498611111111111in"
height="3.379861111111111in"}

## Testing Phase

Figure 8 shows the matrix sorted on the influence the methods have
during the testing phase. Testing is an important phase for ISHEM
applications. The testing methods can have significant effects on system
dependability. Uncovering defects and emergent faults by the
[verification and validation]{.underline} and
[simulation/modeling]{.underline} methods increase overall system
dependability. The connection between the planning, testing, and
operations phases is illustrated by the influences of the [verification
and validation]{.underline}, [knowledge management]{.underline}, [risk
management]{.underline}, and [safety and hazard analysis]{.underline}
methods.

The testing methods share a requirement to have some comparison standard
or specification that they must determine is met by the created system.
Specifications are based on the design requirements, materials are
tested to quality standards, and reliability statistics are compiled
from many repetitions of test and simulations.

> **Figure 8: Classification of ISHEM Methods: Testing Phase**

![](media/image8.wmf){width="6.498611111111111in"
height="3.379861111111111in"}

Opportunities for improvement in the testing phase include ensuring that
the connections with planning, design, and manufacturing provide good
standards and specifications. Acceptance testing by the operators is
also important, and should be conducted as early as possible, using
prototypes and simulations to allow for operator feedback as early in
the system life cycle as possible. Testing of materials, components,
assemblies, and subsystems as they are integrated into larger units and
the final system requires an understanding of the larger system goals
and objectives. These need to be included in test requirements and
verified throughout the system creation process. Testing also should be
conducted on the processes and procedures that will be used to operate
the system as human operators can unknowingly inject faults into the
system.

## Operations Phase

Figure 9 presents the matrix sorted by the influence on dependability
found in the operations phase. The organizational and social methods
again emphasize the link between the operations and planning phases.
During operations the unpredictability of human-system interactions
becomes a significant source of faults, which can be mitigated by
improving the system health systems that affect the human-machine
interfaces. Also notice that it is during the operations phase that data
is collected by some of the primarily research methods. This data
improves future operations by identifying potential failure modes,
improving human-machine interfaces, and discovering emergent faults that
were not anticipated by earlier analysis.

Operations are improved by better planning as mentioned earlier, but
system operators need to have available the data obtained by other ISHEM
methods from all the prior phases. Techniques and tools to capture the
specific results and information which is created throughout the system
life cycle should be developed for system operators. This information
would be especially useful when trying to diagnose and recover from an
off-nominal operating condition.

> **Figure 9: Classification of ISHEM Methods: Operations Phase**

![](media/image9.wmf){width="6.498611111111111in"
height="3.379861111111111in"}

# Summary {#summary .Style-Heading-1-+-12-pt}

This paper has presented a model of the systems life cycle, described
the activities that take place during different phases, and demonstrated
the interconnectivity of these activities. Several methods used in ISHEM
were described and the effect of using these methods on system
dependability was presented. Finally, a matrix between these methods and
the system life cycle was used to reveal connections and opportunities
for improved communications and efficiency when creating dependable
systems.

The creation of dependable complex systems involves many people, from
many specialties, using a wide variety of techniques and tools to ensure
that these systems operate safely, reliably, and consistently over the
entire life cycle. Each phase of the system life cycle contributes
something to the overall dependability of the system. Understanding
these contributions in the context of the larger system will allow
better and more efficient health management systems to be incorporated
into these systems. In turn, better health management systems will
reduce system operations costs, provide better data for research, and
increase the dependability, efficiency, and usability of the systems
being created.

**References**

Blanchard, B. and Fabrycky, W. (1998). [Systems Engineering and Analysis
(3^rd^ ed.).]{.underline} Upper Saddle River, New Jersey: Prentice Hall.

Checkland, P. (1981). [Systems Thinking, Systems Practice.]{.underline}
New York, NY: John Wiley & Sons.

Dosi, G., Hobday, M., Marengo, L., and Prencipe, A. (2003) The Economics
of System Integration: Toward an Evolutionary Interpretation in
Prencipe, A., Davies, A. and Hobday, M. (Eds.). [The Business of Systems
Integration.]{.underline} Oxford: University Press. 95-113.

Kenny, T and Carr, J. (2004) Techniques and Strategies for Reducing
Long-term Operations Cost/Risk During Early Vehicle Design. [NASA White
Paper]{.underline}. May 19,2004

Landers, R. (1963). [Reliability and Product Assurance: A Manual for
Engineering and Mangement.]{.underline} New Jersey: Prentice-Hall.

Neufville, R. *et al.* (2004) Uncertainty Management for Engineering
Systems Planning and Design. [Engineering Systems
Monograph]{.underline}. Boston: MIT.

Pool, R. (1997). [Beyond Engineering: How Society Shapes
Technology.]{.underline} Oxford: University Press.

Rochlin, G. (1997). [The Computer Trap: The Unintended Consequences of
Computerization]{.underline}. Princeton: University Press.

Shenhar, A. and Bonen, Z. (1997). The New Taxonomy of Systems: Toward an
Adaptive Systems Engineering Framework. [IEEE Transaction on Systems,
Man, and Cybernetics -- Part A: Systems and Humans. 27 (2)]{.underline}.
[137-145.]{.underline}

Stephanou, S. (1981). [Management: Technology, Innovation & Engineering
(A Systems Approach).]{.underline} Malibu, CA: Daniel Spencer
Publishers.

[^1]: Graduate Research Assistant, Department of Space Studies, 501
    Clifford Hall, 4149 Campus Road