NASA/TM-2009-215569

NESC-RP-08-31/08-00451

Crew Exploration Vehicle (CEV) Avionics

Integration Laboratory (CAIL) Independent 

Analysis

Mitchell L. Davis/NESC, Michael L. Aguilar/NESC

NASA Goddard Space Flight Center, Greenbelt, Maryland 

Victor D. Mora

Jet Propulsion Laboratory, Pasadena, California 

Victoria A. Regenie/NESC

NASA Dryden Flight Research Center, Edwards, California 

William F. Ritz 

Jet Propulsion Laboratory, Pasadena, California 

February 2009 
The NASA STI Program. . . in Profile

Since its founding, NASA has been dedicated to the 

advancement of aeronautics and space science. The 

NASA Scientific and Technical Information (STI) 

Program Office plays a key part in helping NASA 

maintain this important role. 

The NASA STI Program Office is operated by 

Langley Research Center, the lead center for NASA’s 

scientific and technical information. The NASA STI 

Program Office provides access to the NASA STI 

Database, the largest collection of aeronautical and 

space science STI in the world. The Program Office is 

also NASA’s institutional mechanism for 

disseminating the results of its research and 

development activities. These results are published by 

NASA in the NASA STI Report Series, which 

includes the following report types: 

x TECHNICAL PUBLICATION. Reports of 

completed research or a major significant phase 

of research that present the results of NASA 

programs and include extensive data or 

theoretical analysis. Includes compilations of 

significant scientific and technical data and 

information deemed to be of continuing 

reference value. NASA counterpart of peerreviewed formal professional papers, but having 

less stringent limitations on manuscript length 

and extent of graphic presentations. 

x TECHNICAL MEMORANDUM. Scientific 

and technical findings that are preliminary or of 

specialized interest, e.g., quick release reports, 

working papers, and bibliographies that contain 

minimal annotation. Does not contain extensive 

analysis. 

x CONTRACTOR REPORT. Scientific and 

technical findings by NASA-sponsored 

contractors and grantees. 

x CONFERENCE PUBLICATION. Collected 

papers from scientific and technical 

conferences, symposia, seminars, or other 

meetings sponsored or co-sponsored by NASA. 

x SPECIAL PUBLICATION. Scientific, 

technical, or historical information from NASA 

programs, projects, and missions, often 

concerned with subjects having substantial 

public interest. 

x TECHNICAL TRANSLATION. Englishlanguage translations of foreign scientific and 

technical material pertinent to NASA’s mission. 

Specialized services that complement the STI 

Program Office’s diverse offerings include creating 

custom thesauri, building customized databases, 

organizing and publishing research results ... even 

providing videos. 

For more information about the NASA STI Program 

Office, see the following: 

x Access the NASA STI Program Home Page at 

http://www.sti.nasa.gov

x E-mail your question via the Internet to 

help@sti.nasa.gov 

x Fax your question to the NASA STI Help Desk 

at (443) 757-5803

x Phone the NASA STI Help Desk at 

(443) 757-5802

x Write to: 

 NASA STI Help Desk 

 NASA Center for AeroSpace Information 

 7115 Standard Drive 

 Hanover, MD 21076-1320
NASA/TM-2009-215569

NESC-RP-08-31/08-00451

Crew Exploration Vehicle (CEV) Avionics 

Integration Laboratory (CAIL) Independent 

Analysis

Mitchell L. Davis/NESC, Michael L. Aguilar/NESC

NASA Goddard Space Flight Center, Greenbelt, Maryland 

Victor D. Mora

Jet Propulsion Laboratory, Pasadena, California 

Victoria A. Regenie/NESC

NASA Dryden Flight Research Center, Edwards, California 

William F. Ritz 

Jet Propulsion Laboratory, Pasadena, California 

NASA Engineering and Safety Center 

Langley Research Center 

Hampton, Virginia 23681-2199 

February 2009 
The use of trademarks or names of manufacturers in the report is for accurate reporting and does not 

constitute an official endorsement, either expressed or implied, of such products or manufacturers by the 

National Aeronautics and Space Administration. 

Available from: 

NASA Center for AeroSpace Information 

7115 Standard Drive 

Hanover, MD 21076-1320 

(443) 757-5802 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

1 of 39 

NESC Request No.: 08-00451 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

May 1, 2008 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

2 of 39 

NESC Request No.: 08-00451 

Approval and Document Revision History 

Approved: Original signature on file 5/6/08 

 NESC Director Date 

Version Description of Revision Author Effective Date 

 Initial Release NASA Technical 

Fellow, Avionics, 

GSFC 

05-01-08 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

3 of 39 

NESC Request No.: 08-00451 

Table of Contents 

Volume I: Technical Report 

1.0 Authorization and Notification ........................................................................................ 5 

2.0 Signature Page ................................................................................................................... 6 

3.0 Team List ........................................................................................................................... 7 

4.0 Executive Summary .......................................................................................................... 8 

5.0 Task Overview ................................................................................................................. 10 

5.1 Integration and Test Process ................................................................................. 12 

5.2 Flat-Sat Approach ................................................................................................. 15 

5.2.1 Flat-Sat System Fidelity ........................................................................................ 16 

5.2.2 System Software Development Major Attributes of Integration Labs ................. 18 

5.2.3 Software Development Parallel Analysis, Evaluation, Integration, and Test ....... 18 

5.2.4 Flat-Sat Summary ................................................................................................. 19 

5.3 SAIL Approach ..................................................................................................... 19 

5.3.1 SAIL Summary ..................................................................................................... 28 

5.4 CAIL Concept ....................................................................................................... 28 

5.4.1 CAIL Summary ..................................................................................................... 30 

5.5 Software Test Beds ............................................................................................... 31 

5.5.1 An Example of Distributed Software Development Analysis, Evaluation, 

Integration, and Test ............................................................................................. 31 

5.6 Comparison Summary of SAIL, CAIL, and Flat-Sat ........................................... 32 

6.0 Comments from NESC Preliminary Briefing (March 17, 2008) ................................ 33 

7.0 Findings, Observations, and Recommendations .......................................................... 35 

7.1 Findings................................................................................................................. 35 

7.2 Observations ......................................................................................................... 35 

7.3 Recommendation .................................................................................................. 36 

8.0 Definition of Terms ......................................................................................................... 36 

9.0 List of Acronyms ............................................................................................................. 37 

10.0 References ........................................................................................................................ 38 

List of Figures

Figure 5.0-1. Lunar Reconnaissance Orbiter Breadboard (left), and Lunar Reconnaissance 

Orbiter Spacecraft (right) ...................................................................................... 10 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

4 of 39 

NESC Request No.: 08-00451 

Figure 5.0-2. Solar Dynamics Observatory Flat-Sat Engineering Test Units (left) and Ground 

Controller (right) ................................................................................................... 11 

Figure 5.0-3. Tropical Rainfall Measuring Mission Flat-Sat in Operational Support ................ 11 

Figure 5.0-4. Hubble Space Telescope VEST in Clean Room ................................................... 12 

Figure 5.1-1. Flight System Development Process .................................................................... 13 

Figure 5.1-2. Comparison of the Flat-Sat Approach to the SAIL/CAIL Approach ................... 14 

Figure 5.2-1. Flat-Sat Approach Phasing Relative to the Flight System .................................... 16 

Figure 5.2-2. Notional Diagram of Flat-Sat range of Fidelity .................................................... 17 

Figure 5.3-1. SAIL Shuttle Test Station Vehicle Layout ........................................................... 21 

Figure 5.3-2. SAIL Mission Capability ...................................................................................... 22 

Figure 5.3-3. Shuttle Test Station, including the SAIL Flight System ....................................... 24 

Figure 5.3-4. Middeck of the Forward Avionics Bays ............................................................... 24 

Figure 5.3-5. Aft Avionics Bays with Two SSME Controllers .................................................. 25 

Figure 5.3-6. Launch Processing System ................................................................................... 25

Figure 5.3-7. Test Operations Center ......................................................................................... 26 

Figure 5.3-8. Digital Strip Chart Recorders................................................................................ 26 

Figure 5.3-9. SAIL Verification Process Flow ........................................................................... 27 

Figure 5.4-1. CEV Development Flow ....................................................................................... 29 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

5 of 39 

NESC Request No.: 08-00451 

Volume I: Technical Report 

1.0 Authorization and Notification 

Mr. Doug Cooke, NASA Deputy Associate Administrator for the Exploration Systems Mission 

Directorate (ESMD) at NASA Headquarters, requested a Crew Exploration Vehicle (CEV) 

Avionics Integrated Lab (CAIL) Independent Analysis to compare the strategic differences, risks 

and benefits between the current CAIL approaches to alternative NASA approaches. The NESC 

team worked independent of the Constellation Program (CxP) to evaluate both flight hardware 

and software development. 

The NESC team started their comparison with the Shuttle Avionics Integration Laboratory 

6$,/DQGURERWLFPLVVLRQ3)ODW-Sat ́DSSURDFK

An NESC out-of-board activity was approved February 8, 2008 and Mr. Mitchell L. Davis, 

NASA Technical Fellow for Avionics, was chosen to lead this assessment. A stakeholder 

outbrief was presented to NESC Review Board (NRB) on March 13, 2008. A formal stakeholder 

outbrief was presented to Dr. Richard Gilbrech, Associate Administrator for the Exploration 

Systems Mission Directorate (ESMD) at NASA Headquarters, and guests at NASA Headquarters 

on March 18, 2008. The final NESC CAIL report was presented to the NRB on May 1, 2008. 

The key stakeholder for this assessment is Mr. Doug Cooke. 7KHSURMHFW¶VSRLQWRIFRQWDFWLV

Mr. Aaron Van Baalen, Avionics and Software Test &Verification (T&V) Lead, Orion Project 

Office. 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

6 of 39 

NESC Request No.: 08-00451 

2.0 Signature Page 

Assessment Team Members 

Mr. Mitchell L. Davis Date Mr. Michael L. Aguilar Date 

 

Mr. Victor D. Mora Date Mr. William F. Ritz Date 

Ms. Victoria A. Regenie Date 

 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

7 of 39 

NESC Request No.: 08-00451 

3.0 Team List 

Name Discipline Organization/Location 

Core Team 

Mitch Davis Lead, Technical Fellow Avionics GSFC 

Michael Aguilar Co-Lead, Technical Fellow Software GSFC 

Victor Mora 

Deputy Integration and Test Manager for 

Advanced Mirror Development (AMD) 

Project JPL 

William Ritz NASA SAIL Manager JSC (Retired) 

Victoria Regenie NESC Systems Engineer DFRC 

Support 

Angela Hinton Project Coordinator ATK, LaRC 

Erin Moran Technical Writer ATK, LaRC 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

8 of 39 

NESC Request No.: 08-00451 

4.0 Executive Summary 

Two approaches were compared to the Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) approach: the Flat-Sat and Shuttle Avionics Integration Laboratory (SAIL). 

The Flat-Sat and CAIL/SAIL approaches are two different tools designed to mitigate different 

risks. Flat-Sat approach is designed to develop a mission concept into a flight avionics system 

and associated ground controller. The SAIL approach is designed to aid in the flight readiness 

verification of the flight avionics system. The approaches are complimentary in addressing both 

the system development risks and mission verification risks. 

The CAIL concept is modeled after the SAIL approach with a number of notable differences. 

These include: For the Orion Project, a coordinated, end-to-end box to system level development 

and verification process was not evident in the information provided. For the CAIL concept, a 

coordinated, end-to-end box to system level development and verification process was not 

evident in the information provided. The CAIL concept assumes the flight subsystems will be 

matured for the system level verification prior integration into the CAIL. It was not apparent to 

the NESC team that there was a documented plan to assure unit/sub-system maturity prior to 

installation into the CAIL. Both the CAIL and the Lockheed-Martin (LM) operated Exploration 

Development Laboratory (EDL) documents did not reference any intentional effort to provide an 

engineering team knowledge transfer from the CAIL integration and verification team to the 

ground operations and mission operations teams. The NESC team noted that, based on their 

H[SHULHQFHVXFFHVVIXOSURMHFWVDUHNHHQO\DZDUHRIWKHLUWRROV¶DELOLWLHVDQGOLPLWDWLRQVDQGFDQ

quickly adjust to the dynamically changing risk priority and hardware/software configurations 

during the development phase. Coordination of the verification process is essential to minimize 

the risk of missing key issues and to be prepared for schedule deviations. 

The Flat-Sat approach starts early in the development and is designed to mature into an accurate 

representation of the flight system. It then transitions to supporting flight software development 

and mission operations. The SAIL approach is a high fidelity engineering replication of the 

flight article designed to verify integrated system performance, capture system discrepancies 

from mission to mission, and aid in reducing schedule pressures on the flight article. 

The SAIL approach is to use flight qualifiable or qualified hardware in the flight systems. 

However, depending on unique test requirements, it may use component simulators or 

engineering test units (ETUs). 

The following NESC team findings were identified: 

x A coordinated, end-to-end, box to system level development and verification process was 

not evident for the proposed CAIL concept. 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

9 of 39 

NESC Request No.: 08-00451 

x The CAIL assumption is that the flight subsystems will be matured for the system level

verification. 

x The Flat-Sat and SAIL approaches are two different tools designed to mitigate different 

risks. 

The following NESC team recommendation was provided: 

x Define, document, and manage a detailed interface between the design and development 

(EDL and other integration labs) to the verification laboratory (CAIL). 

The Orion Project Office is actively working this recommendation. 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

10 of 39 

NESC Request No.: 08-00451 

5.0 Task Overview 

To accomplish this quick turnaround assessment (3-week timeframe), the approach taken 

involved a combination of interviews of subject matter experts (SMEs) associated with Flat-Sat 

and SAIL operations and a review of available documentation associated with the proposed 

Orion CAIL concept. Section 10.0 provides the references the NESC team reviewed for this 

assessment. 

The SME interview process started with the Goddard Space Flight Center (GSFC) Integration 

and Test (I&T) Branch Head was interviewed for a detailed summary of the Flat-Sat approach. 

This summary was shared with the NESC team with additional inputs from: Victor Mora, Jet 

Propulsion Laboratory (JPL); the NASA Technical Fellows for Avionics and Software; and Bill 

Ritz, retired SAIL Manager. Mr. Ritz, retired SAIL Manager, provided the team with a detailed 

overview of the SAIL operation and development history. Aaron Van Baalen, Avionics and 

Software T&V Lead, provided a detailed description of the planned CAIL operations approach. 

On March 3, 2008, the Johnson Space Center (JSC) Engineering Directorate, Avionic Systems 

Division, senior leaders and CAIL team leaders met at GSFC for a Flat-Sat discussion and tour 

of the Lunar Reconnaissance Orbiter (LRO) (Figure 5.0-1), Solar Dynamics Observatory (SDO) 

(Figure 5.0-2), Tropical Rainfall Measuring Mission (TRMM) (Figure 5.0-3), and Wilkinson 

Microwave Anisotropy Probe (WMAP) Flat-Sat laboratories. Additionally, the Hubble Space 

7HOHVFRSH¶VVehicle Electrical System Test (VEST), which serves a similar purpose as the 

SAIL/CAIL (Figure 5.0-4), was also visited. At the conclusion of this visit, good understanding 

on the similarities and differences of the various approaches among the NESC team, JSC 

Engineering Directorate senior leaders, and the CAIL team leaders was achieved. 

Figure 5.0-1. Lunar Reconnaissance Orbiter Breadboard (left), and Lunar Reconnaissance 

Orbiter Spacecraft (right) 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

11 of 39 

NESC Request No.: 08-00451 

Figure 5.0-2. Solar Dynamics Observatory Flat-Sat Engineering Test Units (left) and 

Ground Controller (right) 

Figure 5.0-3. Tropical Rainfall Measuring Mission Flat-Sat in Operational Support 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

12 of 39 

NESC Request No.: 08-00451 

Figure 5.0-4. Hubble Space Telescope VEST in Clean Room 

5.1 Integration and Test Process 

For this report, I&T is the process that matures a design concept and develops and integrates the 

flight hardware with ground controller systems into a fully integrated operational system. The 

NESC team found that some terms used to describe the I&T process caused confusion during 

interchanges due to location specific definitions. Therefore, for this assessment, the NESC team 

clarified the I&T definition to enable focusing on the general process and not the specific 

different details. 

 3Development ́UHIHUs to the process of functional testing, interface testing, open-loop 

testing, and close-loop testing necessary to mature a concept into an operational system. 

 3Verification ́UHIHUs to the formal process of demonstrating a design is flight ready and is 

assumed to be interchangeable in the general sense with the terms verification, validation 

and certification. 

 3Flight Avionics System ́UHIHUVWRDOOHOHFWULFDOHOHFWURQLFV\VWHPVWKDWZLOOIO\RQWKH

vehicle. 

 3Ground Controller ́UHIHUVWRWKH ground electrical/electronic systems, tools, and 

procedures that enable the development and operation of the Flight Avionics System. 

During the development process, different combinations of electrical/electronic systems and 

ground systems replicate various flight configurations enabling the design evolution from 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

13 of 39 

NESC Request No.: 08-00451 

concept into a flight-ready system. There is no standard and fixed tool set; rather the tools 

depend directly oQWKHSURMHFW¶VULVNVDQGPLWLJDWLRQSODQ6RPHH[DPSOHVRIWKHVHWRROV include 

software test beds, computer development unit, and simulators. The NESC team noted that, 

based on their experience, the successful projects are the ones who are keenly aware of their 

WRROV¶DELOLWLHVDQGOLPLWDWLRQVDQGFDQTXLFNO\DGMXVWWRWKHG\QDPLFDOO\FKDQJLQJULVNSULRULW\

and hardware/software configurations of the development phase. 

Early in the development cycle of a space flight design, high level requirements are parceled out 

to lower level subsystems, as shown in Figure 5.1-1. The requirements are then translated into 

functions that will be implemented in the specific hardware or software subsystem. Decisions 

are made regarding all aspects of the design, from the electrical circuit architecture to wire 

routing. These decisions will then define the design features. The subsystem is assembled into a 

system with an associated ground controller. Later in the development process, the subsystem 

was confirmed that the initial requirements were met and, more importantly, all other design 

features were documented. The requirements are the minimal set of functions that the system 

must do where the design features are all functions the system can do. 

Figure 5.1-1. Flight System Development Process 

A large portion of these design features are desirable and aid in meeting the requirement. For 

example, the power interface and the command interface to a computer which implements the 

control requirement are necessary design features (e.g. results of the implementation). Some of 

these design features are benign and may not be desirable, but are not at a risk level that would 

dictate removal from the design; rather they lead to a limitation or operational constraint. For 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

14 of 39 

NESC Request No.: 08-00451 

example, a condition in which simultaneously executing commands subsystem-A ON and 

subsystem-B ON under worst case conditions will trip the circuit breaker. Since the ON 

commands are an infrequent operation, one is willing to accept an operational constraint of 200 

milliseconds between commands. The remaining design features are those that are undesirable 

and can lead to a premature anomaly or system failure. Once identified, the undesired feature is 

removed from the design. The undesirable features are discovered throughout the development 

phase by identifying a discrepancy between a predicted or expected subsystem response and the 

actual hardware/software response. 

The Flat-Sat approach is commonly used in the robotic spacecraft development where a single or 

limited number spacecraft or set of spacecraft are built. For the development of the single 

spacecraft, the formal system verification must occur on the flight unit to contain the risk to an 

acceptable level. The Flat-Sat serves three general purposes which align to the different phases 

of the development: I&T development (primary purpose), the flight software development 

(secondary purpose), and the flight operations support and maintenance (tertiary purpose). The 

Flat-Sat approach is optimized for development where the fidelity and configuration control is 

moderated to enable rapid design evolution from an initial concept to a flight ready system. For 

the single or set of missions, the formal verification process occurs on the flight unit and 

therefore there is less need for flight readiness verification on a high fidelity copy. Figure 5.1-2 

compares the Flat-Sat approach to the SAIL/CAIL approach. 

Figure 5.1-2. Comparison of the Flat-Sat Approach to the SAIL/CAIL Approach 

For the SAIL approach, high fidelity and rigorous configuration control enables flight avionics 

verification with high degree of confidence for the first mission and subsequent missions. The 

SAIL approach is optimized to identify subtle discrepancies in the design by a high fidelity 

replication of the flight avionics system. For multi-missions, the SAIL approach provides the 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

15 of 39 

NESC Request No.: 08-00451 

tool for the formal flight readiness verification prior to final installation on the vehicle and 

allows identification and resolution of system level discrepancies from mission to mission. 

5.2 Flat-Sat Approach 

The Flat-Sat begins assembly shortly after the Preliminary Design Review (PDR). It is an 

assembly of key subsystem Engineering Test Units (ETUs), simulators, ground controller and a 

flight-like harness. A ground controlling system is developed in conjunction with the flight-like 

avionics system. Shortly after the Critical Design Review (CDR), the amount of testing on the 

Flat-Sat accelerates as the spacecraft matures. When a major subsystems arrives, extensive 

interface and functional tests are completed which result in discovering discrepancies in the 

requirements or design that lead to improvements in the flight design and increased accuracy of 

the Flat-6DW¶VVLPXODWion. Discovering these discrepancies in this early phase allows for changes 

to be incorporated into the flight design prior to delivery. Since the design engineers and the 

ground operations team are both working concurrently on the Flat-Sat, there is beneficial 

knowledge transferred between the teams. The Flat-Sat approach also enables the concurrent 

development of all integration plans and procedures, the ground controller command files, and 

the telemetry files data base. 

The GSFC practice is to not place a flight unit into the Flat-Sat mainly due to the lack of 

historical pedigree of the associated flight-like avionics. However, JPL does not have this 

practice. It is common to place a Flat-Sat ETU into the flight subsystem as a place-holder while 

the flight unit is elsewhere. Around the time the first Comprehensive Performance Test is 

completed on the flight system, the Flat-Sat would primarily be supporting flight software 

development. When a discrepancy is discovered that results in a hardware change, the corrective 

action is demonstrated on the Flat-Sat before implementation on the flight system. Upon launch, 

the Flat-Sat operation is transferred to the flight operations team for flight operational support 

and software maintenance. Figure 5.2-1illustrates this progression. For large out-of-house 

spacecraft like Earth Observing System (EOS)-Aqua and EOS-Aura, a Flat-Sat is a contractorowned system, with a separate, nearly identical Flat-Sat system delivered for operational support. 

See Figure 5.2-1. 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

16 of 39 

NESC Request No.: 08-00451 

Figure 5.2-1. Flat-Sat Approach Phasing Relative to the Flight System 

5.2.1 Flat-Sat System Fidelity 

The Flat-Sat hardware is comprised of ETUs that are modified to be configurationally identical 

with the current flight design. Additionally, fidelity of the simulators increase as discrepancies 

are discovered and mitigated by design modifications. The fidelity of the wire will vary 

depending on the potential risk. High speed complex interfaces require duplication of the flight 

design and length. Experience has shown that a lower fidelity power harness is an acceptable 

risk and thus the exact flight length is not typically replicated. Figure 5.2-2 describes, in a 

notational sense, variations in fidelity of the Flat-Sat over time. The discrepancies in the ordinate 

axis are defined as system responses that are different from expected responses. As mentioned 

earlier, the discrepancies are a result of design features which can be beneficial, benign or 

undesirable. It is the benign design features which lead to limitation or operational constraints 

and the undesirable design features which are most critical to identify. These hidden 

discrepancies are discovered by operational time and testing of the Flat-Sat system. Note that a 

finite or limited time from initial concept to launch, which is what creates the acceptable risk 

level. 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

17 of 39 

NESC Request No.: 08-00451 

Figure 5.2-2. Notional Diagram of Flat-Sat range of Fidelity 

The Flat-Sat approach is a risk trade bounded by two limits: insufficient fidelity and insufficient 

time. Both impede the ability to identify discrepancies prior to entering the flight system 

verification phase. In the flight system verification phase, a design change halts the verification, 

the hardware is corrected, and time consuming regression testing is required. Therefore, the 

desire is to maximize the discovery of (benign and undesirable) discrepancies prior to the 

verification phase. At one extreme, a Flat-Sat placed in operation too quickly will be of 

insufficient fidelity and will not adequately simulate the flight system, thus the rate of 

discovering discrepancies will result in the detection of an unacceptable number of discrepancies 

upon entering flight system verification. At the other extreme, building an exceptionally high 

fidelity Flat-Sat consumes a significant portion of the available time and impacts configuration 

management complexity. It also becomes cumbersome to reconfigure Flat-Sat systems resulting 

in insufficient time to identify and resolve discrepancies prior to entering the flight system 

verification phase. The goal is to verify all requirements on the flight system. However, those 

requirements that can not be verified on the flight system are verified on the lower fidelity FlatSat. The risk associated with a lower fidelity Flat-Sat is not directly translated to higher mission 

risks. The reason is most requirements are verified on the flight system, and the requirements 

verified on the Flat-Sat are analyzed to be at the appropriate fidelity. 

Hardware and software test automation requires command and control capabilities above and 

beyond flight system operations. External control of test simulators and hardware is required to 

set initial conditions and repeat the test from these initial conditions. The capability to sense, test, 

and record the system response is required for automated test execution and pass-fail evaluation. 

Off-nominal behavior requires insertion or simulation of system faults in hardware and software, 

under control of the test scripts. System performance can be evaluated within the margins and 

limitations of the test environment fidelity. High fidelity portions of the system can be evaluated 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

18 of 39 

NESC Request No.: 08-00451 

for performance, even within an overall low-fidelity test system. Within the Flat-Sat 

environment, knowledge transfer of ground and mission operations is value-added to the 

operation, test, and development personnel. Familiarity of the mission operations equipment, 

procedures, and support software optimizes the quality of this knowledge transfer. 

5.2.2 System Software Development Major Attributes of Integration Labs 

The scope and needs of the laboratory customers directly influence the test bed fidelity. Test bed 

customers include simulation developers, hardware and software integration and test engineers, 

ground operations and ground application developers, mission operators, external hardware 

interface engineers and system integration engineers. Scheduling the use of the system, with 

margin for delivery and failure analysis, determines the required number and capabilities of the 

test beds. 

Simulation developers require access to high fidelity interfaces and models to identify the 

simulation accuracy and limitations of flight and ground systems. The command and control 

interface to these simulations need to be implemented and supported. Hardware and software 

integration and verification requires test suite development, command and control interface 

development, and access to interface signals for diagnostic purposes. Ground operations require 

test suite development time for environment and integration tests, test control and telemetry 

monitoring, and code modification procedures. External users will need access to the test bed for 

acceptance testing. 

Sparing of hardware for the test bed and the time expected to repair the test bed should be 

considered. This drives maintenance personnel and maintenance scheduling requirements.

5.2.3 Software Development Parallel Analysis, Evaluation, Integration, and Test 

Multiple test beds of varied fidelity can be used in parallel to support user demands. An effort 

must be made to detail what subsystems are verified on which test environment. Software 

partitioning the system into subsystems, with the minimal interfaces and the minimal 

dependencies, reduces integration risk and effort when the subsystems are integrated. 

Maximizing the similarity of the test operations interfaces to the actual ground and flight 

operations interfaces provides an early start for ground and operations development and 

knowledge sharing between personnel. Even a low-fidelity test environment can provide early 

insight into development issues. 

Software and software test bed configuration management ranges from minimal early in 

development to rigorous enforcement as the subsystem test beds migrate toward high-fidelity 

system simulations. The configuration management complexity is directly related to the number 

and variety of uses the test bed is required to support. Constantly switching between 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

19 of 39 

NESC Request No.: 08-00451 

configurations to support differing users needs to be addressed in the configuration management 

plan. 

Customers for a test bed can include parties external to a project, with conflicting schedules and 

priorities. Non-NASA-owned test beds can create conflicts and competition over schedule, 

configuration, and access to the use of the test bed. 

5.2.4 Flat-Sat Summary 

The Flat-Sat approach regulates the hardware/software and test bed configuration control process 

to a level that enables rapid development of the hardware and software and, as the design 

matures, the fidelity and configuration control increases appropriately. Mission success is 

directly linked to the earliest possible testing which enables corrective feedback into flight 

hardware/software design prior to fabrication. The I&T process and procedures development 

along with Flat-Sat is another advantage, which reduces the risk of inadvertent mistakes on the 

flight system. Finally, developing the Flat-Sat in conjunction with the eventual ground controller 

allows a solid knowledge transfer between the design engineering and mission operations teams. 

Although the lower fidelity system will allow early discrepancies to escape, past experience has 

shown that overall this is the optimal approach for development and balancing cost and schedule. 

5.3 SAIL Approach 

Relative to Shuttle Avionics Integration Laboratory (SAIL) background, the Space Shuttle Task 

Group and technology program planning in the 1960/1970 time frame included simulation 

testing of an integrated electronics system to ensure compatibility of individual systems. The 

intent would be verification of functions involving several subsystems performing as a whole, 

and would lead to development of operational procedures (human/machine functional 

relationships, etc.) and software design and test. Completeness of the simulations would be such 

that they are faithful analogs of the mission/configuration under test or investigation. 

The Space Shuttle Program (SSP) planning included a systems integration laboratory for 

subsystem development and integration, flight control, and mission verification of flight 

hardware/software (e.g. avionics). All Space Shuttle contractor proposals included 

subsystems/systems integration laboratories. The SAIL was located at the Johnson Space Center 

(JSC) to make use of: existing Government facilities; JSC civil service and support contractor 

expertise; proximity to the software development laboratory; ability to support Shuttle 

operational phase without being committed to a prime contractor (i.e., reduced operational 

costs); and better NASA insight into the development of a computer-controlled, integrated 

avionics system -- a novel idea in the 1971 time frame. For example, the facility and test support 

systems (integrated test area, test operations center, dynamics simulation, etc.) were provided by 

JSC. 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

20 of 39 

NESC Request No.: 08-00451 

The SAIL was a multi-center, multi-system that serviced three separate centers for vehicle I&T 

and operations. The Orbiter flight system and pertinent Ground Support Equipment (GSE) was 

provided by the Space Shuttle prime/integrating contractor. The Marshall Mated Elements 

System (MMES) was supplied by the Marshall Space Flight Center (MSFC) after the Approach 

and Landing Test (ALT), and included flight system components and pertinent GSE and test 

support/simulation systems. The Launch Processing System (LPS) was provided by the Kennedy 

Space Center (KSC). 

The SAIL is used as an integration and verification laboratory. System level integration was 

completed in the SAIL before formal verification. As new elements or design changes are 

implemented on the Space Shuttle, the SAIL is utilized for integration and test prior to formal 

verification. The Phase I breadboard SAIL was activated in the 1973 time frame. This allowed 

early installation of laboratory test systems, integration of early flight subsystems, and 

accomplishment of laboratory performance testing. Major SSP activities supported by SAIL 

included Phase II ALT, Phase III Orbital Flight Test (OFT), and Phase IV Mission Support. A 

significant activity between the completion of Phase II ALT and start of Phase III OFT was the 

Operational Flight Test Changeover which involved complete replacement of the avionics, bays, 

and wire harnesses. 

Other significant support activities included: LPS I&T with the SAIL flight system, preacceptance test of the Remote Manipulator System (RMS) (prior to delivery of the first "arm"), 

Ku-band subsystem integration and test, and multi-function electronic display system 

I&T. During the three phases of SAIL subsequent to Phase I, two test stations were utilized. The 

Space Shuttle test station was configured with flight qualifiable or flight- like avionics hardware 

in a multi-string configuration. This hardware was in a three-dimensional mock-up consisting of 

a cockpit, forward equipment bays, and aft equipment bays. Wire harnesses for this mock-up 

were flight configuration (length, size, material, and routing) except where SAIL facility 

limitations dictate otherwise. Figure 5.3-1 illustrates the vehicle layout in the SAIL Shuttle Test 

Station. The SAIL Shuttle Test Station Orbiter Flight System was designated Orbiter Vehicle 

(OV)-095. The Guidance, Navigation, and Control (GN&C) test station utilizes flight-type 

hardware mounted in racks connected with non-flight cabling. The test station supported GN&C 

ascent and limited on-orbit verification testing. The need for the GN&C test station was due to 

the overall verification test workload. The GN&C test station was decommissioned in the 1983 

time frame and recommissioned in the 1993 time frame, and later decommissioned, due to 

changes in SSP requirements. Figure 5.3-2 illustrates the SAIL Mission Capability. 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

21 of 39 

NESC Request No.: 08-00451 

Figure 5.3-1. SAIL Shuttle Test Station Vehicle Layout 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

22 of 39 

NESC Request No.: 08-00451 

Figure 5.3-2. SAIL Mission Capability 

Overall, the SAIL is a central facility where avionics and related flight hardware (or simulations 

of this hardware), flight software, flight procedures, and associated GSE are brought together for 

integration and mission verification testing. The SAIL supports avionics integration, test, and 

verification from pre-launch to abort phases such as Return-To-Launch Site (RTLS), and landing 

and roll-out. The SAIL configuration supports total integration of the Orbiter avionics, Shuttle 

Mated Elements, Space Shuttle LPS, and Space Shuttle payloads, and near real-time mission 

support. 

 

Top-level functions for the SAIL are: 

1. Full Orbiter ship set of flight configuration avionics. 

2. Sufficient mated element flight configuration avionics to check all interfaces. 

3. Subsystem to subsystem integration that make up the flight systems. 

4. Hardware/software flight system integration to verify mission capability. 

5. Formal test operations and failure reporting. 

6. Rigorous quality control and configuration management. 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

23 of 39 

NESC Request No.: 08-00451 

7. Electrical interfaces and physical space to accommodate payload testing. 

8. Located at JSC to use Government facilities. 

 

The SAIL objectives include: 

 

x Systems Verification 

± evaluation of subsystem interfaces 

± hardware and software integration 

± validation of vehicle ground checkout software 

± evaluation of redundancy management 

± determination of system limitations 

± mated element integration 

± launch processing system integration 

x Mission Verification 

± hardware and software capability 

± flight operating procedures such as in-flight maintenance 

x Operational Support 

± real-time mission support 

± payload integration 

± anomaly investigation 

The SAIL flight system (OV-095) configuration tracks and complies with the vehicle flight 

configuration. This includes the electrical power distribution and control; displays and controls; 

GN&C; data processing: mated elements [Space Shuttle Main Engine (SSME), Solid Rocket 

Booster (SRB, and External tank ET)]; payload interfaces; caution & warning; RMS; range 

safety: and Orbiter wire harness. The philosophy relative to non-avionics was anything that 

exercised flight software where it was not practical to have flight hardware would be simulated 

in the SAIL non-avionics simulation. For Ku-Band, the SAIL provided integration within the 

subsystem as well as subsystem-to-system level integration. Figure 5.3-3 shows the Space 

Shuttle test station, including the SAIL flight system. Figure 5.3-4 shows the middeck of the 

forward avionics bays. Figure 5.3-5 shows the aft avionics bays with two SSME controllers. 

Figure 5.3-6 shows the LPS. Figures 5.3-7 and 5.3-8 show the new/upgrade of the Test 

Operations Center and digital strip chart recorders. 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

24 of 39 

NESC Request No.: 08-00451 

Figure 5.3-3. Shuttle Test Station, including the SAIL Flight System 

Figure 5.3-4. Middeck of the Forward Avionics Bays 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

25 of 39 

NESC Request No.: 08-00451 

Figure 5.3-5. Aft Avionics Bays with Two SSME Controllers 

Figure 5.3-6. Launch Processing System 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

26 of 39 

NESC Request No.: 08-00451 

Figure 5.3-7. Test Operations Center 

Figure 5.3-8. Digital Strip Chart Recorders 

The overall verification process was a joint effort of civil service and contractor personnel. This 

effort was organized by engineering teams usually led at the subsystem level. These teams were 

part of the SAIL test team, as required, and participated from requirements definition through 

test procedure development, test execution, data analysis, and problem identification and 

correction/retest. Figure 5.3-9 illustrates the verification process flow. 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

27 of 39 

NESC Request No.: 08-00451 

Figure 5.3-9. SAIL Verification Process Flow 

The avionics verification process began when the software was released from the software 

production/reconfiguration facility and delivered to the SAIL. It was then integrated and closedloop tested, with the avionics flight hardware in simulated mission sequences. Actual flight 

procedures/flight data files were integrated as appropriate into the SAIL Integrated Test and 

Checkout Procedures (ITCP). One purpose of the SAIL testing was to assure the avionics flight 

hardware and software would safely perform the upcoming mission. The process includes the 

verification of all hardware and software changes since the previous flight and the re-verification 

of the base software capabilities when new software Operational Increment (OI) was released for 

integration and test in the SAIL. One of the key elements of the verification process was the 

audit trail and accountability for all hardware and software changes from requirements to flight 

readiness. This is implemented through avionics engineering teams who represent ascent, onorbit, descent, avionics services, and data processing. An avionics flight system team also 

integrates across the other operations teams. The SAIL test requirements were coordinated 

through each of the avionics teams with a starting point of "what verification testing do you 

require in order to sign a flight readiness statement for your area of responsibility?" One part of 

the process was an evaluation of all hardware and software changes in the respective area of 

responsibility, and a definition of the type of test required to verify the respective system. The 

appropriate hardware changes/modifications were installed in the SAIL. These tests were not 

designed to be part of a test series on the actual flight vehicle. 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

28 of 39 

NESC Request No.: 08-00451 

 

Overall, SAIL provided the capability to identify problems and implement and test corrective 

action using an integration/verification laboratory rather than the utilizing an Orbiter. 

Approximately 300 software problems and approximately 50 hardware problems were identified 

as a result of integration and test in the SAIL prior to the first operational flight test. 

5.3.1 SAIL Summary 

In summary, the SAIL provided the capability to: 

1. Verify the systems functional and compatibility of the integrated Orbiter avionics and the 

associated software. 

2. Verify the human/machine and avionics interfaces, integrated functions, avionics 

capability of the Orbiter, the mated elements (SSME, SRB, and ET) and the LPS. 

3. Verify the avionics interfaces between payloads and the Orbiter. 

4. Perform systems level investigations and verifications for mission unique software, 

mission functional requirements, operational anomalies, and effects of system design 

changes. 

5.4 CAIL Concept 

The CAIL concept is nearly identical to the SAIL approach. It is envisioned as a high fidelity, 

human-in-the-loop, and closed-loop flight simulation environment for the CEV avionics 

integration testing, which can execute mission scenario, ground and flight, nominal and offnominal conditions. 

The Orion CAIL team LGHQWLILHGWKH&$,/¶VSrimary functions1

: 

x Integration testing of the CEV avionics hardware and software and associated internal 

and external CEV interfaces. 

x CxP element-to-element integration testing as the CEV element. 

x Test and analysis operational support during CEV missions. 

The Orion CAIL team identified CAIL¶V secondary functions2

: 

x Artifacts of a high fidelity flight-configuration test rig. 

x Engineering evaluation of proposed designs or subsystem/software changes. 

 

1

Section 10.0, Reference 3 

2

Section 10.0, Reference 3 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

29 of 39 

NESC Request No.: 08-00451 

x Crew training and procedures checkout. 

The NESC team commended the Orion CAIL team for recognizing the need to set a priority of 

CAIL functions and for limiting the scope of new demands. Without these controls, the CAIL 

system would quickly exceed the available resources (e.g. cost and schedule). Although the 

outbriefs indicated there would be two production and one development/maintenance CAIL rigs, 

the Orion CAIL team noted the program plan includes three identical production rigs and no 

development/maintenance rig. The reason for this change was the projected increased nonrecurring engineering cost of the development/maintenance rig. 

A key assumption made by the Orion CAIL team was that all avionics, software and GN&C 

subsystems delivered to CAIL will have been verified at the component and/or assembly level. 

This implies that there will be minimal development activities in the CAIL. To be more precise, 

the elements delivered to the CAIL will already have been operating at an entire system level and 

thus CAIL is not the first time the elements operate at a system level. Figure 5.4-1 indicates the 

CEV development flow is vertical (low- to high-level). 

Figure 5.4-1. CEV Development Flow 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

30 of 39 

NESC Request No.: 08-00451 

The NESC team focused on the depicted flow of elements into the system verification [CAIL, 

Electronics Systems Test Laboratory (ESTL)] and the LM EDL. The NESC team was expecting 

a more iterative process as shown in the SAIL or evolving process as discussed in the Flat-Sat 

approach. Additionally, the EDL is positioned in the development flow as the Flat-Sat, and 

therefore could construe that it performs a similar function. However, it was noted by the Orion 

CAIL team that the EDL is not dedicated to the development process rather the purpose is 

defined by LM as 3'HYHORSDQGDSSO\WHFKQRORJ\ ́DQG30DNH&RQVWHOODWLRQVXFFHVVIXO ́3

. 

Clearly, the development function will make CxP successful, but the development and apply 

technology is more focused on future CxP elements and can compete for utilization time. The 

EDL is aQ3LQYHVWPHQW ́LQWR&[3E\/0DQGFRQWUDFWRUSDUWQHUVHVVHQWLDOO\3/0EXLOGVLW

NASA can XVHLW ́,WZDV not possible to determine the exact role the EDL will play in the flight 

avionics system development. It is important to note that there are four vehicle management 

computer test beds planned for avionics and software development and verification testing at the 

subsystem level. Thus, the Orion CAIL team has several options to fill the role of the 

development tool: dedicate the EDL, increase the fidelity of a vehicle management computer test 

beds, or decrease the fidelity of one of the CAIL systems. 

The requirement4

 of the CAIL portability was raised during CxP and NESC team discussions. 

Portability implies a level of modularity that does not exist in the flight article and appears 

contrary to the necessity to duplicate the flight system. When conflicting requirements exist, a 

priority must be established to guide the desired result. A review of the Spacecraft Test & 

Verification Facility System Requirements Document (CEV-T-082100) did not confirm a clear 

hierarchy. 

5.4.1 CAIL Summary 

The strategic approach of the CAIL is similar to the SAIL approach. Essentially, the approach is 

to create a tool of sufficient fidelity to enable a high fidelity, human-in-the-loop, closed-loop 

flight simulation environment for CEV avionics integration testing with the intent of reducing 

the risk of undiscovered safety discrepancies residing in the flight avionics system. Rigor in 

fidelity and configuration control required for verification of first mission and subsequent 

missions is cumbersome and not optimized for system development. A key assumption is the 

flight units delivered to the CAIL will be ready for system level verification and beyond 

development phase. The CAIL is a tool enabling formal flight readiness verification or preverification for formal verification on the flight article. 

 

3

Exploration Development Lab (EDL) Overview Presentation (Slide 6), Lockheed Martin-owned and operated for Constellation Program and 

internal IR&D of Exploration Capabilities.

4 3/HYHO$)XQFWLRQDO5HTXLUHPHQWV)RUWKH&UHZHG([SORUDWLRQ9HKLFOH&(9$YLRQLFV,QWHJUDWLRQ/DE&$,/ ́7UDQVLWLRQ to 

Sustained Operations: The CAIL test system(s) shall be portable. 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

31 of 39 

NESC Request No.: 08-00451 

The development process from concept to flight system will utilize different tools as the design 

matures and the CAIL verification function must be aligned and coordinated with other planned 

end-to-end tools. It was not evident that the interaction of the different Orion development tools 

has been sufficiently coordinated. The Orion CAIL team recognized the coordination necessity 

and conducted a Technical Interchange Meeting (TIM) on the end-to-end testing in July 2007 

and a follow-on meeting that occurred in the spring of 2008. 

5.5 Software Test Beds 

During a development process a large portion of the time will be spent on software development, 

test and verification with the hardware. The overall development plan will directly influence the 

test bed fidelity and indirectly defines the scope and needs of the laboratory customers. For a 

software test bed there are a large number of users (simulation developers, hardware and 

software integration and test engineers, ground operations and ground application developers, 

mission operators, external hardware interface engineers, and system integration engineers). The 

conglomerate of users dictates a tightly coordinated delivery scheduling and user scheduling. 

5.5.1 An Example of Distributed Software Development Analysis, Evaluation, 

Integration, and Test 

Test beds can be supplied to external developers. The James Webb Space Telescope (JWST) 

exported project-owned test beds to the telescope instrument developers. The test beds consisted 

of commercial off-the-shelf (COTS) processors, high-fidelity hardware interfaces, the GSFCdeveloped Command and Data Handling (C&DH) flight software, the actual mission operations 

ground software, and an identical software development tool suite used at GSFC. The external 

developer is able to deliver flight-ready instrument software, instrument integration tests, and a 

set of mission procedures and the mission operations database. These deliverables are verified at 

their development site in the instrument operational environment. 

The instrument developers are using the test beds to support flight software development, 

environment test development, integration test development, system test procedures, and mission 

operations database and procedures. The exported test bed supported high-fidelity software 

interface verification with the C&DH, early high-fidelity subsystem hardware and software 

integration test development, mission operations database development, and mission operations 

process development. 

The GSFC developer is able to deliver C&DH software updates to the external developers for 

early subsystem integration and verification. A side effect of the exported test bed was the 

increased fidelity of communication between the external developers and GSFC. A shared 

knowledge of the test system developed due to the similarity of the development and test 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

32 of 39 

NESC Request No.: 08-00451 

environments. Shared knowledge of the JWST system developed due to the consistent system 

visibility. Solutions to shared issues were solved by involving all the developers. Expertise 

gained at one site was shared by the other sites. Late development at one site quickly "caught 

up" with the integration schedule through this information sharing, which in turn greatly reduced 

the learning-curve on the test beds. 

5.6 Comparison Summary of SAIL, CAIL, and Flat-Sat 

Figure 5.1-2 indicates the different purposes the Flat-Sat, SAIL and CAIL tools provide in 

relationship to the development cycle: 

x The Flat-Sat is a tool to mature a single mission concept into a flight avionics system and 

associated ground controller. The Flat-Sat fidelity progression enables rapid design 

maturation and earlier delivery enables more time on system with an opportunity to 

identify and correct discrepancies. 

x For the SAIL/CAIL tools, a high fidelity and rigorous controlled tool enables first 

mission and subsequent mission flight avionics verification with a high degree of 

confidence. 7KHWRROV¶KLJKILGHOLW\ enables rapid re-verification from mission to mission 

and provides programmatic risk management along with technical risk mitigation. 

Figure 5.1-2 indicates the different purposes the Flat-Sat, SAIL and CAIL tools provide in 

relationship to the development cycle. The Flat-Sat is a tool to mature a mission concept into a 

flight avionics system and associated ground controller. Then, the software development along 

with mission operations support follows. The Flat-Sat initial lower fidelity enables rapid 

maturity of design and lower system fidelity traded for earlier delivery enables more time on 

system and an opportunity to identify and correct discrepancies. Due to the single mission 

dictating that the formal verification process must occur on the flight article, there is no need for 

formal verification (and thus high fidelity) on the Flat-Sat tool. 

For the SAIL/CAIL tools, a high fidelity and rigorous controlled tool enables first mission and 

subsequent mission flight avionics verification with high degree of confidence. The tools¶ high 

fidelity enables rapid re-verification from mission to mission and provides programmatic risk 

management along with technical risk mitigation. 

These tools are not exclusive; rather, they are complimentary in mitigating different risks 

encountered along the development from concept to operations. The NESC team noted that, 

based on their experience, the successful projects are keenly aware of their available WRROV¶

abilities and limitations and can quickly adjust to the dynamically-changing risk priority and 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

33 of 39 

NESC Request No.: 08-00451 

hardware/software configurations of the development phase. Therefore, it is more important to 

have a complete tool set, than selecting any one individual tool. 

6.0 Comments from NESC Preliminary Briefing (March 17, 2008) 

It was observed by the NESC team that WKHUHTXLUHPHQWIRU3JHRPHWULFDOO\LGHQWLFDO ́in the 

CAIL was not addressed. This requirement sets a threshold on the risks that the CAIL must 

duplicate and thus provide the opportunity to discover these risks/discrepancies. To be 

quantitative of the benefit/cost of this risk threshold, it would require a statistical analysis of 

QXPHURXVSDVWSURMHFWVEDVHGRQWKHLQGLYLGXDOSURMHFWV¶ULVNILGHOLW\RIWKHLUWRROV, and types of 

risks discovered. Unfortunately, this data is not known to exist in any collective format. Based 

on experience and knowledge of the development process, a general relationship between risk 

and system fidelity can be identified. At the highest level, or first order, the design is verified at 

a system level by ensuring the results of the different control loop interactions are as intended 

under nominal conditions and a functional representative system will be required. The functional 

representative system can be black boxes and simulators with no attempt at harness duplication 

and implementation specific nuances. At the second order, the performance around the 

individual control loops is verified under all operational conditions and thus an accurate 

representation of the implementation voltage variation and delays will be required. At this 

second level, only the single control loop requires flight-like assemblies and wires to duplicate 

the intentional implementation variances and the interacting systems can be simulated. At the 

third order, the unintentional LQWHUDFWLRQVEHWZHHQWKHV\VWHPV¶DUHYHULILHGDQGWKXVhigh fidelity 

hardware with geometrically identical configuration is required. At this lower level, the intent is 

to identify the subtle environmental induced interactions such as subsystem-to-subsystem 

thermal interactions, electromagnetic interactions and mechanical motion interactions. It is 

important to point out the first two levels are necessary to thoroughly characterize the normal 

system variations thus enabling the ability to identify the more subtle unintentional interactions. 

For Constellation, the optimal solution is a complete set of development test beds or tools that 

address the risks at various levels of the design. 

Another CxP/NESC team discussion item related to the philosophy of requiring all verification 

on the flight article or allowing verification on a high fidelity flight-like article. Clearly, the 

1$6$SKLORVRSK\RI3WHVWDV\RXIO\fO\DV\RXWHVW ́KROGVWrue in this case and it is desirable to 

complete all verification on the flight article. Completing all verification on the flight article 

mitigates the class of risks that may be unique to that individual flight article. However, there 

are some tests that require specific inputs to close the control loop or conditions that require 

inducing failures or simulated failures. For the Space Shuttle's size and complexity, system level 

verification on the flight vehicle was not practical. In these cases, it is a balance of the risk of 

disassembling the flight article to implement the unique inputs versus the risk of the unknown 

difference between the flight article and the high fidelity flight-like test unit. There are certain 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

34 of 39 

NESC Request No.: 08-00451 

attitude control system tests where it is lower risk to verify on a high fidelity flight-like article. 

Programmatic risks (e.g. schedule and cost) or flight article availability must be weighed against 

the risk of not completing all verifications on the flight article. Other risks that must be 

evaluated include usage limits for the flight article and cost, schedule and complexity of vehiclein-the-loop testing. The risks must be considered on a case by case basis. 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

35 of 39 

NESC Request No.: 08-00451 

7.0 Findings, Observations, and Recommendations 

7.1 Findings 

The following NESC team findings were identified: 

F-1. A coordinated, end-to-end, box to system level development and verification process was 

not evident for the proposed CAIL concept. 

F-2. The CAIL assumption is that the flight subsystems will be matured for the system level

verification. 

x Assumes returning to the previous level is not required. 

F-3. The Flat-Sat and SAIL approaches are two different tools designed to mitigate different 

risks. 

x Flat-Sat is designed to develop a mission concept into a flight avionics system and 

associated ground controller. 

x SAIL is designed to validate the flight avionics system. 

x The approaches are complimentary in addressing both the system development 

risks and mission verification risks. 

7.2 Observations 

The following NESC team observations were identified: 

O-1. %DVHGRQWKHWHDP¶VH[SHULHQFHUHTXLUHGUHTXHVWHGSHDNXVDJHRIDQLQWHJUDWLRQODE

typically exceeds lab availability even in a multi-shift operation. 

O-2. Both the CAIL and the EDL material did not refer to an intentional effort to provide an 

engineering team knowledge transfer to ground operations and mission operations teams. 

O-3. Coordination of the verification process is essential to avoid missing key risks and to be 

prepared for schedule deviations. 

O-4. Partitioning of integration facility into subsystePLQWHJUDWLRQODEVDOORZVIRU3FORQLQJ ́

DQG3H[SRUW ́WRH[WHUQDOGHYHORSHUV

O-5. The CAIL and the SAIL differ in the fidelity of the propulsion hardware and software 

interface systems. 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

36 of 39 

NESC Request No.: 08-00451 

7.3 Recommendation 

The following NESC team recommendation was made: 

R-1. Define, document, and manage a detailed interface between the design and development 

(EDL and other integration labs) to the verification laboratory (CAIL). [F1, F2, O3]

± Interfaces (flight software/hardware, schedule deviations, system discrepancies) 

± Typically defined and documented by CAIL PDR 

8.0 Definition of Terms 

Corrective Actions Changes to design processes, work instructions, workmanship practices, 

training, inspections, tests, procedures, specifications, drawings, tools, 

equipment, facilities, resources, or material that result in preventing, 

minimizing, or limiting the potential for recurrence of a problem. 

Development Process of functional testing, interface testing, open-loop testing and 

close-loop testing necessary to mature a concept to operation system 

Finding A conclusion based on facts established during the assessment/inspection 

by the investigating authority. 

Flight Avionics All electrical/electronic systems that will fly on the vehicle. 

System 

Ground Controller Ground electrical/electronic systems, tools, and procedures that enable the 

development and operation of the Flight Avionics System. 

Lessons Learned Knowledge or understanding gained by experience. The experience may 

be positive, as in a successful test or mission, or negative, as in a mishap 

or failure. A lesson must be significant in that it has real or assumed 

impact on operations; valid in that it is factually and technically correct; 

and applicable in that it identifies a specific design, process, or decision 

that reduces or limits the potential for failures and mishaps, or reinforces a 

positive result. 

Observation A significant factor established during this assessment that supports and 

influences the conclusions reached in the statement of Findings and 

Recommendations. 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

37 of 39 

NESC Request No.: 08-00451 

Problem The subject of the independent technical assessment/inspection. 

Recommendation An action identified by the assessment/inspection team to correct a root 

cause or deficiency identified during the investigation. The 

recommendations may be used by the responsible C/P/P/O in the 

preparation of a corrective action plan. 

Root Cause Along a chain of events leading to a mishap or close call, the first causal 

action or failure to act that could have been controlled systemically either 

by policy/practice/procedure or individual adherence to 

policy/practice/procedure. 

Verification Formal process of demonstrating a design is flight ready and is assumed to 

be interchangeable in the general sense to verification, validation and 

certification. 

9.0 List of Acronyms 

ALT Approach and Landing Test 

AMD Advanced Mirror Development 

BFCS Backup Flight Control System 

C&DH Communications and Data Handling 

CAIL CEV Avionics Integration Laboratory 

CDR Critical Design Review 

COTS Commercial-Off-The-Shelf 

CxP Constellation Program 

EDL Exploration Development Laboratory 

EOS Earth Observing System 

ESMD Exploration Systems Mission Directorate 

ESTL Electronics Systems Test Laboratory 

ET External Tank 

ETU Engineering Test Unit 

GN&C Guidance, Navigation, and Control 

GSE Government Supplied Equipment 

GSFC Goddard Space Flight Center 

I&T Integration and Test 

ITCP Integrated Test and Checkout Procedures 

JPL Jet Propulsion Laboratory 

JSC Johnson Space Center 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

38 of 39 

NESC Request No.: 08-00451 

JWST James Webb Space Telescope 

KSC Kennedy Space Center 

LM Lockheed Martin 

LPS Launch Processing System 

LRO Lunar Reconnaissance Orbiter 

MMES Marshall Mated Elements System 

NASA NASA Aeronautics and Space Administration 

NESC NASA Engineering and Safety Center 

NRB NESC Review Board 

OFT Orbital Flight Test 

OI Operational Increment 

OV Orbiter Vehicle 

PDR Preliminary Design Review 

RMS Remote Manipulator System 

RTLS Return To Launch Site 

SAIL Shuttle Avionics Integration Laboratory 

SDO Solar Dynamics Observatory 

SME Subject Matter Expert 

SRB Solid Rocket Booster 

SSME Space Shuttle Main Engine 

SSP Space Shuttle Program 

T&V Test & Verification 

TIM Technical Interchange Meeting 

TRMM Tropical Rainfall Measuring Mission 

VEST Vehicle Electrical System Test 

WMAP Wilkinson Microwave Anisotropy Probe 

10.0 References 

1. NESC Constellation Avionics Integrated Lab (CAIL) Independent Analysis Outbrief to 

NASA Headquarters, presented by Mitch Davis on March 17, 2008. 

2. Project Orion Spacecraft Test & Verification Facility System Requirements Document 

(CEV-T-082100), CAIL, July 10, 2007. 

3. CEV Avionics Integration Lab (CAIL) Overview to NESC CAIL Independent Analysis, 

Mr. Aaron Van Baalen, Orion, Avionics & Software Test and Verification Lead, 

February 27, 2007. 

4. Shuttle Avionics Integration Laboratory (SAIL), presented by Bill Ritz on March 5, 

2008. 
NASA Engineering and Safety Center 

Report 

Document #: 

RP-08-31 

Version: 

1.0 

Title: 

Crew Exploration Vehicle (CEV) Avionics Integration 

Laboratory (CAIL) Independent Analysis 

Page #: 

39 of 39 

NESC Request No.: 08-00451 

5. Statement of Work, CEV Avionics Integration Lab (CAIL), Attachment J-1, Section 6.4, 

Contract Number: NNJ06TA25C. 

6. Exploration Development Lab (EDL) Overview Presentation, Lockheed Martin-owned 

and operated for Constellation Program and internal IR&D of Exploration Capabilities. 
REPORT DOCUMENTATION PAGE Form Approved 

OMB No. 0704-0188 

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing

data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or 

any other aspect of this collection of information, including suggestions for reducing this burden, to Department of Defense, Washington Headquarters Services, Directorate

for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that

notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently

valid OMB control number. 

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

1. REPORT DATE (DD-MM-YYYY) 2. REPORT TYPE 3. DATES COVERED (From - To) 

4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER 

5b. GRANT NUMBER 

5c. PROGRAM ELEMENT NUMBER 

6. AUTHOR(S) 5d. PROJECT NUMBER 

5e. TASK NUMBER 

5f. WORK UNIT NUMBER 

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION

REPORT NUMBER 

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSORING/MONITOR'S ACRONYM(S)

11. SPONSORING/MONITORING

REPORT NUMBER 

12. DISTRIBUTION/AVAILABILITY STATEMENT

13. SUPPLEMENTARY NOTES 

14. ABSTRACT

15. SUBJECT TERMS 

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 

ABSTRACT

18. NUMBER 

OF

PAGES

19a. NAME OF RESPONSIBLE PERSON

a. REPORT b. ABSTRACT c. THIS PAGE

19b. TELEPHONE NUMBER (Include area code) 

Standard Form 298 (Rev. 8-98)

Prescribed by ANSI Std. Z39-18 

01-02-2009 Technical Memorandum May 2008- May 2008

Crew Exploration Vehicle (CEV) Avionics Integration Laboratory (CAIL) 

Independent Analysis

510505.01.07.01.06

Davis, Mitchell, L.; Aguilar, Michael, L.; Mora, Victor, D.; Regenie, 

Victoria, A.; Ritz, William, F.

NASA Engineering and Safety Center

Langley Research Center

Hampton, VA 23681-2199

L-19577 NESC-RP-08-31/08-00451

National Aeronautics and Space Administration

Washington, DC 20546-0001

NASA

NASA/TM-2009-215569

Unclassified - Unlimited

Subject Category 06 Avionics And Aircraft Instrumentation Availability: NASA CASI (443) 757-5802

Two approaches were compared to the Crew Exploration Vehicle (CEV) Avionics Integration Laboratory (CAIL) approach: the 

Flat-Sat and Shuttle Avionics Integration Laboratory (SAIL). The Flat-Sat and CAIL/SAIL approaches are two different tools 

designed to mitigate different risks. Flat-Sat approach is designed to develop a mission concept into a flight avionics system

and associated ground controller. The SAIL approach is designed to aid in the flight readiness verification of the flight avionics 

system. The approaches are complimentary in addressing both the system development risks and mission verification risks.

The following NESC team findings were identified: The CAIL assumption is that the flight subsystems will be matured for the 

system level verification; The Flat-Sat and SAIL approaches are two different tools designed to mitigate different risks. The 

following NESC team recommendation was provided: Define, document, and manage a detailed interface between the design and

development (EDL and other integration labs) to the verification laboratory (CAIL).

NESC, CEV, CAIL, SAIL, Flat-Sat, EDL, VEST, TRMM, WMAP, LRO

UU UU UU UU 44

STI Help Desk (email: help@sti.nasa.gov)

(443) 757-5802