Chair, Exploration Systems Advisory Committee
Chair, Financial Audit Committee
Appendix C

NASA Advisory Officials (cont’d)

Chair, Minority Business Resource Advisory Committee
Chair, NASA Operations Council

Chair, Planetary Protection Advisory Committee

Chair, Return to Flight Task Group

Chair, Space Flight Advisory Committee

Chair, Space Science Advisory Committee

Chair, NASA Strategic Planning Council

NASA Centers

Director, Ames Research Center
Director, Dryden Flight Research Center
Director, John H. Glenn Research Center at Lewis Field
Director, Goddard Space Flight Center
Director, Jet Propulsion Laboratory
Director, Lyndon B. Johnson Space Center
Director, John F. Kennedy Space Center

Chief Counsel, John F. Kennedy Space Center
Acting Director, Langley Research Center
Director, George C. Marshall Space Flight Center
Director, NASA Engineering and Safety Center
Director, John C. Stennis Space Center

Non-NASA Organizations and Individuals

Office of Management and Budget
Deputy Associate Director, Energy and Science Division
Branch Chief, Science and Space Programs Branch
Government Accountability Office

Director, Defense, State, and NASA Financial Management, Office of Financial
Management and Assurance

Director, NASA Issues, Office of Acquisition and Sourcing Management

16


capture and implement the concept of the third set of eyes as well as eliminate any further
repetitive findings that the SMA function is not sufficiently independent.

Recommendation for Corrective Action

In lieu of implementing the CAIB recommendation, the Chief SMA Officer should

demonstrate that there is a healthy, sustainable, independent oversight function at the
Centers.

Management’s Response

Management concurred with our recommendation and has determined that actions
already taken or underway by SMA will provide strong oversight for Center SMA
functions. The Chief SMA Officer stated that he closely monitors the effective pursuit of
all SMA functions across the Agency and ensures that oversight of program and
institutional operations is independent and thorough. In addition, NASA policy dictates
that everyone in NASA is responsible for the safety of their individual actions, and that
leadership is to be held accountable for the safety of operations entrusted to their control
to the extent of their authority and capability. Lastly, SMA plans to expand the scope of
its audits to include a review of the safety reporting process and SMA is considering
increasing the time spent at NASA Centers by Headquarters SMA personnel. See
Appendix B for the complete text of management's response.

Evaluation of Management’s Response

Management’s actions are responsive to the recommendation, and we consider the
recommendation closed for reporting purposes. However, we believe the SMA functions
will require close monitoring, particularly as NASA heads into the retirement phase of
the Shuttle Program. We will continue to monitor NASA’s implementation of SMA,
focusing on the concerns identified in this report.

10


RISKS ASSOCIATED WITH NASA’S PLAN FOR
TECHNICAL AUTHORITY AND SAFETY
AND MISSION ASSURANCE

Objectives

The overall audit objective was to assess the adequacy of NASA’s plans to address the
organizational causes of the Columbia accident and to ensure potential weaknesses are
appropriately addressed as part of the Agency’s efforts to return the Space Shuttle to
flight. Specific objectives were to evaluate NASA’s plans to (1) implement an
independent Technical Engineering Authority (Recommendation 7.5-1); (2) establish
direct-line authority for Space Shuttle safety within the Headquarters Office of Safety and
Mission Assurance (SMA) (Recommendation 7.5-2); and (3) reorganize the Space
Shuttle Integration Office (Recommendation 7.5-3.)

Background

After a 7-month independent investigation into the February 1, 2003, Space Shuttle
Columbia accident, the Columbia Accident Investigation Board (CAIB) found that the
management practices overseeing the Space Shuttle Program were as much a cause of the
accident as the foam striking the left wing leading edge. Part Two of the CAIB report,
“Why the Accident Occurred,” examined NASA’s organizational, historical, and cultural
factors and how those factors contributed to the accident. The Board concluded that
NASA’s organization did not provide effective checks and balances, did not have an
independent safety program, and had not demonstrated the characteristics of a learning
organization. After comparing the Columbia and Challenger accidents, the CAIB was
not confident that without substantial management changes, corrective actions to the
recommendations alone would improve safety for the Space Shuttle Program.

The CAIB expected to find SMA deeply engaged in the decision processes of Space
Shuttle Program management. Instead, the CAIB found a lack of SMA involvement,
which had prevented safety personnel from providing needed checks and balances. The
Space Shuttle Program had also fallen into a pattern of accepting problems, such as the
external tank shedding foam debris, without sufficient engineering analysis.

To reinvigorate the engineering discipline, the CAIB recommended that NASA establish
a Technical Authority responsible for technical requirements and for building a
disciplined systematic approach to identify, analyze, and control hazards. The CAIB
also recommended that NASA’s Headquarters Office of SMA have direct-line authority
over the entire Space Shuttle safety program. According to the CAIB, both the
independent Technical Authority and the SMA should be independently resourced. The
CAIB additionally recommended that the Space Shuttle Program reorganize and

strengthen the Space Shuttle Integration Office so that the integration function would
encompass all elements of the Space Shuttle Program.


Appendix C

Congressional Committees and Subcommittees, Chairman and Ranking
Minority Member

Senate Committee on Appropriations
Senate Subcommittee on Commerce, Justice, and Science
Senate Committee on Commerce, Science, and Transportation
Senate Subcommittee on Science and Space
Senate Committee on Homeland Security and Governmental Affairs
House Committee on Appropriations
House Subcommittee on Science, State, Justice, and Commerce
House Committee on Government Reform

House Subcommittee on Government Management, Finance, and Accountability
House Committee on Science

House Subcommittee on Space and Aeronautics

17


Final Report
Reference.

the scope of OSMA audits to include all forms of safety communications, not just those in the
SMA world, and 2) the possibility for increasing the ratio of “field time” to “Washington time”
for our Headquarters-based Center and program point of contacts.

Also, the following is some additional feedback from the Office of the Chief Engineer
regarding some errors that they would like to see corrected in the report.

“The statement that the NPD and NPR were withdrawn is incorrect and should be Revised
deleted. The Administrator's letter of November 23, 2004, directing implementation of

Independent Technical Authority (ITA) using these draft documents is still being
followed. Office of Chief Engineer believes this is an important point in that ITA is
still following approved procedures. Also the statement with regards to Technical
Warrant Holders (TWAS) recusing themselves from decisions on work that they had a
prior involvement is not a requirement of ITA and should be deleted. TWAs when
making their ITA decisions are independent which is defined as 1) not organizationally
reporting to the PM, 2) not being funded by the PM and 3) receiving their authority
from the Chief Engineer by a warrant.”

, Please contact the OSMA Audit Liaison Representative, Kelly Kabiri, if you have any

Ws Any questions on ITA, please notify Mr. Walter Hussey on 202-358-0591.

Deleted

ryan O/ or
ce:
Office of Chief Engineer/Mr. Geveden

/Mr. Hussey
Office of Institutions and Management/Mr, Roberts
Kennedy Space Center/Ms. Massey

14


B. Safety and Mission Assurance Office

While the Agency has taken numerous actions since the Columbia accident, the Chief
SMA Officer has diverged from the explicit intent of the CAIB recommendation and
stated that he does not believe direct-line funding or reporting for all SMA personnel is
necessary. As a result, the implementation steps outlined in “NASA Plan for
Implementing Safe and Reliable Operations” (the Plan) do not provide a completely

independent funding and reporting path solely accountable to the Chief SMA Officer at
Headquarters.

Previous Concerns over SMA Independence

The CAIB stated in Recommendation 7.5-2 that the NASA Headquarters Office of SMA
should have direct-line authority over the entire Space Shuttle Program safety
organization and it should be independently resourced. Even before the CAIB, numerous
commissions or studies had comments or recommendations concerning the funding and
reporting independence of the SMA.

In June 1986, the President’s Commission on the Space Shuttle Challenger Accident (the
Rogers Commission) recommended that NASA establish an Office of SMA at NASA
Headquarters and that it should be headed by an Associate Administrator who would
report directly to the NASA Administrator with direct authority for SMA functions
throughout the Agency. While NASA did establish an Associate Administrator for SMA,

neither direct-line reporting nor independent funding for Center SMA functions was ever
established.

In 1990, the General Accounting Office, now the Government Accountability Office
(GAO), recommended that NASA independently fund safety activities through the
Headquarters Office of SMA. NASA chose not to implement the GAO recommendation
because such implementation would (1) lessen the program’s SMA responsibilities,

(2) do away with Center SMA organizations having to defend and support their

requirements to the program, and (3) require an increase in Headquarters SMA
administrative staff.

In 2000, the Shuttle Independent Assessment Team (SIAT) recommended that the NASA
SMA be restored to its previous role of independent oversight rather than simply being a
“safety auditor.” The SIAT stated that NASA SMA was moving away from an oversight
role to one of insight, in which direct involvement was being replaced by periodic
surveillance and audits. In addition, the SIAT was concerned that the movement would
result in (1) fewer independent reporting paths as the NASA presence was withdrawn
from engineering and operations teams, and (2) a reduced ability of the SMA function to
determine and enforce compliance with safety and reporting requirements.

During congressional testimony on May 14, 2003, the CAIB chairman, referring to the
SMA Office before the Columbia accident, stated, “We find the safety organization is on
paper, perfect. But when you bore down a little bit deeper, you don’t find anything


Findings

A. Technical Authority and Warrant System

In response to CAIB Recommendation 7.5-2, NASA issued “NASA Plan for
Implementing Safe and Reliable Operations” (the Plan) as well as draft policy and
procedures for an engineering authority responsible for establishing, monitoring, and
approving technical requirements across the Agency. At the same time, the NASA
Administrator designated the Chief Engineer as NASA’s Technical Authority and
permitted the Chief Engineer to delegate that authority to selected individuals through a
system of warrants. Those who were granted such authority were designated as
Technical Warrant Holders (TWHs). The TWHs were envisioned as the Agency’s
“technical conscience.” The Plan, which was revised on March 7, 2005, outlines the
steps necessary for restoring the engineering discipline and visibility throughout the
Agency. To make the warrant system effective and ensure that the integrity and

independence of the TWH are maintained, procedures and processes will need to be
established and monitored.

CAIB Recommendation

In its August 2003 final report on the Space Shuttle Columbia accident, the CAIB
identified that NASA did not have clear, clean lines of technical authority, responsibility,
and accountability. According to CAIB Recommendation 7.5-1, NASA should:

Establish an independent Technical Engineering Authority that is responsible for
technical requirements and all waivers to them, and will build a disciplined,
systematic approach to identifying, analyzing and controlling hazards
throughout the life cycle of the Shuttle System . . . The Technical Engineering
Authority should be funded directly from NASA Headquarters, and should have
no connection to or responsibility for schedule or program cost.

NASA’s Response to the CAIB

To address Recommendation 7.5-1, NASA issued draft NASA Policy Directive

(NPD) 1240.4, “NASA Technical Authority,” dated November 23, 2004. That draft
NPD! gives the Chief Engineer responsibility and accountability to establish, approve,
and maintain technical requirements, processes, and policy to conduct and oversee high-
tisk technical work. The Chief Engineer delegates that authority to competent,
experienced individuals (known as TWHs) through a system of warrants.

Also issued on November 23, 2004, Draft NASA Procedural Requirements

(NPR) 1240.1, “NASA Technical Warrant System,” discusses and identifies the types of
TWHs: Systems and Discipline. Systems warrant holders provide checks and balances
for mission-related programs and projects by maintaining technical standards, processes,

' NASA is revising the NPD because of the Administrator’s June 23, 2005, request for modifications.
? NASA is revising the NPR because of the Administrator’s June 23, 2005, request for modifications.


Space Shuttle Engineering and Integration Office

As a result of key improvements, the Space Shuttle Program Manager now has an office
with continuous and consistent responsibility to integrate all anomalies, corrective
actions, and risks associated with the Space Shuttle. On December 3, 2003, the Space
Shuttle Program Manager issued the charter for the Space Shuttle Systems Engineering
and Integration Office (SEIO) to explicitly communicate the roles and responsibilities of
that office across the Space Shuttle Program. The SEIO is responsible for the
performance and safety of the Space Shuttle vehicle ground and flight activities when
multiple project elements are involved.

Key improvements of the SEIO charter involved placing the SEIO at a higher
organizational level within the Space Shuttle Program Office structure, increasing the
office manager’s grade level to a level commensurate with the increased responsibility,
and increasing the number of program and contractor employees supporting the office.
For example, the SEIO Manager leads a Systems Integration Control Board and each
program element is a mandatory member of that Board. Additionally, the Propulsion
Systems Engineering and Integration Office at Marshall Space Flight Center and the
Systems Integration Office at Kennedy Space Center receive technical direction and
coordination from and report directly to the SEIO for all integration-related activities.


Appendix C. Report Distribution

National Aeronautics and Space Administration (NASA) Headquarters

Administrator
Deputy Administrator
Administrator Staff Offices
Chief Safety and Mission Assurance Officer
Chief Education Officer
Associate Deputy Administrator for Systems Integration
Chief Scientist
Chief Health and Medical Officer
Director of Advanced Planning
Mission Offices

Exploration Systems Associate Administrator
Space Operations Associate Administrator
Science Associate Administrator

Aeronautics Research Associate Administrator

Mission Support Offices

Chief Financial Officer

Chief Information Officer

Chief Engineer

Associate Administrator for Institutions and Management
Assistant Administrator for Infrastructure and Administration

Director, Management Systems Division

Assistant Administrator for Institutional Planning and Investment
Assistant Administrator for Procurement
Assistant Administrator for Security and Program Protection
Assistant Administrator for Human Capital Management
Assistant Administrator for Diversity and Equal Opportunity

Assistant Administrator for Small and Disadvantaged Business Utilization
General Counsel

Acting Chief of Strategic Communications

NASA Advisory Officials

Chair, NASA Advisory Council

Chair, Aeronautics Research Advisory Committee

Chair, Aerospace Medicine and Occupational Health Advisory Committee
Chair, Aerospace Safety Advisory Panel

Chair, Biological and Physical Research Advisory Committee

Chair, Earth System Science and Applications Advisory Committee
Chair, Education Advisory Committee

15


processes and procedures for monitoring and ensuring that the integrity, competence, and
independence of the individual TWHs are maintained, For example, the Chief Engineer
has not established processes or procedures to:

1. periodically assess the performance of the TWH to ensure integrity and
independence are maintained.

2. ensure consistency and adequacy in the review and application of technical
standards across the Agency.

3. ensure the pressures of being asked to individually challenge a major program do
not discourage subject matter experts from accepting the TWH role. :

4. address succession planning because the warrant system is based on an individual
rather than an organizational concept.

We understand that the Independent Program Assessment Office (PAO) is planning to
review the implementation of the Technical Authority. That review should include the
concems we have listed above. We believe such a review should help the Chief Engineer
identify the processes needed to ensure the strength, effectiveness, and long-term
sustainability of the Technical Authority.

Roles and Responsibilities for TWHs and SMA

The Plan also lacks details concerning the roles and responsibilities of the TWHs and
SMA personnel. Further discussion is needed on which standards and/or requirements
TWHs will own, the definition of ownership, and how engineering and SMA standards
will be integrated to ensure that both technical and safety concerns are appropriately
addressed. Roles and responsibilities for Technical Authority and SMA must be clearly
stated and agreed upon across the Agency.

Conclusion

We believe the organizational structure planned for the technical authority poses some
risks. On the one hand, TWHs—empowered to be NASA’s technical conscience—must
be devoid of program pressures such as cost, schedule, and mission accomplishment.
However, to say that TWHs will be fully immune to program pressures is not realistic.
Procedures and processes will need to be established and monitored to make the warrant
system effective and ensure that the integrity as well as the independence of each TWH is
maintained. Because the implementation of NASA’s Plan continues to be under
development, we are not making any recommendations. We will continue to monitor
NASA’s implementation of the technical authority, focusing on the concerns identified in
this report.

3NASA established the Program Analysis and Evaluation Office at Headquarters in May 2005. The IPAO
was realigned under that office, thus removing it from the Office of the Chief Engineer.


AUDIT REPORT 1G-05-024

RISKS ASSOCIATED WITH NASA’S PLAN FOR
TECHNICAL AUTHORITY AND SAFETY
AND MISSION ASSURANCE

August 19, 2005

OFFICE Libel GENERAL
Released by: Msctx 29
National Aeronautics and Evelyn R. Klemstine/

Space Administration Assistant Inspector General for Auditing



are marginal and powerless in organizations may have useful information or opinions that
they don’t express. Even when these people are encouraged to speak, they find it
intimidating to contradict a leader’s strategy or a group consensus.” Therefore, we feel it

is critical to establish a system whereby safety issues are fully addressed and, if
necessary, forced out into the open.

Third Set of Eyes

If NASA is going to diverge from the strict intent of the CAIB, the Chief SMA Officer
should demonstrate that there is a healthy, sustainable, independent oversight function at
the Centers that will maintain a concept informally known as the “third set of eyes.”

Contractor SMA personnel overseeing the efforts of the work force provide the first set of
eyes. Center and program SMA personnel provide the second set of eyes, overseeing the
contractor and performing their own independent analysis when needed or desired. The
third set of eyes is generally provided by experienced safety professionals or engineers
who are able to independently observe key decision-making processes. They have the
power and authority to go where they want to, when they want to. Those individuals
would be expected to follow their intellectual and safety curiosity, look for undetected or
overlooked issues, listen for dissenting opinions, push for ”root cause” thinking, and ask
probing questions—all in an effort to provide assurance that decision makers have
adequately considered all relevant facts. Those personnel should have no funding or

administrative ties to the Center where they reside or to the Programs based at that
Center.

Although the concept of a third set of eyes is not unique to safety, its importance is
paramount because the results of any undetected mistake brought about by human error
can be catastrophic. Human errors can allow problems to go undetected, unreported, or
reported without sufficient accuracy and emphasis. The third set of eyes should be
considered an integral part of high-risk programs such as the Space Shuttle because an
active third set of eyes decreases the risk of overlooking human error and, ultimately,
undesirable consequences. Both the Rogers Commission and CAIB reported that SMA
personnel were either absent or silent during key decisions prior to both Space Shuttle
accidents. Additionally, the SIAT proposed the following in its report of 2000:

Recent reductions in duties and personnel have resulted in a Safety and Mission
Assurance process that appears largely absent from day-to-day activities of Shuttle
operations and maintenance. Further, the deficiencies observed in risk assessment, risk
communication and problem trending, and the increasing occurrences of “stumble-ons,”
“diving catches” and “escapes” indicates degradation in the Safety & Mission Assurance
system that remains. The SIAT strongly believes in the necessity of an experienced,
well-staffed NASA quality assurance function (“second set of eyes”) and an independent,
empowered NASA Safety & Mission Assurance function (“third set of eyes.”)

The Plan states that Headquarters SMA functional oversight of the Centers will be
improved through enhanced policy and audits. However, internal and external reviews
have shown that relying on audits and surveillance to verify SMA will probably not be
sufficient. No substitute exists for a real-time, completely independent presence not only


there.” He went on to clarify his remarks by saying that key SMA advisors, those who
can bring up alternative points of view, were not adequately or independently funded.
While the 2003 findings and recommendations of the CAIB are not an exact duplication
of previous commissioned studies, a continuous theme has been woven throughout the
years when discussing a lack of SMA independence for funding and reporting.

Changes in the SMA Funding Process

In response to the CAIB report, NASA established at each Center in October 2004
directed service pools that would allocate SMA resources to programs and projects. The
directed service pools are not funded with general and administrative (G&A) funds;
rather, they are funded by NASA programs. We are concerned that funding for the
directed service pools could become more constrained as the Space Shuttle Program
enters its retirement phase and the overall program budget decreases. Retirement
concerns, such as loss of experienced personnel and program vendors as well as lowered
morale, could intensify the need for adequately funded SMA activities at the Centers.
History and lessons learned have demonstrated that a string of successful missions or the
auspices of funding constraints can lead to a reduction in SMA activity.

While the directed service pool is an improvement over the previous SMA funding
mechanisms, SMA officials must still negotiate their budgets with Center Directors and
the program/project management they are responsible for assessing. It is unknown how
the recent realignment of the Center Directors reporting directly to the Administrator’s
office rather than to an MDAA will affect the SMA funding process. Should any changes
to the SMA funding process be made, it is imperative that SMA funding remain
independent from NASA programs.

Changes in SMA Direct-Line Reporting

According to the Plan, NASA’s Chief SMA Officer will have more influence and formal
input into the appraisal of the Center and program SMA senior managers. Although
authority over those SMA managers has increased, the Chief SMA Officer does not plan
to establish a Center SMA presence solely accountable to the Headquarters Office of
SMA. Specifically, all SMA personnel residing at the Centers continue to report to
Center Directors. Furthermore, Center Directors and program managers continue to have
“supervisory status” over Center SMA Directors and Program SMA Managers
respectively. In that regard, the Agency elected to diverge from the CAIB-recommended

position that the Chief SMA Officer have direct-line authority over Shuttle Program
safety.

Even after all the post-CAIB changes detailed in the Plan, SMA personnel are still
required to bypass their formal chain of command to report significant concerns directly
to the Chief SMA Officer. If program or Center personnel are forced to go outside of
their formal chain of command, particularly during key, real-time decision processes,
they could be placed in a compromised position of disagreeing with officials who
evaluate their job performance. As stated in section 8.6 of the CAIB report: “People who


policies, and variances for assigned systems, such as the Space Shuttle, the International
Space Station, or aviation systems. In January 2005, the Chief Engineer advised the
Aerospace Safety Advisory Panel that the role of systems TWHs was a full-time position.
In contrast and in a broader sense, discipline warrant holders maintain technical standards
for specific disciplines such as structures, power, or software engineering. The discipline
warrant may or may not require a full-time TWH.

Risks in Reporting Structure

According to the draft NPR, the TWHs report directly to the Chief Engineer for daily
tasks, although their organizational reporting structure and performance evaluation still
aligns them to a Center Director. For example, TWHs for both the Space Shuttle
Systems warrant and the International Space Station Systems warrant permanently reside
in the Engineering Directorate at the Johnson Space Center (Johnson). The Engineering
Directorate ultimately reports to Johnson’s Center Director. Although Draft NPR 1240.1
states that Center Directors will ensure that TWHs do not have a supervisory reporting
chain to the managers of various programs or projects, an unintended connection to the
program officials responsible for schedule or program cost could occur over time. NASA
has also recognized the potential for conflict between programmatic pressures and

Technical Authority decisions and addressed that concern in the actual warrant, which
states:

This warrant does not circumvent your responsibilities to your operational chain of
authority. However, it does provide you with the authority and accountability to directly
access the Chief Engineer without fear of administrative repercussions in issues affecting
safe and reliable operations.

On April 18, 2005, the NASA Administrator realigned the reporting structure to require
that Center Directors report directly to the Administrator’s office rather than to a Mission
Directorate Associate Administrator (MDAA). Ina June 23, 2005, memorandum, the
NASA Administrator directed a refinement of the policy and approach for the conduct of
Technical Authority at NASA, stating that independent Technical Authority
implementation “will be re-delegated to the Centers by the Chief Engineer [and that]
Center Directors will select systems warrant holders and discipline leads who will have
decision authority on program/project technical matters with the exception of variances to
NASA standards.” The memorandum further stated that variances will be the
responsibility of discipline warrant holders, who will still report to the Chief Engineer via
the Deputy Chief Engineer. We believe these changes will help alleviate our concerns
that programmatic pressure can potentially influence TWHs.

Process to Ensure Sustainability

One of the Plan’s five key principles for the Technical Authority is that authority for
technical requirements must reside in an individual and not an organization.
Consequently, NASA is relying solely on the integrity of the TWH to champion technical
decisions that could adversely affect a major program. While that concept makes clear
that the TWH is accountable, neither the Plan nor the draft NPR 1240.1 addresses


Additional Copies

Visit www.hq.nasa.gov/office/oig/hg/issuedaudits.html or contact the Assistant Inspector
General for Auditing at (202) 358-2572 to obtain additional copies of this report.

Comments on This Report

In order to help us improve the quality of our products, if you wish to comment on the
quality or usefulness of this report, please send your comments to Ms. Jackie White,

Director, Quality Control Division, at Jacqueline. White@nasa.gov or call
(202) 358-0203.

Suggestions for Future Audits

To suggest ideas for or to request future audits, contact the Assistant Inspector General
for Audits. Ideas and requests can also be mailed to:

Assistant Inspector General for Audits
NASA Headquarters
Washington, DC 20546-0001

NASA Hotline

To report fraud, waste, abuse, or mismanagement, contact the NASA OIG Hotline at (800)
424-9183, (800) 535-8134 (TDD), or at www.hg.nasa.gov/office/oig/hq/hotline.html#form;
or write to the NASA Inspector General, P.O. Box 23089, L’Enfant Plaza Station,
Washington, DC 20026, The identity of each writer and caller can be kept confidential,
upon request, to the extent permitted by law.

Major Contributors to the Report
Sandy A. Massey, Deputy Director, Space Operations and Exploration Directorate

Rebecca L. Sharek, Project Manager, Space Operations and Exploration Directorate

18


Table of Contents

Executive Summary

Objectives

Background

Findings
A. Technical Authority and Warrant System
B. Safety and Mission Assurance Office

Appendixes

A. Scope and Methodology
B. Management’s Response

C. Report Distribution

Acronyms Used in the Report

CAIB Columbia Accident Investigation Board
G&A General and Administrative

GAO Government Accountability Office

IPAO Independent Program Assessment Office
MDAA Mission Directorate Associate Administrator
NESC NASA Engineering and Safety Center

NPD NASA Policy Directive

NPR NASA Procedural Requirements

SEIO Systems Engineering and Integration Office
SIAT Shuttle Independent Assessment Team
SMA Safety and Mission Assurance

TWH Technical Warrant Holder

11

13

15


Appendix A. Scope and Methodology

Scope and Methodology
We performed field work at NASA Headquarters, Kennedy and Johnson Space Centers,
and Marshall Space Flight Center. We reviewed various draft plans leading to the
March 7, 2005, “NASA Plan for Implementing Safe and Reliable Operations.” We also
reviewed the following NASA guidance documents:

¢ NPD 1000.3B, “The NASA Organization,” dated July 30, 2004;

e Draft NPD 1240.4, “NASA Technical Authority,” dated November 23, 2004;

¢ Draft NPR 1240.1, “NASA Technical Warrant System,” dated November 23,
2004; and

¢ NASA Interim Directive, “Responsibilities of Program/Project Managers in the
NASA Technical Warrant System,” dated December 21, 2004.

We reviewed internal and external reports related to NASA’s organizational structure,
SMA reporting and funding concerns, and independent verification of Agency programs
and processes. The specific reports reviewed include:

¢ “Report of Roles, Responsibilities, and Structures Team,” (Clarity Team),
June 24, 2004

¢ “Report of the President’s Commission on Implementation of United States Space
Exploration Policy,” (Aldridge Report), June 2004

¢ “Columbia Accident Investigation Board Report,” August 2003

¢ “One NASA Initiative Report,” March 2003

¢ “NASA Space Shuttle Independent Assessment Team Report,” March 7, 2000

¢ GAO Report NSIAD-90-187, “Space Program Safety: Funding for NASA’s
Safety Organizations Should Be Centralized,” August 1990

¢ “Report of the Presidential Commission on the Space Shuttle Challenger Accident
(Rogers Commission),” June 1986

We interviewed engineering, SMA, and Space Shuttle Program representatives from

NASA Headquarters, Kennedy Space Center, Johnson Space Center, and Marshall Space
Flight Center. We attended Aerospace Safety Advisory Panel sessions.

11


Appendix B. Management’s Response

Rony te Att of

National Aeronautics and
‘Space Administration

Headquarters
Washington, DC 20546-0001

July 15, 2005

Offfice of Safety and Mission ms BO

TO: Offic€ of Inspector General
FROM: Chief, Safety and Mission Assurance

SUBJECT: Comments from Draft Report, “Review of Selected NASA Organizational
Issues Related to the Space Shuttle Columbia Accident”

The Office of Safety and Mission Assurance (OSMA) provided comment to your office on
May 4, 2005, regarding the discussion draft audit report, A-04-004-00, “Risks Associated with
NASA’s Plan for Technical Authority and Safety and Mission Assurance.” From the later
June 28, 2005, draft report, one recommendation for corrective action was made to the Chief,
Safety and Mission Assurance (SMA) Office as follows: The NASA Chief SMA Officer

should demonstrate that there is a healthy, sustainable, independent oversight function at
the Centers.

Day-to-day accountability for SMA activities at a Center belongs to the Center Director in
accordance with NASA’s philosophy for managing mission support functions and in line with
reasonable decentralized, practical management procedures, accepted widely in today’s
organizations both inside and outside of government. The Chief SMA Officer closely monitors
the effective pursuit of all SMA functions across the Agency and ensures oversight of program
and institutional operations is independent and thorough. NASA policy dictates that everyone
in NASA is responsible for the safety of their individual actions, and that line and program
leadership is to be held accountable for the safety of operations entrusted to their control to the
extent of their authority and capability. One cannot be held accountable if not assigned
responsibility,

The new Administrator has stated his intent, for organizing the Agency so that each Center
Director will report directly to the Administrator's organizational box. This will provide an
even greater degree of independence of authority between the program and the Center SMA
organizations.

We appreciate the efforts of your audit and its intent as well as your belief that this will assist in
the improvement of the Agency's safety posture. We concur with the recommendation and
have determined that actions already taken or underway by the OSMA demonstrate a will for
providing strong oversight of Center SMA functions. We will adhere to the suggestions made
earlier by our response to the discussion draft. Actions to be taken include: 1) the expansion of

13



makes it clear that Technical Warrant Holders (TWHs) are ultimately accountable for
technical requirements, NASA’s Plan does not include processes and procedures to
monitor the warrant system and ensure that the individual integrity, competence, and
independence of TWHs are maintained. Additionally, further discussion is needed over
which standards and/or requirements TWHs will own, the definition of ownership, and
how engineering and SMA standards will be integrated to ensure that both technical and
safety concerns are appropriately addressed.

We understand that the Independent Program Assessment Office is planning to review
the implementation of the Technical Authority. We believe a review, which addresses
the concerns we have described in this report, should help the Chief Engineer identify the
processes needed to maintain the strength, effectiveness, and long-term sustainability of
the Technical Authority. In addition, as of August 2005, the Technical Authority concept
was being modified. Therefore, we are not making any recommendations. However, we
will continue to monitor NASA’s revision and subsequent implementation of the
Technical Authority, focusing on the concerns described in this report. (Finding A)

NASA’s Plan describes changes to strengthen the independent funding and reporting
paths for SMA functions across the Agency. However, NASA diverged from the explicit
intent of the CAIB recommendation by not implementing direct-line funding or reporting
for Shuttle Program SMA personnel. NASA must now demonstrate that its Plan is
thorough enough to capture and implement the intent of the CAIB recommendation as
well as ensure that the SMA function is sufficiently independent. We will continue to
monitor NASA’s modifications of the SMA function, focusing on the concerns described
in this report. (Finding B)

NASA has taken steps that address CAIB concerns related to the Space Shuttle
Integration Office, including the transformation of that office into the Systems
Engineering and Integration Office, which is solely responsible for the systems
engineering and integration of flight performance of all Space Shuttle elements. Key
improvements included placing the office at a higher organizational level within the
Space Shuttle Program Office structure and increasing the office manager’s grade level to
a level commensurate with the increased responsibility.

Recommendation for Corrective Action. Because NASA has chosen to diverge from
the CAIB recommendation, we recommend that the Chief SMA Officer demonstrate that
there is a healthy, sustainable, independent oversight function at the Centers.

Management Comments. Management concurred and stated that actions already taken
or planned will provide strong oversight for Center SMA functions. The Chief SMA
Officer stated that he closely monitors the effectiveness of SMA functions and that
NASA policy dictates that all employees and their managers share the responsibility for
safety. In addition, SMA plans to review safety reporting processes and is considering
increasing the time spent at NASA Centers by the Headquarters SMA personnel. See
Appendix B for the complete text of management’s response.

ii


1G-05-024 August 19, 2005
A-04-004-00

RISKS ASSOCIATED WITH NASA’S PLAN FOR
TECHNICAL AUTHORITY AND SAFETY
AND MISSION ASSURANCE

Executive Summary

We conducted an audit to review NASA’s plans for implementing the recommendations
the Columbia Accident Investigation Board (CAIB) made related to organizational causes
of the accident. The overall audit objective was to assess NASA’s plans to address the
organizational causes of the accident and ensure potential weaknesses are appropriately
addressed. Specific audit objectives were to evaluate NASA’s plans to (1) implement an
independent Technical Engineering Authority (Recommendation 7.5-1); (2) establish
direct-line authority for Space Shuttle safety within the Headquarters Office of Safety and
Mission Assurance (SMA) (Recommendation 7.5-2); and (3) reorganize the Space
Shuttle Integration Office (Recommendation 7.5-3).

Background. Following the February 1, 2003, loss of the Space Shuttle Columbia, the
NASA Administrator appointed the CAIB to investigate the cause of the accident. In its
August 2003 final report, the CAIB concluded that NASA’s culture and organization had
as much to do with the accident as the physical causes. The CAIB found that NASA’s
system safety engineering and management were separate from mainstream engineering,
hidden in other safety disciplines at NASA Headquarters, and not vigorous enough to
have an impact. In addition, SMA’s ability to oversee operations and communicate
potential problems was limited because oversight and communications were primarily
dependent upon the Space Shuttle Program for funding. In addition, the Space Shuttle

Integration Office was not truly integrating all elements of the entire Space Shuttle
Program.

Prior to returning the Space Shuttle to flight, CAIB Recommendation 9.1-1 required that
NASA prepare a detailed plan for defining, establishing, transitioning, and implementing
Recommendations 7.5-1, 7.5-2, and 7.5-3. Recommendation 9.1-1 was a “Return to
Flight” constraint and was extensively reviewed by the Return to Flight Task Group,
which determined that NASA’s actions were responsive to the recommendation and
closed its review on June 8, 2005.

Results. In November 2004, NASA issued draft policy and procedures for an
engineering authority responsible for establishing, monitoring, and approving technical
requirements across the Agency. Additionally, on March 7, 2005, the Agency issued an
update to “NASA Plan for Implementing Safe and Reliable Operations” that addresses
fundamental changes being made in response to Recommendation 9.1-1 to improve the
safety and reliability of Space Shuttle operations. We believe the organizational structure
planned for the technical authority poses some risks. Specifically, while NASA’s Plan


during critical program decisions but also during the day-to-day activities. The CAIB
noted in Chapter 8, “In both pre-accident periods, events unfolded over a long time and in
small increments rather than in sudden and dramatic occurrences.” We do not believe
that periodic audits or visits by Headquarters-based SMA personnel will be thorough
enough to capture and implement the intent of the CAIB for a real-time, long-term,
independent presence.

The Chief SMA Officer at Headquarters defers accountability for day-to-day SMA
activities to Center Directors in accordance with NASA’s philosophy for managing
mission support functions. The Chief SMA Officer stated that MDAAs and Center
Directors are ultimately accountable for mission success, while the role of the Chief SMA
Officer is to be accountable for providing leadership, policy direction, functional
oversight, assessment, and Agency wide coordination of SMA activities. According to
the Chief SMA Officer, removing SMA from the institutional chain of command could
cause the perception of relieving program managers and Center Directors of their
accountability for ensuring safe and reliable operations.

Although there are numerous organizational alternatives for SMA, we question whether
assigning sole responsibility for SMA activities to Center Directors and MDAAs is
enough to satisfy the full intent of the CAIB. Center and program management’s
accountability for safety will not be compromised by a third set of eyes. Conversely, we
believe that the completely independent third set of eyes will more closely approximate
the intent of the CAIB recommendation.

Conclusion

NASA management has made a conscious decision not to implement fully the latest in a
long series of recommendations concerning independent funding and reporting for SMA.
In lieu of full implementation, NASA has instituted many changes in its organization to
include establishing the Technical Authority, the NASA Engineering and Safety Center
(NESC),' and the Ombuds* concept.

Although various elements of the post-CAIB changes may encompass the concept and
features of an independent third set of eyes, NASA needs to assess whether the function
is better left divided among and between these various elements or centralized into one
entity. However, NASA must now demonstrate that its Plan is thorough enough to

4 The NESC, based at the Langley Research Center, was created shortly after the Columbia accident and
became operational on November 1, 2003. The NESC is an Agency-wide technical resource focused on
engineering excellence. The objective of the NESC is to improve safety by performing i depth
independent engineering assessments, testing, and analysis to uncover technical vulneral ies and to
determine appropriate preventative and corrective actions for problems, trends, or issues within NASA's
programs, projects, and institutions.

The Ombuds program was established to provide the civil service and contractor workforce a confidential
supplemental, and informal channel to communicate critical issues and concerns that could impact safety,
organizational performance, or mission success. The program serves as a channel that addresses issues that
are not the exclusive responsibility of existing administrative systems or offices. The Ombuds reports
directly to the Center Director.



Appendix A

Management Program Control Review

The scope of the audit was limited to a review of the Agency plans to address engineering
and SMA weaknesses as identified by the CAIB.

Audit Work

We performed audit work for this report from October 2003 through June 2005 and in
accordance with generally accepted government auditing standards.

12