**National Aeronautics and Space Administration**

Information Resources

Self Review Process Guide

Developed by the NASA Self-Assessment Redesign Team

March 1996

**Table of Contents**

> **Section**
>
> **Preface 1**
>
> **1. Purpose 2**
>
> **2. Scope 2**
>
> **3. Process Model Description 3**
>
> **4. Self Review Products 5**
>
> **5. Self Review Instructions 5**
>
> **6. Annual Integrated Program Assessment (AIPA) 8**
>
> **7. Sharing Self Review Results 8**
>
> **[Appendix]{.underline}**
>
> **A. IR Oversight 10**
>
> **B. IT Security Oversight 11**
>
> **C. Information Management 12**
>
> **D. IT Management 13**
>
> **E. MIS Management 14**
>
> **F. Self Review Improvement Survey 15**
>
> **G. Format for Self Review Reports 16**
>
> **H. Summary Report Example 23**

PREFACE

Self review is a management tool devised to answer external requirements
for assessment imposed by Congress and the Office of Management and
Budget. These requirements are found in public law and Federal
directives. Principal external requirements for review of information
resources and information technology security include the Information
Technology Management Reform Act of 1996, the Paperwork Reduction Act of
1995, the Government Performance and Results Act of 1993, the Computer
Security Act of 1987, and the Federal Managers' Financial Integrity Act
of 1982.

Information resources, which include information and information
technology, are critical to the successful accomplishment of NASA's
missions. This *Information Resources Self Review Process* *Guide* is a
tool for implementing self-assessment as the primary review process for
assuring compliance with law and continual improvement of information
resources management, and evaluating progress in achieving objectives
set forth in the *NASA Information Resources Strategic Plan*. The
*Guide* itself is a product of teamwork and continual improvement.

In 1995, an intercenter re-engineering team redesigned NASA\'s
information resources management functional area self-assessment. The
team not only redesigned previous methods used for self-assessment but
also integrated two formerly separate evaluation processes for
information resources management and information technology security
into one streamlined annual review. The new methodology de-emphasizes
compliance and promotes a progressive implementation of evolving "best
practices". A risk-based management philosophy underscores commitment to
the idea that there will always be opportunities for improvement.

Ronald S. West

Chief Information Officer

NASA Team Leader: Marie K. Tynan

Team Members:

Ames Research Center: John Humbert and John R. Ray

George C. Marshall Space Flight Center: Kathleen K. Mann

Goddard Space Flight Center: Dennis E. Reed

Headquarters: Bessie B. Berry, Richard W. Carr, and Linda K. Perez

Langley Research Center: Lisa Y. Yu

Lyndon B. Johnson Space Center: Wanda S. Hobley

1\. PURPOSE

The self review process is designed to stimulate thought and the sharing
of ideas that will yield assessments focused on measuring the
contribution of Information Resources Management (IRM) to NASA's mission
performance. It is a means for assessing how well technologies,
procedures, organizations, and initiatives meet internally developed
best practices, with particular emphasis on those outlined as objectives
in the *NASA Information Resources Strategic Plan,* which supports the
*NASA Strategic Plan.* The Center self review reports were redesigned
for the purpose of providing useful improvement information to each
Center as well as contributing more effectively to the *Annual
Integrated Program Assessment (AIPA)* for IRM. The *AIPA* will be used
by senior management to report achievement of performance goals and
identify needed improvement actions and Agency best practices.

2\. SCOPE

The self review process provides the framework for improving controls
and coordinating NASA's review activities. The process identifies five
Information Resources (IR) elements for review:

> I. IR Oversight
>
> II\. Information Technology (IT) Security Oversight
>
> III\. Information Management
>
> IV\. IT Management
>
> V. Major Information System (MIS) Management
>
> These five areas represent a major streamlining of seventeen different
> IRM activities previously identified for self-assessment as well as
> the integration of the IT security review into this re-engineered
> process. IT security is treated as a separate area for self review
> because of its critical function in the protection of NASA's large
> investment in IT. Each IR element is reviewed annually.

Primary customers for the self review are the Center Chief Information
Officers (CIOs), the Center Directors, and the Enterprise CIOs.
Secondary customers include the Agency CIO and the Director, IRM
Division, who prepares the *AIPA* for signature by the Agency CIO and
submission to the Administrator. The self review process is used to
provide continual input to the strategic planning process and for the
improvement of the annual integrated program assessment process.

3\. PROCESS MODEL DESCRIPTION

The integration of the IR self review with NASA\'s IR planning
activities led to the development of an overall process model that
defines key elements of the review process and links them to IR
strategic planning. The annual cycle model is derived from the NASA
continual improvement approach as advocated by NASA Management
Instruction (NMI) 1240.3, Functional Management. It is composed of two
spheres, each representing IR activities as conducted according to a
Plan-Do-Check-Act (PDCA) model.

**Cycle 1 - IR Management**

The first PDCA sphere represents the cycle of IR activities carried out
in the day-to-day IR environment. This sphere is reflected as a closed
loop model.

**Plan:** The cycle begins with strategic plans that are developed for
the Agency and each Center. The strategic plans contain goals and
objectives that can be tracked to corresponding actions. In addition,
the formulation and use of measures are encouraged at all levels to
provide progress data and input for reports on how the Centers and
Agency are achieving their goals.

**Do:** Actions are implemented to achieve the strategic plan goals and
objectives.

**Check:** Throughout the year, program, project, and management reviews
are conducted. Customers provide feedback and employees suggest
improvements. Outside sources such as the Government Accounting Office
(GAO) and the Office of the Inspector General (OIG) perform reviews.
Each Center's IR self review activities and the *AIPA* activities are
part of this cycle.

**Act:** As a result of the various reviews, steps are taken to make
improvements and changes. Also the knowledge gained from these reviews
is incorporated into the Agency and Center strategic plans. It is this
closing of the loop that makes the process model dynamic and capable of
fostering continual improvement in the management of IR.

**Cycle 2 - IR Self Review**

In addition to continual improvement brought about by day-to-day
reviews, NASA has added a formalized self review approach, represented
by the lower PDCA cycle. This approach

accomplishes the following functions:

> · Provides a tool for measuring the Agency\'s degree of success in
> meeting the objectives set forth in the *NASA IR Strategic Plan*
>
> · Institutes the annual Center IR self-review
>
> · Integrates customer feedback into the IR self review.

The lower PDCA cycle shows that the self review process is itself a
closed loop model. The arrows at both sides of this model indicate that
Cycle 1 feeds into Cycle 2 and that the findings of the self review feed
back into the Cycle 1 activities. This information flow can occur at any
step of the PDCA model.

**Plan:** The planning for the performance of the annual self review
includes input from strategic plans and other Cycle 1 activities, as
well as information learned through previous reviews.

**Do:** The collecting and analyzing of data in performing the IR self
review.

**Check:** Evaluating the self review process to assure that it
continually improves.

**Act:** Recommending improvements and changes to the self review
process and providing results to the IR management process.

4\. SELF REVIEW PRODUCTS

This new IR self review process calls for the production by each Center
of five *Center Detailed Reports,* a *Center Summary Report,* and an *IR
Self Review Improvement Survey.* Each of these products is a metric, as
well as a means to effecting continual improvement. The conduct of the
self review, identification of accomplishments and areas for
improvement, and assessment of risk result in information that is
compiled into a detailed report used by the Center and the Center\'s CIO
to initiate improvement activities.

The Center self review team will complete a *Center* *Detailed Report*
for each of the five IR elements. The *Detailed Reports* are summarized
by the Center CIO for submission as the *Center Summary Report* to NASA
Headquarters. The *Summary Report* from each of the Centers provides
input to the development of the *AIPA.* The *Center Summary Reports* and
the *AIPA* serve as progress reports for use by the Agency CIO in
measuring NASA's success in meeting its IR strategic planning
objectives. The reports may further support the Agency CIO's decision
making process by focusing attention on areas of critical concern, thus
assuring that necessary resources are in place and needed changes are
implemented.

The Center self review team for each IR element will also complete the
*IR Self Review Improvement Survey*. The Center CIO will develop the
Center *IR Self Review Improvement Survey* based on the team surveys
from the five IR element self reviews. The survey tool will provide
feedback for continual improvement of the self review process.

Formats for the *Detailed* and *Summary Reports* are included in
Appendix G. An example of a *Summary Report* submission to NASA
Headquarters is included in Appendix H. The *IR Self Review Improvement
Survey* format is in Appendix F.

5\. SELF REVIEW INSTRUCTIONS

The following instructions are provided as a guide to Centers for
conducting the self review and completing the required report documents.

Carefully consider and discuss the IR element reviews with this question
in mind: How can I use the best practices presented in this self review
process to improve management at our Center?

Identify major accomplishments. These performance indicators show
success and will generally include significant cost savings, cost
avoidance, workforce savings, or direct contribution to mission
accomplishments. Significant achievements may also include improved
business and mission planning, more efficient operations, higher quality
of information and processes, more effective decision-making, and better
management reporting. Major accomplishments should emphasize
improvements resulting from pursuing strategic plan goals and
objectives.

**IR Element Review Process**

**Step 1 (Center CIO) Organize the Self Review**

A team approach is recommended to complete each of the IR element self
reviews. The teams should include the IR functional manager (and
sub-function managers as required), staff personnel, and customers of
the function. Participation on the teams from other NASA Centers is
encouraged to obtain as broad a perspective as possible.

Note that the self review includes an assessment of each of the
Center\'s major information systems identified in the Center's latest
submission to the Agency CIO's annual IR strategic planning call. Teams
formed to review MISs should also include the MIS data owner as a team
member.

**Step 2 (Review Team) Review Best Practices**

Best practices for each IR element are located in Appendices A through
E. Consider for each IR element how the Center is employing the best
practices and to what degree, according to the \"level of maturity
scale.\" Performance of each best practice is rated as shown in Appendix
G, *Center Detailed Report*. Centers are encouraged to identify
additional best practices that have been successful in achieving
continual improvements in service to customers, improved efficiency and
cost-effectiveness, improved standardization, etc.

**Step 3 (Review Team) Identify Accomplishments**

Identify the Center's significant accomplishments during the past year
in IR Oversight, IT Security Oversight, Information Management, IT
Management, and MIS Management. Cost savings, cost avoidance,
operational efficiencies, labor savings, cycle time decreases, mission
performance, and standardization improvements are to be identified and
considered in assessing the significance of the accomplishment. Major
accomplishments are to be flagged for inclusion in the *Center Summary
Report*. Please use an asterisk (\*) to identify the items that are
considered major for the *Summary Report.*

**Step 4 (Review Team) Identify Areas for Improvement**

Identify areas for improvement which each team recommends that the
Center undertake in the coming year. Only major improvement areas will
be included in the *Center Summary Report*. Please use an asterisk (\*)
to identify the items that are considered major for the *Summary
Report.*

**Step 5 (Review Team) Assign Risk Rating**

Assign a risk level for each IR element on the line for \"Risk
Assignment\" using the rating scale of High, Medium, or Low as shown in
Appendix G, *Center Detailed Report*.

**Step 6 (Review Team) Complete Improvement Survey**

Complete the *IR Self Review Improvement Survey,* Appendix F. Centers
are requested to complete this survey to provide feedback and input for
future improvements to the IR self review process.

**Step 7 (Review Team) Assess Performance Measures**

Identify performance measures or metrics that have been successful at
measuring continual improvements in service to customers, improved
efficiency and cost-effectiveness, improved standardization, etc. Refer
to *Center Detailed Report.*

**Step 8 (Center CIO) Generate the Center Products**

Provide an overall maturity rating for each IR element based upon the
results of the individual best practice evaluation for each of the IR
elements. Provide an overall risk rating for the Center\'s IR activities
derived from the rating in each area. Center CIOs should use the self
review maturity scale to rate the Center's responsiveness to Agency CIO
focus areas during the year. This gauge provides the Agency CIO with a
snapshot of how the Center CIOs perceive they are implementing change,
CIO focus areas, and priority initiatives.

The *Summary Report* format shown in Appendix G serves as an executive
summary for documenting the Center\'s progress in adoption of best
practices, the overall risk rating for the Center, and the Center\'s
performance measures.

In addition, the Center\'s Major Accomplishments and Areas for
Improvement are to be summarized from the detailed findings. This
information may be included with the *Summary Report* as shown in
Appendix H.

**Step 9 (Center CIO) Assessing the Assessment Process**

Complete the *IR Self Review Improvement Survey* included in Appendix F
from the Center CIO's perspective and include it with your *Summary
Report.*

**Detailed Report Content**

The *Center Detailed Self Review Report* is an aggregation of the
detailed findings/results, obtained through the preceding steps, and
will be retained and used by the Center.

**Center Summary Report Content**

The *Center Summary Report* is intended to be a high-level summary. The
self review results should be formatted as defined in Appendix G. The
summary report will also include narrative on significant
accomplishments and major areas for improvement, and the *Center IR Self
Review Improvement Survey.* An example of the *Center Summary Report,
Results Synopsis of Major Accomplishments and Areas for Improvement,
Survey*, and the associated transmittal memo are shown in Appendix H.

6\. ANNUAL INTEGRATED PROGRAM ASSESSMENT (AIPA)

Center results provide input to development of the *AIPA* by the IRM
Division. The *AIPA* is the result of all the Centers' self reviews and
other audit and performance measurement activities, which are compiled
and analyzed to provide the Agency CIO with an overall assessment of
areas of excellence, areas of risk and identification of best practices.
The *AIPA* also provides input to the Administrator's *Agency
Accountability Report* provided annually to the President and the
Congress as a formal assurance that the Agency's controls are achieving
their intended objectives.

7\. SHARING SELF REVIEW RESULTS

An important part of the self review process is the post-review video
teleconference. This teleconference includes the Agency CIO, the IRM
Division, Enterprise CIOs, Center CIOs, and Center review team members.
The role of the IRM Division is to coordinate and facilitate this forum
for sharing best practices, areas needing improvement, and performance
measures.

**Information Resources**

**Self Review Process**

**Appendix**

**APPENDIX A**

**IR OVERSIGHT**

Definition: Oversight of Information Resources (IR) is the management of
information and related technology resources to assure support to
missions and operational effectiveness.

Objective: NASA\'s IR oversight objective is to assure that IR strategic
and CIO focus area goals are achieved. These goals are achieved through
the establishment of architectures and uniform standards, defining
appropriate measures, and performing investment planning, all of which
contribute to an effective IR life-cycle methodology. The Agency must
ensure that a positive return on its investment in information
technology (IT) is achieved. The focus should be on the elimination of
unnecessary and duplicate information, and the combining or sharing of
services, tools, processes, and other resources. Special emphasis must
be placed on consolidation, sharing, and reuse of information resources.

Best Practices:

1\. Ensure that IR investments are adequately considered as part of the
Center's strategic planning.

2\. Perform IR investment planning by balancing risks and benefits.

3\. Integrate the IR investment strategy into the Center's budgeting
process.

4\. Participate in the development of Agency IT architectures and
standards.

5\. Develop and review IT architectures and standards for the Center and
align with Agency interoperability goals.

*6.* Measure progress toward achieving consolidation, modernization,
interoperability, standardization, increased use of Commercial
Off-the-Shelf (COTS) products, high customer satisfaction, training
goals and a high return-on-investment.

7\. Use performance measures for continual improvement of IR products,
processes and services.

8\. Upgrade the skills and knowledge of the workforce through timely
training on information resources, technologies, and related
disciplines.

9\. Establish an effective IR management structure for supporting the
Agency CIO's vision and strategic focus areas, as well as the Center's
IR objectives.

10\. Ensure that existing assets are leveraged effectively.

**APPENDIX B**

**IT SECURITY OVERSIGHT**

Definition: Information technology (IT) security oversight is the
protection of the integrity, confidentiality, and availability of
information and related IT resources*.* IT security is an integral part
of the effective management of information resources. It is represented
in every IR element in the assessment process and is presented here as a
separate oversight entity, to emphasize its importance and to facilitate
separate reporting requirements.

Objective: NASA's IT security oversight objective is to ensure that
information, computing, and communications services are secure through
managing risks; developing security infrastructure, architecture and
standards; integrating customer issues and concerns; providing customer
training and awareness; and influencing the policy, planning, budgeting,
and performance measurement processes.

Best Practices:

1\. Ensure that security at NASA protects our resources through
implementation of policies, procedures, standards, and performance
measures.

2\. Ensure that a management official authorizes in writing the use of
each major application and general support system based on systems
testing, technical evaluation and certification.

3\. Ensure that controls for security are performed as an integral part
of daily operations.

4\. Ensure that responsibility for security in each system is assigned
to an individual knowledgeable in the IT used in the system and in
providing security for both the IT and the information.

5\. Establish a systematic process to ensure that personnel receive
training and are knowledgeable of and are managing identified IT
security risks.

6\. Ensure that IT architectures, standards and implementation plans
address IT security goals, objectives and strategies.

7\. Integrate security into the life-cycle management of IT planning and
evaluation, budgeting, acquisition, development, operation, and disposal
or re-utilization processes.

8\. Ensure that a systematic process is in place to identify and handle
IT security incidents, which includes effective cost analysis, interface
with supervisory NASA management officials, and interface with the NASA
OIG on all cases that involve criminal activity.

**APPENDIX C**

**INFORMATION MANAGEMENT**

Definition: Information management is the management and control of
NASA's investment in information, including the identification of
information needs and the assurance of sharing and consolidation,
standardization, control, security, and integrity of data created,
stored, manipulated, disseminated and archived/retired. Information
management includes the disciplines of data administration, and
management of scientific and technical information, libraries, records,
forms, mail, directives, printing, and correspondence.

Objective: NASA's information management objective is to ensure data
integrity, accessibility, and availability and to protect the Agency's
investment in information. Information management seeks to ensure that
data created, stored, manipulated, disseminated, and reported
effectively meets customer needs through sharing information
requirements, eliminating redundancies, ensuring standardization,
providing training and awareness, and measuring customer satisfaction.

Best Practices:

1\. Ensure that data management policies and procedures further the
Enterprise goals and objectives.

2\. Manage data/information as a strategic resource in its own right.

3\. Consider information management functions in IR planning and
budgeting.

4\. Ensure that the needs of all customers, including those with
disabilities, are met through information management products and
services.

5\. Apply IT to improve the performance of information management
functions.

6\. Facilitate the accessibility of NASA scientific and technical
information to our customers.

7\. Apply information security techniques to protect the integrity,
confidentiality, and availability of information and guard against
unauthorized disclosure.

8\. Identify opportunities to reduce data redundancies and standardize
implementation at the Center and the Agency.

9\. Provide users and managers with awareness and training on
information life-cycle management (creation, access, dissemination,
storage, and disposition).

10\. Continually improve information management by measuring performance
and customer satisfaction.

**APPENDIX D**

**INFORMATION TECHNOLOGY (IT) MANAGEMENT**

Definition: IT management is the management and control of NASA's
investment in IT which includes components as defined by the
"Information Technology Management Reform Act of 1996,\" Section 5002,
as follows:

> \(A\) The term "information technology", with respect to an executive
> agency means any equipment or interconnected system or subsystem of
> equipment, that is used in the automatic acquisition, storage,
> manipulation, management, control, display, switching, interchange,
> transmission, or reception of data or information by the executive
> agency. For purposes of the preceding sentence, equipment is used by
> an executive agency if the equipment is used by the executive agency
> directly or is used by a contractor under contract with the executive
> agency which (i) requires the use of such equipment, or (ii) requires
> the use, to a significant extent, of such equipment in the performance
> of a service or the furnishing of a product.
>
> \(B\) The term "information technology" includes computers, ancillary
> equipment, software, firmware and similar procedures, services
> (including support services), and related resources.
>
> \(C\) Notwithstanding subparagraphs (A) and (B), the term "information
> technology" does not include any equipment that is acquired by a
> Federal contractor incidental to a Federal contract.
>
> Objective: NASA's IT management objective is to apply uniform
> architecture and standards that will ensure interoperability with a
> continual focus on modernization and life cycle planning to meet
> customer needs.

Best Practices:

1\. Ensure that IT management policies and procedures support NASA's
Enterprises.

2\. Ensure that IT is addressed in the *Center Strategic Plan.*

3\. Assess the applicability of industry trends to the Center's IT
architecture and standards.

4\. Plan for an increased use of COTS products and outsourcing of IT
services.

5\. Assess IT needs based on requirements, utilization of existing
resources, consolidation opportunities and modernization objectives.

6\. Ensure that the IT needs of all customers, including those with
disabilities, are met.

7\. Assure optimum return-on-investment from IT acquisition planning
through standardization, consolidation, and sharing of resources.

8\. Ensure adequate training in use of IT.

9\. Measure the effectiveness of such key IT processes as obsolescence
metrics to improve interoperability.

**APPENDIX E**

**MAJOR INFORMATION SYSTEM (MIS) MANAGEMENT**

Definition: MIS management is oversight of a system that requires
special continuing management attention because of its importance to an
Agency mission, its high development, operating or maintenance costs, or
its significant impact on the administration of agency programs,
finances, property, or other resources. An MIS inventory is identified
by each Center as part of the *Annual Request for Information Resources
Strategic Implementation Plans.*

Objective: MISs should be managed as investments that are essential to
successfully and efficiently achieving NASA\'s mission. Management
should be accomplished through a focus on comprehensive life-cycle
planning, sound risk management, continual improvement in operational
efficiencies, and the use of measures to evaluate performance and
customer satisfaction.

Best Practices:

1\. Ensure the MIS program plan spans the full life cycle of the system,
addressing modernization, architectures, standards and consolidation.

2\. Ensure the MIS information technology security plan addresses risk
management and contingency planning.

3\. Consider asset management strategies for determining if the system
is continuing to meet customer needs and if technology replacement is
required.

4\. Capitalize on opportunities for outsourcing or use of COTS
solutions.

5\. Ensure active customer participation in managing the MIS, especially
the configuration management process.

6\. Measure system performance, improve service to customers, and reduce
operating costs.

**APPENDIX F**

**IR Self Review Improvement Survey**

The self assessment redesign team wishes to ensure that the self review
process is useful, efficient, kept up-to-date, and responsive to the
expressed needs of the NASA community.

Please answer the following questions by using the scale provided,
comparing *last* year\'s *self assessment* with *this* year\'s *self
review*. We would like to use your feedback to make improvements to next
year\'s *IR self review*.

**Scale**

1 - Strongly Disagree 2 - Disagree 3 - Agree 4 - Strongly Agree

SD D A SA

1\. The IR self review was efficient; it made good use of our time.

Last
Year\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....1
2 3 4

This
Year\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....1
2 3 4

2\. The *IR self review* was effective; it enabled us to do a thorough
review.

Last
Year\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....1
2 3 4

This
Year\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....1
2 3 4

3\. The results were useful; they provided us with information needed
for improvements.

Last
Year\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....1
2 3 4

This
Year\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....1
2 3 4

4\. The *IR self review* was responsive to the needs of IR customers.

Last
Year\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....1
2 3 4

This
Year\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....1
2 3 4

5\. What improvements to this process guide would you like us to make
next year?

6\. What did you like or dislike about the reporting approach used this
year?

7\. Recommend additional best practices to be incorporated into this
review guide?

**APPENDIX G**

**Format for Self Review Reports**

***"Center Name"* Summary Report**

**Information Resources Self Review**

**Degrees of Performance of Best Practices**

+-------------+-------------+-------------+-------------+--------------+
| **IR        | **Not       | **Being     | **Being     | **Mature or  |
| Elements**  | supported   | Defined**   | Im          | Institu      |
|             | or          |             | plemented** | tionalized** |
|             | uns         |             |             |              |
|             | tructured** |             |             |              |
+-------------+-------------+-------------+-------------+--------------+
| **IR**      |             |             |             |              |
|             |             |             |             |              |
| **          |             |             |             |              |
| Oversight** |             |             |             |              |
+-------------+-------------+-------------+-------------+--------------+
| **IT        |             |             |             |              |
| Security    |             |             |             |              |
| Oversight** |             |             |             |              |
+-------------+-------------+-------------+-------------+--------------+
| **          |             |             |             |              |
| Information |             |             |             |              |
| M           |             |             |             |              |
| anagement** |             |             |             |              |
+-------------+-------------+-------------+-------------+--------------+
| **IT**      |             |             |             |              |
|             |             |             |             |              |
| **M         |             |             |             |              |
| anagement** |             |             |             |              |
+-------------+-------------+-------------+-------------+--------------+
| **MIS       |             |             |             |              |
| M           |             |             |             |              |
| anagement** |             |             |             |              |
+-------------+-------------+-------------+-------------+--------------+

**Levels of Maturity Scale:**

[Non-supported or unstructured]{.underline} means that the Center has
not defined processes or procedures for implementing the management
practices.

[Being defined]{.underline} means that processes or procedures are being
developed that call for the expected practices, but

they have not yet been put in place.

[Being implemented]{.underline} means that appropriate IR processes and
procedures have been designed and are being

followed in parts of the Center, but they may not be consistently
followed.

[Mature or institutionalized]{.underline} means that the Center has
fully adopted the IR practices called for, applies them

consistently, and improves them through a feedback loop.

**APPENDIX G**

**Format for Self Review Reports**

***"Center Name"* Summary Report**

**Information Resources Self Review**

**Degrees of Implementation for CIO Strategic Focus Areas**

  -------------------- ---------------- ------------ --------------- ---------------------
  **Focus Areas**      **Not supported  **Being      **Being         **Mature or
                       or               Defined**    Implemented**   Institutionalized**
                       unstructured**                                

  **1. Supercomputing                                                
  consolidation**                                                    

  **2. Mainframe                                                     
  consolidation**                                                    

  **3. IT obsolescence                                               
  management**                                                       

  **4. Agency IR                                                     
  Strategic Plan**                                                   

  **5. IT standards &                                                
  interoperability**                                                 
  -------------------- ---------------- ------------ --------------- ---------------------

[Major Accomplishments]{.underline}: In narrative text, specify business
function or enterprise supported and quantify the results.

[Major Improvements]{.underline}**:** In narrative text, describe areas
for improvement and if possible, state the desired results.

**Detailed "*Center Name"* Report**

**IR Oversight Review Response**

**Degrees of Performance of Best Practices**

  ------------- ---------------- ------------- --------------- ---------------------
  **IR          **Not supported  **Being       **Being         **Mature or
  Oversight**   or               Defined**     Implemented**   Institutionalized**
                unstructured**                                 

  **1.**                                                       

  **2.**                                                       

  **3.**                                                       

  **4.**                                                       

  **5.**                                                       

  **6.**                                                       

  **7.**                                                       

  **8.**                                                       

  **9.**                                                       

  **10.**                                                      
  ------------- ---------------- ------------- --------------- ---------------------

**Levels of Maturity Scale:**

[Non-supported or unstructured]{.underline} means that the Center has
not defined processes or procedures for implementing the management
practices.

[Being defined]{.underline} means that processes or procedures are being
developed that call for the expected practices, but

they have not yet been put in place.

[Being implemented]{.underline} means that appropriate IR processes and
procedures have been designed and are being

followed in parts of the Center, but they may not be consistently
followed.

[Mature or institutionalized]{.underline} means that the Center has
fully adopted the IR practices called for, applies them

consistently, and improves them through a feedback loop.

[Significant Accomplishments]{.underline}: narrative text

[Areas for Improvement]{.underline}: narrative text

[Risk Assignment]{.underline}: select one of the following for an
overall rating of this area

Risk Definition:

Low Controls are sufficient to minimize risk, regardless of how much
risk is inherent in a process.

Med. There are several areas of significant risk in spite of existing
controls, but the remaining risk is unlikely to result in

\(1\) loss of life, (2) significant loss or waste of resources, or (3)
endangerment or reduction of the Agency's ability to perform its
mission.

High There are several areas of significant risk in spite of existing
controls, and the remaining risk has the potential to result in (1) loss
of life, (2) significant loss or waste of resources, or (3) endangerment
or reduction of the Agency's ability to perform its mission.

[Successful Measures]{.underline}: provide a brief narrative description
of the criteria used to measure the Center's success in managing the IR
elements.

**Detailed "*Center Name"* Report**

**IT Security Oversight Review Response**

**Degrees of Performance of Best Practices**

+------------+------------+------------+------------+---------------+
| **IT       | **Not      | **Being    | **Being    | **Mature or   |
| Security** | supported  | Defined**  | Imp        | Instit        |
|            | or         |            | lemented** | utionalized** |
| **O        | unst       |            |            |               |
| versight** | ructured** |            |            |               |
+------------+------------+------------+------------+---------------+
| **1.**     |            |            |            |               |
+------------+------------+------------+------------+---------------+
| **2.**     |            |            |            |               |
+------------+------------+------------+------------+---------------+
| **3.**     |            |            |            |               |
+------------+------------+------------+------------+---------------+
| **4.**     |            |            |            |               |
+------------+------------+------------+------------+---------------+
| **5.**     |            |            |            |               |
+------------+------------+------------+------------+---------------+
| **6.**     |            |            |            |               |
+------------+------------+------------+------------+---------------+
| **7.**     |            |            |            |               |
+------------+------------+------------+------------+---------------+
| **8.**     |            |            |            |               |
+------------+------------+------------+------------+---------------+

**Levels of Maturity Scale:**

[Non-supported or unstructured]{.underline} means that the Center has
not defined processes or procedures for implementing the management
practices.

[Being defined]{.underline} means that processes or procedures are being
developed that call for the expected practices,

but they have not yet been put in place.

[Being implemented]{.underline} means that appropriate IR processes and
procedures have been designed and are being

followed in parts of the Center, but they may not be consistently
followed.

[Mature or institutionalized]{.underline} means that the Center has
fully adopted the IR practices called for, applies them consistently,
and improves them through a feedback loop.

[Significant Accomplishments]{.underline}: narrative text

[Areas for Improvement]{.underline}: narrative text

[Risk Assignment]{.underline}: select one of the following for an
overall rating of this area

Risk Definition:

Low Controls are sufficient to minimize risk, regardless of how much
risk is inherent in a process.

Med. There are several areas of significant risk in spite of existing
controls, but the remaining risk is unlikely to result in (1) loss of
life, (2) significant loss or waste of resources, or (3) endangerment or
reduction of the Agency's ability to perform its mission.

High There are several areas of significant risk in spite of existing
controls, and the remaining risk has the potential to result in (1) loss
of life, (2) significant loss or waste of resources, or (3) endangerment
or reduction of the Agency's ability to perform its mission.

[Successful Measures]{.underline}: provide a brief narrative description
of the criteria used to measure the Center's success in managing the IR
elements..

**Detailed "*Center Name"* Report**

**Information Management Review Response**

**Degrees of Performance of Best Practices**

  --------------- ---------------- ------------- --------------- ---------------------
  **Information   **Not supported  **Being       **Being         **Mature or
  Management**    or               Defined**     Implemented**   Institutionalized**
                  unstructured**                                 

  **1.**                                                         

  **2.**                                                         

  **3.**                                                         

  **4.**                                                         

  **5.**                                                         

  **6.**                                                         

  **7.**                                                         

  **8.**                                                         

  **9.**                                                         

  **10.**                                                        
  --------------- ---------------- ------------- --------------- ---------------------

**Levels of Maturity Scale:**

[Non-supported or unstructured]{.underline} means that the Center has
not defined processes or procedures for implementing the management
practices.

[Being defined]{.underline} means that processes or procedures are being
developed that call for the expected practices, but they have not yet
been put in place.

[Being implemented]{.underline} means that appropriate IR processes and
procedures have been designed and are being followed in parts of the
Center, but they may not be consistently followed.

[Mature or institutionalized]{.underline} means that the Center has
fully adopted the IR practices called for, applies them consistently,
and improves them through a feedback loop.

[Significant Accomplishments]{.underline}: narrative text

[Areas for Improvement]{.underline}: narrative text

[Risk Assignment]{.underline}: select one of the following for an
overall rating of this area

Risk Definition:

Low Controls are sufficient to minimize risk, regardless of how much
risk is inherent in a process.

Med. There are several areas of significant risk in spite of existing
controls, but the remaining risk is unlikely to result in (1) loss of
life, (2) significant loss or waste of resources, or (3) endangerment or
reduction of the Agency's ability to perform its mission.

High There are several areas of significant risk in spite of existing
controls, and the remaining risk has the potential to result in (1) loss
of life, (2) significant loss or waste of resources, or (3) endangerment
or reduction of the Agency's ability to perform its mission.

[Successful Measures]{.underline}: provide a brief narrative description
of the criteria used to measure the Center's success in managing the IR
elements.

**Detailed "*Center Name"* Report**

**IT Management Review Response**

**Degrees of Performance of Best Practices**

  -------------- ---------------- ------------- --------------- ---------------------
  **IT           **Not supported  **Being       **Being         **Mature or
  Management**   or               Defined**     Implemented**   Institutionalized**
                 unstructured**                                 

  **1.**                                                        

  **2.**                                                        

  **3.**                                                        

  **4.**                                                        

  **5.**                                                        

  **6.**                                                        

  **7.**                                                        

  **8.**                                                        

  **9.**                                                        
  -------------- ---------------- ------------- --------------- ---------------------

**Levels of Maturity Scale:**

[Non-supported or unstructured]{.underline} means that the Center has
not defined processes or procedures for implementing the management
practices.

[Being defined]{.underline} means that processes or procedures are being
developed that call for the expected practices, but they have not yet
been put in place.

[Being implemented]{.underline} means that appropriate IR processes and
procedures have been designed and are being followed in parts of the
Center, but they may not be consistently followed.

[Mature or institutionalized]{.underline} means that the Center has
fully adopted the IR practices called for, applies them consistently,
and improves them through a feedback loop.

[Significant Accomplishments]{.underline}: narrative text

[Areas for Improvement]{.underline}: narrative text

[Risk Assignment]{.underline}: select one of the following for an
overall rating of this area

**Risk Definition:**

Low Controls are sufficient to minimize risk, regardless of how much
risk is inherent in a process.

Med. There are several areas of significant risk in spite of existing
controls, but the remaining risk is unlikely to result in (1) loss of
life, (2) significant loss or waste of resources, or (3) endangerment or
reduction of the Agency's ability to perform its mission.

High There are several areas of significant risk in spite of existing
controls, and the remaining risk has the potential to result in (1) loss
of life, (2) significant loss or waste of resources, or (3) endangerment
or reduction of the Agency's ability to perform its mission.

**[Successful Measures]{.underline}:** provide a brief narrative
description of the criteria used to measure the Center's success in
managing the IR elements.

**Detailed "*Center Name"* Report**

**MIS Management Review Response**

**Degrees of Performance of Best Practices**

  -------------- ---------------- ------------- --------------- ---------------------
  **MIS          **Not supported  **Being       **Being         **Mature or
  Management**   or               Defined**     Implemented**   Institutionalized**
                 unstructured**                                 

  **1.**                                                        

  **2.**                                                        

  **3.**                                                        

  **4.**                                                        

  **5.**                                                        

  **6.**                                                        
  -------------- ---------------- ------------- --------------- ---------------------

**Levels of Maturity Scale:**

[Non-supported or unstructured]{.underline} means that the Center has
not defined processes or procedures for implementing the management
practices.

[Being defined]{.underline} means that processes or procedures are being
developed that call for the expected practices, but they have not yet
been put in place.

[Being implemented]{.underline} means that appropriate IR processes and
procedures have been designed and are being followed in parts of the
Center, but they may not be consistently followed.

[Mature or institutionalized]{.underline} means that the Center has
fully adopted the IR practices called for, applies them consistently,
and improves them through a feedback loop.

[Significant Accomplishments]{.underline}: narrative text

[Areas for Improvement]{.underline}: narrative text

[Risk Assignment]{.underline}: select one of the following for an
overall rating of this area

**Risk Definition:**

Low Controls are sufficient to minimize risk, regardless of how much
risk is inherent in a process.

Med. There are several areas of significant risk in spite of existing
controls, but the remaining risk is unlikely to result in (1) loss of
life, (2) significant loss or waste of resources, or (3) endangerment or
reduction of the Agency's ability to perform its mission.

High There are several areas of significant risk in spite of existing
controls, and the remaining risk has the potential to result in (1) loss
of life, (2) significant loss or waste of resources, or (3) endangerment
or reduction of the Agency's ability to perform its mission.

[Successful Measures]{.underline}: provide a brief narrative description
of the criteria used to measure the Center's success in managing the IR
elements.

**APPENDIX H**

**Summary Report Example**

May, 1996

To: NASA Headquarters

AO/Chief Information Officer

All applicable Enterprise CIOs

Thru: Center Director

From: Chief Information Officer

Subject: Information Resources (IR) Self Review for FY96

The Center has conducted the FY96 Information Resources self review.
Summary results are enclosed, including the Summary Report matrix and
narrative descriptions of significant accomplishments and significant
areas for improvement. Also included is the Center's response to the IR
self review improvement survey.

**[Significant Accomplishments:]{.underline}**

· Automation of on-line access to management instructions.

· Re-engineering of the Shuttle Mission Control Center (MCC), through
the application of advanced IT systems and architectures.

**[Areas for Improvements:]{.underline}**

· Develop a process to ensure prompt deletions of computer access when
grants are terminated.

Based on the "degree of performance" of best practices and areas for
improvement identified in this self review, the risk that the Center
will not meet its IR strategic goals is \_\_\_\_\_\_\_ (High, Med., or
Low).

Signed

Center CIO

Enclosures:

1\) Summary Report

2\) IR Self Review Results

3\) IR Self Review Improvement Survey

"***Center Name"* Summary Report**

**Information Resources Self Review**

**Degrees of Performance of Best Practices**

+------------+------------+------------+------------+---------------+
| **IR       | **Not      | **Being    | **Being    | **Mature or   |
| Elements** | supported  | Defined**  | Imp        | Instit        |
|            | or         |            | lemented** | utionalized** |
|            | unst       |            |            |               |
|            | ructured** |            |            |               |
+------------+------------+------------+------------+---------------+
| **IR**     |            |            | **X**      |               |
|            |            |            |            |               |
| **O        |            |            |            |               |
| versight** |            |            |            |               |
+------------+------------+------------+------------+---------------+
| **IT       |            |            |            | **X**         |
| Security   |            |            |            |               |
| O          |            |            |            |               |
| versight** |            |            |            |               |
+------------+------------+------------+------------+---------------+
| **I        |            | **X**      |            |               |
| nformation |            |            |            |               |
| Ma         |            |            |            |               |
| nagement** |            |            |            |               |
+------------+------------+------------+------------+---------------+
| **IT**     |            | **X**      |            |               |
|            |            |            |            |               |
| **Ma       |            |            |            |               |
| nagement** |            |            |            |               |
+------------+------------+------------+------------+---------------+
| **MIS      |            |            | **X**      |               |
| Ma         |            |            |            |               |
| nagement** |            |            |            |               |
+------------+------------+------------+------------+---------------+

**Levels of Maturity Scale:**

[Non-supported or unstructured]{.underline} means that the Center has
not defined processes or procedures for implementing the management
practices.

[Being defined]{.underline} means that processes or procedures are being
developed that call for the expected practices, but they have not yet
been put in place.

[Being implemented]{.underline} means that appropriate IR processes and
procedures have been designed and are being followed in parts of the
Center, but they may not be consistently followed.

[Mature or institutionalized]{.underline} means that the Center has
fully adopted the IR practices called for, applies them consistently,
and improves them through a feedback loop.

**Degrees of Implementation for CIO Strategic Focus Areas**

  -------------------- ---------------- ------------ --------------- ---------------------
  **Focus Areas**      **Not supported  **Being      **Being         **Mature or
                       or               Defined**    Implemented**   Institutionalized**
                       unstructured**                                

  **1. Supercomputing                                                **X**
  consolidation**                                                    

  **2. Mainframe                                                     **X**
  consolidation**                                                    

  **3. IT obsolescence                  **X**                        
  management**                                                       

  **4. Agency IR                                     **X**           
  Strategic Plan**                                                   

  **5. IT standards &                   **X**                        
  interoperability**                                                 
  -------------------- ---------------- ------------ --------------- ---------------------

[Major Accomplishments]{.underline}: In narrative text, specify business
function or enterprise supported and quantify the results.

[Major Improvements]{.underline}: In narrative text, describe areas for
improvement and if possible, state the desired results.

**Enclosure #2**

IR Self Review Results

[Major Accomplishments]{.underline}: In a narrative text paragraph,
specify business function or enterprise supported and quantify the
results.

· Information Management

Center automation of on-line access to management instructions resulted
in increased accessibility, currency, and cost savings. Direct savings -
maintaining management instructions on-line reduced printing, editing,
and binding cost by \$500,000 per year. Indirect savings - labor hours
were reduced by eliminating hours to maintain instruction changes and
deletions; distribution costs were reduced; and all instructions are
current and available on demand.

· Mission Control Center (MCC) Re-engineering

Re-engineering of the Shuttle Mission Control Center (MCC), through the
application of advanced IT systems and architectures, has resulted in
significant improvements in responsiveness to mission requirements while
reducing cost by approximately \$75 million over the next five years.
This modernization is based upon the use of high performance COTS
engineering workstations inter-connected in a distributed computing
network, replacing the custom engineered hardware and mainframe-based
systems. This COTS approach has significantly reduced hardware and
software cost and the on-going support/maintenance labor requirements.
The incorporation of industry standards in this modular design also
greatly facilitates the infusion of new technologies to maintain the
long-term cost effectiveness of the system. The simplified architecture
further facilitates modifications needed to meet flight-specific
experiments, reducing lead time from approximately one year to a few
days.

[Major Improvements]{.underline}: In narrative text paragraph, describe
areas for improvement and, if possible, state the desired results.

· IT Security

Efficient elimination of expired ID's. Develop a process to ensure
prompt deletions of computer access when grants are terminated. Reduce
security risks by eliminating inactive accounts.

**Enclosure #3**

**IR Self Review Improvement Survey**

The self assessment redesign team wishes to ensure that the self review
process is useful, efficient, kept up-to-date, and responsive to the
expressed needs of the NASA community.

Please answer the following questions by using the scale provided,
comparing *last* year\'s *self assessment* with *this* year\'s *self
review*. We would like to use your feedback to make improvements to next
year\'s *IR self review*.

**Scale**

1 - Strongly Disagree 2 - Disagree 3 - Agree 4 - Strongly Agree

SD D A SA

1\. The IR self review was efficient; it made good use of our time.

Last
Year\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....1
2 3 4

This
Year\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....1
2 3 4

2\. The *IR self review* was effective; it enabled us to do a thorough
review.

Last
Year\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....1
2 3 4

This
Year\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....1
2 3 4

3\. The results were useful; they provided us with information needed
for improvements.

Last
Year\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....1
2 3 4

This
Year\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....1
2 3 4

4\. The *IR self review* was responsive to the needs of IR customers.

Last
Year\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....1
2 3 4

This
Year\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....1
2 3 4

5\. What improvements to this process guide would you like us to make
next year?

6\. What did you like or dislike about the reporting approach used this
year?

7\. Recommend additional best practices to be incorporated into this
review guide?