<table style="width:100%;">
<colgroup>
<col style="width: 3%" />
<col style="width: 1%" />
<col style="width: 45%" />
<col style="width: 1%" />
<col style="width: 6%" />
<col style="width: 15%" />
<col style="width: 20%" />
<col style="width: 1%" />
<col style="width: 0%" />
<col style="width: 0%" />
<col style="width: 0%" />
<col style="width: 1%" />
</colgroup>
<tbody>
<tr class="odd">
<td></td>
<td colspan="4"></td>
<td colspan="4"></td>
<td colspan="3"></td>
</tr>
<tr class="even">
<td></td>
<td colspan="4"></td>
<td colspan="4"><img src="media/image1.png"
style="width:1.44931in;height:1.42431in" /></td>
<td colspan="3"></td>
</tr>
<tr class="odd">
<td></td>
<td colspan="4"></td>
<td colspan="4"></td>
<td colspan="3"></td>
</tr>
<tr class="even">
<td></td>
<td colspan="6"><p><strong>Office of Information Resource
Management</strong></p>
<p><strong>Office of the Assistant Secretary for Management and
Budget</strong></p>
<p><strong>Department of Health and Human Services</strong></p></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="odd">
<td></td>
<td colspan="6"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="even">
<td></td>
<td colspan="6"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="odd">
<td></td>
<td colspan="6"><p>HHS IRM Guidelines for</p>
<p>Capital Planning and Investment Control</p></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="even">
<td></td>
<td colspan="6"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="odd">
<td></td>
<td colspan="6"></td>
<td colspan="3"></td>
<td colspan="2"></td>
</tr>
<tr class="even">
<td></td>
<td colspan="8"><strong>January 8, 2001</strong></td>
<td colspan="3"></td>
</tr>
<tr class="odd">
<td></td>
<td colspan="5"></td>
<td colspan="3"></td>
<td colspan="3"></td>
</tr>
<tr class="even">
<td></td>
<td colspan="5"></td>
<td colspan="3"></td>
<td colspan="3"></td>
</tr>
<tr class="odd">
<td></td>
<td colspan="5"></td>
<td colspan="3"></td>
<td colspan="3"></td>
</tr>
<tr class="even">
<td></td>
<td colspan="5"></td>
<td colspan="3"></td>
<td colspan="3"></td>
</tr>
<tr class="odd">
<td></td>
<td colspan="5"></td>
<td colspan="3"></td>
<td colspan="3"></td>
</tr>
<tr class="even">
<td></td>
<td colspan="2"><strong>Project:</strong></td>
<td colspan="6">HHS IRM Guideline</td>
<td colspan="3"></td>
</tr>
<tr class="odd">
<td></td>
<td colspan="2"><strong>Document Number:</strong></td>
<td colspan="6">HHS-IRM-2000-0001-GD</td>
<td colspan="3"></td>
</tr>
<tr class="even">
<td></td>
<td colspan="2"></td>
<td colspan="6"></td>
<td colspan="3"></td>
</tr>
<tr class="odd">
<td></td>
<td colspan="2"></td>
<td colspan="6"></td>
<td colspan="3"></td>
</tr>
<tr class="even">
<td colspan="2"></td>
<td colspan="2"></td>
<td colspan="4"></td>
<td colspan="3"></td>
<td></td>
</tr>
</tbody>
</table>

Table of Contents

A. Guideline A - Model Process [3](#guideline-a---model-process)

A.1. Requirements Management [4](#requirements-management)

A.1.1 Business Need Identification
[4](#a.1.1-business-need-identification)

A.1.2 Project Formulation [4](#a.1.2-project-formulation)

A.1.3 Performance Measures [5](#a.1.3-performance-measures)

A.1.4 Cost-Benefit and Alternatives Aanalysis
[6](#a.1.4-cost-benefit-and-alternatives-analysis)

A.1.5 Return on Investments [6](#a.1.5-return-on-investments)

A.1.6 Economic Sensitivity Analysis
[7](#a.1.6-economic-sensitivity-analysis)

A.1.7 Gap Analysis [7](#a.1.7-gap-analysis)

A.1.8 Risk Assessment [7](#a.1.8-risk-assessment)

A.2. Investment Decision Management [8](#investment-decision-management)

A.2.1 Concept Review Based on Mission Impact
[8](#a.2.1-concept-review-based-on-mission-impact)

A.3. Systems Development [9](#systems-development)

A.3.1 Systems Life Cycle [9](#a.3.1-systems-life-cycle)

A.3.2 Project Management [10](#a.3.2-project-management)

A.3.3 Technical Strategy and Plan
[10](#a.3.3-technical-strategy-and-plan)

A.3.4 Configuration Management [11](#a.3.4-configuration-management)

A.3.5 Measuring Performance [11](#a.3.5-measuring-performance)

A.4. Operations and Maintenance [12](#operations-and-maintenance)

A.5. Project Performance Evaluation
[12](#project-performance-evaluation)

B. Guideline B: The Raines' Rules [13](#guideline-b-the-raines-rules)

C. Guideline C: HHS IT Capital Planning and Investment Control/Budget
Formulation Time Line
[14](#guideline-c-hhs-it-capital-planning-and-investment-controlbudget-formulation-time-line)

D. Guideline D: Financial Criteria [15](#guideline-d-financial-criteria)

E. Guideline E: Monte Carlo Analysis
[16](#guideline-e-monte-carlo-analysis)

F. Guideline F: Earned Value [17](#guideline-f-earned-value)

G. Guideline G: The Capability Maturity Model
[18](#guideline-g-the-capability-maturity-model)

# Guideline A - Model Process

This model process is to be used as guidance to assist the OPDIVs in
managing their CPIC processes and procedures. OPDIVs may adopt their own
specific process. Some of the outputs specified below are required to be
completed and available for Departmental ITIRB review.

The IEEE/EIA[^1] U.S. implementation of the ISO/IEC[^2] *12207 Standard
for Information Technology -- Software Life Cycle Processes* defines a
framework for software life-cycle processes from concept through
retirement. It is especially suited for acquisitions because it
recognizes the distinct roles of acquirer and supplier.

ISO 12207 describes five primary processes: acquisition, supply,
development, maintenance, and operation; eight supporting life cycle
processes: documentation, configuration management, quality assurance,
verification, validation, joint review, audit and problem resolution;
and four organizational life cycle processes: management,
infrastructure, improvement and training. The full standard is available
from IEEE - [http://www.ieee.org](http://www.ieee.org/). DoD has adopted
12207 in place of their Software Development Life Cycle Specifications
(e.g. DoD 2167, DoD 2167A, DoD 498).

A similar approach will be used in capital planning for all projects
subject to HHS IRM Policy \# HHS-IRM-2000-001, "Capital Planning and
Investment Control." OPDIVs should adopt this or similar processes for
projects under their direct control.

-   Projects will be divided into five major stages:

-   Requirements Management

-   Investment Decision Management

-   Systems Development

-   Operations and Maintenance

-   Project Performance Evaluation

At each of the first four stages, a business case is developed, refined
and updated. The business case consists of documents outlining the
requirements, return on investment (ROI), cost, scope, schedule, and
risk associated with the project. After the project is initially
approved by the ITIRB, the business case is generally updated annually.
This may be required more frequently for specific projects.

The output from each of the first three stages provides the information
necessary to make the decision to move forward with the project, by the
Department and/or the OPDIV's ITIRB. Figure A "ITIRB Review Process with
Required Products of HHS IRM Policy for Capital Planning and Investment
Control" illustrates the stages and outputs expected from the OPDIVs
during the ITIRIB process.

The following sections provide detail on the recommended content and
documentation for ITIRB review.

## Requirements Management

In order to identify and therefore manage, the requirements of any
project, a particular sequence of steps is usually followed. A business
need is identified and an initial business case for the project
including cost, scope, schedule, and risk is developed.

### A.1.1 Business Need Identification

All business needs flow from the mission of the OPDIV. As the mission
changes, additional projects reflecting new or changed emphasis on IT
are identified. IT is often used as a tool to implement the solution to
the changes.

The first step in developing an IT investment project is to identify a
specific business need and preliminary, high-level system requirements.
A high level, rough order of magnitude (ROM) determination of resource
and schedule requirements is conducted. In addition, pre-coordination
with stakeholders and other interested parties (i.e., executive
management, budget staff, etc.) can assist by providing necessary
information to each party. As the system concept is developed, the
general scope of the project is established.

### A.1.2 Project Formulation

As a part of the Federal budget cycle, a formal call for the submission
of proposed IT projects is generally issued by the OPDIV CIO, in
September, when the organization is developing the operating plan for
the upcoming fiscal year. The IT budget, at its highest level, is simply
the aggregate of all of the approved IT projects for a given year, with
possible exceptions for emergency situations (i.e., new legislation,
unexpected system failure). To ensure that the HHS CIO is working with
up-to-date information regarding the Department's IT investments and
expenditures during the annual Budget Review Board sessions, OPDIVs must
update and submit an OMB Exhibit 53, "Agency IT Investment Portfolio"
and any necessary OMB Circular A-11 forms 300B, "Capital Asset Plan and
Justification."

HHS IT projects include new investments, ongoing investments, and
ongoing operations and maintenance systems, as well as grant programs
with significant IT components. See the timeline in Guideline C, "HHS IT
Capital Planning Investment Control/Budget Formulation Time Line".

The project manager prepares a spending/procurement plan that lists the
specific purchases, including operations and maintenance costs,
anticipated for the coming year. One purpose of the plan is to ensure
timely, efficient procurement of planned IT acquisitions throughout the
year. Another purpose of the plan is to assure availability of funds in
the appropriate quarter of the fiscal year.

A business case containing a complete description of the project must be
created and justified for the project. The business case includes:

A business needs statement that defines the specific need(s) this system
will fulfill, clearly demonstrates how the project supports the OPDIV
and HHS strategic plans, business functions, and mission-critical
functions, and HHS IT architecture.

-   Return on investment and cost-benefit/alternatives analysis.

-   A risk assessment and mitigation plan.

-   A technical strategy and plan.

-   An economic sensitivity analysis.

-   A performance measurement plan that includes an identification,
    definition, collection and tracking plan.

-   A spending and procurement plan, including IV&V funds and testing
    funds.

-   A project plan, a schedule, and a work breakdown structure.

A project plan and schedule list the major milestones of the project.
The work breakdown structure lists the hierarchical arrangement of the
work to be performed.

The paragraphs below cover some of the required input to complete the
eight documents delineated above.

### A.1.3 Performance Measures

Performance measures shall be established to demonstrate a successful
outcome by designating accountability for the results and by striving
for concrete results. All systems developed and deployed since 1996
require the establishment and tracking of appropriate performance
measures. Performance measures are established to track the success of
the project. Each performance measure, shall be reported and monitored,
and should demonstrate no more than a ten (10) percent deviation from
the target. Example measures include:

\(1\) *Amount of contract award*: Is the amount of the initial contract
award within the budget? Is the overall projected contract value, at the
time of award, within the budget?

\(2\) *Baseline or current performance or level of service:* How will
the current environment or performance be improved.

\(3\) *Implementation status:* Describe the service level achieved
compared to the level planned. Will the project produce the anticipated
return on investment? Are deliverables for each quarter being received
on time?

\(4\) *Performance of hardware/software or other purchases:* Are
hardware and software packages working well for the agency? Are the
hardware and software contributing to project success? For example, is
down time excessive?

\(5\) *Schedule:* How does the current schedule compare with the project
schedule?

*Risk.* How does projected risk compare to actual risk?

\(7\) *Definition of benefits and subsequent measure:* Are the
definitions and measures appropriate?

\(8\) *Benefits:* Are the benefits being achieved? For example, are they
producing the anticipated cost savings and cost avoidance?

\(9\) *Meeting business need:* Has the project meet the identified
business need?

The OPDIV should add any other performance measures that are appropriate
or necessary.

### A.1.4 Cost-Benefit and Alternatives Analysis

The cost/benefit analysis should identify at least three alternatives
considered to implement the proposed development or change request, and
identify the option with the greatest benefit to the agency. For each
alternative, the project manager should show how the project's
functional requirements would or would not be met; estimates of the
project life cycle costs, and the anticipated benefits or the return,
including the corresponding analyses that were conducted to develop the
estimates.

A complete cost/benefit analysis should include the following for each
alternative considered:

\(1\) Identify and quantify costs, both recurring and non-recurring for
investment and operations and maintenance.

\(2\) Identify assumptions and constraints that were used when
developing the figures.

\(3\) Identify both quantitative and qualitative benefits.

\(4\) Evaluate alternatives using net present value techniques,
including a discount factor, ROI and cost-benefit ratio.

\(5\) Conduct a gap analysis, which refers to a need that is not being
fulfilled.

Costs and benefit risk elements Include, but are not limited to:

-   Up-front costs - hardware, software, costs to design and develop the
    project, and transition costs.

-   Ongoing costs - salaries, software upgrades, training, supplies.

-   Operations and maintenance.

-   Indirect costs - initial productivity losses and computer support
    for project management costs, architecture support costs and change
    control costs.

-   Tangible benefits - benefits that can be quantified, such as
    reducing costs, increasing productivity, decreasing cycle time or
    improving service quality.

-   Intangible benefits - benefits that may be easy to identify but that
    are difficult to quantify, like faster, more efficient decision
    making, greater data accuracy, improved data security, reduced
    burden, or increased organizational knowledge.

### A.1.5 Return on Investments

The ROI of the project shows how much the project benefits the
organization in comparison to the savings or cost avoidance. Anticipated
return can be the basis for measuring project performance. Prototypes
and pilots can help quantify anticipated gains in performance or
reductions in cost by testing expectations on a small scale.

All enhancements are subject to an ROI justification, and in general,
the industry convention is that, if the annual cost of enhancing an
application exceeds 20 percent of its replacement cost, or 20 percent
modification to code, there may be less risk and less cost if the system
is replaced.

Present value and net present value (NPV) analyses are used to compare
investment alternatives that occur over multiple years. See Guideline D,
"Financial Criteria."

### A.1.6 Economic Sensitivity Analysis

The sensitivity analysis refers to the relative magnitude of change in
one or more elements of an economic analysis. Due to the uncertainties
in the analysis, it is necessary to know more than the results using one
set of conditions especially if a recommendation would change if one or
more of the input variables to the cost-benefit analysis changed.
Sensitivity analysis expresses cash flows in terms of unknown variables
and calculation of the consequences of wrongly estimating the variables.
These variables may be cost estimates, implementation schedule length,
requirements, configuration of equipment or software, conversion costs,
or other types of assumptions.

The "Monte Carlo" method of simulation is a useful example. Many
products can be used to conduct a sensitivity analysis, depending on the
OPDIV needs, and the scope and scale of the project purchase. The method
was originally developed for computer simulations of nuclear fission,
which were used to test whether an atom bomb was feasible. Refer to
Guideline E, "Monte Carlo Analysis."

### A.1.7 Gap Analysis

When an IT project is initiated, a gap may exist between the current
state and the desired state. A gap analysis compares and examines the
current system functionality against the proposed system and
requirements. If the proposed system meets the current system
functionality or new requirements, there is a fit. If it does not, then
there is a gap. If the proposed system exceeds the requirements or
current system functionality, then there is "value added."

Once a list of unique functionality has been developed for each
alternative, the analysis can begin to identify benefit categories. Each
alternative will not necessarily provide the same benefits, nor will
they necessarily provide each benefit in equal amounts.

### A.1.8 Risk Assessment

Risk assessment and risk management are ongoing activities throughout
the life of the project. The project owner is responsible for
recognizing risks and of developing risk mitigation strategies. Again,
prototypes and pilots can minimize risk by substantiating assumptions
and testing technical solutions on a small scale. Typical categories of
risk include:

Project risks - the size of the investment, the project size and
longevity, scalability, project interdependencies, or the experience of
the project team in managing similar projects.

Organizational risks - the impact on agency programs if the project does
not proceed, management commitment to the project, or political
expectations or legislative requirements to complete the project.

Technical risks - risks associated with maintaining skilled staff,
hardware and software dependencies, application software, other
infrastructure needs, and security vulnerabilities and safeguards.

To deal effectively with potential risks, the project owner must develop
a risk assessment plan that includes:

Risk identification - documenting events that could jeopardize the
successful completion of the project within the quality, cost, and
schedule defined for the project.

Risk analysis - estimating the impact of risk on the quality, cost, and
project schedule and seeking ways to reduce the impact of a risk event
prior to its occurrence.

Risk monitoring and reassessment - reporting on the progress of the
project, deleting risks as necessary to reflect the current situation,
identifying new risks, and beginning mitigation action to reduce risk.
Deviation by more than 10% from the performance measures requires risk
mitigation.

Risk progress reporting - developing complete documentation on
identified risk events, mitigation actions, and success/failure of these
actions.

Finally, if the OPDIV is considering more than one investment, all
candidate systems must be ranked and prioritized by the strength of the
business needs.

### 

## Investment Decision Management

All projects must be reviewed and prioritized for funding, and proceed
based on mission and cross-functional impact.

### A.2.1 Concept Review Based on Mission Impact

The highest priority projects will be those that have the greatest
impact on the OPDIV mission or on other dependent agencies.

The project may be important to the OPDIV, but the OPDIV does not
consider it critical. In this case, the OPDIV may "wait-list" the
project in favor of another, more important project, or until the OPDIV
makes funding available, or until the OPDIV can assure that an
appropriately skilled project team is available.

The OPDIV's ITIRB must evaluate each project concept based on the
project's impact and the probable benefit to the OPDIV and/or other
OPDIVs, or HHS as a whole. The review will result in a rating for each
system. The rating will then be used to form a priority ranking of the
candidate systems. The budget should allow for a mix of projects,
including proposed projects, projects under development, and operational
systems. The portfolio should ideally be a mix of mission-related
systems, administrative systems, and infrastructure.

The OPDIV ITIRB process for ranking and funding projects shall build in
project management costs, IV&V funds and independent testing funds
up-front for the largest and most critical projects. Actual costs for
operations and maintenance shall be calculated for five years, and shall
include hardware and software, technical refreshment, and training
funds, at a minimum.

If full funding is not received, the project owner may need to reduce
the level of service or activities proposed to reflect the new lower
amount. If the project is not approved at all, the project owner may
wait and resubmit the project at a later date, or refine the
implementation strategies based on guidance provided by the board. The
ITIRB may find that the project is under-funded and cannot achieve
success without more money. In any case, the business alternatives
analysis and/or cost-benefit analysis shall be updated and resubmitted
if the project is intended to proceed.

## Systems Development

Each project shall use system life cycle and development planning.
Systems development shall include planning for any training,
configuration management, quality assurance, risk management, and any
other collateral tasks associated with the project. A technical strategy
and the associated plan shall be developed since the project is mature
enough at this point for detailed design.

### A.3.1 Systems Life Cycle

All systems design, development, and maintenance efforts must follow a
minimal set of activities, documentation, and review requirements.

"System Life Cycle (SLC)" is defined as a structured development
approach with defined activities, phases, products, and reviews that
provide a standard to support the development of systems from
identification through implementation, operation, maintenance, and
eventual retirement. The systems life cycle process is a basic
requirement for systems development. There are a variety of life cycle
models, such as: waterfall, spiral, evolutionary, decomposition (or
stepwise refinement), and formal transformation. The choice among the
models is made based on the specific project. Once chosen, a change in
method being employed must be justified to the ITIRB.

To improve information management to support these business operations,
the SLC shall guide technology investments and process improvements. The
SLC incorporates objectives, formal documentation, and key reviews
throughout its major activities and phases that allow managers to make
key decisions about systems development and information technology
control. Planning, oversight, and approval processes within the SLC
support the development of high-quality systems, delivered on time and
within budget.

### A.3.2 Project Management

The project manager is responsible for tracking and assessing the actual
performance against the established goals and projections. Reporting on
project performance should be done throughout the life of the project
and is the basis for monitoring the progress of the project against
projected costs, schedules, performance, and deliverables. If a project
is projected to be or is actually late or over cost, a corrective plan
and adjustments and improvements must be made for the project to
continue. Early identification and correction of any problems that arise
are critical. Project managers will report performance using earned
value techniques (see Guideline F, "Earned Value"). The project manager
is responsible for:

\(1\) Application of the specific measures included in the project plan
to track and compare estimated versus actual values for costs,
schedules, benefits, risks, and project success.

\(2\) Documentation of any changes in the nature and scope of the
requirements between the original requirements and the current business
conditions, and changes in the business assumptions.

\(3\) Preparing "lessons learned" and identifying process improvements
that can be applied to later phases of the project or similar projects.

Project managers are responsible for staffing, how staff will be used,
and how long the project will take. A complete project schedule should
include:

-   The sequence in which the project tasks will be performed.

-   The performance milestones and expected outcomes within each task.

-   The number of people who will be required to accomplish the tasks.

-   The specific assignments of tasks to people.

-   The dates each person will begin and end work on the project and any
    periods when personnel will not be working full time on the project.

-   The beginning and concluding dates for each task.

Project managers should continuously compare the actual costs and
benefit data for the project to the initial figures submitted in the
project plan. If there have been approved changes in the project as a
result of changing environments, budgets, or priorities, the HHS IT
repository should be revised to reflect these changes with appropriate
documentation and rationale.

### A.3.3 Technical Strategy and Plan

OPDIV program officials/project managers must work closely with
technical experts to ensure that all system requirements are fully
developed. The technical plan must address the technical parameters of
the project in detail, including the platform chosen and the expected
impact on the infrastructure. The assumptions regarding the level of
data processing resources, the amount of staffing support anticipated,
and who will provide the support (contractor staff or OPDIV staff) must
be included. In addition, the amount of operations and maintenance
support must be considered. The plan must address the options
considered, and provide a justification of why the specific models or
types of hardware, software, network, and telecommunications purchases
noted in the spending plan were selected.

### A.3.4 Configuration Management

All HHS system proposals constitute formal change requests. The change
requests will be designated critical, routine, or administrative. Any
proposed new system development or upgrade must be documented with a
change request, and, eventually, a business case. A program level change
management process is critical to control and document changes to
systems. The OPDIV shall create and operate a change control board. In
most cases, change control will exist to manage changes concerning a
related family of systems.

Change management occurs throughout all phases of the system life cycle.
Changes during maintenance of existing systems cycle forward into new
business needs and the need to modify existing systems or develop new
systems. All changes must be tracked and monitored for adherence to the
HHS ITA, interaction with other systems and organizations (crosscutting
systems only), HHS and OMB policies, the CCA, and other pertinent
policies and legislation. Substantive changes that meet ITIRB thresholds
need to go through the ITIRB process.

Specifically, the OPDIV shall:

-   Identify and submit requests for information technology as change
    requests, identifying the nature of the change, if it has not
    already been documented

-   Track the progress of all change requests in the system.

-   Document thoroughly all system changes during the system life cycle.

-   Track and monitor all systems for adherence to the ITA, interaction
    and dependencies with older systems, HHS and OMB policies, the CCA,
    and other relevant policy and legislation.

-   Report the status of change requests and systems under development
    or upgrade, including projects that are behind schedule, and the way
    in which the project will be corrected.

### A.3.5 Measuring Performance 

OPDIVs must establish a system to measure the performance and cost of an
operational asset against the baseline established in the planning
phase. This information will allow agency managers to optimize the
performance of capital assets and identify the need for new investments.
Analysis of the cost of asset ownership is defined as the total of all
costs incurred by the owners and users to obtain the benefits of a given
acquisition. Ownership costs, such as operations and maintenance,
service contracts, and disposition can easily consume up to 80 percent
of the total life-cycle costs.

## Operations and Maintenance

Proper maintenance is essential to the life of an asset. If an asset is
not properly maintained, its useful life can be shortened dramatically,
reducing the return on the taxpayers' investment. Day-to-day operation
and maintenance of any asset must be carefully planned. In addition the
projected maintenance cost of the asset must be factored into the
procurement of that asset, to make a best-value determination when
selecting between competing proposals, and tracked throughout its life
cycle.

Factors affecting the successful implementation of an Operations and
Maintenance plan include:

-   Sign-offs on the plan by key personnel to instill responsibility.

-   Training of user staff.

-   Tracking of labor and material costs.

The OPDIV should have a close relationship with the vendor
representative on maintenance the vendor is required, by contract, to
keep the equipment in good operating condition. The maintenance
provisions of the contract is the basis for negotiating a maintenance
with the vendor(s).

## Project Performance Evaluation

The evaluation phase occurs once the project has been completed, and
consists of an assessment of the project\'s success or failure at each
major phase. The Contracting Officer may seek past performance data on
an annual basis, during the contract performance period. The IT project
owners should make sure that their input to such data is consistent with
the past performance evaluation data that they collect in the HHS IT
repository. Past performance data about the project shall be collected
in the HHS IT repository and analyzed to compare expected results
against actual benefits and returns. The past performance evaluation
will be based on:

The project's overall effectiveness in meeting the original objectives.

The identification of the specific benefits that have been achieved,
including whether these benefits match the projections and the reasons
for any discrepancy.

A comparison of actual costs incurred against the projected costs.

A determination of how well the project met time schedules and a
determination of management and user perspectives on the project, if
necessary.

An evaluation of issues that require further attention.

A summary of "lessons learned" for use in providing future project
management.

# Guideline B: The Raines' Rules

1.  Support core/priority mission functions that need to be performed by
    the Federal Government.

2.  Be undertaken by the requesting agency because no alternative
    private sector or governmental source can efficiently support the
    function.

3.  Support work processes that have been simplified or otherwise
    redesigned to reduce costs, improve effectiveness, and make maximum
    use of commercial, off-the-shelf technology.

4.  Demonstrate a projected return on the investment that is clearly
    equal to or better than alternative uses of available public
    resources. Return may include: improved mission performance in
    accordance with GPRA measures, reduced cost, increased quality,
    speed or flexibility, and increased customer and employee
    satisfaction. Return should also be adjusted for risk.

5.  Be consistent with Federal, agency, and bureau information
    architectures which integrate agency work processes and information
    flows with technology to achieve the agency's strategic goals,
    reflect the processes and information flows, reflect the agency's
    vision, and specify standards that enable information exchange and
    resource sharing.

6.  Reduce risk by avoiding or isolating custom-designed components to
    minimize the potential adverse consequences to the overall project,
    using fully tested pilots, simulations or prototypes, establishing
    clear measures accountability for project progress, and securing
    buy-in by program officials who will use the system.

7.  Be implemented in phased, successive chunks as narrow in scope and
    brief in duration as practicable, each of which solves a specific
    part of an overall mission problem and delivers a net benefit.

8.  Employ an acquisition strategy that appropriately allocates risk
    between government and contractor, effectively uses competition,
    ties contract payments to accomplishments, and takes maximum
    advantage of commercial technology.

# Guideline C: HHS IT Capital Planning and Investment Control/Budget Formulation Time Line

**April/May** HHS ITIRB meetings held and recommendations to the CIO
completed.

> Begin budget planning process for Budget year PLUS 1 (18 months away).
> (CY+2 or Budget Year \[BY\] + 1.

**June/July** HHS Budget Review Board meetings and funding decisions
made.

(CY+2 or Budget Year +1)

**August** HHS ITIRB meeting.

> Development of Information Technology (IT) operating plan for the
> upcoming fiscal year (CY + 1).
>
> Draft Secretary's initiative papers and draft Department OMB budget
> due.
>
> (BY + 1)

**September** Final Secretary's initiative papers due.

> New HHS budget transmitted to OMB. (BY + 1)

**October** Beginning of the new fiscal year.

Brief OMB on Department's budget.

**November** HHS ITIRB meeting.

> OMB passback received in the Office of Budget.

**December** Department appeals of OMB passback.

Negotiations with OMB on budget requests and appeals.

> HHS budget appeals completed and OMB budget decisions finalized**.**
>
> (BY +1)

**January** Final budget all-purpose table due; includes all years.

Final budget reference book and budget in brief due.

Final Congressional justifications due.

State of the Union Address

**February** HHS ITIRB

President submits budget to Congress.

**March** OPDIV investment reviews completed and documentation, in
accordance with the OMB passback instructions.

Budget briefings for Congressional staff.

Appropriations hearings begin.

# Guideline D: Financial Criteria

Financial criteria are required for mission-related and administrative
IT projects (proposed, ongoing, and existing). Two major but relatively
simple financial analyses are used to evaluate projects. These analyses
are Net Present Value (NPV)
$\text{NPV} = \sum_{i - 1}^{n}\frac{\text{values}_{i}}{(1 + \text{rate})^{i}}$
and Return on Investment.

Present value (PV) and net present value (NPV) analyses are used to
compare investment alternatives that occur over multiple years. PV
requires that all quantifiable benefits and costs are brought back to
current day dollar values; i.e., "present" value. By defining all costs
and benefits in current dollar amounts, various alternatives can be
compared directly. An analysis becomes a net present value when the
analyst subtracts the project's PV costs from the forecasted PV
benefits.

When estimating costs over time, a person initially estimates dollar
amounts in current day dollars. This is known as using constant base
year dollars to estimated costs. However, this method is not accurate
simply due to the time value of money. To account for this timing, or
opportunity, difference, and analyst multiplies each yearly cost and
benefit by a yearly discount factor. Once each dollar is multiplied by
the discount rate, the dollar is considered to be in "present value"
dollars. The equation used to multiply the cost/benefit amounts is: P =
F 1/(1 + I)n, where 1 = discount rate, and n = year of project (base
year = 0).

PV is needed in addition to Return on Investment because Return on
Investment does not account for the magnitude of the savings, just the
relative savings ratio.

Return on Investment is simply the rate of return for a given
investment. Considering financial criteria only, and given a number of
investment opportunities, the goal of capital planning is to construct
an investment portfolio with the highest overall Return on Investment.
For these purposes, Return on Investment is defined as NPV divided by
present value costs, expressed as a Percent, or ROI = TDB --TDC/TDC,
where TDB = total discounted benefits, and TDC = total discounted costs.

# Guideline E: Monte Carlo Analysis

A sensitivity analysis refers to the relative magnitude of change in one
or more elements of an economic analysis. Due to the uncertainties in
the analysis, it is necessary to know more than the results using one
set of conditions, especially if a recommendation would change if one or
more of the input variables to the cost-benefit analysis changed.
Sensitivity analysis expresses cash flows in terms of unknown variables
and calculation of the consequences of wrongly estimating the variables.
These variables may be cost estimates, implementation schedule length,
requirements, configuration of equipment or software, conversion costs,
or other types of assumptions.

The "Monte Carlo" method of simulation is one example. The name of this
method was assigned because the random number used to generate
information for the trials is independent for each trial, like spins of
a roulette wheel. The method was originally developed for computer
simulations of nuclear fission, which were used to test whether an atom
bomb was feasible.

The Monte Carlo simulation is a mathematical technique for numerically
solving differential equations. It is extensively used in finance for
such tasks as pricing derivatives or estimating the value at risk of a
portfolio. The Monte Carlo method is typically used in a financial
situation. Because many financial problems are complex, this method is
frequently used.

A typical Monte Carlo simulation is used to solve problems which require
that one or more statistics of a probability distribution be calculated.
For example:

(1) The price of an option is the risk neutral discounted mean of the
    probability distribution for the option's value at expiration.

(2) Value at risk is the upper bound on a confidence interval for the
    probability distribution of the loss/profit that a portfolio will
    realize over a specified horizon.

    One example is how a complex option might be priced. Suppose the
    option's value is dependent on underliers, a stock index, and an
    exchange rate. The Monte Carlo simulation might be used to price
    such an option. It can randomly generate 10,000 scenarios for the
    value on the option's expiration date, and the two underliers, and
    will do so in a manner that is consistent with an assumed joint
    probability distribution of the two variables. It can also determine
    what the option's expiration value would be under each of the
    scenarios, and form a histogram of these results, generating an
    approximation for the probability distribution of the option's
    expiration value. The discounted mean of the histogram is the
    estimated option price.

    More information on the Monte Carlo method may be found at
    [http://contingencyanalysis.com.]{.underline}

# Guideline F: Earned Value

In 1967, DoD established the earned value concept. This concept:

(1) compares the value of work accomplished during a time period with
    the work scheduled for that period.

(2) uses the value of work done as the basis for estimating the cost and
    time to complete.

(3) alerts program managers to potential problems sooner than
    expenditure alone can.

    There are two major objectives of an earned value system to:

\(1\) encourage contractors and government staff to use effective
internal cost and schedule

management control systems.

\(2\) permit the government to be able to rely on timely data produced
by those systems for determining product-oriented contract status.

The essence of earned value management is that at some level of detail
appropriate for the degree of technical, schedule, and cost risk or
uncertainty associated with the program, a target value (e.g., budget)
is established for each scheduled element of work.

The project owner must ensure that both contractor and government plans,
budgets, and scheduled work are established in increments that
constitute a performance measurement baseline. Milestones defined as
deliverables are concrete and measurable and can demonstrate work
accomplished. These are the "planned value." The sum of work completed
and the cost for direct and indirect effort is the "earned value."

An understanding of several terms is necessary to understand earned
value calculations. These are "budgeted cost of work scheduled," or
BCWS; "budgeted cost of work performed," or BCWP. As an example, if BCWS
-- BCWP = ACWP, or Actual Cost of Work Performed," the work scheduled
equals the work performed, and there is no deviation. However, if a work
deviation occurs, the cost variance will be BCWP -- ACWP. Indices may
also be used to measure deviations. A financial deviation is noted as
ACWP/BCWS. Other helpful terms include Earned Value (EV) and Actual Cost
(AC). As an example, EV/AC yields a cost performance index and EV-budget
equals schedule variance.

To illustrate, assume that a contract calls for an IT system to be
developed in four weeks at a cost of \$4 million. After three weeks of
work, only \$2 million has been spent. By analyzing planned versus
actual expenditures, it appears that the project is under-running the
estimated costs. However, an earned value analysis reveals that the
project is in trouble, because even though only \$2 million has been
spent, the system is only 25% completed. On the basis of work completed,
the project will cost \$8 million (\$6 million to complete the remaining
75% of the work, and the work will ultimately take a total of 12 weeks.)

To learn more about earned value, please refer to the EIA-747 standard
entitled, "Earned Value Management Systems." You may also wish to
participate in the Office of Grants and Acquisition Management "Early
Warning Project Management System Workshop."

# Guideline G: The Capability Maturity Model

To fully implement the Clinger-Cohen Act, the Department urges the
OPDIVs to strive, at a minimum, to meet the Software Engineering
Institute (SEI) Capability Maturity Model (CMM) level 2, the repeatable
level, of this five-step model. In level 2, basic project management
processes are established. Cost, schedule, and functionality are tracked
and reported. The necessary process discipline is in place to repeat
earlier successes on projects with similar applications.

The SEI CMM is important because each maturity level provides a layer in
the foundation for continuous process improvement. Achieving each level
of the maturity model institutionalizes software processes, resulting in
an overall increase in the process capability of the organization. As
the Department's abilities and processes continue to mature, OIRM will
ask the OPDIVs to meet additional levels of the CMM.

CMM assists organizations in maturing their people, processes, and
technology assets to improve long-term business performance. The SEI has
developed CMM for software, people, and software acquisition, and
assisted in the development of the CMM for system engineering and
integrated product development.

For more information, contact
[http://www.sei.cmm.org](http://www.sei.cmm.org/)[.]{.underline}

[^1]: IEEE: Institute of Electrical and Electronic Engineers: EIA:
    Electronic Industries Association

[^2]: ISO: International Organization for Standardization; IEC:
    International Electrotechnical Commission