2002

DoD HPC

Modernization Program

Tools for Discovery

Annual Report
Approved for public release; distribution is unlimited. Approved for public release; distribution is unlimited.

For more information about the DoD HPC Modernization Office and the DoD HPC

Modernization Program, visit our Web site at http://www.hpcmo.hpc.mil .hpcmo.hpc.mil

Front cover captions (from top to bottom):

Accurate Modeling of Lossy Frequency Selective Surfaces for Real-Time Design by John Meloling,

Wendy Massey, and David Hurdsman

Smith chart of boundary admittance using past approximations (green) and more accurate solutions

(blue)

Mixed Electromagnetic — Circuit Modeling and Parallelization for Rigorous Characterization of

Cosite Interference in Wireless Communication Channels by Costas Sarris

Electric field magnitude iso-surface on a military vehicle staircase model at 50 MHz

Absolute Scalability of Space/Time Domain Decomposition in Computational Structural

Dynamics for Virtual Modeling and Testing Applications by R. Kanapady and K. K. Tamma

Solid modeling of keel beam

Novel Electron Emitters and Nanoscale Devices by Jerry Bernholc

Electron distribution near a tip of a BN/C nanotube in a field emitter configuration

Detached-Eddy Simulation of Massively Separated Flows over Aircraft by Kyle Squires, James

Forsythe, Scott Morton, Kenneth Wurtzler, William Strang, Robert Tomaro, and Philippe Spalart

Flow over the delta wing at 27° angle of attack. Isosurface of vorticity
A Report by the Department of Defense

High Performance Computing Modernization Program Office

JULY 2003

Department of Defense

High Performance Computing Modernization Program

2002 ANNUAL REPORT

The Department of Defense’s High Performance Computing Modernization Program entered into its

ninth year in 2002, successfully realizing the Congressional and senior Defense leadership vision of

creating a pervasive supercomputing environment that the Department’s scientists

and engineers routinely use as they conduct the research,

development, test, and evaluation activities needed to

maintain our national defense posture.

We saw several procedural changes take effect in 2002,

largely based on applying the lessons learned in

previous years. We continued to examine the best

means of maximizing our return on investments

through benchmarking and differing acquisition

strategies. We awarded a major contract, the

Defense Research and Engineering Network follow-on

contract, and began planning for transition and

upgrade. We selected our industry partners to assist

us in running the four major shared resource

centers. Our Software Applications Support component was transformed

through the use of portfolios that span multiple scientific disciplines, and

training of the community was expanded with implementation of the

Online Knowledge Center.

As the year began, we as a nation were faced with the reality of the

war on terrorism. As the year progressed, the ongoing work of the

Department’s scientists and engineers, using High Performance

Computing Modernization Program resources, made a material

difference in liberating Afghanistan and strengthening our Homeland

Defensive posture. High performance computing environments

supported national priority work on thermobaric weapons and

unmanned aerial, land-based, and submersible vehicles. Scientists

using our resources have addressed chemical and biological threats,

including anthrax and chemical detection and decontamination.

This annual report provides the program’s users and stakeholders with a concise

summary of the state of the program, details our progress in achieving our

objectives, and includes a brief discussion of some of the challenges we will

address in the years ahead.

Cray J. Henry

Director

High Performance Computing Modernization Program

MESSAGE FROM THE DIRECTOR

THE HPCMP MANAGEMENT TEAM

Dr. Robert Peterkin

Chief Scientist

Dr. Leslie Perkins

SAS

Mr. Brad Comes

HPC Centers

Dr. Larry Davis

Deputy Director

Mr. Rodger Johnson

Networking Ms. Jann Ensweiler

Outreach

Mr. John Blair

Financial

•1•

TABLE OF CONTENTS ○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○

MESSAGE FROM THE DIRECTOR

THE HPCMP MANAGEMENT TEAM

SECTION 1—OVERVIEW AND ANALYSIS ............................................ 3

Program Mission ........................................................................................... 4

Program Vision.............................................................................................. 4

Introduction.................................................................................................. 5

HPCMP Goals ........................................................................................................... 6

Program Accomplishments ...................................................................................... 7

Background ................................................................................................................................ 7

2002 Accomplishments............................................................................................................. 7

Other Collaborative Accomplishments................................................................... 8

Using Program Evaluation ......................................................................... 10

Summary of Performance Data ............................................................................. 10

Performance Results ............................................................................................... 10

Considering Future Trends......................................................................... 11

Applying Lessons Learned...................................................................................... 11

Looking Ahead ........................................................................................................ 11

Financial Analysis ....................................................................................... 12

FY 2002 Budget Resources: HPCMP Appropriations......................................... 13

Obligations and Costs............................................................................................ 13

Financial Trends ...................................................................................................... 14

SECTION 2 – PERFORMANCE RESULTS ............................................. 17

PROGRESS TOWARD STRATEGIC GOALS AND OBJECTIVES .................. 19

FY 2002 Performance.................................................................................. 19

Table of Contents
•2•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

SECTION 3—MANAGEMENT ACCOMPLISHMENTS AND

CHALLENGES............................................................................. 25

Accomplishments....................................................................................... 27

Contract Awards .................................................................................................... 27

HPC Centers ........................................................................................................... 28

Networking and Security ....................................................................................... 29

Software Applications Support ............................................................................. 30

Program Oversight...................................................................................... 32

SECTION —FINANCIAL STATEMENT ................................................ 33

Section 4 – FY 2002 Annual Financial Statements .................................. 35

Balance Sheet As of March 31, 2003 ..................................................... 36

Cash Flow Statement: October 1, 2001–September 30, 2002 ............ 37

Income Statment: October 1, 2001–September 30, 2002 ................... 38

SECTION 5—APPENDICIES ............................................................. 39

Appendix A – Program Evaluations........................................................... 41

Performance Evaluation Assessment Review ....................................................... 42

Appendix B – HPCMP Integrated Program Strategy ............................... 43

Appendix C – Acronyms and Abbreviations ............................................ 44
○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○

○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○

SECTION 1

OVERVIEW AND

ANALYSIS
PROGRAM MISSION

The HPCMP’s mission is to deliver world-class

commercial, high-end, high performance computational

capability to the DoD’s science and technology and test

and evaluation communities facilitating the rapid

application of advanced technology into superior

warfighting capabilities.

PROGRAM VISION

The HPCMP’s vision is to provide a pervasive culture

among DoD’s scientists and engineers where they

routinely use advanced computational environments

to solve the most demanding problems.
•5•

SECTION 1 — OVERVIEW AND ANALYSIS ○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○

SECTION 1 — OVERVIEW AND ANALYSIS

INTRODUCTION

The Department of Defense’s High Performance Computing

Modernization Program (HPCMP) provides the Department’s

scientists and engineers with an extraordinary computational

environment to further national defense objectives. In less than

ten years, the Program has produced an outstanding computing

environment that routinely uses high

performance computing resources to solve the

Department’s most challenging scientific and

engineering problems. This, in turn, helps the

United States ensure military advantage and

warfighting superiority on the 21st century

battlefield.

Congressional investment in and support of

the HPCMP has caused cultural changes in the

fundamental way science and engineering are

pursued. In 1993, the Department had just

over 180 gigaflops of computational power to

support the science and technology

community. Through sound management

practices, a deliberate user based

requirements identification process,

appropriate oversight, and an integrated

approach, we have expanded those

capabilities to over 27,000 gigaflops in 2002.

Similarly, basic wide area networking services

linking the laboratories over government

owned, government operated assets have

transformed into a commercial “cloud”

providing a secure, high bandwidth capability

currently reaching OC-48 on the backbone with the potential for

OC-768. The DoD processes for conducting research,

development, test, and evaluation have been redesigned to

include physics based simulation in support of, or in place of,

physical experimentation and test. This change would not have

been possible without the hardware, networks, and software

provided by the HPCMP, as well as the vigorous training and

knowledge transfer the Program has fostered. As connectivity

and computational power have improved, projects have gone
•6•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

from being single Service, single-discipline projects to Joint

Service, multi-discipline projects.

The HPCMP is a continuous critical technology insertion program.

Our mission is to deliver world-class commercial, high-end, high

performance computational capability to the DoD’s science and

technology and test and evaluation communities facilitating the

rapid application of advanced technology into superior

warfighting capabilities. Our vision is to create a pervasive

culture among DoD’s scientists and engineers where they

routinely use advanced computational environments to solve the

most demanding problems. The emphasis is on leveraging the

national trends fostered by existing Federal investments in high

performance computing and communications programs,

information technology, high performance computing research

and development programs, and other related research,

development, and test and evaluation efforts.

We are guided by five broad strategic goals to achieve that vision.

These goals, and the progress we are making in meeting them,

are discussed in detail in Section 2.

The Program consists of three components — HPC Centers,

Networking, and Software Applications Support. These

components are interdependent, supporting different technology

areas, having different contractual relationships, and having

different baseline goals. The Director provides overall guidance

and sets individual cost, schedule, and performance goals for

each project manager. The Program Office staff monitors and

evaluates the progress of each element towards meeting the

program goals. We evaluate the effectiveness of each program

component by measuring actual cost and schedule performance

versus planned cost and schedule performance and through the

measurement of actual outcomes verses planned outcomes.

Periodically, each component undergoes formal review with the

Director.

The DoD high performance computers have been used to solve

urgent problems associated with conflicts involving the US since

the program’s inception. For example, in Bosnia, the US Army

Corps of Engineers used DoD HPCs to model flooding on the

Sava River. This information allowed our forces to establish

ground positions out of harms way and showed us where to build

transportation routes across the area to maintain mobility.

Another example is the rescue of a downed pilot in Kosovo. High

performance computing-enabled simulations of Joint Search and

Rescue operations allowed the rescuers to practice the scenario

before undertaking the mission.

HPCMP GOALS

3⁄4 Provide the best commercially

available high-end HPC

capability

3⁄4 Acquire and develop jointneed HPC applications,

software tools, and

programming environments

3⁄4 Educate and train DoD’s

scientists and engineers to

effectively use advanced

computational environments

3⁄4 Link users and computers sites

via high-capacity networks,

facilitating user access and

distributed computing

environments

3⁄4 Promote collaborative

relationships among the DoD

HPC community, the National

HPC community and Minority

Serving Institutions (MSIs) in

network, computer, and

computational science
•7•

SECTION 1 — OVERVIEW AND ANALYSIS ○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○

Program Accomplishments ccomplishments

Background

In 2000, the Office of the Under Secretary of Defense

(Acquisition, Technology and Logistics) chartered an overarchingintegrated product team (O-IPT) to review the overall program

and the DoD projects it supports. The O-IPT

performed a comprehensive zero-based review

on the need for the HPCMP. The results of the OIPT review were overwhelming showing that high

performance computing is essential in developing

our future warfighting systems.

House Committee Report 107-298 DoD

Appropriations Bill, 2002, reaffirmed the need for

High Performance Computing in supporting DoD

core functions. Specifically mentioned were:

operational weather forecasting, dispersion of

airborne contaminants, armor design, design of

large aircraft or ship structures, and studies of

weapons effects.

In FY 2000, we started to reeingineer our

business process for procuring supercomputers like “Technology

Insertion-2001 (TI-01)”—to take a more corporate approach to

acquisition and to investigate the value of alternative purchasing

options for upgrades (which would be made starting in FY 2001).

In FY 2001, the acquisition plan for TI-02 continued this initiative,

incorporating lessons learned from TI-01. In this upgrade, we

negotiated integration fees in advance with the Major Shared

Resource Center (MSRC) integrators and established alternatives,

pricing, and performance through a set of benchmarks and two

sequential requests for quotes. In FY 2002, a further step in the

evolution of the technology insertion process (TI-03) eliminated

integrating contractors and the fees resulting from that process.

An interagency agreement was established with General Services

Administration (GSA) and a joint team established alternatives,

pricing, and performance through a set of system benchmarks

and two sequential Requests for Quote.

2002 Accomplishments

In 2002, the High Performance Computing Modernization

Program had over 4,000 users working on over 600 projects at

100 sites throughout the continental United States, Alaska, and

Hawaii. The user base was diverse, including representatives of
•8•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

each of the Services, several DoD agencies, industrial partners,

and academia.

The Program continued to build on the successes of past efforts

during 2002. A few highlights include:

3⁄4 The program sustained existing capability and continued

modernizing HPC systems, storage, and scientific visualization

capabilities to fulfill a significant portion of the projected

research and development, and laboratory and center HPC

requirements. Generally the program replaces 25% of the

HPC equipment annually.

3⁄4 Network services provided under

the Defense Research and Engineering

Network (DREN) Intersite Services

Contract (DISC) began to transition to

the follow-on service provider.

Operation of security systems and

enhancements continued.

Collaborative work continued with the

Federal networking community and

standards associations to ensure DREN

remains compatible with future

technology change. FY 2002 funding

reductions delayed implementation of

Virtual Private Network technology.

3⁄4 Within Software Applications

Support (SAS), development efforts in

the Common HPC Software Support

Initiative (CHSSI) program continued to

mature as some CHSSI projects were

completed and others began. The

Programming Environment and

Training (PET) effort provided

computational and computer science

support to the DoD HPC user community through interaction

and collaborative projects with academic and industrial

partners. A program was established to develop technologies

and methodologies to protect and limit end-use of high

performance computing applications software while

minimizing the burden on authorized end-users. FY 2002

funding reductions delayed some software development and

protection efforts.

Other Collaborative Accomplishments ccomplishments

Members of the HPCMP community have sought to leverage the

knowledge and experience of other federal agencies and activities
•9•

SECTION 1 — OVERVIEW AND ANALYSIS ○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○

to foster communications and to build strong and viable

supercomputing environments at the national level. Using

interagency agreements and Memoranda of

Understanding, the High Performance Computing

Modernization Program Office (HPCMPO) staff

has worked collaboratively with several of the

foremost federal supercomputing agencies,

including the National Aeronautics and Space

Administration, the National Science Foundation,

the Department of Energy’s (DOE) laboratories,

and Federally Funded Research and Development

Centers such as MIT’s Lincoln Laboratory and the

MITRE Corporation. The HPCMP community participates in

several working groups that focus on supercomputing issues and

furthers US supercomputing interests, such as the National

Coordination Office’s High End Computing and Computation

Working Group, the Large Scale Networking Coordinating

Group’s Joint Engineering Team, and the work done by several

federal agencies on the National Security Agency led

“Operational Requirements Working Group Report on High

Performance Computing Requirements for the National Security

Community”.

The HPCMP has been and continues to be involved in several

collaborative opportunities, including participating in:

3⁄4 Performance modeling and the creation of benchmarking

suites in collaboration with NSF, DOE, and NSA;

3⁄4 Review of HPCMP Challenge Project proposals by non-DoD

activities;

3⁄4 Planning activities for the HPC User

Forum and the Joint Army-NavyNASA-Air Force (JANNAF) Propulsion

Conference, which facilitates the

exchange of technical information and

coordination of research and

development of missiles, guns, and

space propulsion;

3⁄4 The Grid Computing initiative, helping identify areas for joint

collaboration, streamlining security requirements, and

establishing standards;

3⁄4 Homeland defense initiatives on critical IT infrastructure

protection;

3⁄4 The Hawaii Intranet Consortium, which seeks to share existing

resources among DoD activities and other independent

network initiatives to minimize costs; and

3⁄4 The Next Generation Internet, Fednets, and Internet 2.
•10•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

USING PROGRAM EVALUATION

Summary of Performance Data mance Data

The Director periodically meets with the project

managers to assess the program’s status. In addition,

the Joint Interoperability Test Command (JITC) provides

independent assessment of the operational

effectiveness and suitablity of the program. The

Director, with the cooperation of the project managers,

has developed both output and outcome metrics that

measure the success of the modernization effort. Key

output metrics are captured in an annual report

published by the HPCMPO, the HPC Systems Metrics

Report, which includes both usage and turnaround

times on all HPCMP shared resource systems for all

computational projects. Monthly use and turnaround

time analysis is used to adjust operational policies and

procedures to ensure that computational resources are being

delivered to DoD’s highest priority computational projects. In

addition, monitoring usage of HPCMP systems ensures their

efficient and effective use. When combined with requirements

and allocation data, this information serves as an important input

in determining the acquisition of future HPC resources.

Performance Results mance Results

The performance data discussed above is actually a subset of the

overall metrics developed and monitored by the program. In

particular, the DREN and CHSSI components actively monitor

tailored metrics appropriate to their activities. This is discussed in

detail in Section 2.

As noted in the program’s Systems Metrics Report, data from FY

2002 shows that HPCMP usage continued to grow at a rate of

almost 40% compared to FY 2001. DoD Challenge Projects

continued to use resources at an increasing rate. These projects

used almost all of their allocations for FY 2002. Usage of

Service/Agency allocations increased significantly compared to FY

2001. Efforts to ensure a more efficient matching of

requirements to resources should result in continuing increases

into future years.

In addition to Challenge Project and standard Service/Agency

workload, the HPCMP also supported several urgent and highpriority Service/Agency projects, including several related to reconstruction of the Pentagon following the terrorist attack and

anti-terrorist technology. Altogether, these high-priority and
•11•

SECTION 1 — OVERVIEW AND ANALYSIS ○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○

urgent projects amounted to approximately 100,000 hours of

computer time on various program resources and were delivered

with minimal turnaround times.

Since the total workload on HPCMP systems increased,

turnaround times on HPCMP systems grew somewhat during FY

2002. We expect a significant improvement of expansion factors

in early FY 2003 as major new HPC capabilities are added.

CONSIDERING FUTURE TRENDS

Applying Lessons Learned

HPCMP management at the centers and at the HPCMPO

routinely assess lessons learned to modify processes, to improve

acquisition strategies, and to achieve the Program vision to

“provide a pervasive culture...routinely use advanced

computational environments to solve the most

demanding problems”. Benchmarking

activities focusing on the performance of

hardware helps guide our acquisition

decisions, and we are currently analyzing key

applications codes in a systematic fashion to

guide users to the most advantageous

computing environment. Periodic reviews of

each of the Program’s components lead to

modifications in our business practices, such

as the restructuring of the PET program to

make it a program-wide resource.

Looking Ahead

As the program enters its tenth year, the

success of the initial effort promises a very

strong future. With the continued support of the Congress and

senior department leadership, the HPCMP community looks

forward to continued improvement of the Department’s

supercomputing environment and the increasing sophistication of

the user base. The DoD HPCMP community, working in concert

with other Federal agencies to identify future trends and

requirements, anticipates more extensive cooperation with those

agencies. As a result of applying lessons learned, the HPCMPO

is carefully considering restructuring a major component, the

CHSSI, to encourage user involvement and enhance its relevance

to the whole user base. The program will continue to use

synthetic benchmark methods to improve overall understanding

of our applications and model performance. These activities will
•12•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

promote the efficient and effective use of supercomputing

resources for national defense.

FINANCIAL ANALYSIS

The HPCMP is a technology program and a major defense

acquisition program (MDAP) (Acquisition Category ID) under the

cognizance of the Deputy Under Secretary of Defense for Science

and Technology. The program is included in the DoD e-business

architecture within the operational, systems, and technical views.

The program provides supercomputing services for DoD scientists

and engineers using commercial standards and protocols.

The HPCMP Director is APDP Program

Management level III certified. He uses

integrated project teams to execute

acquisition and deployment of high

performance computing assets, provides

overall guidance, and sets individual cost

schedule and performance goals for each

project manager. The Program Office

monitors and evaluates the progress each

element achieves toward meeting the

program goals.

The HPCMP funds (1) capitalization,

sustainment, and operations at the MSRCs;

(2) annual capitalization for selected DCs;

(3) wide area network services for the DoD

HPC community; (4) development of key

HPC software; and (5) expert HPC services

from leading academic institutions.

The HPCMP has multiple contracting

officers assigned in support of different efforts. The program

office uses contracting officers at the GSA in support of HPC

equipment and services purchases and uses contracting officers at

various DoD installations in support of service contracts. This

support is necessary because the program requires multiple

contracts and contract types with an ongoing need to ensure that

state-of-the-art technical capabilities are made available to DoD

scientists and engineers in a timely manner. Contracts are a

combination of firm fixed price, cost and/or indefinite delivery/

indefinite quantity. All procurement awards are made for

commercially available systems. Acquisitions are accomplished

competitively to the fullest extent possible and encourage the

inclusion of small, disadvantaged businesses and Minority Serving

Institutions (MSIs).
•13•

SECTION 1 — OVERVIEW AND ANALYSIS ○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○

FY 2002 Budget Resources: HPCMP

Appropriations Appropriations

Obligations and Costs Obligations and Costs

The financial manager conducts semi-annual reviews with each

major component manager and major field activity to review

actual cost performance against budgeted cost goals in a tailored

work breakdown structure format with special attention on

variance analysis. Significant variances are reported to the

Program Director and corrective actions taken. The Program

receives approximately $250,000,000 each year in funding

appropriated for the Department of Defense. Cash Flow during

2002 is illustrated by the following Cash Flow Statement.

High Performance Computing Modernization Program

Cash Flow Statement

October 1, 2001 - September 30, 2002

F 2 iscal Year 2002

R e evenue

Research, Development and Engineering Funding

President's Budget $188,376,000

Congressional Funding ($4,852,000)

Department of Defense Reprogramming - In $0

(Less Department of Defense Reprogramming - Out) ($4,110,000)

Net Research, Development and Engineering Funding $179,414,000

Procurement Funding

President's Budget $50,763,000

Congressional Funding $25,000,000

Department of Defense Reprogramming - In $0

(Less Department of Defense Reprogramming - Out) ($452,000)

Net Procurement Funding $75,311,000

N e et Revenue $ 0 254,725,000

E e xpense

Major Shared Resource Center Operations $66,463,215

Major Shared Resource Center Upgrades $40,536,779

Distributed Center Operations $23,908,600

Distributed Center Upgrades $34,774,221

Defense Research and Engineering Network $31,592,700

Software Initiatives $57,449,486

N e et Expense $ 0 254,725,000

B 0 alance (As of September 30, 2002) $
•14•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

Since the program began in 1994, validated requirements have

always exceeded the computing capability available to address

those requirements. This occurs: 1) because the use of science

based models and simulations to answer research questions and

solve engineering problems has dramatically grown; and 2)

because fully funding the HPC requirement is unaffordable given

the entire scope of activities the DoD budget must address. While

fiscal resources do not fully meet the computational requirements

of the science and technology and test and evaluation

communities, the returns’ provided are substantial and are

allocated to the highest priority projects. The program has

consistently met approximately half of the validated requirements

because of major cost performance improvements in computer

technology since 1994. These shortfalls can be seen in the

following 2002 Income Statement.

Financial Trends

In FY 2003 sustainment of capability

and the continued modernization of

HPC systems, storage, and scientific

visualization capabilities will be

greatly improved at the MSRCs

through new technical services

contracts. We plan to acquire or

upgrade systems at three to five

DCs. Network services provided

under the American Telephone and

Telegraph contract will fully transition

to MCI WorldCom, Inc., providing

substantially high services for

essentially the same cost.

Development of shared scalable

applications supporting software will

continue. The DoD HPC user

community will be supported by PET

efforts. The Software Protection

Initiative will continue to mature.

The following chart breaks out

planned spending program-wide during 2003.

Finally, as stated earlier, the program requires multiple contracts

and contract types. The charts on page 16 display spending by

vendor in 2002 and planned spending by vendor in 2003. There

is an anomaly in the Site Preparation, Infrastructure and

Acquisitions in Negotiation category caused by continued

negotiations affecting final award of the congressionally added
•15•

SECTION 1 — OVERVIEW AND ANALYSIS ○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○

High Performance Computing Modernization Program

Income Statement

October 1, 2001 - September 30, 2002

Note 1: Note 1 Expenses include travel; supplies; government and contractor salaries and training; maintenance of hardware and software; studies and

analysis; annual operations investments; communications, utilities, facilities lease and facilities maintenance.

Note 2: Note 2: Software initiatives are separated into 2 distinct categories - expenses associated with research and development, management,

education/training and expert services; and capitol assets resulting from developed software.

Note 3: Note 3: Depreciation for HPC hardware is calculated using a 42 month straight-line depreciation method. Current HPC technology

development results in predictable obsolescence. Generally after 42 months of use, HPC systems are retired with little or no residual value. Fiscal

year 2002 depreciation includes the 12 month value calculated for all systems in the inventory between October 1, 2001 through September 30,

2002.

Note 4: Note 4: Depreciation for HPC software is calculated using a 60 month straight-line depreciation method. A period of 60 months is used

because it is the typical life cycle of HPC software before significant modifications are required. Fiscal year 2002 depreciation includes the 12

month value calculated for all software in the inventory between October 1, 2001 through September 30, 2002.

Note 5: Annual program Note 5: investments in system hardware have not been made at levels sufficient to maintain stable equipment inventories. For

several years depreciated values have not been offset by new assets.

F r 2 iscal Year 2 2 00

I e ncom

R g esearch, Development and Engineering Funding

Major Shared Resource Center Operations $66,463,215

Distributed Center Operations $23,908,600

Defense Research and Engineering Network $31,592,700

Software Initiatives $57,449,486

P g rocurement Funding

Major Shared Resource Center Upgrades $40,536,779

Distributed Center Upgrades $34,774,221

Defense Research and Engineering Network $0

Software Initiatives $0

T e otal Income $ 0 254,725,000

E e xpense

R g esearch, Development and Engineering Funding

Major Shared Resource Center Operations $66,463,215

Distributed Center Operations $23,908,600

Defense Research and Engineering Network $31,592,700

Software Initiatives $37,096,224

D r e ep s c iation of Capital Assets

Hardware $88,508,286

Software $19,711,952

T e 2 otal Expense $2 7 6 7,280,977

Balance (As of September 30, 2002) (See Note 5) -$12,555,977
•16•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

funding for the Army High Performance Computing Research

Center (AHPCRC).

The program continues to deploy, sustain, and

upgrade commercially available high

performance computing environments and

networking services in support of DoD

laboratories and test facilities. The ensuing

activities of the HPCMP have substantially

improved the Department’s computational

capabilities with the objective of providing the

DoD the technology to ensure dominance on the

battlefield by the early fielding of the most

advanced computing capability available.
○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○

SECTION 2

PERFORMANCE

RESULTS

○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○

•19•

SECTION 2 — PERFORMANCE RESULTS ○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○

SECTION 2 — PERFORMANCE RESULTS

PROGRESS TOWARD STRATEGIC GOALS

AND OBJECTIVES

FY 2002 PERFORMANCE

One of the major goals and activities of the HPC Modernization

Program is to continually upgrade HPC capabilities at its four

MSRCs. The technology insertion process (TI-XX, in which “XX” is

the fiscal year) annually requests proposals

for upgrades and new HPC systems from

each HPC hardware vendor; evaluates

their responses; and provides

recommendations to senior

management on which HPC systems to

acquire for that particular year. The

evaluation of responses to hardware

proposals considers several major

categories, including system usability,

confidence in the proposing vendor,

performance, and price.

Performance evaluation of HPC

systems is based on a set of

benchmark codes, which are in

turn, firmly based on user

requirements. These requirements

determine the set of application

benchmark codes and test cases,

and these are re-evaluated each

year. In addition to this set of

application benchmark codes,

the benchmark suite also

contains a robust set of synthetic

benchmarks, which are

designed to independently test

performance of HPC systems across several important

system attributes, such as central processing unit performance,

Goal 1: Provide the best commercially available high-end HPC capability
•20•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

memory performance, and input/output performance. These

synthetic benchmarks provide information on how HPC systems

are likely to perform on future applications, while the set of

application benchmarks is designed to provide performance on

current and projected HPCMP workload. Together they are used

to create an overall performance score for each HPC system

proposed for acquisition.

Performance and price/performance are considered as separate

evaluation criteria; each is important in its own right. Overall

price/performance is determined for a proposed set of HPC

system acquisitions with the help of an optimization procedure

which takes both price and performance into

account on individual application test cases. This

optimizer produces an overall price/performance

score for each proposed set of alternative systems

by shifting workload among proposed systems to

maximize overall performance for a specified total

acquisition cost. In addition, constraints are

placed on the optimizer such that the overall

application performance is spread among

individual application test cases according to the

desired percentage of total performance on these

cases as determined by user requirements. This

overall price/performance score for each proposed

set of alternatives, in addition to the total

performance score for the entire set, is one of the

major criteria used to make final acquisition

decisions. The strong dependence of these decisions on explicit

user requirements, as reflected by the benchmark suite, ensures

that new HPC capability will be used efficiently and effectively by

our user base.

Since the average useful life of any given HPC system is

approximately four years, we continually use our technology

insertion process and emerging benchmarks as the basis for

hardware acquisitions. During 2002 we funded three large

Compaq HPC systems with a total of 1,504 processors and a 256

processor Origin HPC system from SGI. Through equipment

consolidation, we also created the world’s largest Cray T3E

(1,888PE) at the Engineer Research and Development Center.
•21•

SECTION 2 — PERFORMANCE RESULTS ○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○

The Program is acquiring and developing joint-need HPC

applications, software tools, and programming environments

through the Software Applications Support (SAS) component. SAS

consists of three facets: the Common High Performance

Computing Software Support Initiative (CHSSI), the Programming

Environment and Training (PET), and the Software Protection

Initiative. CHSSI and PET enable HPCMP

users to accomplish their work more

efficiently. CHSSI participants redesign legacy

software to run on new computer

architectures. Software that is scalable,

portable, and accurate helps the

Department’s scientists and engineers take

full advantage of the new computer systems’

significant capabilities. Through PET, the

program’s systems are made more userfriendly and users are afforded training and

tools to be more productive.

CHSSI funds and oversees in-house

development of militarily significant scientific

and engineering codes. The CHSSI Project

Manager acts as executive agent, ensuring

high priority, technically feasible projects are

selected and effectively managed to reduce

risks inherent in such software development efforts. The

HPCMPO solicits proposals annually from the Service and Agency

science and technology and test and evaluation executives who

submit their proposals prioritized for military relevance. Subject

matter experts rate the proposals for technical merit and limited

risk. The highest ranking proposals, which do not duplicate or

overlap existing efforts, are selected subject to the availability of

funds. Projects selected typically have short (three years or less)

development periods; each project’s funding averages $500,000

annually.

The PET initiative enables the DoD high performance computing

user community to make the best use of the computing capacity

the HPCMP provides and to extend the range of DoD technical

problems solved on HPC systems. PET is enhancing the total

Goal 2: Acquire and develop joint-need HPC applications, software tools,

and programming environments

Goal 3: Educate and train DoD’s scientists and engineers to effectively

use advanced computational environments
•22•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

capability and productivity of the program’s user

community through training, collaboration, tool

development, software development support,

technology tracking, technology transfer, and outreach

to users. PET is responsible for gathering and

deploying the best ideas, algorithms, and software

tools emerging from the national high performancecomputing infrastructure into the DoD user community.

PET accomplishes its goals through three approaches:

3⁄4 One-on-one software programming support to

users

3⁄4 Development and transfer of new computational

technologies to DoD

3⁄4 Formal and informal training

Since the DoD HPC community is widely dispersed, the program

uses the Defense Research and Engineering Network (DREN) to

link users to each other and to the HPC centers. High capacity

network connectivity between sites and users is critical. Since its

inception, DREN has evolved from a full mesh DS-3 network to

an OC-48 based cloud supporting asynchronous transfer mode

(ATM), internet protocol (IP), and MPLS services.

DREN provides these services to a variety of

clientele throughout the Department.

Activities in 2002 were focused on positioning

ourselves for the next 10 years. 2003 is the year

that DREN is moving from an ATM backbone to

MPLS (multi-protocol layer switching). In

January 2003, we accepted delivery of a new

high capacity network. Over the next year, we

will transition all sites from the old DREN ATM

backbone to this new DREN MPLS backbone.

The contract for services is an IDIQ firm-fixedprice vehicle with a 3-year base and 7 one-year

options, with a ceiling of $450 million. This

contract will provide both IP and ATM services

over the core and will have jumbo frame

capacity, IPv6, multi-cast, and a number of other

Goal 4: Link users and computers sites via high-capacity networks,

facilitating user access and distributed computing environments
•23•

SECTION 2 — PERFORMANCE RESULTS ○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○

capabilities that will be important in the future. There are also

provisions for light wave capabilities as soon as this new

technology stabilizes.

Our resources and our community of users and stakeholders

have continuously evolved since the program’s inception. One

component of the program has been our outreach activities.

Outreach raises the Department’s awareness of the support

available through the program. It promotes the appropriate use

of HPCMP resources within the DoD and provides a conduit for

exchange of scientific and engineering technical information

between and among the Services, Agencies, and other Federal

agencies. Outreach also encourages our national infrastructure

and youth to become more proficient in the high technology

scientific and engineering disciplines needed to maintain US

technical superiority. Outreach provides the stakeholders and

taxpayers with a means of understanding what Congressional

investment in HPC supports.

Equally important, Outreach activities give the HPC user

community a means of providing feedback to the Program’s

managers and decision makers. Through site visits, new user

Goal 5: Promote collaborative relationships among the DoD HPC

community, the National HPC community and Minority Serving Institutions

(MSIs) in network, computer, and computational science
•24•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

surveys, conferences, advisory groups, and independent

assessments, the program benefits from the observations,

recommendations, and criticisms it receives.

We promoted a vigorous outreach program in FY 2002. The High

Performance Computing Modernization Program Office

participated in twelve international, national, and regional

conferences throughout the United States, including those

sponsored by the Armed Forces Communications and

Electronics Association, the National Defense Industrial

Association, the American Institute of Aeronautics and

Astronautics, and the Institute of Electrical and

Electronic Engineers. Individual centers were routine

participants in similar conferences and symposia,

raising the awareness of the Department, its academic

and industrial partners, and its allies to the potential of

high performance computing.

Each center has an ongoing outreach program that

addresses the needs of their community and potential

customers. As a Program, our centers have reached

out to Service Academies and institutions of higher

learning to familiarize more of our nation’s future

leaders with the potential of high performance

computing. Several cadets participate in internship

programs at our centers, including Maui and Alaska.

Similarly, the Army’s AHPCRC in Minneapolis, MN,

runs a summer internship program, allowing students

from Minority Serving Institutions an opportunity to

gain hands on experience with HPCMP resources and

methodologies. Annually, students attend the intensive

program, which emphasizes the use of simulation and

modeling, parallel computing, and graphics and

visualization to solve real world problems of interest to

the Army. The Institute’s purpose is not only to train students in

these areas but also to encourage them, especially women and

minorities, to pursue graduate studies or careers and HPC.”1

1 AHPCRC web site http://www.arc.umn.edu/education/summer-inst.html
○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○

○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○

SECTION 3

MANAGEMENT

ACCOMPLISHMENTS

AND CHALLENGES

•27•

SECTION 3 — MANAGEMENT ACCOMPLISHMENTS AND CHALLENGES ○○○○○○○○ ○○○○○○○○○○○○○○○○○○○○○○○○○○○○

SECTION 3 — MANAGEMENT

ACCOMPLISHMENTS AND CHALLENGES

ACCOMPLISHMENTS

Contract Awards

The following contracts were awarded by our organization in

2002:

In addition, we worked towards award of the following contracts

in early 2003:

C t ontract

T t arge

(Millons)

C g eiling

(Millons)

HPC Design:

Instrumental, Inc., Bloomington, MN

GS04T02DEA0030, FFP and T&M

Award: October 31, 2002 $53.3 $53.3

DREN:

WorldCom Gov't Mkts, McLean, VA

DCA200-02-R-5008, IDIQ FFP

Award: April 4, 2002

Definitized: June 6, 2002 $ 0 200.0 $275.

ARSC Support Services:

University of Alasksa, Anchorage, AK

IDIQ Task Order, Cost Reimbursable

Award: October 14, 2002

D 5 efinitized: October 14, 2002 $ 5 49. $49.

C t ontract

T t arge

(Millons)

C g eiling

(Millons)

ARL MSRC Support Services:

Raytheon, Aberdeen, MD

GSA FTS Mill GWA, CPFF

Award: January 31, 2003 $90.0 $90.0

ASC MSRC Support Services:

Computer Sciences Corp., Wright-Patterson AFB, OH

GSA FTS Mill GWA, CPFF

A 2 ward: January 31, 2003 $ 2 52. $52.

GSA:

Multiple Awards, Multiple

A 9 ward: January 15, 2003 $ 9 46. $46.
•28•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

HPC Centers HPC Centers

During 2002, we were able to significantly increase the

computational capabilities at the four Major Shared Resource

Centers (MSRC) by nearly 48%. In actual values, the MSRCs

added additional systems that provide 4.8 trillion floating-point

operations during the year. Also during 2002, the

HPCMP conducted a new procurement for the four

follow-on integration contracts at the MSRCs. The

awards are to be announced in 2003.

FY 2002 saw the implementation of new policies

concerning the Distributed Centers. Recognizing

that distributed centers had historically been

awarded equipment to meet the needs of a specific

project or mission need, we reviewed the existing

centers to determine if that need had been

successfully addressed. Where feasible, centers

that had satisfied their schedule, cost, and

requirements goals were released from their formal

association with the program, thus lessening their

reporting and oversight burden. By September

2002, nine centers had made the transition. The

remaining centers were categorized to reflect their

actual relationship to the program – as “allocated”

or “dedicated” distributed centers. In both cases,

the allocated and dedicated distributed centers continue to

receive HPC resources funded through the HPCMP.
•29•

SECTION 3 — MANAGEMENT ACCOMPLISHMENTS AND CHALLENGES ○○○○○○○○ ○○○○○○○○○○○○○○○○○○○○○○○○○○○○

Networking and Security

In April 2002, we awarded a Firm Fixed-Price (FFP), Indefinite

Delivery/Indefinite Quantity (IDIQ) contract to MCI WorldCom

Communications, Inc. The contract is a follow-on contract to the

DREN Intersite Services Contract (DISC) and provides state-ofthe-art WAN services to support our program and DoD users in

various technical communities. As part of the process of

transitioning this contract to MCI, a comprehensive

implementation and transition plan was completed in July 2002.

A Comprehensive Security Assessment (CSA) was completed on

the new network in December 2002. As a result of this

assessment, the Designated Approving Authority

granted MCI WorldCom an interim authority to

operate in January 2003. After the CSA, Initial

Performance Capability (IPC) tests were completed

on the new backbone. The IPC tests required the

new carrier to set up 10 Service Delivery Points

(SDPs) and then prove that those SDPs and the

network backbone met the requirements for

availability, latency, and throughput. The tests were

successful and the backbone was accepted for

service in January 2003.

As a result of switching to the new backbone, OC12 gateways for IP and ATM traffic were activated in

Los Angeles, California, and Washington, DC in

December 2002. This allowed the exchange of

network traffic between the old DREN backbone and

the new DREN backbone.

A technology insertion for a one box solution at

SDPs with nine DREN core nodes was completed

and approved in September 2002. The single IP/

MPLS box allows the integration of ATM and IP

traffic to be sent across a single connection to their

core. This approach maximizes the use of MPLS

label switch pads to provide virtual networks for the

segregation of DREN traffic.

A new port mirroring technique was engineered that allows a

single intrusion detection device to monitor multiple paths and

interfaces at internet access points. Peering services between

NIPRNET and DREN have been upgraded to a 100Mbps

connection.
•30•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

Software Applications Support Software Applications Support

In FY 2002, eight new projects were selected from thirty-six

proposals submitted by the Services and Agencies. These new

projects will begin development in FY 2003 and are expected to

provide the DoD scientific and engineering

communities with critical software to support

chemical and biological defense (identification,

transport, and containment), electronic battlefield

environment planning and simulations, and the

design of new materials.

From CHSSI’s inception to the end of FY 2002, 51

scientific and engineering code development efforts

were completed. In FY 2002 alone, CHSSI project

leaders completed 12 projects and released 19

codes to the user community.

CHSSI’s return on investment is the capability of our

scientists and engineers to effectively tackle more

and more complex problems and deliver

technological solutions demanded to support today’s military.

This annual report will focus on one facet of CHSSI’s contributions

to the DoD - modeling and simulation. CHSSI codes have helped

improve the speed, complexity and accuracy of military

simulations - materials, space and earth weather, littoral

environments, weapons systems, and simulations for the

battlefield. CHSSI codes allow engineers to

model and test weapons stores and how they

separate from aircraft before the stores are tested

in actual flight. CHSSI codes provide accurate

models of blast effects and contaminant

transport. Codes released in FY 2002 will be

used for space-weather forecasting and

nowcasting; modeling radar-based sensing of

surface and subsurface targets, including land

mines, unexploded ordnance, and vehicles;

modeling of 3-D rectangular arrangements such

as the pulsed plasma micro-thruster for

microsatellite propulsion; modeling and

simulating large-scale military communications

and tactical signal intelligence platforms; weather

forecasting model improvements; and simulating

large scale, heterogeneous, communication

networks.

PET’s focus was placed on the ten HPCMP

computational technology areas (CTA) and five

PET cross-cutting areas (Computational
•31•

SECTION 3 — MANAGEMENT ACCOMPLISHMENTS AND CHALLENGES ○○○○○○○○ ○○○○○○○○○○○○○○○○○○○○○○○○○○○○

Environment, Enabling Technologies, Collaborative Distance

Learning Technologies, Online Knowledge Center, and Education,

Outreach, and Training Coordination) with a broad HPCMP-wide

management approach.

During FY 2002, PET supported 52 training events (including

several workshops and seminars), with an average attendance of

15 students. Evaluations of formal classes in the latter portion of

FY 2002 averaged scores of approximately 4.4 on a scale of 0 to

5. It is anticipated that FY 2003 will see a marked increase in the

number of such events, as 30 classes are planned for the first

quarter. These events, as well as the many thousands of hours of

one-on-one user programming support, were provided by

leading scientists drawn from many of our leading universities.

PET has also provided significant

outreach to, and training of,

students and faculty at MSIs,

including summer institutes, summer

intern programs, and faculty/

graduate student participation in PET

development projects. During FY

2002, ten MSIs participated in PET.

Also in FY 2002, PET supported 40

high performance computing

software and algorithm development

projects, mostly university-based,

encompassing the 10 CTAs and 4 of

the 5 cross-cutting areas,

representing a value of

approximately $4.5M, with the intent

of transferring the best available

applicable high performance computing software technologies to

the DoD. Among these projects was the development and

deployment of a HPCMP-wide On-line Knowledge Center (OKC).

The OKC provides information on all aspects of PET, including

delivered reports, sources of user help, a training calendar, and

class registration. While still in the process of being populated

with PET information, the OKC will be the source of all PET

information in the future.

One of the major challenges of DoD using, to the fullest extent

possible, modern high performance computing platforms is the

development of complex physics-based software that can make

use of tens to thousands of commodity processors to achieve the

performance that these platforms provide and DoD requires. PET

is facing this challenge with a robust, multi-faceted program,

making use of the best intellectual talent available.
•32•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

PROGRAM OVERSIGHT

In May 2002, the HPCMP was designated a major defense

acquisition program (MDAP) under the oversight cognizance of

the OUSD(AT&L). With this designation came the requirement for

submitting the Consolidated Acquisition Reporting System (CARS)

documentation, which the HPCMPO has complied with. During

the previous eight years of its existence, the DoD HPCMP was

designated a major automated information system (MAIS)

program under the oversight cognizance of the Assistant Secretary

of Defense (Command, Control, and Computers). The HPCMP

has successfully met all documentation preparation requirements

and passed all major milestone decisions (Milestone 0–Post

Milestone II) required under the Office of the Secretary of Defense

oversight. In addition, the HPCMP has submitted the required

Clinger-Cohen Act (CCA) documentation to support a program

certification in FY 2003. The Director, Operational Test and

Evaluation has continuously provided input and oversight to the

HPCMP test processes and execution as well as administer and

analyze user satisfaction with the HPCMP resources. The JITC has

found that the HPCMP “remains Effective & Suitable for its charter

to ‘provide High Performance Computing to the DoD S&T and T&E

communities’”.2

2 HPCMPO/JITC 2002 Test Activities Outbrief, JITC HPC Test Team, Sept. 2002
○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○

○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○

SECTION 4

FINANCIAL

STATEMENT

•35•

SECTION 4 — FINANICAL STATEMENT ○○○○○○○ ○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○

SECTION 4 — FY 2002 ANNUAL

FINANCIAL STATEMENTS

The HPCMP evaluates the effectiveness of each program

component by measuring actual cost and schedule performance

versus planned cost and schedule performance and through the

measurement of actual outcomes verses planned outcomes. The

MSRC contractors submit several reports regularly including a

monthly and quarterly cost performance report (CPR) and

quarterly contract funds status report (CFSR). Each contract

specifies, as a deliverable, a work breakdown structure (WBS) to

facilitate the on-going review of smaller task components. Cost/

schedule status reports are one of the primary tools used for

oversight management of the MSRCs.

The HPCMP has provided a significant supercomputing capability

to the Department of Defense scientific community. The balance

sheets on the next two pages show the cumulative value of the

program.
•36•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

High Performance Computing Modernization Program

Balance Sheet

As of March 31, 2003

(1) Research, Development and Engineering Funding used to develop inventory software.

(2) Office of Management and Budget Circular A-11, Section 300 - Planning, Budgeting, Acquisition, and Management of Capital

Assets, (Paragraph 300.4), defines capital assets as land, structures, equipment, intellectual property (e.g., software), and information

technolgy (including IT service contracts) that are used by the Federal government and have an estimated useful life of two years or

more. Therefore, manpower is treated as a capital asset.

(3) Small consumable items such as computer tapes and supplies are considered as expense items and not carried as inventory items.

A y ssets and Equity

Hardware

Less: Depreciation

$882,616,000

Fiscal Year 1994-1999

Fiscal Year 2000:

Fiscal Year 2001:

Fiscal Year 2002:

Fiscal Year 2003:

$582,372,505

$68,125,714

$34,036,286

$10,758,714

$0 $187,322,781

Software (1)

Less: Depreciation

$117,029,993

Fiscal Year 1994-1997:

Fiscal Year 1998:

Fiscal Year 1999:

Fiscal Year 2000:

Fiscal Year 2001:

Fiscal Year 2002:

Fiscal Year 2003:

$45,801,067

$17,789,985

$15,667,400

$9,509,220

$5,222,404

$2,035,326

$0 $21,004,591

Manpower Contracts (2 & 3)

Software Development

Exercised Contract Value $20,540,146

Less: Value Consumed

Remaining Exercised Value

$18,141,285

$2,398,861

Maintenance Contract (2 & 3)

Hardware Maintenance

Fiscal Year 2003:

Less: Value Consumed

Hardware Maintenance

Fiscal Year 2003:

Software Maintenance

Fiscal Year 2003:

Less: Value Consumed

Software Maintenance

F : iscal Year 2003

$26,037,825

$22,225,871

$10,000,169

$ 5 8,406,697 $5,405,42

Intellectual/Operations

Government Labor

Fiscal Year 2003:

Less: Value Consumed

Government Labor

Fiscal Year 2003

Contract Labor

Fiscal Year 2003:

Less: Value Consumed

Contract Labor

Fiscal Year 2003:

$18,023,028

$16,227,846

$78,947,917

$ 8 68,534,770 $12,208,32

T y otal Assets and Equity $ 6 228,339,986

L s iabilities

Uncompleted Software

Development $2,398,861

Maintenance Contract Liabilities

March 31, 2003

Hardware

Fiscal Year 2003:

Software

Fiscal Year 2003:

$3,811,953

$1,593,472

Intellectual/Facilities Expenses

March 31, 2003

Government Labor

Fiscal Year 2003:

Contract Labor

Fiscal Year 2003:

$1,795,181

$10,413,147

T s 4 otal Liabilities $ 4 20,012,614

T y otal Program Equity $ 2 208,327,372

T y otal Liability and Program Equity $ 6 228,339,986
•37•

SECTION 4 — FINANICAL STATEMENT ○○○○○○○ ○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○

High Performance Computing Modernization Program

Cash Flow Statement

October 1, 2001 - September 30, 2002

F 2 iscal Year 2002

R e evenue

Research, Development and Engineering Funding

President's Budget $188,376,000

Congressional Funding ($4,852,000)

Department of Defense Reprogramming - In $0

(Less Department of Defense Reprogramming - Out) ($4,110,000)

Net Research, Development and Engineering Funding $179,414,000

Procurement Funding

President's Budget $50,763,000

Congressional Funding $25,000,000

Department of Defense Reprogramming - In $0

(Less Department of Defense Reprogramming - Out) ($452,000)

Net Procurement Funding $75,311,000

N e et Revenue $ 0 254,725,000

E e xpense

Major Shared Resource Center Operations $66,463,215

Major Shared Resource Center Upgrades $40,536,779

Distributed Center Operations $23,908,600

Distributed Center Upgrades $34,774,221

Defense Research and Engineering Network $31,592,700

Software Initiatives $57,449,486

N e et Expense $ 0 254,725,000

B 0 alance (As of September 30, 2002) $
•38•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

High Performance Computing Modernization Program

Income Statement

October 1, 2001 - September 30, 2002

F r 2 iscal Year 2 2 00

I e ncom

R g esearch, Development and Engineering Funding

Major Shared Resource Center Operations $66,463,215

Distributed Center Operations $23,908,600

Defense Research and Engineering Network $31,592,700

Software Initiatives $57,449,486

P g rocurement Funding

Major Shared Resource Center Upgrades $40,536,779

Distributed Center Upgrades $34,774,221

Defense Research and Engineering Network $0

Software Initiatives $0

T e otal Income $ 0 254,725,000

E e xpense

R g esearch, Development and Engineering Funding

Major Shared Resource Center Operations $66,463,215

Distributed Center Operations $23,908,600

Defense Research and Engineering Network $31,592,700

Software Initiatives $37,096,224

D r e ep s c iation of Capital Assets

Hardware $88,508,286

Software $19,711,952

T e 2 otal Expense $2 7 6 7,280,977

Balance (As of September 30, 2002) (See Note 5) -$12,555,977

Note 1: Note 1 Expenses include travel; supplies; government and contractor salaries and training; maintenance of hardware and software; studies and

analysis; annual operations investments; communications, utilities, facilities lease and facilities maintenance.

Note 2: Note 2: Software initiatives are separated into 2 distinct categories - expenses associated with research and development, management,

education/training and expert services; and capitol assets resulting from developed software.

Note 3: Note 3: Depreciation for HPC hardware is calculated using a 42 month straight-line depreciation method. Current HPC technology

development results in predictable obsolescence. Generally after 42 months of use, HPC systems are retired with little or no residual value. Fiscal

year 2002 depreciation includes the 12 month value calculated for all systems in the inventory between October 1, 2001 through September 30,

2002.

Note 4: Note 4: Depreciation for HPC software is calculated using a 60 month straight-line depreciation method. A period of 60 months is used

because it is the typical life cycle of HPC software before significant modifications are required. Fiscal year 2002 depreciation includes the 12

month value calculated for all software in the inventory between October 1, 2001 through September 30, 2002.

Note 5: Annual program Note 5: investments in system hardware have not been made at levels sufficient to maintain stable equipment inventories. For

several years depreciated values have not been offset by new assets.
○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○

○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○

APPENDICIES

SECTION 5

•41•

SECTION 5 — APPENDICIES ○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○○

SECTION 5 — APPENDICIES

APPENDIX A – PROGRAM EVALUATIONS

The Joint Interoperability Test Command (JITC), as the

independent test agency, has responsibility for performing the

oversight for the HPCMP test and evaluation activities. The test

requirements include system acceptance testing at the MSRCs and

Allocated DCs, System Acceptance Tests and full operational

capability assessment for the Dedicated DCs, and software testing

for the software initiative.

To support the ongoing operational assessment of the program,

the JITC team reviews test plans, witnesses test activities, reviews

site’s test reports, and writes the independent test report for the

activity. They also administer the HPCMP User Satisfaction Survey

and conduct user interviews. The team collects information from

each center, its users, and in many cases, the Service/Agency

Approval Authorities in order to fulfill this requirement.

Performance Evaluation Assessment mance Evaluation Assessment

Review

In addition to the testing activities, each Distributed Center that

receives new funding is required to undergo a Performance

Evaluation Assessment Review. These reviews are part of the

required HPCMP review process to evaluate HPCMP information

technology (IT) investments under the Government Performance

Results Act and Clinger-Cohen Act. These reviews were

established in 2002 and superceded the previous PostDeployment Evaluation and Assessment Process.

The first set of these reviews was held on June 11, 2002, as part

of the HPCMP Users Group Conference (UGC) for sites that

received funding in FY 2000–2001 and FY 2002. A

representative of a major project which served as the justification

for funding each DC was required to give a technical

presentation showing how the DC was used to satisfy the project’s

computational requirements. The DC HPCMP Technical

Evaluation Panel evaluated the presentation to determine whether

the DC enabled the project to meet the goals which had been

proposed for the DC.
•42•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

FY 2002 awarded centers were required to give a milestone chart

as part of the technical presentation to show where the site/

project was in the acquisition process (i.e., funding received,

obligation of funds, acquisition, installation, etc.).
•43•

SECTION 5 — APPENDICIES ○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○○

APPENDIX B – HPCMP INTEGRATED

PROGRAM STRATEGY
•44•

HPC MODERNIZATION PROGRAM ANNUAL REPORT ○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○

2-D two-dimensions (dimensional)

3-D three-dimensions (dimensional)

AHPCRC Army High Performance Computing Research Center

APDP Acquisition Professional Development Program

ATM asynchronous transfer mode

CFSR contract funds status report

CHSSI Common High Performance Computing Software Support Initiative

CIO Chief Information Officer

CPR cost performance report

CSA Comprehensive Security Assessment

CTA Computational Technology Area

DC Distributed Center

DISC DREN Intersite Services Contract

DoD Department of Defense

DOE Department of Energy

DREN Defense Research and Engineering Network

FFP Firm Fixed-Price

FOC full operational capability

FY fiscal year

GSA General Services Administration

HPC high performance computing or high performance computer

HPCMP High Performance Computing Modernization Program

HPCMPO High Performance Computing Modernization Program Office

I2 Internet 2

IDIQ Indefinite Delivery/Indefinite Quantity

I/O input/output

IP Internet protocol

IPC Initial Performance Capability

IT information technology

JANNAF Joint Army-NASA-Air Force

JITC Joint Interoperability Test Command

Mbps million bits per second

MDAP major defense acquisition program

MPI Message Passing Interface

MPLS multi-protocol layer switching

APPENDIX C – ACRONYMS AND ABBREVIATIONS
•45•

SECTION 5 — APPENDICIES ○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○○ ○○○○○○○○○○○○○○○○○○○○

MSI Minority Serving Institutions

MSRC Major Shared Resource Center

NSA National Security Agency

NSF National Science Foundation

O-IPT overarching-integrated product team

OKC On-line Knowledge Center

OSD Office of the Secretary of Defense

PET Programming Environment and Training

SAS Software Applications Support

SDPs service delivery points

TI-01 Technology Insertion-01

TI-02 Technology Insertion-02

UAG User Advocacy Group

US United States

WAN wide area network

WBS work breakdown structure


Department of Defense

High Performance Computing Modernization Office

1010 North Glebe Road, Suite 510

Arlington, VA 22201

Phone: (703) 812-8205

Facsimile: (703) 812-9701

http://www.hpcmo.hpc.mil