Modernization Plan 2002

High Performance Computing

Tools for the 21st Century

DoD HPC

MODERNIZATION

PROGRAM
Approved for public release; distribution is unlimited.

For more information about the DoD HPC Modernization Office and the DoD HPC

Modernization Program, visit our Web site at http://www.hpcmo.hpc.mil

Front cover captions (clockwise starting from top left):

An Air Force Pararescueman with the 38th Rescue Squadron, Moody Air Force Base,

Georgia, makes a static jump from an HC-130 over the B-16 range at Fallon Naval Air

Station, Nev., June 19, 2001 (U.S. Air Force photo)

Guided MLRS pressure contours and particle traces (DoD Mission Success Stories 2000

pg. 98)

An HH-60 helicopter heads back to home station after successfully hitting a simulated

target at range B-17 during a hellfire demonstration for the exercise June 22, 2001 (U.S.

Air Force photo)

The U.S. Navy Arleigh Burke class guided missile destroyer USS Cole (DDG 67) taken

September 14, 2000, approximately one month before being attacked by a terrorist-suicide

mission during the early morning hours of October 12th, 2000 (U.S. Navy photo)

DoD Mission Success Stories 2000 pg. 117

This figure shows the results of a calculation of an incendiary payload burning in a

structure containing a BW agent simulant. (The walls of the structure facing the viewer are

not shown.) The colored circles represent solid products from the combustion of the

incendiary, where the color indicates the temperature of each particle. The colored

contours plotted on solid surfaces show the air temperature adjacent to the surface. The

black arrows indicate velocity plotted on a plane intersecting the computational domain.

(DoD Mission Success Stories 2000 pg. 109)
DEPARTMENT OF D EFENSE

High Performance Computing

Modernization Program

MODERNIZATION PLAN 2002

February 2002


EXECUTIVE SUMMARY

Under the auspices of the Deputy Under Secretary of Defense (Science and Technology), the

High Performance Computing Modernization Program continues to focus on acquiring, managing,

and sustaining modern commercial high performance computing (HPC) resources. High

performance computing resources allow the United States to develop and deploy superior

weapons, warfighting capabilities, and mission support systems.

The 2002 High Performance Computing Modernization Plan includes a program overview,

several recent examples of defense projects successfully employing high performance computing

resources, and a section on program accomplishments and future plans, with an emphasis on

fiscal years 2002 and 2003.

The program’s overarching goal continues to be the exploitation of high performance

computing technology for military advantage. The strategy remains to acquire and sustain worldclass high performance computing and network capabilities to support defense scientists and

engineers. The program supports over 600 high performance computing projects, and over 4,300

scientists and engineers located at more than 100 Department of Defense (DoD) laboratories,

universities, test centers, and industrial sites.

The program puts advanced technology in the hands of U.S. forces quicker, cheaper, and with

greater certainty of success. Today’s weapons programs, such as the Joint Strike Fighter,

Unmanned Aerial Vehicles, the Bradley Tank, and the Avenger Missile Program have benefited

from HPC. Benefits include innovative materials, advanced design concepts, improved designs,

more accurate simulations, and more efficient tests. High performance computing is a key factor

in many major DoD acquisition programs. The use of high performance computing technology

reduces the costs and time required for systems analysis, design, development, test, and

deployment; helps avoid environmental damage; and improves the integration and effectiveness of

complex weapons systems. As DoD continues to reform and reengineer its acquisition processes,

high performance computing assets, along with high-fidelity scalable models and simulations,

reduce the number of expensive prototypes. Modeling and simulation are being used to explore

detailed design options and to identify testing priorities earlier in the design cycle.

Future weapons systems, such as high power microwave weapons, are benefiting through

basic research. Plasma physics, molecular engineering, high-energy materials, and advanced

signal processing are some of the challenges currently addressed by the program.

3

Contents

Executive Summary ......................................................................................... 1

Program Mission and Vision .............................................................................. 7

PROGRAM OVERVIEW.........................................................9

Introduction .................................................................................................. 9

Program Scope............................................................................................... 9

User Requirements and Resource Allocations ....................................................... 10

Computational Technology Areas ...................................................................... 12

Program Goals .............................................................................................. 14

Program Strategies ......................................................................................... 14

Program Components ..................................................................................... 16

High Performance Computing Shared Resource Centers ..................................... 16

Major Shared Resource Centers ............................................................... 16

Distributed Centers ............................................................................... 17

Networking.............................................................................................. 18

Defense Research and Engineering Network (DREN) ..................................... 18

Security .............................................................................................. 19

Software Applications Support...................................................................... 20

Common High Performance Computing Software Support Initiative (CHSSI) ....... 21

Programming Environment and Training (PET) ............................................. 21

Software Protection ............................................................................... 21

PROGRAM ACCOMPLISHMENTS AND FUTURE PLANS ................... 27

Introduction ................................................................................................. 27

High Performance Computing Shared Resources Centers ........................................ 28

Major Shared Resource Centers .................................................................... 28

Major Shared Resource Centers, Fiscal Year 2001 ........................................ 28

Major Shared Resource Centers, Fiscal Year 2002 and 2003 ........................... 30
4

2002 HPC Modernization Plan

Distributed Centers .................................................................................... 31

Distributed Centers, Fiscal Year 2002 ........................................................ 31

Distributed Centers, Fiscal Year 2003 ........................................................ 31

Networking .................................................................................................. 35

Defense Research and Engineering Network (DREN) ......................................... 36

DREN, Fiscal Year 2002 ......................................................................... 36

DREN, Fiscal Year 2003 ......................................................................... 36

Security .................................................................................................. 37

Software Applications Support .......................................................................... 37

Common High Performance Computing Software Support Initiative (CHSSI) ........... 38

CHSSI, Fiscal Year 2002 ......................................................................... 38

CHSSI, Fiscal Year 2003 ......................................................................... 39

Programming Environment and Training (PET) .................................................. 39

PET, Fiscal Year 2002............................................................................. 39

PET, Fiscal Year 2003............................................................................. 40

User Requirements and Resource Allocations ....................................................... 40

DoD High Performance Computing Challenge Projects .......................................... 40

Program Oversight and Review ......................................................................... 41

Acquisition................................................................................................... 46

Program Funding ........................................................................................... 46

ACRONYMS ..................................................................49
5

CONTENTS

ILLUSTRATIONS

FIGURES

Figure 1. High Performance Computing Shared Resource Centers ............................ 10

Figure 2. Integrated Program Strategy of the Department of Defense High Performance

Computing Modernization Program ............................................................... 11

Figure 3. DREN Connections, High Performance Computing Shared Resource Centers,

and Other Network Access Points ................................................................ 19

Figure 4. PET Team Members ........................................................................... 40

TABLES

Table 1. Examples of High Performance Computing Projects and Their Value to the

Warfighter ............................................................................................... 12

Table 2. Computational Technology Areas .......................................................... 13

Table 3. Computational Technology Area Leaders................................................. 14

Table 4. Common HPC Software Support Initiative Projects and Principal Investigators

(Fiscal Year 2002) ...................................................................................... 22

Table 5. Major Shared Resource Centers High Performance Computing Capability

(Fiscal Year 2002) ...................................................................................... 29

Table 6. Distributed Centers High Performance Computing Capability

(Fiscal Year 2002) ...................................................................................... 32

Table 7. Distributed Centers ............................................................................ 35

Table 8. CTA Component Functional Areas .......................................................... 41

Table 9. DoD High Performance Computing Challenge Projects (Fiscal Year 2002) ....... 42

Table 10. Department of Defense High Performance Computing Modernization

Program Funding (Fiscal Years 1997–2007) ..................................................... 48
PICTURE CAPTIONS (from right to Left):

A B-1B Lancer from the 28th Air Expeditionary Wing heads for Afghanistan after receiving fuel from a KC-10

Extender from the 60th Air Expeditionary Group in the late evening, Nov. 27, 2001, Operation Enduring Freedom.

(U.S. Air Force photo by Tech. Sgt. Cedric H. Rudisill) (Released). Photo by TECH. SGT. CEDRIC H RUDISILL,

1ST COMBAT CAMERA SQ.

KANDAHAR (December 28, 2001) — At the U.S. Marine Corps Base in Kandahar, Afghanistan, Marines on a

Light Armored Vehicle (LAV) prepare to go on patrol as an AH1W Super Cobra helicopter flys overhead. U.S.

Marines are in Afghanistan operating in support of Operation Enduring Freedom. U.S. Navy photo by Chief

Photographer’s Mate Johnny Bivera, Fleet Combat Camera Atlantic (RELEASED). Photo by PHC JOHNNY BIVERA,

FLEET COMBAT CAMERA ATLANTIC.

133rd Airlift Wing, Minnesota Air National Guard, Minneapolis-St. Paul. Flag at half mast in front of the

headquarters building on Sept. 20, 2001. (U.S. Air Force photo by Tech. Sgt. Devona Maher) (Released). Photo b:

TSGT DEVONA MAHER, 133 COMMUNICATIONS FLIGHT.






	






		



	


	

















		



























 











	














	






	
	

	


	 

9

The High Performance

Computing Modernization

Program (HPCMP) was

initiated in 1993 in response

to congressional direction to

modernize the Department

of Defense (DoD)

laboratories’ high

performance computing

(HPC) capabilities. Early on,

senior leaders recognized

this technology as critical to

our nation’s future defense.

Today, the HPCMP fields

world-class, commercial

high performance computing

resources to support the full

science and technology

(S&T) and test and evaluation

(T&E) communities. These

resources include a balanced

set of major shared resource

centers, smaller focused

distributed centers, wide

PROGRAM

OVERVIEW

Introduction area networking services,

and HPC software

applications support. Figure

1 shows the HPC center

locations and Figure 2

depicts the integrated

program strategy.

PROGRAM SCOPE

The HPCMP scope is

bounded both in terms of the

user community it serves and

the technological capability

that it delivers. By

concentrating the majority of

resources at a small number

of HPC centers, the program

has been able to provide

computing capabilities that

could not have efficiently

been obtained and sustained

by individual Services or
10

2002 HPC Modernization Plan

Agencies. This sharing of

resources reduces overall

acquisition and sustainment

costs and fosters

collaboration and

cooperation across the DoD

S&T and T&E communities.

The scope of the user

community is defined by the

Congress (Public Law 104-

61, December 1, 1995, 109

Statute 665, Sec. 8073) to be

“(1) the DoD Science

and Technology sites

under the cognizance

of the DDR&E, (2) the

DoD Test and

Evaluation centers

under the Director,

Test and Evaluation

OUSD (A&T), and (3)

the Ballistic Missile

Defense

Organization.”

The program maintains a

strict focus on providing high

performance computing.

The definition of “high

performance computing”

changes as the technology

continues to evolve. This

evolution requires that the

program periodically reassess

its acquisition plans to

ensure that it is not acquiring

“departmental” class

systems, which remain the

responsibility of the local

organization. The rapid

evolution of high

performance computing

requires that the program

focus on delivering improved

capability early in a weapon

systems life cycle. This

allows the DoD to maintain

the technological edge

required to analyze, design,

produce, and deploy

advanced weapons systems

and capabilities to the

warfighter—before similar

computational capabilities

are available to our

adversaries.

USER REQUIREMENTS

AND RESOURCE

ALLOCATIONS

From its beginning, the

HPCMP was designed to

address the continuing highend computing needs of the

DoD S&T program.

Congress later expanded the

program’s scope to include

the T&E community. The

program currently has a user

base of over 4,300 scientists

and engineers located at

more than 100 DoD

laboratories, test centers,

universities, and industrial

sites. In order to focus on

the needs of those scientists

and engineers, the program

office has developed a multilevel methodology for

determining user

requirements and for

establishing resource

allocations that reflect

Departmental priorities.

The requirements process,

developed and employed by

the program, includes an

annual survey of the DoD

S&T and T&E communities,

coupled with site visits to

discuss requirements and

provide information to user

groups. Working with

representatives of the user

community, the program

office identifies and tracks

requirements at the

computational project level.

The program also obtains

feedback through user

Figure 1. High Performance Computing Shared Resource Centers
11

PROGRAM OVERVIEW

satisfaction surveys, annual

user group meetings, and

functional and technical

advisory panels.

The High Performance

Computing Modernization

Program Office (HPCMO)

tracks HPC systems

performance metrics

throughout the year at the

computational project level

to ensure projects receiving

allocations are able to use

the resources effectively and

efficiently. This process

provides feedback to the

Services and Agencies

allowing resources to be

reallocated to the projects

best positioned to make

progress. The overall

integrated process furnishes

accurate and timely data on

which to base program

decisions.

The DoD computational

community has become

more sophisticated in its use

of HPC resources.

Consequently, the program

needed a representative,

reflective, and forwardlooking set of benchmarks.

In 2000, the HPCMO formed

a software analysis and

benchmark team to oversee

development of a new set of

benchmarks. The program’s

new benchmark suite is

more representative of the

application codes run by

DoD’s S&T and T&E

communities on HPCMP

systems. Every year, the

benchmark suite is reviewed

and updated. It is used as

an important criterion in the

acquisition of HPC systems.

The new benchmarks were

augmented in fiscal year

2001 and implemented in

the last system acquisition

process.

Twenty to twenty-five

percent of the program’s total

resources are dedicated each

year to a set of DoD HPC

Challenge Projects. These

computationally intensive,

high-priority projects are

selected annually through a

rigorous technical and

mission relevance

Figure 2. Integrated Program Strategy of the Department of Defense High Performance Computing Modernization Program
12

2002 HPC Modernization Plan

Table 1. Examples of High Performance Computing Projects and Their Value to the Warfighter

evaluation. Services and

Agencies allocate the

remaining resources through

Service and Agency unique

processes.

Challenge Project efforts

produce and support key

enabling technologies,

capabilities, and

demonstrations expressed by

the Defense Technology

Objectives (DTOs). These

enabling DTOs, in turn,

support Joint Vision 2020

and the thirteen Joint

Warfighting Capability

Objectives (JWCOs)

promulgated by the Joint

Requirements Oversight

Council of the Joint Chiefs of

Staff. While not allinclusive, JWCOs provide

focus, priority, and a

common reference point for

much of the DoD’s research,

test, and evaluation efforts.

They are described in detail

in the annual Joint

Warfighting Science and

Technology Plan (JWSTP).

Table 1 provides a few

examples of HPC projects

and their value to the

warfighting community.

COMPUTATIONAL

TECHNOLOGY AREAS

The DoD HPCMP user

community is organized

around ten broad

computational technology

areas (CTAs). Each CTA has

a designated leader, a

prominent DoD scientist or

engineer, working in the

discipline. Table 2 provides

a brief description of each

CTA and Table 3 lists the

current CTA leaders.

Project Value to the Warfighter and Joint Warfighting

Capability Objectives Supported

Operational Wave Forecasting in Scalable

Environments

Enables development of a state-of-the-art wave modeling

technology which can provide timely forecasts to military

operations. (Joint Readiness and Logistics and Sustainment of

Strategic Systems)

Real-Time Processing of Hyperspectral Sensor

Data

Enables scientists to evaluate the next generation of passive

s .l tandoff chemical warfare agent detectors These detectors wil

assess and disseminate threat information more quickly by

providing the needed data analysis in real-time.

(Chemical/Biological Warfare Defense and Protection and

Counter Weapons of Mass Destruction)

CFD-Based Rotorcraft Improvement Studies Enables the prediction of loads, performance, and acoustics of

r ;e otor aircraft in various modes of operation allows th

improvement of rotor aircraft designs; and answers the question

of whether to field new aircraft or modernize the existing fleet.

(Force Projection/Dominant Maneuver)

Ordnance Detonation and Penetration

Mechanics Modeling

Enables the modeling of high-impact explosive warheads,

reducing the number of live tests required, at a significant

savings over live ordnance testing. (Precision Fires, Joint

Readiness and Logistics and Sustainment of Strategic Systems)

W ) eapons of Mass Destruction ( t WMD Defea

Payload Development

Enables the development of payloads for air-dropped munitions

which destroy or deny the use of WMD while minimizing the

associated collateral effects. (Precision Fires,

Chemical/Biological Warfare Defense and Protection and

Counter Weapons of Mass Destruction)

Contaminant Transport Modeling in Complex

Urban Environments

Enables the development of a contaminant transport model to

study the effects of chemical/biological agents released into the

atmosphere; results can support consequence management

operations. (Military Operations on Urbanized Terrain, Joint

Readiness and Logistics and Sustainment of Strategic Systems,

Chemical/Biological Warfare Defense and Protection and

Counter Weapons of Mass Destruction)
13

PROGRAM OVERVIEW

Table 2. Computational Technology Areas

CTA Description

CSM Computational Structural

Mechanics

Covers the high resolution, multi-dimensional modeling of materials and

structures subjected to a broad range of loading conditions including

static, dynamic, and impulsive. CSM encompasses a wide range of

engineering problems in solid mechanics such as linear elastic stress

analysis. CSM is used for basic studies in continuum mechanics, stress

analysis for engineering design studies, and structural and material

response predictions to impulsive loads.

CFD Computational Fluid

Dynamics

Covers high performance computations whose goal is the accurate

numerical solution of the equations describing fluid and gas motion.

CFD is used for basic studies of fluid dynamics for engineering design of

complex flow configurations and for predicting the interactions of

chemistry with fluid flow for combustion and propulsion.

CCM Computational Chemistry

and Materials Science

Covers the computational research tools used to predict basic properties

of new chemical species and materials which may be difficult or

impossible to obtain experimentally. Within DoD, quantum chemistry and

molecular dynamics methods are used to design new chemical systems

and solid state modeling techniques are employed in the development of

new high-performance materials.

CEA Computational

Electromagnetics and

Acoustics

Covers the high-resolution, multi-dimensional solutions of Maxwell's

equations as well as the high-resolution, multi-dimensional solutions of

the acoustic wave equations in solids, fluids, and gases. Some DoD

applications include calculating the electromagnetic fields about antenna

arrays, the electromagnetic signatures of tactical ground, air, sea and

space vehicles, the electromagnetic signature of buried munitions, high

power microwave performance, as well as the interdisciplinary

applications in magnetohydrodynamics and laser systems.

CWO Climate/Weather/Ocean

Modeling and Simulation

Is concerned with the accurate numerical simulation of the earth's climate

including the simulation and forecast of atmospheric variability and

oceanic variability. CWO includes the development of numerical

algorithms and techniques for the assimilation of in-situ and remotely

sensed observations into numerical prediction systems.

SIP Signal/Image Processing Emphasizes research, evaluation, and test of the latest signal processing

c se oncepts directed toward these embedded system . This will enable th

t, t raditional expensive military-unique 'black boxes' required to implemen

high-speed signal/image processing to be replaced by commercial-offthe-shelf (COTS) HPC-based equipment.

FMS/

C4I

Forces Modeling and

Simulation

Focuses on force level modeling and simulation for training, analysis,

and acquisition. The acquisition domain includes research and

development, test and evaluation, and production and logistics. The

overarching Simulation Based Acquisition domain is included.

EQM Environmental Quality

Modeling and Simulation

Covers the high-resolution, three-dimensional Navier-Stokes modeling of

hydrodynamics and contaminant and multi-constituent fate/transport

through the aquatic and terrestrial ecosystem and wetland subsystems,

their coupled hydrogeologic pathways, and their interconnections with

numerous biological species.

CEN Computational Electronics

and Nanoelectronics

Uses advanced computational methods to model and simulate complex

electronics for communications, command, control, electronic warfare,

signal intelligence, sensing, and related applications. Use of high

performance computing assets enables the DoD electronics community to

solve complex problems, explore new concepts, gain insight and

improved understanding of the underlying physics, perform virtual

prototyping, and test new ideas.

IMT Integrated Modeling and

Test Environments

Addresses the application of integrated models, simulation tools and

techniques with live tests and hardware-in-the-loop simulations. IMT also

provides proof-of-concept by using technology-based war gaming

modeling and software integration tools for the simulation of weapon

component subsystems and systems in a virtual operational context.
14

2002 HPC Modernization Plan

PROGRAM GOALS

The Program has five

overarching goals to achieve

the program’s vision:

(1) Provide the best

commercially available

high-end HPC capability.

(2) Acquire and develop

joint need HPC

applications, software

tools, and programming

environments.

Table 3. Computational Technology Area Leaders

Computational Technology Area Leader

Computational Structural Mechanics

M d r. Michael Giltru

D , efense Threat R i eduction Agency A , lexandr a AV

Computational Fluid Dynamics

Dr. Robert Meakin

A d rmy Aviation a A n Missile Command, Moffett Field, C

Computational Chemistry and Materials

Science

D r r. Ruth Pachte

A g ir Force Research Laboratory, Materials/Manufacturin

Directorate, Wright-Patterson Air Force Base (AFB), OH

Computational Electromagnetics and

Acoustics

Dr. Robert Peterkin

Air Force Research Laboratory, Directed Energy Directorate,

Kirtland AFB, NM

Climate, Weather, and Ocean Modeling and

Simulation

D t r. William Burnet

Naval Oceanographic Office, John C. Stennis Space Center,

MS

Signal/Image Processing

D n r. Anders Sulliva

Army Research Laboratory, Aberdeen Proving Ground, MD

Forces Modeling and Simulation/C4I D n r. Larry Peterso

Space and Naval Warfare Systems Center, San Diego, CA

Environmental Quality Modeling and

Simulation

Dr. Jeffrey Holland

Engineer Research and Development Center, Vicksburg, MS

Computational Electronics and

Nanoelectronics

Dr. Barry Perlman

Army Communications-Electronics Command Research

Development and Engineering Center, Ft. Monmouth, NJ

Integrated Modeling and Test Environments

Mr. Jere Matty

Arnold Engineering Development Center, Arnold Air Force

Base (AFB), TN

(3) Educate and train DoD’s

scientists and engineers

to effectively use

advanced computational

environments.

(4) Link users and computer

sites via high-capacity

networks, facilitating user

access and distributed

computing environments.

(5) Promote collaborative

relationships among the

DoD HPC community,

the national HPC

community and Minority

Serving Institutions (MSIs)

in network, computer,

and computational

science.

PROGRAM STRATEGIES

The program implements the

following strategies to

achieve the five program

goals:
15

PROGRAM OVERVIEW

Goal Strategies

Provide the best

commercially available

high-end HPC

capability.

• Provide a balanced set of commercially available heterogeneous computing platforms and HPC

systems to meet the full range of DoD requirements.

• Permit the optimum mapping of requirements to system types.

• Conduct an annual benchmarking process that relates HPC systems performance to the program's

computational requirements.

• Build complete HPC environments, including large computing systems, software, and support

expertise at a few DoD laboratories designated as major shared resource centers (MSRCs) to support

a wide community of users.

• Place modest-sized HPC systems at selected distributed centers (DCs) when that allows for either

innovation or mission support not provided by a major shared resource center.

• Support efforts to build complementary HPC capabilities and technologies throughout government,

academia, and industry that are applicable to defense S&T and T&E requirements.

• Continually acquire and upgrade equipment and support services to ensure that DoD HPC centers

provide world-class, commercial, state-of-the-art capabilities.

Acquire and develop

joint need HPC

applications, software

tools, and

programming

environments.

• Encourage use of standards for defense research, development, test, and evaluation software to

ensure future transitions and advancements in software technology are applied in an efficient, costeffective manner.

• Focus on applications software projects designed to overcome technological inhibitors that may

delay the effective use of scalable high performance computers.

• Support software components used in specific computational technology areas or by a subset of

HPC users across the community.

• Identify software suites to be shared across major shared resource centers and selected distributed

centers.

Educate and train

DoD's scientists and

engineers to effectively

use advanced

computational

environments.

• Provide user education in scalable computer and computational sciences, software applications

and optimization, and code conversion.

• Promote the formation of DoD-sponsored interdisciplinary teams and collaborative groups to

determine how best to support each computational technology area and to leverage academia and

industry expertise in HPC for the solution of DoD problems.

• Develop shared application software in support of high-priority and broadly based computational

technology area needs.

• Aggressively transfer expertise and knowledge among the DoD HPC user communities.

• Broaden the knowledge base of the DoD HPC user community based on new applications and their

capabilities.

Link users and

computer sites via

high-capacity

networks, facilitating

user access and

distributed computing

environments.

• Deploy a secure, robust, wide area network (WAN) specifically designed to meet the needs of

advanced high performance computing in exploiting high fidelity computer models and simulations.

• Provide Gigabit range networking services using emerging technologies such as multi-protocol

label switches, packet over SONET, packet over lightwave, and lightwave services.

• Provide a low latency, high bandwidth WAN suitable for the establishment of truly distributed

computational grids, next generation mass storage of data, and the ability to do remote disaster

recovery.

• Offer DoD scientists and engineers the ability to securely and dynamically allocate bandwidth

among different applications and transmission protocols with minimal setup.

Promote collaborative

relationships among

the DoD HPC

community, the

national HPC

community and MSIs

in network, computer,

and computational

science.

• Ensure that users in the Department's S&T and T&E communities remain cognizant of, interact with,

and leverage internal Defense Department initiatives.

• Ensure that Defense S&T and T&E users remain cognizant of, collaborate with, and leverage other

government, academia, industry, and each other's HPC efforts.

• Maintain cooperative contact with the appropriate committees, subcommittees, working groups, and

advisory panels associated with the national high performance computing efforts.

• Provide HPC resources to support defense applications that directly relate to dual-use technologies

and National Challenges, such as environment, medical, digital libraries, and manufacturing

processes and products.

• Ensure the widest possible success of DoD HPC Challenge Projects. Leverage the nation's HPC

infrastructure to benefit the warfighter.
16

2002 HPC Modernization Plan

PROGRAM COMPONENTS

The program is organized into three components to achieve its goals. The components are

HPC shared resource centers, networking, and software applications support. The components

allow the program to focus on the most efficient means of supporting the S&T and T&E

communities’ requirements. The components align to the program goals as follows:

HIGH PERFORMANCE

COMPUTING SHARED

RESOURCE CENTERS

Major Shared Resource

Centers

The HPCMP operates four

large major shared resource

centers (MSRCs) that enable

the DoD S&T and T&E

communities to effectively

use the full range of HPC

resources. Each MSRC is

operated and maintained by

a team of government and

contractor personnel. The

four MSRCs are:

• Aeronautical Systems

Center (ASC), WrightPatterson AFB, OH

• Army Research

Laboratory (ARL),

Aberdeen Proving

Ground, MD

• Army Engineer Research

and Development

Center (ERDC),

Vicksburg, MS

• Naval Oceanographic

Office (NAVO), John C.

Stennis Space Center,

MS

Each MSRC includes a robust

complement of high-end,

high performance computing

and communications

systems. System capabilities

include scientific

visualization, peripheral and

archival mass storage

devices, and support staff.

MSRCs support a wide range

of computational projects.

In the past, a key element of

support provided to users by

the MSRCs has been the

Programming Environment

and Training (PET) effort.

This effort has transitioned

from separate contracts at

the four MSRCs to an

integrated, program-wide set

HPC Shared

Resource Centers Networking

Software Applications

Support

G- y oal 1 HPC Capabilit X X

G , oal 2 - HPC Applications

Software Tools, and

Support

X X

G d oal 3 - Education an

Training X X X

G g oal 4 - Networkin

Environment X

G e oal 5 - Collaborativ

Relationships X X X
17

PROGRAM OVERVIEW

of competitively awarded

contracts. Extensive training

to meet a wide variety of

users needs and

incorporating state-of-the-art

system tools and HPC

technology facilitate the

efficient use of the various

HPC systems. This support

is a collaborative partnership

established by the HPCMP

with leading civilian HPC

centers and academic

institutions. Such rapid

collaboration fosters

innovation and draws

additional HPC technology

and expertise into the DoD.

MSRCs also focus on training

the DoD user base,

identifying HPC technology

opportunities, and

introducing these

opportunities into all HPC

centers’ computing

environments.

Distributed Centers

Distributed centers (DCs)

provide HPC capability to

local and remote portions of

the program’s user

community. Modest-sized

systems are deployed to DCs

where there is a significant

advantage of using a local

HPC system and the

potential for advancing DoD

applications. DCs leverage

HPC expertise or address

problems that cannot be

readily solved at the MSRCs.

The DCs are linked by highspeed communications to

the MSRCs and remote users.

Thus, they augment the

MSRCs, forming the total

DoD HPCMP computational

capability.

Some DCs primarily focus on

real-time data processing/

simulation, signal image

processing, and embedded

systems applications. These

centers provide dedicated

computational resources

targeted to support local

mission requirements. These

centers include:

• Air Armament Center

(AAC), Eglin AFB, FL

• Air Force Flight Test

Center (AFFTC),

Edwards AFB, CA

• Air Force Research

Laboratory, Information

Directorate (AFRL/IF),

Rome, NY

• Air Force Research

Laboratory, Sensors

Directorate (AFRL/SN),

Wright-Patterson AFB,

OH

• Arnold Engineering

Development Center

(AEDC), Arnold AFB,

TN

• Joint National

Integration Center

(JNIC), Schriever AFB,

CO

• Naval Air Warfare

Center Aircraft Division

(NAWCAD), Patuxent

River, MD

• Naval Air Warfare

Center Weapons

Division (NAWCWD),

China Lake, CA

• Redstone Technical Test

Center (RTTC),

Huntsville, AL

Impact to DoD: High performance computers and high-bandwidth, lowlatency, networking technologies provide a capability to virtual prototype realtime systems and metasystems impacting military sensors, communication, and

C2 systems. The technology supports simulating air traffic control

communication, drug-enforcement surveillance, and port monitoring.

System Virtual

Prototyping, a

screen capture

from the

ADVISR realtime, threedimensional

visualization

system
18

2002 HPC Modernization Plan

• Space and Naval

Warfare Systems Center

(SSCSD), San Diego,

CA

• Tank Automotive

Research, Development

and Engineering Center

(TARDEC), Warren, MI

• White Sands Missile

Range (WSMR), White

Sands Missile Range,

NM

Other centers focus on nonreal-time computing, short

turnaround requirements,

Challenge Projects, and early

access technology transfer.

These centers provide

significant resources to

supplement MSRCs and are

configured to support

multiple tasks and process

multiple requests. The DCs

are:

• Arctic Region

Supercomputing Center

(ARSC), Fairbanks, AK

• Army High Performance

Computing Research

Center (AHPCRC),

Minneapolis, MN

• Maui High Performance

Computing Center

(MHPCC), Kihei, HI

• Naval Research

Laboratory (NRL-DC),

Washington, DC

• Space and Missile

Defense Command

(SMDC), Huntsville, AL

NETWORKING

Defense Research and

Engineering Network

The Defense Research and

Engineering Network (DREN)

is DoD’s recognized research

and engineering network.

Figure 3 graphically displays

the DREN connections for

the program’s HPC centers

and other wide area network

(WAN) access points. The

DREN is a robust, highspeed network that provides

connectivity between the

HPCMP’s geographically

dispersed user sites and

shared resource centers.

Since users and resources are

scattered throughout the

United States, strong

interconnectivity with other

major networks and high

performance test beds at key

exchange points are critical

for optimal use of DoD high

performance computers.

The networking services of

DREN are provided as a

contract service. The service

provider has built the DREN

as a virtual private network

The colored squares indicate those

elements for which we have

developed tight-binding parameters,

and show the ground-state crystal

structure of each element. The

figure also indicates the accuracy of

physical properties determined by

the method. These accuracies are

comparable to first-principles

results. Finally, molecular dynamics

capabilities are shown in the green

box.

Impact to DoD: DoD researchers are using high performance computers to develop an efficient

computational method for studying materials properties which will lead to the design of new materials

for DoD needs.
19

PROGRAM OVERVIEW

(VPN) over a public

infrastructure. The DREN

provides for digital data

transfer services between

defined service delivery

points (SDPs). SDPs are

specified in terms of WAN

bandwidth access, network

protocols [Multi PROTOCAL

LABEL switching, Internet

Protocol, Asynchronous

Transfer Mode], and local

connection interfaces.

DREN supports bandwidths

from DS-3 at user sites to

OC-12 at selected DC and

MSRCs. Future bandwidths

will scale to OC-192 and

beyond. DREN services may

be at any point in the

continental United States,

Alaska, or Hawaii, or other

overseas sites. Incorporating

the best operational

capabilities of both the DoD

and the commercial

telecommunications

infrastructure, the DREN is

the official DoD long-haul

network for computational

research and testing in

support of DoD’s S&T and

T&E communities.

Since its inception, DREN

has been active in

transferring leading edge

network and security

technologies across DoD

and other federal agencies.

Security

The program’s security-indepth model is one of the

most effective security

programs in use

today. The

security-in-depth

model uses

layers of

defense, ranging

from network

protections such

as access

control lists and

intrusion

detection to

individual

systems with

very strong

authentication

and

identification,

strong

configuration

management,

periodic

physical and logical

examination, and active

monitoring. The program

office implements security at

the HPC site level to protect

HPC resources and users and

at the network level for the

HPC community.

The security posture of each

HPC site is verified before it

is authorized to connect to

the DREN. While the

HPCMP retains overall

security policy and oversight

responsibilities, security

certification and

accreditation are done at the

individual site level with

accreditation delegated to

the host Services. The host

Services certify user sites

while the HPCMP is

responsible for certifying

HPC centers. A security test

Figure 3. DREN Connections, High Performance Computing Shared Resource Centers,

and Other Network Access Points
20

2002 HPC Modernization Plan

and evaluation team, led by

organizations such as the

Space and Naval Warfare

Systems Center, San Diego,

performs HPC center

certification. The HPCMP

also performs site assistance

visits annually and

periodically sponsors

independent National

Security Agency (NSA)

security assessments.

HPC centers and user sites

with classified computing

requirements are accredited

and certified at the

appropriate classification

level. The HPCMP requires

that copies of memoranda of

agreement, in accordance

with Department of Defense

Directive 5200.28, be on file

before allowing classified

communications between

user and

provider sites.

All classified

communications

use Type 1 NSA

approved

encryption

devices.

For unclassified

processing, the

HPCMP

requires strong

identification

authentication

to ensure

access only to

authorized

users.

Authentication

uses a

combination of Kerberos

software, one-time pass

codes (currently SecurID

cards), and encrypted shell

access like KTelnet or SSH

Secure Shell software. This

provides increased protection

from:

• disruption of service to

authorized users;

• unauthorized

disclosure, copying

(theft), or alteration of

users’ data;

• unauthorized access to

HPC centers’

computers; and

• misuse of HPC centers’

computers.

The HPCMP works closely

with other DoD

organizations to ensure that

the DREN and all connected

sites meet Service and Office

of the Secretary of Defense

security regulations and

requirements. At the

network level, the HPC

Computer Emergency

Response Team (CERT)

monitors the DREN using

network intrusion detection

systems running Department

of Energy developed Joint

Intrusion Detection and other

software. Access control

lists on external network

devices further protect DREN

users from known sources of

attack.

SOFTWARE APPLICATIONS

SUPPORT

“Software Applications

Support” is new terminology

that captures the

evolutionary nature of the

Program’s efforts to

accomplish goals 2,

“Acquire and develop joint

need HPC applications,

software tools, and

programming environments”,

and 3, “Educate and train

DoD’s scientists and

engineers to effectively use

advanced computational

environments”. As the

program has evolved, the

potential synergy of refocusing programming

environment and training

(PET) activities caused the

HPCMP to realign PET from

a shared resource center

centric orientation to a

broader program-wide focus.

Impact to DoD: High performance computing

enables the development and design of advanced

energetic materials through predicting/tailoring

properties and performance by reducing the need

for large scale test and evaluation and reducing the

development and life costs through smart design.

MolecularModeling Performance Modeling

Advanced Weapons Systems for Army After Next
21

PROGRAM OVERVIEW

This realignment offers

significant opportunities for

leveraging and collaboration

with CHSSI. In 2001, PET

transitioned from separate

contracts at the four MSRCs

to an integrated, programwide set of competitively

awarded contracts.

Common High

Performance Computing

Software Support Initiative

The Common High

Performance Computing

Software Support Initiative

(CHSSI) provides DoD

scientists and engineers

efficient, scalable, portable

software codes, algorithms,

tools, models, and

simulations that run on a

variety of HPC platforms.

CHSSI, which is organized

around ten computational

technology areas, involves

several hundred scientists

and engineers working in

close collaboration across

government, industry, and

academia. CHSSI teams

include algorithm

developers, applications

specialists, computational

scientists, computer

scientists, engineers, and end

users. Software developed

by CHSSI is CTA-specific or

multi-disciplinary. Portfolios

combining multi-disciplinary

projects focus on readily

understood warfighter needs.

Developing software for

scalable HPC systems is

technically challenging and

labor intensive. CHSSI helps

the DoD take advantage of

existing and future

computing and

communications capabilities

by building software with an

emphasis on reusability,

scalability, portability, and

maintainability. In addition,

CHSSI is producing a new

generation of world-class

scientists and engineers

trained in scalable software

techniques that will reduce

the future costs of doing

business and increase our

future defense capabilities.

Table 4 lists the current 35

CHSSI projects with principal

investigators.

Programming Environment

and Training

The Programming

Environment and Training

(PET) component enables the

DoD HPC user community

to make the best use of the

computing capacity the

HPCMP provides and to

extend the range of DoD

technical problems solved on

HPC systems. PET enhances

the total capability and

productivity of users through

training, collaboration, tool

development, software

development support,

technology tracking,

technology transfer, and

outreach.

PET activities consist of

short- and long-term strategic

efforts. Short-term efforts

include on-site training and

short collaborative projects

with users. Examples of

long-term efforts are distance

learning and

metacomputing. The PET

initiative allows the DoD to

shape the direction of HPC

through participation and

partnership.

Part of the PET approach is

to marshal elite and readily

accessible university teams

by creating an information

conduit between DoD HPC

users and top academic

experts. PET teams are able

to both anticipate and

respond to the needs of DoD

users for training and

assistance in advancing the

programming environment.

The teams have a strong onsite presence and interact

daily with HPCMP staffs and

users, with additional

support by senior level

researchers at universities.

Software Protection

Software protection is a

focused research and

development effort to design

and implement specific

mechanisms to HPC software

applications to protect the

intellectual content of these

codes from unauthorized

use.
22

2002 HPC Modernization Plan

Table 4. Common HPC Software Support Initiative Projects and Principal Investigators

(Fiscal Year 2002)

Project Principal

Investigator

Organization

Computational Structural Mechanics

A Next Generation Scalable Finite Element

Software to Describe Fracture and

Fragmentation of Solids and Structures

A.M. Rajendran Army Research Laboratory,

Aberdeen Proving Ground, MD

Computational Fluid Dynamics

Parallelizing SOCRATES - The Air Force Research

Laboratory Direct Simulation Monte Carlo Flow

Field, Chemistry, and Radiation Code

Thomas Smith Air Force Research Laboratory,

Propulsion Directorate, WrightPatterson AFB, OH

Unstructured Overset Grid Framework for

Moving Body Applications

J d ere Matty Arnol Engineering Development

Center, Arnold AFB, TN

Computational Chemistry and Materials Science

Tight Binding Molecular Dynamics Dimitris

Papaconstantopoulos

Naval Research Laboratory,

Washington, DC

Molecular Dynamics for Energetic and NonEnergetic MaterialsRichard Mowrey Naval Research Laboratory,

Washington, DC

Computational Electromagnetics and Acoustics

A Scalable, Dynamically Adaptive Mesh

Software Package for General

Magnetohydrodynamic and Space Weather

Modeling

Sprios Antiochos Naval Research Laboratory,

Washington, DC

High-Resolution Modeling of Acoustic Wave

Propagation in Atmospheric Environments

D. Keith Wilson Army Research Laboratory,

Aberdeen Proving Ground, MD

MACH3: Portable, Scalable, Parallel-Computing

Magnetohydrodynamics Software for Multimaterials in Complex 3-D Geometry

Robert Peterkin Air Force Research Laboratory,

Directed Energy Directorate,

Kirtland AFB, NM

Scalable, Adaptive Large-Scale Structural

Acoustics

Joseph Bucaro Naval Research Laboratory,

Washington, DC

Scalable Computer Ocean-Acoustic Propagation

Engine (SCOPE)

Stanley Chin-Bing Naval Research Laboratory, John C.

Stennis Space Center, MS

Software Development for Electromagnetic

Sensing of Surface and Subsurface Targets:

Simulation and Signal Processing

Anders Sullivan Army Research Laboratory, Adelphi,

MD
23

PROGRAM OVERVIEW

Table 4—Continued. Common HPC Software Support Initiative Projects and Principal Investigators

(Fiscal Year 2002)

Project Principal

Investigator

Organization

Signal/Image Processing

Acoustic Analysis Workbench Using Windows

NT

K s eith Bromley Space and Naval Warfare System

Center, San Diego, CA

Image Fusion and Signal/Image Processing R , ichard Linderman Air Force Research Laboratory

Information Directorate, Rome, NY

Infrared Search and Track Processing for

Missile Surveillance

D s ennis Cottel Space and Naval Warfare System

Center, San Diego, CA

Climate, Weather, and Ocean Modeling and Simulation

Development of Two Complete Numerical

Weather Prediction Systems for Heterogeneous

Scalable Computing Environments

T , homas Rosmond Naval Research Laboratory

Monterey, CA

Satellite Radiance Variational Data

Assimilation: Code Migration to Scalable

Architectures

F , rank Ruggiero Air Force Research Laboratory

Space Vehicles Directorate,

Hanscom AFB, MA

Scalable Ocean Models with Data Assimilation A . lan Wallcraft Naval Research Laboratory, John C

Stennis Space Center, MS

Scalable Weather Research and Forecast

(WRF) Model Development

J t erry Wegiel Air Force Weather Agency, Offut

AFB, NE

Forces Modeling and Simulation/C4I

Parallelization of the Joint Interim Mission

Model (JIMM)

David Mutschler Naval Air Warfare Center Aircraft

Division, Patuxent River, MD

Simulation of Communications Networks Using

Commercial-Off-The Shelf (COTS) Software

Albert Legaspi Space and Naval Warfare Systems

Center, San Diego, CA

Environmental Quality Modeling and Simulation

Simulation of Land Use Evolution and Toxics

Transport on Military Installation Readiness,

Training, Testing, and Security

J d ames Westervelt Engineer Research an

Development Center, Vicksburg, MS

Computational Electronics and Nanoelectronics

Efficient Numerical Solutions to Large-Scale

Military Tactical Communications Problems via

a Scalable Time Domain Method (Sc-MRTD)

Robert Pastore Army Communications-Electronics

Command, Ft. Monmouth, NJ

Integrated Modeling and Test Environments

Simulation and Software Implementation of

Non-Uniformity Correction (NUC) for MultiElement Infrared Scene Projector (IRSP) Arrays

Kenneth LeSueur Redstone Technical Test Center,

Redstone Arsenal, AL

HPC in Concurrent Engineering, Modeling and

Testing

Andrew Mark Army Research Laboratory,

Aberdeen Proving Ground, MD

Model-Based Test Data Validation Mark Chappell Arnold Engineering Development

Center, Arnold AFB, TN
24

2002 HPC Modernization Plan

Table 4—Continued. Common HPC Software Support Initiative Projects and Principal Investigators

(Fiscal Year 2002)

Project Principal

Investigator

Organization

Portfolio Modeling and Simulation of the Electronic Battlefield Environment (EBE), Dr. Barry Perlman,

Leader

Design of Multifunctional Multi-Mode

Reconfigurable Antenna Systems for Sensing

and Communications

Gregory Creech Air Force Research Laboratory,

Sensors Directorate, WrightPatterson AFB, OH

Electromagnetic Environmental Effects Toolkit Rosemary Wenchel Naval Information Warfare Center,

Washington, DC

EM Design General Framework for 21st Century

Integrated Military Platforms

John Rockway Space and Naval Warfare Systems

Center, San Diego, CA

Portfolio Hyperspectral Image Exploitation (HIE), Dr. Richard Linderman, Leader

High Performance Computing Software

Framework for Hyperspectral Imaging

George Ramseyer Air Force Research Laboratory,

Information Directorate, Rome, NY

Adaptive Spectral Reconnaissance Program

Algorithms for HPC

Eric Coolbaugh Space and Naval Warfare Systems

Center, San Diego, CA

Automatic Target Detection in Hyperspectral

Imagery Using Principal Components Analysis

Gary Stolovy Army Research Laboratory,

Aberdeen Proving Ground, MD

HPC PALM - High Performance Computing

Pairwise Adaptive Spectral Matched Filter for

Hyperspectral Image Exploitation

Michael Eismann Air Force Research Laboratory,

Sensors Directorate, WrightPatterson AFB, OH

Scalable Software for the Analysis (Including

Automatic Target Recognition (ATR)) and

Compression of Hyperspectral Data

Jeffery Bowles Naval Research Laboratory,

Washington, DC

Atmosphereic Correction for Hyperspectral

Imaging Exploitation

Gail Anderson Air Force Research Laboratory,

Space Vehicles Directorate,

Hanscom AFB, MA

Portfolio High Fidelity Simulation of Littoral Environments (SLE), Dr. George Heburn, Leader

Multi-Level Parallelization Alan Wallcraft Naval Research Laboratory, John C.

Stennis Space Center, MS

Model Integration Framework M t atthew Bettencour Naval Research Laboratory, John C.

Stennis Space Center, MS

Simulation of Surf Zone Waves and Currents Jane Smith Engineer Research and

Development Center, Vicksburg, MS

S n imulation of Coastal and Estuarine Circulatio Cheryl Blain Naval Research Laboratory, John C.

Stennis Space Center, MS

Near-Shore Sediment Transport Timothy Keen Naval Research Laboratory, John C.

Stennis Space Center, MS

Scalable Modeling of Tidal Riverine

Environments

Charlie Berger Engineer Research and

Development Center, Vicksburg, MS

Advanced Surface Water-Groundwater

Connectivity with Linkage to Atmospheric

Outputs

Stacy Howington Engineer Research and

Development Center, Vicksburg, MS

M&S Environmental Server Richard Allard Naval Research Laboratory, John C.

Stennis Space Center, MS
25

PROGRAM OVERVIEW

Table 4—Continued. Common HPC Software Support Initiative Projects and Principal Investigators

(Fiscal Year 2002)

Project Principal

Investigator

Organization

Portfolio Interdisciplinary Computational Environment for Weapon Target Interactions (WTI), Dr. Raja

Namburu, Leader

Penetrator-Target Interaction Raju Namburu Army Research Laboratory,

Aberdeen Proving Ground, MD

Blast-Structure Interaction Michael Giltrud Defense Threat Reduction Agency,

Alexandria, VA

Optimization of Weapon-Target Systems Design Ernest Baker Engineer Research and

Development Center, Vicksburg, MS

and Army Research Laboratory,

Aberdeen Proving Ground, MD

Portfolio Material by Design (MBD), Dr. Margaret Hurley, Leader

Design of New Materials Using Computational

Chemistry and Materials Science

Jerry Boatz Air Force Research Laboratory,

Materials and Manufacturing

Directorate, Edwards AFB, CA

Materials Design Software Suite Noam Bernstein Naval Research Laboratory,

Washington, DC

Scalable Quasi-Continuum Software Margaret Hurley Army Research Laboratory,

Aberdeen Proving Ground, MD

Portfolio System of Systems Simulation (SOS), Mr. Rick Cozby, Leader

Distributed Information Enterprise Modeling

and Simulation (DIEMS)

Robert Hillman Air Force Research Laboratory,

Information Directorate, Rome, NY

Virtual Communication Links Robert Pritchard Naval Air Warfare Center, Aircraft

Division, Patuxent River, MD

Virtual Prototyping and Accelerated Test of DoD

Composite Material Combat Systems for LongTerm Prediction of Structural Integrity

Andrew Mark Army Research Laboratory,

Aberdeen Proving Ground, MD

Portfolio Sensor/Scene Processing and Generation (SPG), Dr. Kenneth LeSueur, Leader

Fast Line-of-Sight Imagery for Target and

Exhaust-Plume Signatures (FLITES)

Charles Coker Air Force Research Laboratory,

Propulsion Directorate, Edwards

AFB, CA

Scene Generation With Plume Signature for

Hardware-in-the-Loop Test Applications

Dan Benedict Arnold Engineering Development

Center, Arnold AFB, TN

OpenIR: Validated Infrared Graphics

Rendering Libraries for Scalable Parallel

Supercomputers

John Andujar Naval Air Warfare Center, Aircraft

Division, Patuxent River, MD

Hyperspectral Polarimetric (HSIPOL) Scalable

Target and Background Signature Model

Jason Coker Air Force Research Laboratory,

Munitions Directorate, Eglin AFB, FL

Parallel Scene Generation/Electromagnetic

Modeling of Complex Targets in Complex

Clutter and Propagation Environments

Gerard Genello Air Force Research Laboratory,

Sensors Directorate, WrightPatterson AFB, OH

27

Since its inception in 1992,

the HPCMP has matured

from vector-based computers

such as the Cray-2 and

Control Data Cyber 960, to

the massively parallel Cray

T3E and SGI Origin 2000.

Laboratories were once

connected by local area

networks (LANs) and Wide

Area Networks (WANs) with

commercial point to point

links at data rates of 1.544

million bits per second

(Mbps) (T 1) or less. At

present, these laboratories

are connected by the stateof-the-art Defense Research

and Engineering Network

(DREN) with asynchronous

transfer mode (ATM) and

Internet protocol (IP) services

providing DS3 connectivity

to the user sites and most

PROGRAM

ACCOMPLISHMENTS

AND FUTURE PLANS

Introduction distributed centers. DREN

provides OC3 to OC12

connectivity to the four

Major Shared Resource

Centers (MSRCs) and

selected DCs with data

transfer rates of up to 1397

Mbps.

Along with this maturity

have come valuable lessons

learned allowing the

program to streamline many

of its processes. This year,

many of the contracts have

expired, such as the PET and

DREN contracts, and the

new processes in place allow

the program to more

effectively and efficiently

select new contractors. The

acquisition process has also

been streamlined. A new

acquisition process has been
28

2002 HPC Modernization Plan

established which allows the

HPCMP to buy HPC

hardware and services at the

best value to the

government. This section

provides details on what the

program has accomplished

this past year and what it

plans to do in the future.

HIGH PERFORMANCE

COMPUTING SHARED

RESOURCE CENTERS

MAJOR SHARED

RESOURCE CENTERS

The High Performance

Computing Modernization

Program (HPCMP) continues

to modernize the HPC

capabilities of the MSRCs. A

key focus area for the next

several years is an effort to

build the centers’ wide area

computing environment

(computational grids) with an

emphasis on peripheral and

archival mass storage

capabilities. The program

also provides user training.

Interaction with other

government agencies,

industry, and academic

centers is encouraged, so

that defense scientists and

engineers can take advantage

of the latest high

performance computing

expertise, capabilities,

standards, and infrastructure.

Since the first high

performance computing

systems were fielded by the

program in 1993, the total

peak computational high

performance computing

capability at the MSRCs

increased from 30 gigaflops

to just over 8,892 gigaflops

at the end of fiscal year

2001. The program will

continue to modernize the

centers to sustain the worldclass, high performance

computing infrastructure.

During fiscal year 2002, the

gigaflops of peak computing

capability added to the

MSRCs will again be

substantial, measuring in the

teraflops. (“Peak teraflops”

represent a simple measure

of growth. The term,

however, does not consider

architecture specific features,

nor does it provide a

measure of actual

performance delivered).

Table 5 shows the high

performance computing

systems’ capability installed

at the MSRCs as of February

2002.

Major Shared Resource

Centers, Fiscal Year 2001

In fiscal year 2001 the

program introduced a new

acquisition strategy, called

Technology Insertion 2001

(TI-01). The previous

process described hardware

and software upgrades in

terms of Performance Levels.

Performance Levels I, II, and

III were embedded in the

original integration contracts

at the four MSRCs. This

approach took the DoD HPC

environment at the MSRCs

from a few modest high

performance computing

systems to a world-class

HPC capability in just five

years. TI-01 continued that

history of success, but rather

than using site-specific

contracts, TI-01 was based

on a program-wide

perspective. TI-01 placed

more emphasis on overall

composition of HPC user

requirements, and how to

satisfy as much of those

requirements as possible

within the funds available.

Defining the optimal solution

required the examination of

the computational

requirements in several

different ways – by

classified/unclassified

processing environments, by

computational technology

area, by Challenge/nonChallenge projects, and by

the types of systems needed.

The TI process begins with a

call for proposals that

outlines the terms and

conditions of the intended

purchase, identifies a set of

benchmarks and prescribes

the required format for

vendor responses. The call

requests balanced HPC

system proposals with rough

order of magnitude pricing.

The vendors are notified of

the intent to solicit proposals

through public

announcements and are
29

PROGRAM ACCOMPLISHMENTS AND FUTURE PLANS

Note: Current as of February 2002.

Table 5. Major Shared Resource Centers High Performance Computing Capability

(Fiscal Year 2002)

Major Shared Resource Center Processors

Total Memory

(gigabytes)

Total

Capability

(gigaflops)

Aeronautical Systems Center (ASC), WrightPatterson AFB, OH

SGI Origin 2000/195

SGI Origin 2000/195

IBM SP/375

Compaq ES-40/833

Compaq SMP EV-67, ES-320

Compaq ES-45

128

384

528

64

128

836

128

192

528

64

128

836

51

154

792

107

205

1,672

Army Research Laboratory (ARL), Aberdeen

Proving Ground, MD

SGI Origin 2000/300

SGI Origin 2000/250

Sun E10000

Sun E10000

IBM SMP/375 16-way

SGI SN1/400

IBM NH2

256

128

128

128

512

768

768

256

112

128

128

512

512

576

153

64

102

102

768

615

1,152

Engineer Research and Development Center

(ERDC), Vicksburg, MS

Cray T3E-1200

SGI Origin 3000

Compaq-ES-40

Compaq ES-45

768

512

256

512

200

512

256

512

921

410

426

1,024

Naval Oceanographic Office (NAVO), Stennis

Space Center, MS

IBM SMP/375

Cray T3E-900

Cray SV1EX-4

Cray SV1

Sun E10000 64-way SMP

1,336

1,088

64

32

64

1 8 ,28

400

256

32

64

1 2 ,93

9 76

128

38

51
30

2002 HPC Modernization Plan

invited to attend an

informational meeting to

ensure that all interested

parties understand the

ground-rules. Proposals

received are evaluated based

on benchmark performance,

relative cost-performance,

usability, and confidence

criteria. The Collective

Acquisition Team (CAT),

comprised of site

representatives, HPCMO

staff and a user

representative, construct

purchasing options that

provide high value to the

program. A second request

for quotes is then issued to

solicit best and final pricing

for specific systems that

make up the highest value

options. Final

recommendations are

presented to the Source

Selection Authority (SSA) for

action. Factors such as local

user requirements, site

expertise, power and space

limitations, and the number

of different architectures that

exist at a site contribute to

the recommended allocation

of equipment among the

MSRCs. To maintain an

emphasis on buying larger

capability systems,

procurement resources are

targeted for two major and

two minor purchases each

year at the four MSRCs. This

allows time for sites to

integrate new HPC hardware

between major purchases

and build infrastructure or

fine-tune HPC systems

during their “off-years”.

The trend for corporate-wide

initiatives to address

common issues gathered

momentum during fiscal year

2001. The process of

building the centers’ wide

area computing environment

continued, including the

beginning of a grid

computing effort. The

program established test beds

that allow a central queue to

receive computational work

(jobs). The queue

determines which

computational environment

in the available pool (which

could be geographically

separated by 1000’s of miles)

will run the job fastest.

Cooperation activities with

other Federal Agencies will

continue to develop this

technology. The program

will continue to provide user

training and interaction with

other government agencies,

industry, and academic

centers so that defense

scientists and engineers can

take advantage of the latest

high performance computing

expertise, capabilities,

standards, and infrastructure.

Major Shared Resource

Centers, Fiscal Years

2002 and 2003

The program will continue

the success of the TI-01

process by implementing

Technology Insertion 2002

(TI-02). TI-02 is a

continuation of carefully

Impact to DoD: DoD scientists, using high performance

computers, are able to examine environmental impacts without

endangering aquatic environments

Bottom dissolved

oxygen (mg/l) is plotted

in plain view from a

Chesapeake Bay

simulation, where the

red areas indicate

depleted dissolved

oxygen that typically

occurs in the summer

and early fall.
31

PROGRAM ACCOMPLISHMENTS AND FUTURE PLANS

planned and successfully

executed enhancements to

the HPC infrastructure. The

specific goals of TI-02 are to

improve the program’s ability

to satisfy documented user

requirements; to base

acquisitions on programwide criteria; to refine the

benchmark suite to support

this and future acquisitions;

and to ensure the

government obtains the best

value.

The centers will gain

additional high performance

computing systems, mass

storage, and scientific

visualization capabilities.

The multi-year process of

building the MSRC’s wide

area computing environment

will continue as well with a

continuing emphasis on grid

computing.

DISTRIBUTED CENTERS

Distributed centers (DCs) are

selected annually for a onetime investment in high

performance computing

equipment. Typically, the

equipment is no longer in

the high performance

computing class after two to

three years. If a DC is not

selected for an upgrade in

computational capability

during this period, the center

may be transitioned from the

HPCMP to the parent Service

or Agency under the HPCMP

program’s transition policy.

The DUSD(S&T) made

awards to the Naval

Research Laboratory, the

Army High Performance

Computing Research Center,

and the Arctic Region

Supercomputing Center DCs

to purchase high

performance computer

equipment totaling $35.15

million during fiscal year

2001.

Table 6 shows the fiscal year

2002 high performance

computing systems at the

DCs. The total peak gigaflop

computational capability at

the distributed centers was

about 4,500 gigaflops at the

end of fiscal year 2001.

Table 7 lists the program’s

fiscal year 2001 DCs, which

are unchanged from fiscal

year 2000.

Distributed Centers,

Fiscal Year 2002

Investment decisions at the

DCs are determined through

an annual competitive

process. Selected DCs

(Arctic Region

Supercomputing Center

(ARSC), Fairbanks, AK and

Maui High Performance

Computing Center

(MHPCC),Kihei, HI) received

RDT&E funding as part of the

HPCMP annual budget. Five

DCs were competitively

selected to receive HPC

capital investments funds

from the HPCMP in fiscal

year 2002: Air Force

Research Laboratory,

Information Directorate,

(AFRL/IF), Rome, NY; Arnold

Engineering Development

Center (AEDC), Arnold AFB,

TN; Naval Air Warfare

Center, Aircraft Division

(NAWCAD), Patuxent River,

MD; Naval Research

Laboratory (NRL),

Washington, DC; and the

U.S. Army Space and Missile

Defense Command (SMDC),

Huntsville, AL. In addition,

a significant investment will

be made at the Army High

Performance Computing

Research Center (AHPCRC).

The investments will support

mission-specific, technical

challenges at each center.

During fiscal year 2002, the

HPCMO will continue to

work with the Services,

Agencies, HPCAP, and other

stakeholders to refine the

policies and processes used

to determine the suitable mix

of distributed centers and the

appropiate level of oversight

they require. The HPCMO’s

focus will be to ensure

adequate resources are

available to support Service/

Agency priority projects, and

to alleviate administrative

burdens.

Distributed Centers,

Fiscal Year 2003

During fiscal year 2003, the

HPCMO will implement the

changes developed in fiscal

year 2002. The program will

continue to review the
32

2002 HPC Modernization Plan

Table 6. Distributed Centers High Performance Computing Capability

(Fiscal Year 2002)

Note: Current as of February 2002.

Distributed Centers Processors

Total Memory

(gigabytes)

Total

Capability

(gigaflops)

Air Armament Center (AAC), Eglin AFB, FL

SGI Origin 2000

SGI Onyx Reality

SGI Onyx2

20

32

36

20

8

36

10

13

22

A s ir Force Flight Test Center (AFFTC), Edward

AFB, CA

SGI Onyx2

SGI Onyx2

SGI Origin 3800

24

24

64

9

9

48

16

16

54

Air Force Research Laboratory Information

Directorate (AFRL/IF), Rome, NY

Sky Excalibur 384 26 640

Air Force Research Laboratory Sensors

Directorate (AFRL/SN), Wright-Patterson AFB,

OH

Mercury MP-510

Mercury MP-510

64

96

16

24

205

307

Arctic Region Supercomputing Center

(ARSC), Fairbanks, AK

Cray SV1EX

Cray T3E

IBM P3/SMP

32

272

200

8

68

92

64

230

276

Army High Performance Computing Research

Center (AHPCRC), Minneapolis, MN

Cray T3E-1200 1,088 544 1,300

Arnold Engineering Development Center

(AEDC), Arnold AFB, TN

HP V2500

HP V2500

HP V2500

SGI Origin 2000

32

64

32

64

32

64

32

32

70

140

70

38
33

PROGRAM ACCOMPLISHMENTS AND FUTURE PLANS

Table 6—Continued. Distributed Centers High Performance Computing Capability

(Fiscal Year 2002)

Distributed Centers Processors

Total Memory

(gigabytes)

Total

Capability

(gigaflops)

J (C oint National Integration Center J , NI )

Schriever AFB, Colorado Springs, CO

SGI Origin 2400

SGI Origin 2000

SGI Origin 2000

SGI Origin 2000

64

32

32

128

32

16

16

128

32

16

16

77

Maui High Performance Computing Center

(MHPCC), Kihei, HI

IBM P3/SMP (16-way)

IBM P2SC

IBM SP3

IBM P2SC

IBM P2SC

IBM Netfinity Cluster (933)

IBM Netfinity Cluster (933)

IBM Netfinity Cluster (933)

IBM P2SC

IBM P2/SMP (8-way)

IBM P3/SMP (16-way)

IBM P3/SMP (8-way)

716

160

320

32

32

520

8

56

19

64

64

32

358

80

160

8

20

260

4

28

4

8

32

16

1,074

102

480

15

21

478

7

52

9

14

96

48

Naval Air Warfare Center Aircraft Division

(NAWCAD), Patuxent River, MD

SGI Onyx2

SGI Onyx2

SGI Onyx3

SGI Onyx3

S ) GI Origin 2000 (4 systems

28

28

32

32

30

16

8

12

12

4

12

11

23

23

24

Naval Air Warfare Center Weapons Division

(NAWCWD), China Lake, CA

SGI Onyx2

SGI Onyx2

32

32

12

12

20

20

Naval Research Laboratory (NRL),

Washington, DC

SGI Origin 3800

Sun Ultra

128

64

128

15

107

20

Note: Current as of February 2002.
34

2002 HPC Modernization Plan

Table 6—Continued. Distributed Centers High Performance Computing Capability

(Fiscal Year 2002)

Note: Current as of February 2002.

Distributed Centers Processors

Total Memory

(gigabytes)

Total

Capability

(gigaflops)

Redstone Technical Test Center (RTTC),

Redstone Arsenal, AL

SGI Origin 2000

SGI Origin 2000

SGI Origin 3800

SGI Origin 3800

SGI Onyx2

SGI Oniyx2

SGI Origin 2400

32

32

32

32

16

16

16

13

13

16

16

8

6

8

33

33

29

29

16

16

12

Space and Missile Defense Command

(SMDC), Huntsville, AL

S ) GI Origin 2000 (Sim Cen

S ) GI Origin 2000 (ARC

S ) GI Origin 2000 (ARC

S ) GI Origin 2000 (ARC

S ) GI Origin 2000 (ARC

S ) GI Origin 2000 (ARC

S 8 GI Origin 3 00

Cray SV1E

128

96

96

16

48

48

64

16

32

30

48

8

12

12

64

32

52

55

59

12

29

20

59

32

Space and Naval Warfare Systems Center

(SSCSD), San Diego, CA

HP V2500

HP V2500

HP Superdome

HP Superdome

SGI Onyx2

16

16

48

48

24

24

24

48

48

12

28

28

105

105

9

Tank-Automotive Research, Development and

Engineering Center (TARDEC), Warren, MI

SGI Power Challenge

SGI Onyx2

28

32

6

31

11

16

White Sands Missile Range (WSMR), NM

SGI Origin 2000

SGI Origin 2000

SGI Origin 2000

64

64

64

24

24

24

32

32

32
35

PROGRAM ACCOMPLISHMENTS AND FUTURE PLANS

progress and achievements

of previously selected DCs,

and (providing the fiscal year

2003 budget is sustained)

continue to make

investments in high

performance computing

equipment at DCs.

NETWORKING

DEFENSE RESEARCH AND

ENGINEERING NETWORK

The Defense Research and

Engineering Network (DREN)

is a robust, high-speed

network that provides

connectivity between the

HPCMP’s geographically

dispersed user sites and

shared resource centers.

DREN provides gateways to

many existing military and

civilian networks, allowing

Table 7. Distributed Centers

Center Location

Most Recent

Upgrade

Real-Time Focused Centers

Air Armament Center (AAC) Eglin AFB, FL 1999

Air Force Flight Test Center (AFFTC) Edwards AFB, CA 2000

Air Force Research Laboratory, Information Directorate

(AFRL/IF) Rome, NY 2002

Air Force Research Laboratory, Sensors Directorate

(AFRL/SN) Wright-Patterson AFB, OH 2000

Arnold Engineering Development Center (AEDC) Arnold AFB, TN 2002

Joint National Integration Center (JNIC) Schriever AFB, CO 1999

Naval Air Warfare Center Aircraft Division (NAWCAD) Patuxent River, MD 2002

Naval Air Warfare Center Weapons Division

(NAWCWD) China Lake, CA 1999

Redstone Technical Test Center (RTTC) Huntsville, AL 2000

Space and Naval Warfare Systems Center (SSCSD) San Diego, CA 1999

Tank Automotive Research, Development and

Engineering Center (TARDEC) Warren, MI 1999

White Sands Missile Range (WSMR) White Sands Missile Range, NM 1998

Non-Real-Time Focused Centers

Arctic Region Supercomputing Center (ARSC) Fairbanks, AK 2002

Army High Performance Computing Research Center

(AHPCRC) Minneapolis, MN 2002

Maui High Performance Computing Center (MHPCC) Kihei, HI 2002

Naval Research Laboratory (NRL-DC) Washington, DC 2002

Space and Missile Defense Command (SMDC) Huntsville, AL 2002
36

2002 HPC Modernization Plan

access to the program’s

resource centers from

contractor and university

facilities.

The wide area networking

portion of the HPC

community is in the process

of being accredited at the

Sensitive but Unclassified

system level. Each enclave,

both classified and

unclassified, is already

accredited at the edges.

In March 2000, the DREN

underwent formal program

review; evaluation

comments characterized the

initiative as outstanding. In

August 2000, the Assistant

Secretary of Defense

(Command, Control,

Communications and

Intelligence) recognized

DREN as the DoD’s research

and engineering network.

A Request for Proposal was

released in January 2001 for

a 10-year follow-on contract.

The new DREN contract is

expected to be awarded in

the second quarter of fiscal

year 2002 and will be a best

value award under full and

open competition. The

contract will have a base

period of three years and

seven one-year options. It

is anticipated that the new

vendor will provide a very

high capacity, state-of-the-art

backbone with new high

speed interfaces, more

stringent performance

parameters, and new

services such as IPv6,

increased security, and

maximum flexibility for

technology insertions.

DREN, Fiscal Year 2002

During fiscal year 2002, the

major emphasis will be the

transition to the new DREN

contract. Network

connectivity will be

sustained and bandwidth

increased. DREN will

continue to support test bed

environments and facilitate

the distributed computing

environments across the

program’s MSRCs and

selected DCs. The long term

objective of this effort is to

optimize sharing of valuable

DoD HPCMP resources

across centers by supporting

dynamic, real-time,

ubiquitous computing

environments; providing

better protection during

times of disaster or extended

outages; and by eliminating

single points of failure.

DREN, Fiscal Year 2003

During fiscal year 2003, the

HPCMP will continue to

increase bandwidth and to

support test bed

environments and facilitate

the distributed computing

environments across the

program’s MSRCs and

selected DCs. The very

high-bandwidth DREN will

be the key to success, since

individual centers will need

immediate, bandwidthintensive connectivity to

Impact to DoD: DoD engineers are using high performance computers to

better understand the thrust and lift generation mechanisms in flapping foil

propulsion which impacts submersibles, maneuvering, and the aerodynamics

of MAVs.

Surface

pressure

distribution

on a micro

air vehicle

(MAV)
37

PROGRAM ACCOMPLISHMENTS AND FUTURE PLANS

move increasingly larger data

sets and resource sharing at

low latencies.

SECURITY

Security at HPCMP sites and

on the DREN continues to

evolve and improve. During

1999, the HPCMP

implemented the Uniform

Use-Access Policy for

Unclassified DoD HPCMP

Resources to ensure that

security requirements for

obtaining accounts at all

HPCMP shared resource

centers are adequate to

protect HPC resources. In

September 2001, the

Uniform Use-Access Policy

for Unclassified DoD

HPCMP Resources was

revised to address Foreign

National (FN) use of

computer systems. The

revision states that, while

interim access is permitted

for most new users based on

initiation of the appropriate

trustworthiness application, a

FN is not allowed interim

access. The program also

instituted independent NSA

assessments at MSRCs and

selected DCs. These

assessments provided an

independent review and

verification of the sites’

security postures.

Centralized network

intrusion detection systems

(NIDS) coverage has been

expanded to all HPC centers

and at access points where

the DREN meets the Internet

and other networks. The

HPCMP Computer

Emergency Response Team

(CERT) is in place to support

all HPCMP users. The

HPCMP continued a joint

project with the Department

of Energy (DOE) to improve

the DOE-owned Joint

Intrusion Detection (JID)

software used on the HPCMP

NIDS. HPCMP researchers

at the Army Research

Laboratory modified the JID

software so that it can

monitor high-speed

Asynchronous Transfer Mode

(ATM) network traffic. The

project improvement

increases the number of

locations that perform

intrusion detection and

allows the program to

monitor all DREN traffic.

The project is complete and

the ATM JIDS are now

available in OC-3 and OC12 versions. The program

has also increased the level

of security filtering done

across the DREN. In concert

with the Services and

Defense Information Systems

Agency (DISA), access

control lists (ACLs) on

devices at network access

points have been expanded

to provide increased

protection from known

sources of attack. HPCMP

ACLs are coordinated with

the Services and DISA lists to

ensure coverage that meets

all DoD criteria.

SOFTWARE

APPLICATIONS SUPPORT

Software Applications

Support contains three areas:

the Common High

Performance Computing

Software Support Initiative

(CHSSI), which has been

producing efficient

Impact to DoD: Naval engineers are using high performance

computers to reduce design costs and support improvements in the

design and operational safety of submarines.

Turning maneuver

induced by a 10

degree rudder

deflection

(selected views)
38

2002 HPC Modernization Plan

applications software codes

for the last several years,

Programming Environment

and Training (PET), which is

realigned from the HPC

Shared Resource Centers,

and Software Protection.

PET activities will continue

to be associated with both

Shared Resource Centers’

and Software Applications

Support activities.

COMMON HIGH

PERFORMANCE

COMPUTING SOFTWARE

SUPPORT INITIATIVE

The program office

promulgated a call for

proposals for CHSSI projects

to DoD laboratories and test

centers in fiscal year 2000.

Numerous proposals were

received and evaluated.

Seventeen new CHSSI

projects were selected for

initiation at the start of fiscal

year 2001. CHSSI

completed the fiscal year

2001 selection process and

started three new portfolios:

Materials by Design, Sensor/

Scene Processing and

Generation, and System-ofSystems Simulation.

CHSSI continued to develop,

test, and release software in

fiscal year 2001 to take

advantage of advances in

high-speed computing and

communications technology,

especially in distributed

computing, wide-area

computing environments,

massively parallel high

performance computing

architectures, and remote

desktop collaboration.

CHSSI, Fiscal Year 2002

The initiative provides

scientists and engineers,

especially those in the test

and evaluation community,

an easily understandable

view of what the program

has done, what the program

is doing, and what the

program hopes to

accomplish. A uniform and

user-friendly reporting system

is being established for the

CTA Leaders. This will

provide an accessible status

of projects and portfolios,

reduce the need for special

data calls, and provide

HPCMP management with

relevant data. An extranet

page was established to keep

CTA Leaders up to date on

key events and issues, to

share progress and concerns,

and to provide an accessible

repository of policy and

procedures documents.

There were six new CHSSI

projects selected in fiscal

year 2002. The next call for

project proposals will be

released in May 2002.

Software Applications Support (Starting Counter Clockwise):

Simulation of the flow unsteadiness on the tail-buffet of the

Comanche Helicopter. Picture of a Comanche Helicopter. Scalable

Global Weather Forecast System: CWO project to develop and

validate scalable and portable atmospheric models using the Navy

Global Atmospheric Prediction System (NOGAPS). Mine clearing

apparatus. Grid design of a mine clearing blade subjected to land

mine explosion.
39

PROGRAM ACCOMPLISHMENTS AND FUTURE PLANS

CHSSI, Fiscal Year 2003

Software development,

testing, and deployment will

continue in fiscal year 2002.

Some CHSSI projects will

reach completion and be

deployed to the user

community. The HPCMO

will solicit and evaluate

proposals for new projects.

These new projects would

allow the DoD to develop

and transition additional key

DoD applications software to

work effectively on new

scalable architectures.

PROGRAMMING

ENVIRONMENT AND

TRAINING

The Programming

Environment and Training

(PET) vision is to enhance

HPCMP user productivity by

providing world-class

technical support and

training through strategic

partnerships, and focused

enabling-tool development

and deployment. The goals

of PET are to train and

support new and existing

DoD HPCMP users with the

knowledge and tools to

maximize productivity.

Other goals are to foster

collaboration across

functional, Service, Agency,

and national HPC

communities, and to

facilitate the application of

HPC and high performance

networks on new areas of

interest to the DoD.

To accomplish these goals,

PET provides an integrated

portfolio (short term and long

term) of computational

science and engineering

activities [support, training,

targeted development,

Minority Serving Institutions

(MSIs), graduate and postdoctoral programs, summer

internships] conducted by

leading HPC institutions and

organizations. PET also

identifies, evaluates,

recommends, adapts,

develops, and deploys (in

concert with the shared

resource centers) enabling

software tools and

technologies into the shared

resource center user

environments. PET promotes

and deploys collaborative

and distributed computing

environments to facilitate

virtual teaming and

computing.

The PET activity was

previously a subset of the

initial MSRC integration

contracts. A full and open

competition was determined

to be the optimum approach

to acquiring these services

for a follow-on effort. A

Request for Proposal was

released in December 2000.

Two contracts were awarded

in June 2001 to contractors

were High Performance

Technologies, Inc. (HPTi) and

MOS University Consortium.

These contracts, which

consist of a three-year base

period with five one-year

options, have a potential

value of $144 million.

These two contracts, with

representation from over 20

universities and industry

partners (Figure 4), were

awarded under full and open

competition.

The PET activity is now

divided into four

components to facilitate

government oversight. Each

component will execute a

program that supports the

designated functional areas

listed in Table 8 on page 41.

Functional areas were

grouped to encourage

synergy among related CTAs,

collaboration and interaction

between CTA and crosscommunity functions (e.g.,

enabling technologies), and

to balance the workload

across the four components.

PET, Fiscal Year 2002

Programming Environment

and Training activities will

continue to emphasize

distance learning and

distributed training, ensuring

that defense scientists and

engineers can take advantage

of expanded high

performance computing

capabilities as they become

available.
40

2002 HPC Modernization Plan

PET, Fiscal Year 2003

The HPCMP will continue to

refine the PET activity based

on feedback obtained from

users, management, and the

contractors. This work will

assist the DoD scientists and

engineers in exploiting the

available HPCMP assets.

USER REQUIREMENTS

AND RESOURCE

ALLOCATIONS

The HPCMP will improve

requirements and user base

information gathering from

DoD computational projects.

Analysis of that data supports

program planning and

operational decisions. A

user-friendlier, web-based

version of the requirements

questionnaire was developed

and will be deployed in

fiscal year 2002.

The program will continue

its process of allocating

valuable HPC resources

among the Services and

Agencies. This process

reserves approximately 25%

of total HPCMP resources for

DoD Challenge Projects.

The remaining 75% is

allocated by each Service/

Agency among its individual

computational projects. In

fiscal year 2002, the

program will extend its

formal Service/Agency

allocation process to include

not only the MSRCs, but

selected DCs as well.

DOD HIGH

PERFORMANCE

COMPUTING

CHALLENGE PROJECTS

As discussed above,

approximately 25% of the

program’s total

computational resources are

allocated annually to highpriority Service and Agency

projects with very large

computational requirements.

These mission-critical,

computationally intensive

DoD Challenge Projects are

approved by the Deputy

Under Secretary of Defense

(Science and Technology).

The DoD Challenge Projects

Allocation Board—with

representatives from each

Service, selected Defense

Agencies, and the broader

HPCMP community—

evaluates the proposals.

Forty-one projects were

selected for implementation

in fiscal year 2002. Many of

the projects use multiple

hardware platforms and in

some cases, multiple shared

resource centers.

Computationally, these

projects were allocated an

average of 72 gigaflops-years

for fiscal year 2002. Table 9

lists the fiscal year 2002

DoD HPC Challenge

Projects. New projects will

be selected during fiscal year

2002 for implementation in

fiscal year 2003, continuing

Figure 4. PET Team Members
41

PROGRAM ACCOMPLISHMENTS AND FUTURE PLANS

the annual selection and

evaluation process.

The HPCMO will continue

to address the special needs

of the DoD Challenge

Projects. The program office

will continue to promote the

use of CHSSI software in

DoD Challenge Projects.

Each year, special DoD

Challenge Project sessions

will be organized at a set of

four national or international

meetings, including the

HPCMP Users Conference.

User satisfaction survey data

will be analyzed to

determine the differences

between DoD Challenge

Project user responses and

responses from all other

users.

PROGRAM OVERSIGHT

AND REVIEW

During fiscal year 1998, the

HPCMP performed the Initial

Operational Test and

Evaluation of its systems in

compliance with federal

statutes and Defense

Department regulations. The

Joint Interoperability Test

Command (JITC) served as

the independent test activity

and performed the initial

operational evaluation of the

program. The command

witnessed the evaluation of

two MSRCs, two DCs, and

two CHSSI code

development projects as well

as site acceptance testing of

the DREN.

In January 1999, the JITC

issued an Independent

Evaluation Report stating the

HPC centers and network are

operationally effective and

operationally suitable. The

report stated,

The HPCMP is

operationally effective

and suitable when

operated by its

intended users in its

operational

environment. Users

generally agreed the

HPCMP has

revolutionized their

ability to conduct

scientific and

engineering research.

[emphasis added]

They indicated that

problems, while

needing attention, are

minor to the HPCMP’s

overall contribution to

their research efforts.”

The results of the evaluation

represent a significant step

for the program in the formal

oversight process. The

Independent Evaluation

Report for CHSSI projects

was also favorable. The

annual test report to

Congress by the Director,

Table 8. CTA Component Functional Areas

Component

Number

CTAs and Functional Areas

1 Climate/Weather/Ocean Modeling and Simulation

Environmental Quality Modeling and Simulation

Computational Environment

2

Forces Modeling and Simulation/C4I

Integrated Modeling and Test Environments

Signal/Image Processing

Enabling Technologies

3

Computational Fluid Dynamics

Computational Structural Mechanics

PET Online Knowledge Center

Education, Outreach, and Training Coordination

4

Computational Chemistry and Materials Science

Computational Electromagnetics and Acoustics

Computational Electronics and Nanoelectronics

Collaborative and Distance Learning Technologies
42

2002 HPC Modernization Plan

Table 9. DoD High Performance Computing Challenge Projects (Fiscal Year 2002)

Project P ) roject Leader(s Organization

Computational Structural Mechanics

Evaluation and Retrofit for Blast Protection in

Urban Terrain

James Baylot, Tommy Bevins,

James O'Daniel, Young Shon,

David Littlefield, and

Chrisopher Eamon

Engineer Research and

Development Center, Vicksburg,

MS

Modeling Complex Projectile Target Interactions Kent Kimsey and Gordon

Filbey, Jr.

Army Research Laboratory,

Aberdeen Proving Ground, MD

Three-Dimensional Modeling and Simulation of

Bomb Effects for Obstacle Clearance

Alexandra Landsberg Naval Surface Warfare Center,

Indian Head Division, Indian

Head, MD

Computational Fluid Dynamics

3-D Computational Fluid Dynamics Modeling of

the Chemical Oxygen-Iodine Laser (COIL)

Timothy Madden Air Force Research Laboratory,

Kirtland Air Force Base (AFB), NM

Active Control of Fuel Injectors in Full Gas Turbine

Engines

Suresh Menon Army Research Office, Research

Triangle Park, NC

Airdrop System Modeling for the 21st Century

Airborne Warrior

Keith Stein, Richard Benney,

Tayfun Tezduyar, James

Barbour, Hamid Johari, and

Sonya Smith

Army Soldier & Biological

Chemical Command, Natick, MA

Analysis of Full Aircraft with Massive Separation

Using Detached-Eddy Simulation

Kenneth Wurtzler, Robert

Tomaro, William Strang,

Matthew Grismer, Capt James

Forsythe, Kyle Squires, and

Phillipe Spalart

Air Force Research Laboratory,

Wright-Patterson AFB, OH

Applied CFD in Support of Aircraft-Store

Compatibility and Weapons Integration

James Brock, Jr. Air Force Seek Eagle Office, Eglin

AFB, FL

High-Fidelity Analysis of UAVs Using Nonlinear

Fluid/Structure Simulations

Reid Melville and Miguel

Visbal

Air Force Research Laboratory,

Wright-Patterson AFB, OH

High-Resolution Adaptive Mesh Refinement (AMR)

Simulation of Confined Explosions

John Bell, Charles

Rendleman, William

Crutchfield, and Allen Kuhl

Defense Threat Reduction Agency,

Alexandria, VA

Hybrid Particle Simulations of High Altitude

Nuclear Explosions in 3-D

Stephen Brecht Defense Threat Reduction Agency,

Alexandria, VA

Large-Eddy Simulation of Steep Breaking Waves

and Thin Spray Sheets Around a Ship: The Last

Frontier in Computational Ship Hydrodynamics

Ki-Han Kim, Dick Yue, and

Douglas Dommermuth

Naval Surface Warfare Center,

Carderock Division, Bethesda, MD

Large-Eddy Simulation of Tip-Clearance Flow in

Stator-Rotor Combinations

Parviz Moin Office of Naval Research,

Arlington, VA

Modeling of Microbubble Drag Reduction Physics Parney Albright, James

Uhlman, and Leonard Iman

Defense Advanced Research

Project Agency, Arlington, VA

Multiphase CFD Simulations of Solid Propellant

Combustion in Modular Charges and

Electrothermal-Chemical (ETC) Guns

Michael Nusca and Paul

Conroy

Army Research Laboratory,

Aberdeen Proving Ground, MD
43

PROGRAM ACCOMPLISHMENTS AND FUTURE PLANS

Table 9—Continued. DoD High Performance Computing Challenge Projects

(Fiscal Year 2002)

Project Project Leader(s) Organization

Computational Fluid Dynamics (continued)

Numerical Modeling of Wake Turbulence for Naval

Applications: Vortex Dynamics and Late-Wake

Turbulence in Stratification and Shear

Steve Ardent, Donald Delisi,

David Fritts, Maria-Pascale

LeLong, James Riley, Robert

Robins, and Joe Werne

Office of Naval Research,

Arlington, VA

Parallel Simulations of Weapons/Target

Interactions Using a Coupled CFD/Computational

Structural Dynamics (CSD) Methodology

Joseph Baum Defense Threat Reduction Agency,

Alexandria, VA

Three-Dimensional, Unsteady, Multi-Phase CFD

Analysis of Maneuvering High Speed

Supercavitating Vehicles

Robert Kunz, Jules Lindau,

and Howard Gibeling

Office of Naval Research,

Arlington, VA

Time-Accurate Aerodynamics Modeling of

Synthetic Jets for Projectile Control

Jubaraj Sahu, Sukumar

Chakravarthy, and Sally Viken

Army Research Laboratory,

Aberdeen Proving Ground, MD

Time Accurate Computational Simulations of Ship

Airwake for DI, Simulation and Design

Applications

Susan Polsky Naval Air Warface Center Aircraft

Division, Patuxent River, MD

Unsteady RANS Simulation for Surface Ship

Maneuvering and Seakeeping

Ki-Han Kim, Joe Gorski, Fred

Stern, Lafayette Taylor, and

Mark Hyman

Naval Surface Warfare Center,

Carderock Division, Bethesda, MD

Computational Chemistry and Materials Science

Characterization of DoD Relevant Materials and

Interfaces

Emily Carter Air Force Office of Scientific

Research, Bolling AFB, DC

Computational Chemistry Models Leading to

Mediation of Gun Tube Erosion

Cary Chabalowski, Margaret

Hurley, Dan Sorescu, Roger

Ellis, Donald Thompson,

Betsy Rice, and Gerald

Lushington

Army Research Laboratory,

Aberdeen Proving Ground, MD

First Principles Studies of Technologically

Important Smart Materials

Andrew Rappe Office of Naval Research,

Arlington, VA

Interactions of Chemical Warfare Agents with

Acetlycholinesterase

William White E l dgewood Chemical & Biologica

C , enter, Aberdeen Proving Ground

MD

Multiscale Simulations of High Temperature

Ceramic Materials

Rajiv Kalia, Aiichiro Nakano,

and Priya Vashista

Air Force Office of Scientific

Research, Bolling AFB, DC

Multiscale Simulation of Nanotubes and Quantum

Structures

Jerzy Bernholc Office of Naval Research,

Arlington, VA

Multi-Scale Simulations of High Energy Density

Materials

Jerry Boatz Air Force Research Laboratory,

E A dwards AFB, C

New Materials Design Jerry Boatz, Ruth Pachter,

Mark Gordon, Gregory Voth,

and Sharon Hammes-Schiffer

Air Force Office of Scientific

Research, Bolling AFB, DC
44

2002 HPC Modernization Plan

Table 9—Continued. DoD High Performance Computing Challenge Projects

(Fiscal Year 2002)

Project Project Leader(s) Organization

Computational Electromagnetics and Acoustics

Airborne Laser Challenge Project II Wilbur Brown, Jeremy Winick,

and Robert Beland

Air Force Research Laboratory,

Kirtland AFB, NM

Directed High Power RF Energy: Foundation of

Next Generation Air Force Weapons

M. Joseph Arman Air Force Research Laboratory,

Kirtland AFB, NM

Radar Signature Database for Low Observable

Engine Duct Design

Kueichien Hill Air Force Research Laboratory,

Wright-Patterson AFB, OH

Seismic Signature Simulations for Tactical Ground

Sensor Systems and Underground Facilities

Mark Moran, Stephen

Ketcham, George McMechan,

Stig Hestholm, Roy

Greenfield, Thomas Anderson,

James Lacombe, Michael

Shatz, Robert Haupt, Richard

Lacoss, and Robert Greaves

Cold Regions Research and

Engineering Laboratory, Hanover

NH

Signature Modeling for Future Combat Systems Raju Namburu, Theresa

Gonda, John Nehrbass, Peter

Chung, Maggie Hurley, Steve

Bunte, William Coburn,

Christopher Kenyon, Calvin

Lee, John Escarsega, and

William Spurgeon

Army Research Laboratory,

Aberdeen Proving Ground, MD

Climate, Weather, and Ocean Modeling and Simulation

1/32 Degree Global Ocean Modeling and

Prediction

Alan Wallcraft, Harley

Hurlburt, Robert Rhodes, and

Jay Shriver

Naval Research Laboratory, John

C. Stennis Space Center, MS

Basin-scale Prediction with the HYbrid Coordinate

Ocean Model

Eric Chassignet Office of Naval Research,

Arlington, VA

Coupled Environmental Model Prediction (CEMP) Wieslaw Maslowski, Julie

McClean, Albert Semtner,

Rk a obin To makian, Yuxi

Z d hang, Ruth Preller, an

Steve Piacsek

Naval Postgraduate School and

Naval Research Laboratory,

Monterey, CA

Coupled Mesoscale Modeling of the Atmosphere

and Ocean

Richard Hodur Naval Research Laboratory,

Monterey, CA

High Fidelity Simulation of Littoral Environments G J eorge Heburn a y nd effer

Holland

Naval Research Laboratory, John

C. Stennis Space Center, MS and

Engineer Research and

D V, evelopment Center, icksburg

MS

Submerged Wakes in Littoral Regions Patrick Purtell, Roger Britey,

Joseph Gorski, and Douglas

Dommermuth

Office of Naval Research,

Arlington, VA

Signal/Image Processing

Automatic Target Recognition Performance

Evaluation

Timothy Ross Air Force Research Laboratory,

Wright-Patterson AFB, OH
45

PROGRAM ACCOMPLISHMENTS AND FUTURE PLANS

Coupled Environmental Model

Prediction (CEMP)

Wieslaw Maslowski and Julie McClean

Impact to DoD: Realistic simulation of the

present-day sea ice thickness distribution is

crucial to predicting a possibility of partial/

seasonal or full removal of permanent sea ice

cover in the Arctic Ocean during the next

century.

Evaluation and Retrofit for Blast

Protection in Urban Terrain

James Baylot

Impact to DoD: Provides an improved

methodology for evaluating safety of forces

from terrorist attack and for designing

retrofits to improve safety.

Impact to DoD: Allows better

prediction of turbulence in

massively separated flows

around complete aircraft

reducing the risk and cost of

designing future aircraft.

Analysis of Full Aircraft with Massive Separation

Using Detached-Eddy Simulation

Matthew Grismer
46

2002 HPC Modernization Plan

Operational Test and

Evaluation contained no

significant findings about the

program.

Following the test period,

acquisition oversight of the

program was formally

delegated from the Assistant

Secretary of Defense

(Command, Control,

Communications and

Intelligence) to the Deputy

Under Secretary of Defense

(Science and Technology)

(DUSD(S&T)).

In December 1998, the

Under Secretary of Defense

for Acquisition, Technology,

and Logistics formed an

Integrated Product Team (IPT)

under the leadership of the

DUSD(S&T). The IPT, after a

year of review, reaffirmed the

critical impact high

performance computing is

having on the DoD.

Specifically, they concluded

HPC is a critical enabling

tool that decreases the time

to solution, enables new and

innovative approaches, and

allows the solution of highly

complex, previously

unsolvable problems. They

further noted that the current

HPCMP not only leverages

common resources for both

the S&T and T&E

communities, it also

promotes the transition of

technology from science and

technology (S&T) to test and

evaluation (T&E).

In April 1999, the

HPCMP established a

process to evaluate the

effectiveness of

distributed center

selections. That

process is repeated

annually (1999, 2000,

and 2001).

The program will also

continue an aggressive

test program to ensure

defense scientists and

engineers receive high

performance

computing assets that are

operationally effective and

suitable for their work efforts.

This was also confirmed

during fiscal year 2001 as

the Director continued his

internal performance

assessment of the major

program components.

ACQUISITION

In fiscal year 2001, the

program office awarded

contracts for Technology

Insertion 2001 (TI-01), and

Programming Environment

and Training; the Defense

Research and Engineering

Network contract will be

awarded in fiscal year 2002.

The program office is

developing an acquisition

strategy for follow-on MSRC

support contracts and is

implementing the TI-02

acquisition. The program

office will also be analyzing

the possibility of a programwide software licensing

effort.

PROGRAM FUNDING

Funding amounts are based

on the High Performance

Computing Modernization

Program Office’s estimate

completed in September

2001. The HPCMP is

projected to spend $1.73

billion for fiscal years 2001–

2007. Table 10 shows the

funding profile by year and

major spending category.

Within the overall program,

the funding level currently

supports only a portion of

the validated high

performance computing

requirements (e.g., in fiscal

year 2002, the program is

Impact to DoD: High performance

computing enables more accurate

prediction of high frequency, high energy

dynamic environments for T&E of Army

missiles.
47

PROGRAM ACCOMPLISHMENTS AND FUTURE PLANS

providing approximately

40% of the computing

capability needed by the

user community).

Procurement funding

increases provided by

Congress in the fiscal year

2000 and 2001 Department

of Defense Appropriations

Acts, allowed the program to

support approximately 40%

of the documented

requirements in fiscal year

2002. Acquisition Decision

Memoranda are issued by

the Deputy Under Secretary

of Defense (Science and

Technology) annually.

A significant fraction of the

projected user requirements

cannot be adequately

addressed by the current

funding profile. Therefore,

as part of its strategic

planning efforts, the HPCMP

staff continues to examine

options to ensure that the

highest priority DoD needs

are met in light of these

shortfalls. An extensive HPC

resource allocation process

ensures the program’s

resources are allocated by

the Services and Defense

Agencies to the highest

priority projects of the

Defense Department.

The continued stable funding

of the CHSSI and the MSRC

sustainment budget lines

reflect the importance that

the program places on

software, integration, and

related shared services to its

HPC user community. These

investments will continue to

guarantee that the program’s

focus remains on enabling

users to address warfighter

needs, not only by

maintaining the support

infrastructure, but also by

providing effective

mechanisms for continued

leverage of both national and

industrial investments in

HPC-based productivity.
48

TABLE 10. Department of Defense High Performance Computing Modernization Program Fu

(Fiscal Years 1997–2007)

Fiscal Year 1997 1998 1999 2000 2001 2002 2003 2004 200

Procurement (Capital

Investments) 122.7 87.1 91.4 95.4 79.0 75.3 49.5 50.4 51.9 

Research, Development, Test and Evaluation

Networking (IncludesSecurity) 16.0 21.7 17.0 28.6 34.5 31.2 32.9 30.4 30.9 

Software Applications

Support 21.1 20.9 23.6 20.5 21.9 60.5 54.8 57.4 58.0 

Major Shared ResourceCenters 52.8 72.1 87.6 85.8 88.2 75.0 78.5 78.0 77.5 

Distributed Centers 28.5 24.3 23.4 26.5 2 1 7. 16.8 22.5 23.4 23.9 

PROGRAM TOTAL 241.1 226.1 2 0 43. 2 8 56. 2 7 50. 2 8 58. 2 83 .1 2 5 39. 242.
49

Acronyms

AAC Air Armament Center

ACLs access control lists

AEDC Arnold Engineering Development Center

AFB Air Force Base

AFFTC Air Force Flight Test Center

AFRL/IF Air Force Research Laboratory Information Directorate

AFRL/SN Air Force Research Laboratory Sensors Directorate

AHPCRC Army High Performance Computing Research Center

ARL Army Research Laboratory

ARSC Arctic Region Supercomputing Center

ASC Aeronautical Systems Center

AT&L Acquisition, Technology and Logistics

ATM Asynchronous Transfer Mode

CAT Collective Acquisition Team

CCM Computational Chemistry and Materials Science

CEA Computational Electromagnetics and Acoustics

CEMP Coupled Environment Model Prediction

CEN Computational Electronics and Nanoelectronics

CERT Computer Emergency Response Team

CFD Computational Fluid Dynamics

CHSSI Common High Performance Computing Software Support Initiative

COTR Contracting Officer’s Technical Representative

COTS Commercial-off-the-Shelf

CSM Computational Structural Mechanics

CTA Computational Technology Area

CWO Climate/Weather/Ocean Modeling and Simulation

DCs distributed centers

DDR&E Director, Defense Research and Engineering

DISA Defense Information Systems Agency

DoD Department of Defense

DOE Depatment of Energy
50

2002 HPC Modernization Plan

DREN Defense Research and Engineering Network

DTOs Defense Technology Objectives

DUSD(S&T) Deputy Under Secretary of Defense (Science and Technology)

EBE Modeling and Simulation of the Electronic Battlefield Environment

EQM Environmental Quality Modeling and Simulation

ERDC Engineer Research and Development Center

FMS Forces Modeling and Simulation/C4I

FN foreign national

HIE Hyperspectral Image Exploitation

HPC high performance computing

HPCMO High Performance Computing Modernization Office

HPCMP High Performance Computing Modernization Program

HPTi High Performance Technology, Inc.

IMT Integrated Modeling and Test Environments

IPT Integrated Product Team

JID joint intrusion detection

JITC Joint Interoperability Test Command

JNIC Joint National Integration Center

JWCOs Joint Warfighting Capability Objectives

JWSTP Joint Warfighting Science and Technology Plan

MBD Materials by Design

MHPCC Maui High Performance Computing Center

Mbps million bits per second

MSIs Minority Serving Institutions

MSRCs major shared resource centers

NAVO Naval Oceanographic Office

NAWCAD Naval Air Warfare Center Aircraft Division

NAWCWD Naval Air Warfare Center Weapons Division

NIDS network intrusion detection systems

NRL-DC Naval Research Laboratory-Washington, DC

NSA National Security Agency

PET Programming Environment and Training

RTTC Redstone Technical Test Center

S&T science and technology

SIP Signal/Image Processing

SLE High Fidelity Simulation of Littoral Environments
51

ACRONYMS

SMDC Space and Missile Defense Command

SOS Systems of Systems Simulations

SPDs service delivery points

SPG Sensor/Space Processing and Generation

SSA Source Selection Authority

SSCSD Space and Naval Warfare Systems Center, San Diego

T&E test and evaluation

TARDEC Tank Automotive Research, Development and Engineering Center

TI Technology Insertion

TI-01 Technology Insertion 2001

TI-02 Technology Insertion 2002

USD(AT&L) Under Secretary of Defense (Acquisition, Technolgy, and Logistics)

VPN virtual private network

WAN wide area network

WSMR White Sands Missile Range

WTI Interdisciplinary Computational Environment for Weapon Target Interations


Department of Defense

High Performance Computing Modernization Office

1010 North Glebe Road, Suite 510

Arlington, VA 22201

Phone: (703) 812-8205

Facsimile: (703) 812-9701

http://www.hpcmo.hpc.mil