DoD HPC Modernization Office

1010 North Glebe Road, Suite 510

Arlington, VA 22210

Phone: (703) 812-8205

FAX: (703) 812-9701

http://www.hpcmo.hpc.mil
contents

iii Foreword

1 Executive Summary

3 Program Description

15 Program Accomplishments

21 HPC Contributions to Mission Success

29 Implementation Strategy

Cover captions (left to right):

Top row –

Snapshot from a centroid molecular dynamics simulation of the

proton transfer complex (blue and green) in water (red and white)

Cut planes and isosurfaces of mass density (in kg/m3) of radially

compressing plasma at 50 ns. The classical Rayleigh-Taylor

spike and bubble pattern is easily recognized. The strong

azimuthal variations are an indication of important 3-D physics.

Simulated Arctic ice thickness (meters) for June 30, 1992

Center row –

General configuration for picture-frame-mounted surface acoustic

wave resonator

Parallel computation of scattered fields from a fighter for radar-crosssection prediction

Bottom –

Satellite image reconstruction: on-line compensation by adaptive

optics, with postprocessing on the 400-node SP2 at the Maui

High Performance Computing Center
i

HIGH PERFORMANCE

COMPUTING

MODERNIZATION PLAN

March 1997
iii
1

1

High Performance Computing (HPC) is a key

enabling technology that is essential to maintain and

extend the United States’ technological advantage in

warfighting systems. The HPC Modernization Program is the major force designed to improve the

Department of Defense’s (DoD) ability to exploit the

computation necessary to sustain technological

superiority on the battlefield. Managed by the Director, Defense Research and Engineering (DDR&E), the

program is establishing a world-class, nationwide

integrated infrastructure to support the defense

research, development, test, and evaluation communities.

The HPC Modernization Program is creating

HPC environments at shared resource centers that are

accessible to local and remote scientists and engineers

via a high-speed network. Advised by user groups, the

program is underwriting the development of application building blocks, visualization tools, system utilities,

and mathematical libraries. The adaptation of broadly

based, widely used, scalable applications and algorithms to address growing science, technology,

developmental test and evaluation requirements, along

with continued training of users as new architectures

and concepts evolve, is a vital part of the program.

The program’s four major shared resource centers are

establishing collaborative partnerships with several

civilian HPC centers in order to draw civilian expertise

into the DoD.

Fiscal year 1996 saw the award of five major

contracts to support the modernization of DoD’s HPC

capability. Four of these contracts will procure integration services to provide multiple large computer

systems and expert support staffs at four major shared

resource centers. The fifth contract will provide high

speed, high bandwidth connectivity to the program’s

HPC centers. With an average annual budget of

approximately $220 million, the program supports the

research activities of more than 4200 scientists and

engineers at nearly 100 defense laboratories, test

centers, academic institutions, and industrial sites.

The successful award of the contracts is a major

accomplishment for the HPC Modernization Program,

but there is still much work ahead. While past efforts

have been focused on acquisition activities, our efforts

are now directed toward developing seamless computing environments across our centers. A robust suite of

programming and developing tools, shared file systems, visualization capabilities, and interactive collaborative environments are needed to maximize the

productivity of our most valuable resources—our

scientists and engineers.

EXECUTIVE SUMMARY
3

PROGRAM DESCRIPTION

3

Introduction

High performance computing (HPC) has played

and will continue to play a major role in the ability of

the United States to maintain and increase the

technological superiority of its warfighting support

systems. Very high fidelity modeling and simulation

are the keys to rapid insertion of advanced technology into the design, test, and evaluation processes.

The modernization of the Department of Defenses’

(DoD) HPC capability will allow solutions to a wide

variety of military-unique applications that require

tremendous computing power.

Defense researchers are developing high-end

applications that include calculating stealth aircraft

signatures; modeling, at the molecular level, the flow

of air or water across the surface of weapons

systems; predicting sea lane weather; and simulating

tests augmented with live testing. The use of HPC

technology has already led to lower costs for system

deployment, helped avoid damage to the environment, and improved the effectiveness of complex

weapons systems. As DoD transitions its weapons

systems design and test process to rely more heavily

on modeling and simulation, the nation can expect

many more examples of the profound effects that

the HPC capability has on both military and civilian

applications. Better use of computing technology is

necessary to provide the United States with the

military advantage required for operation in the 21st

century.

Background

In 1991, Congress noted that DoD laboratories

lagged behind the Department of Energy, National

Aeronautics and Space Administration, and National

Science Foundation supercomputer centers in HPC

capability. Congress directed the DoD to establish a

program to overcome this deficiency. This prompted

the establishment of the DoD HPC Working Group,

which developed the HPC Modernization Plan

(delivered to Congress in May 1992). This group

consisted of representatives from the military services, Defense Special Weapons Agency, Defense

Advanced Research Projects Agency, and the Office

of the Assistant Secretary of Defense for Command,

Control, Communications, and Intelligence.1 The

DoD High Performance Computing Modernization

Office (HPCMO) was subsequently established and

staffed in fiscal year 1994 to ensure that the program supports the needs of the DoD science, technology, developmental test and evaluation programs.

The achievements of the HPC Modernization

Program are already visible in the success stories of

our laboratories and test centers. The expectations

for this program are that it will exert the force and

the impetus needed by our scientists and engineers

to provide new solutions to the problems of national

defense within ever-constrained resources.

1

For a detailed description of these activities, refer to the Department of Defense High Performance Computing Modernization Plan,

March 1992 and June 1994.

Vision and Goals

The High Performance Computing modernization vision is to enable the United

States to maintain its technological supremacy over its adversaries in weapon

systems design and to foster the flow of

this technology into warfighting support

systems.

Under the auspices of the Director, Defense

Research and Engineering (DDR&E), the High

Performance Computing Modernization Program is

the major force that improves DoD’s ability to

exploit computation to sustain technological superiority. This effort will increase the DoD’s HPC and

communication capabilities to a level equal to or

greater than those available in the foremost academic research centers and industry. The HPC

Modernization Program involves creating HPC

environments at four large centers and several

smaller centers accessible to local and remote

scientists and engineers via high-speed networks.

The ability to provide for the adaptation of broadly

based, widely used, scalable applications and algo-
4

rithms to address growing science, technology,

developmental test and evaluation requirements is an

integral component of the program, along with

continued training of users as new architectures and

concepts evolve. The program pursues continuous

interaction with the national HPC infrastructure,

including academe, industry, and other government

agencies to capture computing innovation and rapidly

transfer it to the science, technology, developmental

test and evaluation communities. The goals of the

HPC Modernization Program are to:

• Buy the best commercially available high

performance computers,

• Buy and develop software tools and programming environments,

• Expand and train the DoD user base,

• Link users and computer sites via high-speed

networks (create collaborative work environments), and

• Exploit the best ideas, algorithms, and software tools of the national HPC infrastructure.

Management Strategies

Modernization of DoD’s HPC capability and

fulfillment of the programs’ vision and goals requires

a strategy that addresses all aspects of HPC. The

programs’ management strategies are:

(1) Goal—buy the best commercially available

high performance computers.

• Provide a balanced set of commercially

available heterogeneous computing

environments for HPC systems to meet

the full range of DoD requirements and to

permit optimum mapping of requirements

to system types.

• Build complete HPC environments,

including large computing systems, at DoD

laboratories and research developmental

test and evaluation centers designated as

major shared resource centers to support

the community of users by providing large

job service and establishing software

environments and other support expertise.

• Place modest-sized high performance

computational systems at selected, distributed centers, when that placement enriches the overall DoD capability for high

performance computing.

• Support efforts designed to provide

complementary HPC capabilities and

technologies that are applicable to DeHPC Modernization Plan

fense science, technology, developmental

test and evaluation requirements.

(2) Goal—buy and develop software tools and

programming environments.

• Focus on standards in DoD research

development test and evaluation software

to ensure that future transitions and

advancements in software technology are

applied in an efficient and cost effective

manner.

• Focus on applications-oriented software

initiatives designed to overcome the

technological inhibitors that delay the

effective use of many scalable parallel

high performance computers.

• Support software components used in

specific computational areas or by a

subset of users across the community.

• Identify software suites to be shared

across selected major shared resource

centers and distributed centers.

(3) Goal—expand and train the DoD HPC user

base.

• Provide user education in computer and

computational sciences, software applications and optimization, and code conversion.

• Promote the formation of DoD-sponsored

interdisciplinary teams and collaboration

groups to determine how best to support

each computational area and to leverage

academe and industry expertise in HPC

for the solution of DoD problems.

• Develop shared-application area-specific

software in support of high-priority and

broadly needed computational application

areas.

• Aggressively transfer expertise and

knowledge among the user community.

Expand the knowledge of the user

community as applications require.

(4) Goal—link users and computer sites via highspeed networks (create collaborative work

environments).

• Deploy robust and cost effective network

connectivity consistent with HPC traffic

requirements among and between DoD

researchers and HPC assets and between

the DoD and external research and

development communities.
5

• Provide high-speed networking to

connect users to a variety of centers, as

required.

• Encourage remote usage

• Ensure that the highest priority applications are served.

(5) Goal—exploit the best ideas, algorithms, and

software tools of the national HPC infrastructure.

• Ensure that DoD science, technology,

developmental test and evaluation users

remain cognizant of, interact with, and

leverage internal DoD HPC initiatives,

such as the Defense Advanced Research

Projects Agency technology development

efforts and the software development

initiatives of the Defense research offices.

• Ensure that DoD science, technology,

developmental test and evaluation users

remain cognizant of, collaborate with,

and leverage other Government agencies’ and each other’s HPC efforts.

• Maintain cooperative contact with the

Computing, Information and Communication, Research and Development

Subcommittee of the Federal High

Performance Computing and Communications Program through the High End

Computing and Communications Working Group.

• Provide HPC environments to support

Defense applications that are directly

related to dual-use technologies and

National Challenges, e.g., environment,

medical, automotive, and manufacturing

processes and products.

• Ensure the availability of HPC environments to support Defense applications

that address DoD Challenge Projects

needs, e.g., meteorological, oceanographic, and environmental modeling;

nature of new materials; structure of

biological and chemical phenomena; and

energy efficient vehicles and airplanes.

• Leverage the nation’s HPC infrastructure

to benefit Defense research and development.

PROGRAM DESCRIPTION

 2

The HPC Modernization Program identifies science, technology, developmental test and evaluation as functional areas in which the

program provides support to all encompassing HPC requirements.

resources for defense science, technology, developmental test and evaluation programs.2 The communities supported by this program have been defined

by the Congress to include:

“. . . capability at (1) the DoD Science and

Technology sites under the cognizance of

the Director, Defense Research and Engineering, (2) the DoD Test and Evaluation

centers under the Director, Test and Evaluation (Office of the Under Secretary of

Defense, Acquisition and Technology),

and (3) the Ballistic Missile Defense Organization . . .”

(Public Law 104-61, December 1, 1995, 109

Statute 665, Sec. 8073).

By limiting the program’s scope to HPC systems

and focusing the majority of computational resources

at a few consolidated shared resource centers, the

program is able to acquire and operate high-end

resources that cannot easily be obtained by individual

services or agencies. The definition of “high performance” changes with the evolution of technology.

The program’s mission statement is specifically

written to follow technological improvements,

effectively transitioning to more powerful systems as

previous levels of capability become affordable at the

laboratory or project level.

Computational Technology Areas

The HPC Modernization Program user base

includes over 4000 scientists and engineers at 100

defense laboratories, test centers, universities, and

industry sites across the nation. Users within the

program have organized themselves into 10 computational technology areas. Table 1 lists the 10

Computational Technology Areas and their descriptions. Each computional technology area has a

designated leader who is a prominent DoD scientist

or engineer working within the scientific domain of

the computional technology area. Each computional

technology area leader chairs a working group

comprised of scientists and engineers from each of

the services, with participation by Defense agencies

as appropriate. These computational technology

area working groups serve as focused user groups to

drive program investment strategies for hardware,

software, and support services decisions. Table 2 lists

the computional technology areas and the leaders.

In addition to being an effective tool for directing

major facilities investments, organizing around the

Scope

The HPC Modernization Program provides and

sustains high-end high performance computing
6

computational technology areas also provides an

effective template for assessing both the importance

and the opportunities of HPC to sustain U.S.

warfighting technical superiority. For example, the

degree of linkage between the activities of the HPC

Modernization Program user community and

warfighting objectives can be demonstrated by crossreferencing the computational technology areas to the

specific warfighting priorities—the Joint Warfighting

Capability Objectives determined by the Joint Chiefs

of Staff. Table 3 shows the correlation between Joint

Warfighting Objectives and work under way in each of

the computational technology areas.

HPC Modernization Plan

Table 1 - Computational Technology Area Descriptions

technology, developmental test and evaluation user

communities. A vigorous requirements analysis process has been implemented, which includes annual

validation of requirements to provide accurate, timely

data upon which to base program decisions. The

requirements analysis process includes an extensive

program of surveys sent to every user and every

known potential user, coupled with visits to selected

DoD HPC user sites to discuss requirements with and

provide program information to individual user

groups. In addition to the formal requirementsgathering process, the program also obtains feedback through user satisfaction surveys, annual user

group meetings, and a users advisory council that

works with a site managers council to establish

operational policies and procedures at the program

centers.

User Requirements

One of the most critical program activities has

been the constant dialog with the defense science,

Computational Structural Mechanics covers the highresolution multidimensional modeling of materials and

structures subjected to a range of loading conditions

including static, dynamic, and impulsive loads. Example

applications for this area include structural analysis,

structural acoustics, ship response to underwater

explosion, and weapon system lethality/survivability.

Computational Chemistry and Materials Science

focuses on the basic properties of new chemical species

and materials that may be difficult or impossible to obtain

experimentally, such as molecular geometry, intermolecular forces, reaction energies, and mechanical

properties. Applications include new rocket propellants

and better materials for weapons systems.

Computational Fluid Dynamics, the accurate

numerical solution of the equations describing fluid and

gas motion, finds its primary use in the analysis and

engineering design of complex systems. Specific

applications include studies of submarine maneuverability

in shallow water, the dynamic response of military stores

during launch, the redesign of aircraft tail components to

reduce fatigue, and simulations for improved fire/

explosion safety and destruction of obsolete munitions.

Environmental Quality Modeling and Simulation

focuses on the solution of coupled, multidimensional

equations for surface, overland, and subsurface hydrodynamics and transport, plus the response thereto of biotic

species. Applications for this area include assessment of

ecosystem impacts from DoD activities, cleanup of

environmental impacts resulting from past DoD practices, and establishment of effective management of

natural and cultural resources under DoD stewardship.

Integrated Modeling and Test Environments uses

simulation techniques in conjunction with live-fire tests,

hardware-in-the-loop testing, and evaluations.

Applications include real-time trajectory calculations,

digital scene generation, and the simulation of weapon

system components in a virtual operations context.

Forces Modeling and Simulation covers the use of

command, control, communications, computer, and

intelligence for battle space management, complex

engagement training, and real-time decision-making

support. Areas of potential impact for DoD include

precision strike planning and force-level simulations.

Computational Electromagnetics and Acoustics

investigates the high-resolution multidimensional solution

to the equations of electromagnetics and acoustic wave

propagation in different media. Applications include

antenna array design, electromagnetic and acoustic

signature identification, surveillance and communication

enhancement, countermine measures, and optimizing

stealth aircraft.

Computational Electronics and Nanoelectronics

involves network and field theory, linear/nonlinear

analysis, synthesis and formal design methods to model

and simulate complex electronic devices, integrated

circuits, and systems of devices. Areas impacted by these

codes include device synthesis, optimization of highfrequency sensors and processors, design of tomorrow’s

faster computers, and advanced communications.

Climate/Weather/Ocean Modeling and Simulation

uses numerical computation to model the Earth’s

climate, oceanic variability, and atmospheric changes.

Numerical simulations provide strategic weather forecasting and long-term climatological prediction.

Signal/Image Processing includes the extraction of

useful information from sensor output in real time. DoD

applications in areas of surveillance, communication, and

electronic warfare include the development of advanced

radars, sonars, and image-processing systems.
PROGRAM DESCRIPTION 7

Figure 1 shows the HPC requirement for a few

key DoD problems. In addition to the problems

identified in the figure, increasing fidelity will be

required for newly emerging HPC applications, such

as model-driven test and evaluation and high-fidelity

forces modeling and simulation. Other important

hardware requirements, such as on-line file storage,

archival data, and network bandwidth will require

increases in capabilities to support growing requirements.

Computational Technology Areas

Table 3 - Correspondence Between Joint Warfighting Capability Objectives

and HPC Modernization Program Computational Technology Areas

Table 2 - Computational Technology Areas and Leaders

from 1996 Requirements Analysis

Computational Technology Area L s eader and Location E-mail Addres

Computational Structural Mechanics (CSM) K l ent Kimsey, Army Research Laboratory kimsey@arl.mi

Computational Fluid Dynamics (CFD) J l ay Boris, Naval Research Laboratory boris@lcp.nrl.navy.mi

Computational Chemistry and Materials Science (CCM) Scott Wierschke, Phillips Laboratory wierscsg@quantum.ple.af.mil

Computational Electromagnetics and Acoustics (CEA) J l oe Shang, Wright Laboratory shang@fim.wpafb.af.mi

Climate/Weather/Ocean Modeling and Simulation (CWO) Joe McCaffrey, Naval Research Laboratory –

Stennis Space Center mccaffrey@nrlssc.navy.mil

Signal/Image Processing (SIP) R l ichard Linderman, Rome Laboratory lindermanr@rl.af.mi

Forces Modeling and Simulation (FMS)/C4I

Bob Wasilausky, Naval Command, Control

and Ocean Surveillance Center, Research

Developmental Test and Evaluation Division

wasilaus@nosc.mil

Environmental Quality Modeling and Simulation (EQM) Jeff Holland, U.S. Army Corps of Engineers

Waterways Experiment Station hollanj@exl.wes.army.mil

Computational Electronics and Nanoelectronics (CEN) Barry Perlman, Army Research Laboratory –

Fort Monmouth bperlman@ftmon.arl.mil

Integrated Modeling and Test Environments (IMT) A l ndrew Mark, Army Research Laboratory amark@arl.mi

Joint

Warfighting Objectives C D SM C M F C A C C O E C P W S S I F M M E N Q C T E IM

Information Superiority hhhh h

Precision Force h hh hh

Combat Identification h h

Joint Theater Missile Defense hhhhhhh hh

Military Operations in Urban Terrain hhhhhh hh

Joint Readiness h hh

Joint Countermine hhhhhhhh

Electronic Warfare h h

Information Warfare h hh

Chemical/Biological Agent Detection hhhhhhh

Real-Time Logistics Control h h

Counter-Proliferation hhh hhhh h
8

The overall conclusion of this requirements

analysis is that a complete HPC environment must be

provided to take full advantage of HPC capabilities

within the DoD research, development test and evaluation program as it supports development of defense

warfighting systems. This environment requires a

broadly accessible infrastructure that includes (a) a

variety of well-balanced high-end computational platforms; (b) a full complement of systems and applications software by scientists and engineers in each of

the 10 computational technology areas; (c) software

tools that facilitate the transition of much of the DoD

computational workload to scalable systems; (d) a

reliable high-speed network that links the computational scientists, engineers, and analysts at their user

sites with the computational resources located at the

HPC Modernization Program’s shared resource

centers; (e) an aggressive training program that

continues the education and expansion of the DoD

HPC user base to accelerate the exploitation of HPC

capabilities. Progress must be balanced across all

program activities to optimize the impact of the HPC

Modernization Program on the DoD research, development test and evaluation program in its support of

the defense warfighting mission.

Fig. 1 - Requirements for key DoD problems

Computer Memory 100 GBytes

1 GBytes

10 GBytes

1 TBytes

10 TBytes

100 TBytes

Computer Speed 

FY97

1 GFlops 10 GFlops 100 GFlops 1 TFlops 10 TFlops

FY99 FY01

Ocean Modeling

Submarine Hydrodynamics

Automatic Target Recognition

100 Atoms

500 

Atoms

1000 Atoms

1/16th degree grid

1/64th degree 

grid

1/32nd degree grid

Real-time

EM Signature Modeling

Limited Capability

Reduce Test Risk

Certification 

(Minimum test) 

Reduce # of Tests

Steady Flow

Unsteady

Flow

Armor Optimization

50 Atom

cluster

Advanced Materials Design

Aircraft Stores Certification

Vehicle Weight/

Vulnerability Reductions Partial 

Optimization

HPC Modernization Plan

Program Initiatives

The DoD modernization effort is a multifaceted

process consisting of a phased acquisition plan for

HPC centers, coupled with the appropriate level of

computational software development and wide-area

network support. The program has three initiatives:

HPC centers, which consist of major shared resource

centers and distributed centers, networking, and

software support.

High Performance Computing Centers

Major Shared Resource Centers

To most effectively and efficiently use the full

range of HPC assets, the HPC Modernization Program provides for the establishment and sustainment

of four large major shared resource centers that have

a wide range of HPC capabilities and are designed to

support the broad DoD laboratory and developmental

test and evaluation communities. Each of the

program’s major shared resource centers has been

designated to support specific computional technology

areas. Each center’s hardware and software configurations, programming environments, and training efforts
9

are focused to support the needs of the designated

computional technology areas for that center. They

include various types of computing systems, scientific

visualization capabilities, peripheral and archival

storage, as well as providing the expertise in the use

of these assets. These large centers provide DoD

HPC users with stable hardware and software, with

an emphasis on supporting large computational

requirements/problems. Software includes programming tools focused on specific computational

technology areas for each center. The unique

features at each major shared resource center are the

programming environment and training component.

This effort, in addition to extensive training on a

wide variety of user needs, will also provide system

tools to facilitate the effective and efficient use of the

HPC systems. This will be accomplished through the

collaborative partnerships the major shared resource

centers have established with several leading civilian

HPC centers in order to draw needed civilian

Table 4 - Major Shared Resource Centers

PROGRAM DESCRIPTION

expertise into DoD. Table 4 lists the four major

shared resource centers.

Table 5 - Distributed Centers

(as of March 1997)

*Distributed centers funded for upgrades for fiscal year 1997.

**An additional distributed center for fiscal year 1997.

C n enter Locatio

Air Force Development Test Center (AFDTC) Eglin Air Force Base, Florida

Arnold Engineering Development Center (AEDC)* Arnold Air Force Base, Tennessee

Army High Performance Computing Research Center (AHPCRC) University of Minnesota, Minnesota

Maui High Performance Computing Center (MHPCC) Kihei, Hawaii

Naval Air Warfare Center (NAWC)* Patuxent River, Maryland

Naval Command, Control and Ocean Surveillance Center (NCCOSC) San Diego, California

Naval Research Laboratory (NRL)* Washington, District of Columbia

Naval Undersea Warfare Center (NUWC) Newport, Rhode Island

Redstone Technical Test Center (RTTC)** Huntsville, Alabama

Rome Laboratory (RL) Rome, New York

Space and Strategic Defense Command (SSDC) Huntsville, Alabama

Tank-Automotive Research Development and Engineering Center (TARDEC) Warren, Michigan

White Sands Missile Range (WSMR)* White Sands Missile Range, New Mexico

Distributed Centers

Distributed centers provide HPC capability to a

specified local and remote portion of the HPC

Modernization Program user community. Modestsized HPC systems are deployed to distributed

centers where there is a significant advantage to

having a local HPC system and where there is a

potential for advancing DoD applications using

investments in the HPC capabilities and resources.

Distributed centers are deployed to leverage specific

HPC expertise or to address problems that cannot

be solved at the major shared resource centers, such

as real-time data processing, signal image processing, and embedded systems applications. Table 5

shows the current distributed centers and their

locations. In addition to the distributed centers in

C n enter Locatio

Aeronautical Systems Center (ASC) Wright-Patterson Air Force Base, Ohio

Army Corps of Engineers Waterways Experiment Station (CEWES) Vicksburg, Mississippi

Army Research Laboratory (ARL) Aberdeen Proving Ground, Maryland

Naval Oceanographic Office (NAVO) Stennis Space Center, Mississippi
10 HPC Modernization Plan

Table 5, the Arctic Region Supercomputer Center

has been funded by Congress in fiscal years 1996

and 1997 and is providing computational resources

to the HPC Modernization Program user community. The Arctic Region Supercomputer Center is not

formally designated as a distributed center because it

lacks a DoD sponsor. The distributed centers are

linked by medium- to high-speed communications to

the major shared resource centers and remote users.

Thus, they augment the major shared resource

centers to form the total DoD HPC capability.

of interconnecting pathways (as portrayed in Fig. 2).

Service levels at each site may be adjusted as

networking requirements change. Adding and

deleting sites from the network and increasing

service levels can be easily accomplished.

The DREN Intersite Service contract will

provide wide-area connectivity initially at up to 155

megabits per second with upgrade potential to 2.4

gigabits per second. Service levels offered by the

contract will support the growing networking

requirements. Larger problem sizes, remote interactive visualization of computations, interactive

collaborations, and heterogeneous computing across

systems at several provider sites will drive the

demand for much greater network bandwidth.

The networking initiative is supported by the

Defense Information Systems Agency and the

Defense Advanced Research Projects Agency and

includes the Defense Simulation Internet requirements. Currently an interim networking capability

(see Fig. 3), which merged the Army Supercomputer Network and the Air Force Supercomputer

Network with selected Navy sites and the Defense

Special Weapons Agency network hubs, supports

access to existing resources through T-3 (45 megabits per second) backbone and T-1 (1.5 megabits

per second) tail circuits. This interim network is

being phased out as new network services become

available under the DREN Intersite Services Contract.

Fig. 2 - DREN Intersite Services Contract - Connectivity “Cloud”

DREN Intersite Services Contract (DISC)

Major Shared

Resource Centers

Distributed Centers

Remote Sites

Types of Service Delivery 

Points (SDP)

• IP

• ATM

• Circuit Emulation

• Gateway

Networking

A robust, high-speed network is essential to

ensure that the program’s HPC capabilities can be

effectively used by a large, geographically dispersed

user community. Approximately 1000 of the 4000

HPC Modernization Program users reside at the four

major shared resource centers, making the majority

remote users.

The Defense Research and Engineering Network

(DREN) will provide the connectivity to link the HPC

Modernization Program users to the program’s

shared resource centers to each other and to other

HPC-supporting national networks. DREN is an

expandable set of network services that can be

accessed upon demand. DREN uses a public communications grid; networking capability is specified as a

given level of service at each site, with no definition
11

Software Support priate use of communication interface, and graphics

visualization standards across DoD. Further, the

CHSSI goals are to promote the development of

new software tools and application area specific

software and to leverage HPC expertise and assets

located in industry, academia, and other federal

laboratories in addition to DoD facilities.

The Common HPC Software Support Initiative

is organized around the 10 computational technology areas. The applications software projects were

determined by the Computational Technology Area

working groups. Table 6 lists the projects and the

principle investigator in each computational technology area.

The software support initiatives will provide

personnel funding to develop applications for DoD

laboratories and test centers within the programming

environments available at the major shared resource

centers. The software development involves

multilaboratory and interservice teams drawn from

their respective computational technology area

working groups. These teams will be responsible for

development, dissemination and technology transfer

and be made up of algorithm and code developers,

applications specialists, and end users. The teams

may include partners from academia, industry, and

other government agencies. The CHSSI will provide

support for approximately 40 projects over the next

5 years. Academic partnerships are encouraged to

exploit sabbatical, visiting-scientist, post-doctoral,

and graduate student arrangements. They may

include other government agency and commercial

Sprint-NAP

DoD Host Site

Internet Service Provider

DSWA Hub

Natick

Rome Lab

NUWC-NPT

West Point

ARL-Monmouth

Benet Lab

AFOSR

Naval 

Academy

ARDEC

Belvoir RDEC

Hanscom

ARL-Adelphi

AEDC

Eglin

SSDC

NOAA

NSWCPanama City

Tyndall

AMRIID

PLK

LANL

FCDNA

Edwards

NAWC-WPN 

Pt Mugu

NAWC-WPN

China Lake

NRL-MRY

ARSC

Alaska

Brooks

NPG School

NSWCDahlgren AF 

Academy

MICOM/RTTC

DISA-RCC

FIX-West

MAE-West

SDSC

AHPCRC

MHPCC

ARL-APG

NCCOSC

WPAFB

NRL-DC

CEWES

NAVO

DISA-CISS

WRAIR NTF

Maxwell AFB

Wahiawa

DSW HQ

CAA

HPCMO

Ft. Benning

Dugway

NFESC

Pt Hueneme

NSWC -

Carderock

E

T

H

E

R

N

E

T

WSMR

TARDEC

NAWC-AD

Ft Leavenworth

SMC

Holloman AFB

dual

T1

dual T1

45 Mbps (T3)

1.5 Mbps (T1)

Not DREN funded

Shared (T3)

10Mbps (ethernet)

Sprint-NAP

MAE-East

Fig. 3 - The Interim Network—point-to-point connectivity

The Common HPC Software Support Initiative

(CHSSI) is an applications software development

effort designed to provide DoD computational

scientists and engineers with applications software

tools essential to exploit high end, scalable HPC

systems. Scalable parallel hardware provides price/

performance advantages over traditional serial vector

hardware and offers greater memory sizes, which

allow larger problems to be solved. Defense-specific

scalable applications software is essential to realize

the advantages of scalable parallel systems.

This initiative leverages the expertise of the DoD

scientists and engineers and allows them to work as

specialized teams to resolve common software

needs. The initiative encourages cooperation and

collaboration, not only among DoD computational

scientists, but also among scientists in other Federal

departments, industry, and academia. The CHSSI

products will be a suite of multipurpose codes that

run efficiently on the new generation of high-end

systems. These products will facilitate a large fraction

of the computational workload in support of DoD

warfighting requirements. This initiative provides the

requisite underlying software to support the development of defense applications capable of exploiting

the vast potential of scalable systems.

The goals for the CHSSI are to develop critical

DoD software applications that can execute in a

scalable computing environment and foster reuse of

software tools and application components, approPROGRAM DESCRIPTION
12

Table 6 - Computational Technology Areas, Projects, and Principal Investigators

HPC Modernization Plan

partners. Teams may employ conventional support

contractor arrangements provided there are no proprietary restrictions on resulting software or capabilities. Although all of these team arrangements are

possible, the overriding purpose of the initiative is to

enhance and create expert groups within DoD that

span laboratories, organizations, and services.

Computational

Technology Areas P r roject P s rincipal Investigato E-mail Addres

Computational

Structural

Mechanics

Small Deformation Structural Mechanics

Large Deformation Structural Dynamics

Scalable Algorithms for Shock Physics

Structure-Medium Interaction Model

Gordon Everstine

Raju Namburu

Kent Kimsey

Mark Emery

geversti@oasys.dt.navy.mil

namburr@exl.wes.army.mil

kimsey@arl.mil

emery@lcp.nrl.navy.mil

Computational

Fluid Dynamics

FAST3D - Global Virtual-Cell Embedding Gridding

COBALT - Unstructured Gridding

FEFLO - Unstructured Gridding

OVERSET - Chimera Gridding

MERCURY - Block Structured Gridding

ARL Zonal Navier Stokes - Block Structured Gridding

Jay Boris

Bill Strang

Ravi Ramamurti

Robert Meakin

Steve Scherr

Roger Strawn

boris@lcp.nrl.navy.mil

strang@fim.wpafb.af.mil

ravi@lcp.nrl.navy.mil

meakin@nas.nasa.gov

scherr@fim.wpafb.af.mil

strawn@merlin.arc.nasa.gov

Computational

Chemistry and

Materials Science

Car-Parinello Methods for Solids

Quantum Chemistry

Tight-Binding Molecular Dynamics

Classical Molecular Dynamics

D.J. Singh

Jerry Boatz

D.A. Papaconstantopoulos

Ruth Pachter

singh@witan.mbvlab.wpafb.af.mil

jerry@helium.ple.af.mil

papacon@dave.nrl.navy.mil

pachterr@ml.wpafb.af.mil

Computational

Electromagnetics

and Acoustics

Automatic Target Recognition and Scene Generation

Code

Electromagnetic Interaction Code

Spectral Domain Low Observable Component

Design Code

Synthetic Sensor Code for Fusion

Time Domain Code

High Resolution Computational Electromagetics and

Acoustics Code

Magnetohydrodynamic Code

Jeff Hughes

Jay Rockway

Kueichien Hill

Russ Burleson

Helen Wang

Miguel Visbal

Robert Peterkin

jah@witan.mbvlab.wpafb.af.mil

rockaway@nosc.mil

hillkc@sga254.wpafb.af.mil

rburleso@mbvlab.wpafb.af.mil

helen_wang@mlngw.chinalake.navy.mil

visbal@fim.wpafb.af.mil

bob@ppws07.plk.af.mil

Climate/Weather/

Ocean Modeling

and Simulation

Ocean Models with Domain Decomposition

Scalable Global Weather Forecast System

Global and Regional Wind Wave Modeling

Steve Piacsek

Tom Rosmond

Robert Jensen

piacsek@nrlssc.navy.mil

rosmond@nrlmry.navy.mil

jensen@madmax.cerc.wes.army.mil

Signal/Image

Processing

Radar

Sonar

Synthetic Aperture Radar/Image Formation

Processor

Automatic Target Recognition

Infrared/Optical Image Processing

Rich Linderman

Bob Bemecky

Christopher Yerkes

David Gadd

Lynda Graceffo

lindermanr@rl.af.mil

bernecky@starbase.nl.nuwc.navy.mil

yerkes@nosc.mil

dgadd@mbvlab.wpafb.af.mil

graceffo@nvl.army.mil

Forces Modeling

and Simulation/

C4

I

Scalable HPC Environment for Command, Control,

Communications, Computers, and Intelligence

Simulation Leveraged Acquisition Test and Evaluation

Efficient Parallel Discrete Event Simulation for

Analysis

HPC Frameworks for Warming and Training

Simulations

Guy Leonard

Henry Ng

Bill Smith

Larry Peterson

gleonard@nosc.mil

hng@relay.nswc.navy.mil

smith@ait.nrl.navy.mil

ljp@nosc.mil

Environmental

Quality Modeling

and Simulation

Structured-Unstructured Modeling

Scalable Multi-Dimensional Contaminant Transport

and Fate Modeling for Surface Water and Surface

Water/Groundwater Interactions

Scalable Parallel Implementation of DoD

Groundwater Modeling System

Robert Bernard

Mark Dortch

Fred Tracy

bernard@hl.wes.army.mil

dortchm@exl.wes.army.mil

tracy@exl.wes.army.mil

Computational

Electronics and

Nanoelectronics

Microwave Circuit Simulation

Power Semiconductor Simulation

Parallel Very High Speed Integrated Circuit

Hardware Description Language Simulation

Electromagnetic Solvers

Dave Rhodes

Robert Pastore

John Hines

Leo DiDomenico

rhodes@arl.mil

pastore@doim6.monmouth.army.mil

hines@el.wpafb.af.mil

ldidomenico@arl.mil

Integrated

Modeling and Test

Environments

Real-time Synthetic Test Environment Representation

High Fidelity Physics Based Models for Testing

Simulation Based Design and Test Technology

Harry Heckathorn

Joel Mozer

Andrew Mark

harry@vader.nrl.navy.mil

mozer@plh.af.mil

amark@arl.mil
PROGRAM DESCRIPTION 13

The HPC Modernization Program is establishing a

world-class, nationwide, integrated infrastructure to support the HPC needs of the defense science, technology,

developmental test and evaluation communities. As depicted in Fig. 4, the program has an integrated strategy

to provide complete computing environments to increase

DoD scientists’ and engineers’ productivity. Connecting

the program’s HPC centers and users is a high-speed,

high-bandwidth network. The major shared resource centers operate the large HPC systems available to the entire HPC Modernization Program community. Each major shared resource center provides a full range of resources, including hardware, software, data storage,

archiving, visualization, training, and expertise in specific

computational technology area. The distributed centers

supplement the major shared resource centers to form

the total HPC Modernization Program’s computing capability. These centers typically have smaller high performance computers to develop and test particular high end

applications. Software support is an important program

element. Advised by user groups, the program is underwriting the development of application building blocks,

visualization tools, system utilities, and mathematical libraries. The combination of the software support effort

with the HPC centers, networked together with users,

will constitute a state-of-the-practice capability with which

our scientists and engineers will routinely conduct worldclass reseach, development, test, and evaluation.

SOFTWARE SUPPORT INITIATIVE

NETWORKING

HPC CENTERS

SCIENTIFIC

VISUALIZATION

S&T

and

DT&E

APPLICATIONS

SUPPORT

APPLICATIONS

COMMERCIAL

OFF-THE-SHELF SOFTWARE

PROGRAMMING

ENVIRONMENT AND TRAINING

SUPPORT SERVICES

INTERNAL NETWORKING

WIDE AREA CONNECTIVITY

COMPUTER

SYSTEMS

• VECTOR

• SCALABLE PARALLEL

• CLUSTERED

 WORKSTATIONS

• DISTRIBUTED FILE

 SYSTEMS

• ROBOTICS ARCHIVAL

 SYSTEMS

STORAGE

Fig. 4 - DoD HPC Modernization Program - Integrated Program Strategy
15

PROGRAM ACCOMPLISHMENTS

 The HPC Modernization Program made significant progress over the past 2 year towards its goal of

modernizing DoD’s HPC capability. Concurrent

acquisitions to provision the four major shared resource centers and provide robust, wide-area networking were initialized, announced, and executed. Contracts for the Major Shared Resource Centers acquisitions were successfully awarded, as shown in Table 7.

The DREN Intersite Service Contract was awarded to

AT&T in July 1996.

The program selected and began funding the

common application software development projects to

ensure that an efficient, robust set of applications

building blocks will be available to support a broad base

of the RDT&E computational workload. Processes for

user input regarding requirements, upon which the

program is based, were expanded. To ensure that

important computationally intensive DoD problems

receive the necessary program resources, new allocation policies were initiated. An outcome of these new

allocation policies was the selection of a set of DoD

“Challenge” projects, which will receive approximately

20% of the program’s computing cycles. These

projects will result in significantly improved models to

address important DoD problems such as determination of blast damage on structures for

counterproliferation and counter-terrorism scenarios,

disposal of expired munitions, mine warfare in littoral

waters, and finer calculation of stealth signatures to

improve invisibility across the electromagnetic spectrum.

 The integration contracts awarded for the four

major shared resource centers will provide for hardware, software, maintenance, operations, and expert

support staffs. Built into the major shared resource

centers multiyear contracts is the option of upgrading

to higher capability as technology and resources

become available. Each of the contracts includes a base

computational capability, Performance Level I, and

two options for upgrades, referred to as Performance

Levels II and III. Table 8 shows the initial systems

populating the centers and their aggregate peak

Table 7 - Major Shared Resource Center Contract Award Dates

O n rganization L r ocatio C e ontracto Award Dat

Army Corps of Engineers

Waterways Experiment Station Vicksburg, Mississippi N 6 ichols Research Corporation March 199

Air Force Aeronautical

Systems Center Wright-Patterson Air Force Base, Ohio N 6 ichols Research Corporation May 199

N i aval Oceanographic Office S s tennis Space Center, Mississipp G 6 rumman Data System May 199

A d rmy Research Laboratory A s berdeen Proving Ground, Marylan R 6 aytheon E-System August 199

15

Table 8 - Major Shared Resource Center Performance Level 1 Capability

C s enter HPC System

Total Site

Capability

(Peak gigaflops)

Army Corps of Engineers Waterways

Experiment Station C 1 ray Y-MP,* Cray C90,* Cray T3E, SGI PCA, SGI Origin 19

Naval Oceanographic Office C 1 ray 90,* Cray T90, SGI PCA(2), Cray J932SE 5

Army Research Laboratory S 8 GI PCA,* SGI Origin (3), Cray J932 (2), Cray T90(2) 8

Aeronautical Systems Center I 3 ntel Paragon,* Cray C90, IBM SP, SGI PCA 7

* HPC systems installed prior to the 1996 integration contract awards
16

computational performance. In most cases, all

systems are now present and operational at the

centers.

One of the key features of the major shared

resource centers contracts is the programming

environment and training effort. Each integrator has

established collaborative partnerships with several

civilian HPC centers as a mechanism for rapidly

capturing innovation and bringing needed expertise in

HPC into DoD. Organized around the program’s 10

computational technology areas, this effort will focus

on training the DoD user base, identifying HPC

technological opportunities, and introducing these

opportunities into the program’s shared resource

centers’ computing environments.

Augmenting the computing capabilities at the

major shared resource centers, the HPC Modernization Program established seven new distributed

centers and upgraded existing systems at four distributed centers over the 3-year period of fiscal years

1995 through 1997. These distributed centers will

apply HPC to a variety of DoD problems including

real-time data analysis, man-in-the-loop and hardwarein-the-loop testing and evaluation, and development

of specific HPC applications. Table 9 shows the

centers which were selected for new or upgraded

systems for fiscal years 1995 through 1997.

Tables 10 and 11 show the program’s total HPC

capability as of March 1997. As a result of the major

shared resource centers contract awards and the fiscal

years 1995 and 1996 distributed centers implementations, total peak capability increased from 239

gigaflops in fiscal year 1994 to over 983 gigaflops as

of January 1997. Additional capability will be available in fiscal years 1997 and 1998, as the program

executes the Performance Level II upgrades at the

major shared resource centers and completes the

fiscal years 1997 and 1998 distributed centers’ new

acquisitions/enhancements.

Connecting the centers will be the Defense

Research and Engineering Network, for which a

contract was awarded to AT&T. With approximately

75% of the program’s user base located at sites

other than the major shared resource centers, a

robust wide area network is essential to ensuring that

our scientists and engineers can make efficient use of

the program’s HPC capabilities. The DREN is a

virtual private network and will replace the interim

DREN, which has been operational since 1993.

Initial bandwidth will vary from 10 megabits per

second to 155 megabits per second (OC3), with

upgrade potential of 2.4 gigabytes per second

(OC48) over the 5-year life of the contract. Ten sites

are scheduled to be implemented by the spring 1997

and the entire 60 government sites by the end of

1997. In addition to government sites, DREN will

provide gateways to many existing military and

civilian networks. Contractor and university facilities

not at government locations will link with the

program’s centers via these gateways.

Funding for the CHSSI projects began in March

1996. This effort will provide a suite of multipurpose

codes that will perform on a range of HPC platforms

with attention to portability to other systems, both

present and future. Approximately 40 projects in 9

computational technology areas were selected by the

computational technology leaders. The projects will

be developed by teams chosen by the computational

HPC Modernization Plan

Table 9 - Distributed Centers for Fiscal Years 1995 through 1997

C n enter Locatio

Air Force Development Test Center (AFDTC) Eglin Air Force Base, Florida

Arnold Engineering Development Center (AEDC) Arnold Air Force Base, Tennessee

Army High Performance Computing Research Center (AHPCRC) University of Minnesota, Minnesota

Maui High Performance Computing Center (MHPCC) Kihei, Hawaii

Naval Air Warfare Center (NAWC) Patuxent River, Maryland

Naval Research Laboratory (NRL) Washington, District of Columbia

Naval Undersea Warfare Center (NUWC) Newport, Rhode Island

Redstone Technical Test Center (RTTC) Huntsville, Alabama

Space and Strategic Defense Command (SSDC) Huntsville, Alabama

Tank-Automotive Research Development and Engineering Center (TARDEC) Warren, Michigan

White Sands Missile Range (WSMR) White Sands Missile Range, New Mexico
PROGRAM ACCOMPLISHMENTS 17

Table 10 - DoD HPC Modernization Program Computational

Resources at the Major Shared Resource Centers

Performance Level 1 Final (as of March 1997)

Number Total Disk Total

Location DoD HPCMP System of Memory Storage Capability

Processors (gigabytes) (gigabytes) (Peak gigaflops)

Aeronautical Intel Paragon 368 11 115 26

Systems Center Cray C90 16 8 202 16

IBM SP 80 68 990 25

SGI PowerChallenge 16 8 202 6

Total Site Capability (gigaflops) 73

Army Corps of Cray C90 16 8 228 16

Engineers Cray Y-MP 8 1 115 3

Waterways Cray T3E 256 33 370 154

Experiment Station SGI PowerChallenge 32 16 202 12

SGI Origin 2000 16 8 196 6

Total Site Capability (gigaflops) 191

Army Research SGI PowerChallenge 96 16 232 29

Laboratory SGI Origin 2000 32 12 206 13

Cray T90 4 4 480 7

SGI Origin 2000 (Secret) 32 12 206 13

Cray T90 (Secret) 4 4 240 7

Cray J932 (Special Access) 16 8 252 3

Cray J932 (Special Access) 16 8 252 3

SGI Origin 2000 32 12 412 13

 (Special Access)

Total Site Capability (gigaflops) 88

Naval Cray C90 16 8 330 16

Oceanographic Cray T90 6 4 224 11

Office SGI PowerChallenge 36 12 212 13

Cray J932SE (Classified) 12 4 224 2

SGI PowerChallenge 24 8 210 9

 (Classified)

 Total Site Capability (gigaflops) 51

Total Major Shared Resource Centers Capability (gigaflops) 403

technology area leaders. The teams are drawn from

the respective computational technology area working

groups and include partners from academia, industry,

and other government agencies. Additional projects in

the Integrated Modeling and Test Environments

computational technology area are planned for

initiation in the 2nd quarter of fiscal year 1997.

The HPC Modernization Program held a CHSSI

public information meeting in March 1996, to announce the common HPC software support initiative

projects. This meeting provided an opportunity to

discuss plans for the effort and to encourage interactions with academia, industry, and other government

agencies. A CHSSI Workshop was held in June 1996

for the CHSSI development teams at which programming standards’ guidelines and performance metrics

for the projects were presented.

Advised by the program’s user advocacy group

(Shared Resource Centers Advisory Panel) and the

service/defense agencies executives’ representatives

to the program’s HPC Advisory Panel, the HPC

Modernization Office established a resource allocation

policy to effectively distribute the program’s computing resources. The new policy, scheduled to be fully

implemented in fiscal year 1998, has two parts:

allocation of a specified portion of the program’s total
18 HPC Modernization Plan

Table 11 - DoD HPC Modernization Program Computational

Resources at the Distributed Centers (DCs)

(as of March 1997)

Number Total Disk Total

Location DoD HPCMP System of Memory Storage Capability

Processors (gigabytes) (gigabytes) (Peak gigaflops)

Air Force Cray T3D 128 8 140 19

Development SGI Onyx 32 8 40 12

Test Center

Army High TMC CM-5 896 29 116 115

Performance

Computing

Research Center

Arnold Engineering Convex C4640 4 2 100 3

Development Center Convex C4640 (Classified) 4 2 100 3

Convex C3880 8 2 40 2

Convex Exemplar SPP-2000 32 4 50 13

Maui High IBM SP 400 64 822 106

Performance IBM SP (Classified) 80 8 227 21

Computing Center IBM SP/SMP 115 15 553 39

Naval Air Warfare SGI PowerChallenge 40 12 300 15

Center

Naval Command, Intel Paragon (Classified) 336 14 128 25

Control and Ocean Convex Exemplar SPP-1000 32 8 160 6

Surveillance Center

Naval Research TMC CM-500e 256 33 400 41

Laboratory TMC CM-500e 32 4 50 5

Convex Exemplar SPP-2000 64 16 150 46

SGI Origin 2000 64 16 85 25

Naval Undersea Cray T3D 64 4 76 10

Warfare Center

Rome Laboratory Intel Paragon 321 21 80 30

Tank-Automotive SGI PowerChallenge 64 16 385 24

Research,

Development and

Engineering Center

White Sands TMC CM-500 128 16 96 20

Missile Range

Total Distributed Centers Capability (gigaflops) 580

Total HPC Modernization Program Capability (gigaflops) 983
19

computing resources to support very large,

computationally intensive problems and allocation of

the remaining computing cycles to the remaining set

of service/agency requirements projects, with 30%

each for the Army, Navy, and Air Force and the

remaining 10% for other DoD agencies. Each service

then distributes its share according to its needs.

The allocation of computing resources to support

large, computationally intensive problems resulted in

the selection of a set of projects for fiscal year 1997,

referred to as DoD Challenge Projects. These projects

were selected via a competitive process by a selection

board that included reviewers from outside of DoD.

Selection was based on DoD priority requirements,

scientific merit, and potential for progress. These

projects will receive approximately 20% of the computing cycles available at the major shared resource

centers and selected distributed centers. The initial set

of DoD Challenge Projects are summarized in Table

12. Allocations averaging 6 gigaflop years per project

PROGRAM ACCOMPLISHMENTS

Table 12 - Fiscal Year 1997 DoD Challenge Projects

P n roject Organizatio Primary

CTA

HPC

System(s)

1 y /16th Degree Global Ocean Model N O aval Research Laborator C E W Cray T3

A First Principles Computation of the Process

of Deflagration-to-Detonation Transition N D aval Research Laboratory C 5 F TMC CMA y irborne Laser Challenge Project P A hillips Laborator CE Cray T3E

IBM SP

Analysis of Jet Interaction Phenomena for the

Theater High Altitude Area Defense

Interceptor

P D rogram Executive Office - Air Missile Defense C 0 F Cray C9

Automatic Aerodynamic Design for Complete

Aircraft Configurations A D ir Force Office of Scientific Research C E F Cray T3

B y -1B Radar Cross Section Prediction W A right Laborator CE

Cray T90

SGI PCA

SGI Origin

Design of New Materials Using Computational

Chemistry

Air Force Office of Scientific Research and

Phillips Laboratory

C P CM IBM S

Impacts of Subsurface Heterogeneity on

Military Site Cleanup

Army Corps of Engineers Waterways Experiment

Station E 5 QM TMC CMKhobar Towers Bomb Damage and Water

Tamping Studies

Army Corps of Engineers Waterways Experiment

Station C E SM Cray T3

Modeling of Complex Projectile-Target

Interactions A M rmy Research Laboratory CS

Cray-2

Cray T90

SGI PCA

Near Real-Time Multispectral Sensor

Simulation Using Parallel Ray-Tracing

A T rmy Research Laboratory I A M SGI PC

O y pen-Air Detonation of Expired Munitions N D aval Research Laborator CF

IBM SP

SGI PCA

Exemplar

Parallel Simulations of Flow-Structure

Interactions O D ffice of Naval Research CF Cray T3E

IBM SP

Simulation of Explosions for CounterProliferation and Counter-Terrorism Scenarios D M efense Special Weapons Agency EQ

SGI PCA

Cray C90

Cray T3E

Time-Domain Computational Ship

Hydrodynamics O D ffice of Naval Research CF

IBM SP

SGI PCA

TMC CM-5

Cray C90

Towards the Virtual Prototyping of Radio

Frequency Weapons P A hillips Laboratory CE IBM SP

Cray C90
20 HPC Modernization Plan

were made for the January-September 1997 time

period. A yearly competition for DoD Challenge

Projects is planned.

The 1995 and 1996 annual DoD HPC Users

Group Meetings were held. The 1995 meeting was

sponsored by and hosted at the Naval Research

Laboratory Distributed Center in Washington, D.C.;

the 1996 meeting was sponsored by the Aeronautical Systems Center’ Major Shared Resource Center

and hosted by the National Center for

Supercomputing Applications (NCSA) in UrbanaChampaigne, Illinois. Both meetings, with over 100

attendees, provided information on the HPC Modernization Program initiatives and offered a forum

for interaction among the program’s center management staffs, program office staff, and users. The

1996 Users Group Meeting, held at NCSA, also

provided an opportunity for the program’s shared

resource centers’ staffs and users to interact with

NCSA staff.

The HPC Modernization Program is a major

automated information system program and, as

such, must meet all Major Automated Information

Systems Review Council (MAISRC) life-cycle management milestone requirements. In April 1994, the

HPC Modernization Office received Milestone 0

approval to initiate the program, and Milestone 1

was granted in February 1995. The Major Automated Information Systems Review Council conducted a formal Milestone 2 review of the HPC

Modernization Program in December 1995. The

Milestone Decision Authority (MDA) issued the

System Decision Memorandum on 4 January 1996.

This milestone approval allowed substantial progress

toward meeting the program’s goals. In the subsequent months, the program awarded contracts for

four major shared resource centers, six distributed

centers, and wide-area-networking services and

released funds to support the Common HPC Software Support Initiative projects.

The Program Office has established several

focused Integrated Product Teams (IPTs) and continues to work through the IPT process to accomplish

its goals. During 1996, 11 IPT sessions were held.

The sessions resolved testing, cost, and security

issues. The last IPT session was an approval, seniorlevel Working IPT (WIPT). The program submitted

the Single Acquisition Management Plan, with

appropriate attachments, and requested approval to

execute planned fiscal year 1997 implementations

for the Common HPC Software Support Initiative

and the fiscal years 1997 and 1998 distributed

centers. The Common HPC Software Support

Initiative implementations and the fiscal year 1997

distributed centers were subsequently approved.
21

HPC CONTRIBUTIONS TO MISSION SUCCESS

21

High performance computing enables detailed

computer-based design, virtual prototyping, and

improved numerical modeling in order to significantly shorten the acquisition cycle and reduce the

costs of weapons system procurement. High-end

computational capabilities impact all aspects of DoD

research, development, test, and engineering—from

basic research in the laboratories to testing and

evaluating at the test centers. The extent of this

impact is accelerating. Computational modeling and

simulation allow the exploration of more options, at

a fraction of the cost, than with traditional theoretical

and experimental methods. The use of high performance computing, coupled with judiciously chosen

experimental data points, provides a powerful tool

for improving the development process of DoD

warfighting and support systems.

Although the DoD HPC Modernization Program

has fielded HPC capabilities for approximately 3

years, a number of major accomplishments have

already resulted from the use of high-end computing

in the DoD science and technology program. In

1995 and 1996, the HPC Modernization Program

published Contributions to DoD Mission Success

From High Performance Computing. This section

highlights a few of the most notable successes from

the 1996 edition. The accomplishments were

selected based on scientific content and application

to the DoD mission. The 12 success stories highlighted here are contributed from the Army, the Air

Force, the Navy, and the Defense Special Weapons

Agency. The stories span 8 of the 10 DoD computational technology areas and include work performed

using several of the new, scalable HPC systems. The

accomplishments presented show the crucial role that

high-end computing plays in a variety of defense

application areas:

• Ship Structural Response to Underwater

Explosions - CSM

• Glass Window Design to Mitigate Terrorist

Threat - CSM

• C-17 Paratrooper Operation - CFD

• SADARM Submunition Collision Solution -

CFD

• Elimination of Explosives and Propellants

Through Environmentally Safe Open-Air

Detonations - CFD

• Conducting and Semiconducting Polymers by

Design - CCM

• Determination of Blast Damage on Structures -

CSM/CFD

• Advanced Warfighting Concepts Using Plasmas - CEA

• Cofired Ceramic Package for a Ka-Band

MMIC Phase Shifter - CEN

• Embedded High Performance Computing for

real-time Target Detection and Tracking - SIP

• Global Ocean Modeling - CWO

• Pollutant Cleanup Strategies within DoD -

EQM

Additional applications software projects are being

developed for use on new, scalable high-end systems

under the DoD HPC Modernization Program’s

software support initiative.
22 HPC Modernization Plan

Fig. 6 - Bending failure of singlepane glass window under blast

loading

Fig. 5 - Cutaway view of finite-element model of

equipment rooms

Ship Structural Response to Underwater

Explosions

The Navy’s need for a modernized minehunting

platform, for use in potential Persian Gulf-like

conflicts, necessitated a quick assessment of a new

platform’s resistance to underwater explosions. The

urgency for the procurement resulted in the decision

to forego a costly and time-consuming experimental

assessment program. The system had to be assessed

independent of traditional “design-build-test” experimental techniques. HPC technology was identified as

the only available means by which the technical

requirements could be met within the time frame and

allowed budget. The needed level of detail required

the development of new models for two main

equipment rooms. (See Fig. 5.) As a result of the

successful implementation of this shock assessment

using HPC systems, the Navy identified a number of

critical design deficiencies. The Navy’s analysis

persuaded the shipbuilder to acknowledge these

deficiencies and to accept responsibility for them.

The use of high-end computing was a major step

toward assuring that the design of the coastal

minehunter system met the requirements for shockhardening. In the process of implementing this HPC

solution, the Navy has gained insight into the

behavior of complex composite structures subjected

to underwater explosions.

Glass Window Design to Mitigate Terrorist

Threat

Because of recent terrorist attacks, both at home

and abroad, assessment of the vulnerability of

existing government buildings to blast loadings has

become a mission priority. One important aspect of

this assessment is the prediction of the lethality of

glass fragments to human occupants. Injury to

humans due to glass fragments has been demonstrated to be dependent on fragment size and

velocity. Average fragment size can be determined

experimentally, but fragment velocities due to blast

loadings are difficult to measure. To quickly assess

the effects of charge weights and standoff distances

to variations of glass thickness and strength, a matrix

of finite element analyses was created using HPC

systems. Glass variables included window size, single

or double-pane thickness, and strength. Results from

the analyses were used to develop equations for

predicting the velocity of glass particles based on

window size, thickness, strength, and blast characteristics. (See Fig. 6.) These results can be used to

evaluate the threat to building occupants from glass

fragments due to terrorist’s bombs and to determine

the best methods of retrofitting existing buildings to

diminish the human risk from flying glass fragments.
HPC CONTRIBUTIONS TO MISSION SUCCESS 23

Fig. 7 - Streamlines behind aircraft at typical speed

and configuration for paratrooper exit

Fig. 8 - Mach contours showing separation of submunitions and collision avoidance, based on CFD

simulation of design modifications

C-17 Paratrooper Operation

The C-17 aircraft was designed to accommodate

a mass exit of 102 fully equipped paratroopers. In

August 1994, during routine development flight

testing of mass paratrooper operations, a pair of

Army jumpers encountered a high-altitude parachute

entanglement (both jumpers landed safely). The

Army and Air Force immediately began work to

understand and resolve the phenomenon that caused

this type of encounter. This phenomenon is not

unique to the C-17. To understand the phenomenon

and develop a solution that would allow the C-17 to

satisfy operational requirements, both the Army and

Air Force formed a joint Executive Independent

Review Team. Using both the DoD and the National

Aeronautics and Space Administration HPC resources, the Air Force developed computational fluid

dynamics simulations to provide insight for the

Executive Independent Review Team on the salient

features of the C-17 and C-141 flowfields. The

Executive Independent Review Team subsequently

oversaw water tunnel and wind tunnel tests to

confirm the best operational configuration and to

satisfy Army requirements. The final configuration

was verified in April 1995 when over 300 jumpers,

spanning three sorties, safely exited a C-17 at the

Army’s Yuma Proving Ground, Arizona. Figure 7

shows streamlines behind the C-17 aircraft predicted

by computational simulation for typical speeds and

configurations of paratrooper exit.

SADARM Submunition Collision Solution

The Sense and Destroy Armor (SADARM)

system is an artillery-delivered (155 mm), tank-killing

system with sensors designed to find and direct

penetrators to the relatively “soft” top of armored

vehicles. During the development cycle of this

program, problems were experienced with in-flight

collisions that damaged and rendered useless the

expelled submunitions. Through the use of DoD

HPC assets and an advanced multibody computational fluid dynamics capability, the underlying

physics, which led to the submunition collisions, was

modeled and understood. The computed results for

the original design, shown in Fig. 8, indicate regions

of low speed flow (blue) and high-speed flow (redyellow). The distribution of the variations in flow

around the paired munitions effectively drew them

together. A new design with drag fins was developed

and tested by HPC simulation. This result was

subsequently verified by a less costly test schedule

made possible by the simulation. This success helped

reverse a pending decision by the Defense Acquisition Board to discontinue the system. Instead, low

rate production was approved for SADARM. The

accomplishment resulted in savings of time and

dollars. Most significantly, this breakthrough in 3-D,

time-dependent computational fluid dynamic methodology suggests the need for and value of engaging

the services of HPC modeling in the early phases of

the development cycle.
24 HPC Modernization Plan

Fig. 9 - Potential octagon design of a containment facility for open-air detonation of

expired and obsolete munitions

Elimination of Explosives and Propellants

Through Environmentally Safe Open-Air

Detonations

Environmentally safe disposal of unexploded

conventional, chemical, and biological ordinance

presents a substantial challenge worldwide. In

addition to the over 800,000 tons awaiting disposal,

over 11 million U.S. acres contain significant

unexploded ordinance concentrations. Because of

their age and composition, more than 60% of this

demilitarized inventory is not amenable to disposal

by means other than by open-air detonation or

open-air burning. Human health and ecosystems

concerns severely restrict and sometimes prohibit

open-air detonation and burning. An ongoing joint

Army, Navy, and Environmental Protection Agency

effort has been established to develop ways to

improve the environmental safety by the use of new

open-air detonation and burning technologies. HPC

resources and analyses contributed substantially to

the design of a pilot-scale incinerator structure now

being constructed at Dugway Proving Ground.

Simulations were performed on over 20 proposed

design options. The tests identified and resolved a

number of problems for which no other practical

means of analysis existed. Open-air detonation and

burning designs developed by using this HPC-aided

approach provide potential to allow the disposal of

unexploded ordinance in an environmentally safe

manner. Figure 9 is an example of a simulated openair containment/detonation facility.

Fig. 10 - Structure and electron density of a pushpull polyene calculated by using PAIMD on the

TMC CM-5

Conducting and Semiconducting Polymers by

Design

The list of applications for organic materials that

conduct electricity or modify the behavior of light is

growing continuously. Sensor protection, electromagnetic shielding, lightweight batteries, fast optical

communication, or replacements for metal wiring are

some of the applications being considered by the

DoD. However, in many cases, there are no known

materials that can meet all of the requirements

imposed on them by the various applications.

Hence, new materials and materials concepts are

needed to try to fill these voids. Computational

chemistry plays an increasingly important role in the

search for new optical and conducting materials.

Accurate quantum chemical techniques are

computationally intensive, and the computational

resources required grow rapidly with the molecular

size. A new, highly parallelized computer code for

predicting molecular structures and properties that

combines accurate ab initio quantum mechanical

calculations with molecular dynamics simulations has

been developed and is being tested on large molecules with promising conduction properties, such as

shown in Fig. 10. Several massively parallel systems

provided by the DoD HPC Modernization Program

were used by the Air Force to develop the computational chemistry codes that are now being used to

investigate the properties of new conducting and

semiconducting polymers.
HPC CONTRIBUTIONS TO MISSION SUCCESS 25

Fig. 12 - Cut planes and isosurfaces of mass density

(in kg/m3) of radially compressing plasma at 50 ns.

The classical Rayleigh-Taylor spike and bubble

pattern is easily recognized. The strong azimuthal

variations are an indication of important 3-D physics.

y

x z

0.31500

0.27435

0.23894

0.20810

0.18124

0.15785

0.13748

0.11973

0.10428

0.09082

0.07910

0.06889

Advanced Warfighting Concepts Using

Plasmas

It is accepted as fact that understanding the flow

of air around an aircraft is vital to aircraft design and

control. For similar reasons, understanding the

behavior of an ionized gas, or plasma, in an environment that supports electric and magnetic fields is

crucial to a variety of advanced warfighting concepts,

including directed energy weapons based on microwave radiation and advanced hypersonic aircraft and

missiles. Plasmas are subject to many instabilities that

make control of their components particularly

difficult. Until now, time-dependent numerical

simulation of dense plasma phenomena in three

spatial dimensions has been too coarse and too time

consuming to be of great use. Only with the advent

of effective algorithms that take advantage of a

modern, distributed HPC environment were we able

to simulate the complex physical phenomena that

take place in a plasma, as shown in Fig. 12. The Air

Force validated the computational and physical

algorithms used in the numerical plasma simulation

through experimental comparisons. Now the same

simulation techniques are being used on a variety of

advanced weapon concepts that require control over

the plasma environment.

Fig. 11 - Three-dimensional MAZe CFSD simulation of a

bomb detonation in a structure (air pressure contours

on wall surface at 1 ms)

Determination of Blast Damage on Structures

Accurate blast damage assessments of large,

complex structures have become a greater concern

since the Gulf War. In contrast to testing large

structures for blast resistance, which is very expensive, detailed HPC simulations offer a unique,

accurate, and cost-effective approach to survivability

assessment. To enable an accurate blast damage

assessment, a fully integrated computational fluid

dynamics and computational structural dynamics

methodology was developed under the sponsorship

of the Defense Special Weapons Agency. The

integrated methodology enables the simultaneous

determination of the following sequence of events:

the detonation and fragmentation of a penetrating

conventional warhead, the blast and fragment

transport through the structure, the forces exerted

on the structure by the impinging shocks and

fragments, the deformation and motion of the

structure, and the structure’s failure modes. Figure

11 shows the application of this methodology to an

explosion at the World Trade Center.
26 HPC Modernization Plan

Embedded HPC for Real-Time Target

Detection and Tracking

Airborne surveillance platforms, proven effective

during operation Desert Storm, are challenged to

detect and track even smaller and slower targets both

on the ground and in the air. New fiscal requirements

further constrain researchers to increase the system

flexibility and decrease the total system cost simultaneously . The military-unique “black boxes” currently

used for the real-time radar signal processing are

expensive and lack the flexibility to implement the

latest generation of signal processing techniques.

Leveraging commercial HPC technology provides

an order of magnitude improvement in both system

price and flexibility and also demonstrates a transition path from laboratory simulations to real-time

implementation onboard aircraft. In this case, one of

the latest techniques for detecting small targets from

airborne radars, known as Space-Time Adaptive

Processing, was implemented by the Air Force on

a DoD HPC system and then ported to a smaller,

ruggedized supercomputer that flew on a test

aircraft. Figure 14 shows the result of Space-Time

Adaptive Processing. The supercomputer met the

challenge of processing 16 channels of radar data

under real-time constraints and sustained 60% of its

peak theoretical performance during the calculations.

The flexibility of the embedded supercomputer

allowed all the functions from raw data collection

through target detections to be implemented instead

of the traditional sequencing of “black boxes.”

Fig. 13 - The 18-40 GHz MMIC

package for phase shifter chip and

electric field distribution in the

package

Cofired Ceramic Package for a Ka-Band

Microwave Monolithic Integrated Circuit Phase

Shifter

Microwave monolithic integrated circuit components are an integral part of military sensors, detectors, and weapons systems. These components are

encased in hermetic packages to reduce electromagnetic interference and minimize radiation leakage. A

variety of packages based on cofired ceramic technology have been developed for use on high frequency

applications. In order to integrate the microwave

monolithic integrated circuit component, an accurate

high-frequency characterization of the electromagnetic

environment around the package is required. Until

now, package modeling has been limited by the large

size and complexity of the problem, thus leading to

long design cycles, low yield, and high cost. For a

single package, the frequency characterization task

can require 40 hours of computation on conventional

serial processors. Using a scalable parallel system, this

task was reduced to 1 hour. Figure 13 is a diagram of

a microwave monolithic integrated circuit package and

a sample output of the model used. The greatly

improved computational capability allows for better

design optimization, thus leading to more effective

materiel for the warfighter.

Fig. 14 - Clutter prior to and after

Space-Time Adaptive Processing
HPC CONTRIBUTIONS TO MISSION SUCCESS 27

Fig. 15 - Sea surface height snapshot for 21 January

1986 simulated by a Pacific Ocean model with 1/16°

resolution for each variable. Contour interval is 12 cm.

Global Ocean Modeling

Global ocean modeling is a Grand Challenge

problem, generally thought to require computational

capabilities beyond the availability of current-generation HPC systems. By careful choice of computational methods, the Navy has been able to run a

global ocean model on existing DoD HPC systems.

The world’s most sophisticated global ocean model

was demonstrated by using multiple HPC resources.

Figure 15 is a snapshot of sea surface height (comparable to satellite altimeter measurements) predicted

by the model. An important milestone in the field of

global ocean monitoring is the ability to run a fullscale global ocean model by producing a prediction

system capable of providing detailed descriptions of

ocean activity. This capability is important to military

operations, including mine warfare in littoral (coastal)

regions, antisubmarine warfare, search, and rescue.

Commercial applications include ship routing, fishery

forecasting, and global environmental monitoring,

e.g., pollutant-spill risk assessments, El Niño forecasts, and global change studies.

Fig. 16 - Desktop visualization of the DoD Groundwater Modeling System, showing a conceptualized

contaminant plume (surface features and one

observation well superimposed)

Pollutant Cleanup Strategies within DoD

The cleanup of contaminated groundwater

resources are a prime concern at over 7,500 DoD

sites. Environmental quality modeling and simulation

technology, shown in Fig. 16 as executed on HPC

resources, are keys to the timely evaluation, design,

and operation of groundwater remediation systems

that are both technically proficient and cost effective.

By using environmental quality HPC technology, the

best cleanup strategies are selected and, in turn,

optimized for site-specific conditions prior to actual

field implementation. Following field implementation, HPC technology provides the basis for refining

or enhancing the operation of remediation technologies. As an example, use of such technology at a

single DoD site (Schofield Barracks, Hawaii) resulted

in the formulation of a cleanup strategy whose costs

are less than estimated for a nonoptimized design.

Perhaps even more important, regulatory acceptance

of the modeling and simulation results from this

installation were made possible and contributed

directly to the acceptance of the more cost-effective

cleanup strategy.
29

IMPLEMENTATION STRATEGY

29

The HPC Modernization Program is now

entering the operational capability phase of an

implementation continuum that was initially documented with the publication of the first HPC Modernization Plan in 1992 and updated in 1994.

During the first phase of the program, existing

DoD laboratories were selected for placement of

HPC resources, including a small number of experimental parallel systems. The Major Automated

Information System Review Council approval is

required for every major event throughout the

program life and initiated the documentation that

resulted in a $1.4 billion delegation of procurement

authority from the General Services Administration.

During the second phase of the program, the

program office established two interim major shared

resource centers and the Interim Defense Research

and Engineering Network. The program also obtained Major Automated Information System Review

Council approval of acquisition milestone decisions

and completed five major acquisitions—four to

establish integration contractors for each of the four

major shared resource centers and a telecommunications services contract to provide the Defense

Research and Engineering Network. The HPC

Modernization Office began an ongoing requirements collection and analysis effort to determine the

immediate and long-term HPC requirements within

the scope of the program. The results of the initial

requirements analysis and the overall acquisition and

implementation plans of the program were documented in the second HPC Modernization Plan,

published in 1994. The major contract awards were

consummated in calendar year 1996.

The third phase, documented in this plan, is

focused on delivering the HPC infrastructure to DoD

users. The focus in earlier phases was a combination

of procurement of interim capabilities and planning

and acquisition activities required to effect the 1996

contract awards. The focus today is on enabling the

users to deliver computational science results to the

warfighter. Therefore, the implementation is principally concerned with (1) the applications needed to

support the warfighter mission, (2) the ongoing

upgrade of HPC technology to support user capabilities, including both center facilities and network

services; and (3) the management of the program to

assure a continuing environment for innovation and

leadership in the application of computational

science to defense challenges.

Fiscal Year 1997 Planned Program

As a result of the completion of acquisition

actions in fiscal year 1996, all four major shared

resource centers have begun operations. By the end

of calendar year 1997, all program elements will be

operational, and network services will be operational

across the user community. The specific program

plans, by program initiative, are described below.

HPC Centers Initiative. The program will

continue the modernization and sustainment of the

HPC centers, as follows:

Distributed Centers - The program will assess

and prioritize HPC requirements for distributed

centers in fiscal years 1997 and 1998 and acquire

and deploy new systems or upgrades to existing

systems that were approved for implementation in

fiscal year 1997, as part of the fiscal year 1997/

1998 review process.

Major Shared Resource Centers - Additional

HPC systems, storage, and scientific visualization

capabilities will be acquired to populate and upgrade

the established major shared resource centers to help

meet the projected HPC requirements of the laboratories and research and development centers. Over

a 2-year period (i.e., fiscal years 1997 and 1998),

major shared resource center contract options will

be exercised to upgrade performance at the major

shared resource centers, minimally tripling the initial

computing capability at each site. Fiscal year 1997

marks the first year of this 2-year effort.

Networking Initiative. During fiscal year

1997, the Defense Research Engineering Network’s

initial performance capability will become operational, providing 10 megabits per second to 155

megabits per second services to the user community
30 HPC Modernization Plan

in the first year. The initial performance capability,

10 existing nodes in the interim network, will be

upgraded to 10 megabits per second or 155 megabits per second. Acceptance of the initial capability is

scheduled for the 4th quarter of fiscal year 1997.

Software Support Initiative. Development

efforts will continue in the software support initiatives. Each project will be reviewed in accordance

with established performance metrics and projectspecific milestones. The integrated modeling and test

environment projects were initiated in the 2nd

quarter of fiscal year 1997. All of the 43 initial

CHSSI projects will be funded and fully under way

during this fiscal year. At least one of the projects will

enter formal alpha testing as specified in the CHSSI

Test and Evaluation Master Plan. A workshop will be

held in June 1997 for the project developers to

review programming standards and performance

metrics.

Fiscal Year 1998 Planned Program

During fiscal year 1998, the infrastructure will be

fully operational. In addition to normal management

and sustainment of the program, the HPC Modernization Program will continue to focus on enabling

users to transition to scalable systems. Continued

leverage of national HPC investments, particularly in

software, will continue to be a priority of the

program’s management. The description of planned

expenditures as documented in support to the

President’s budget includes the following.

HPC Centers Initiative. The program will

continue the modernization and sustainment of the

HPC centers, as follows:

Distributed Centers - The program will acquire

and deploy new systems or upgrades to existing

systems that were approved for fiscal year 1998

implementation. Proposals will be solicited from

DoD services and agencies for fiscal year 1999

implementation.

Major Shared Resource Centers - The program

will sustain the existing capability and continue the

modernization process by acquiring additional HPC

systems, storage, and scientific visualization capabilities; these are needed to populate and upgrade the

established major shared resource centers to meet

the projected HPC requirements of the laboratories

and research and development centers. Over a 2-

year period (i.e., fiscal years 1997 and 1998), major

shared resource center contract options will be

exercised to upgrade performance at the major

shared resource centers, minimally tripling the initial

computing capability at each site. Fiscal year 1998

marks the second year of this 2-year effort.

Networking Initiative. The primary service

provided by the Defense Research Engineering

Network contract will continue operations, providing

10 megabits to 622 megabits of services to the user

community. Options to increase bandwidth at

selected sites will be executed. Transition from the

Interim Defense Research Engineering Network to

Defense Research Engineering Network for all

remaining sites is scheduled for completion by late

fiscal year 1998.

Software Support Initiative. Development

efforts in the software support initiative will continue,

as will the projects approved for the integrated

modeling and test environment computational

technology area. The common HPC software

support initiative will continue to develop shared

scalable applications-supporting software to exploit

scalable HPC assets. Projects will be reviewed in

accordance with established performance metrics

and project-specific milestones. During fiscal year

1998, approximately half of the CHSSI projects will

enter the formal alpha testing phase. Depending on

the outcome of this testing, many of these projects

will then enter beta testing and early use by a

broader but selected group of users. These users will

not only help to evaluate the efficacy and scientific

accuracy of the software but will also form the core

team for technology transfer and dissemination to

the broader community after successful completion

of the test process.

Fiscal Year 1999 Planned Program

During fiscal year 1999, the HPC Modernization

Program will continue to focus on user enablement

via ongoing management of the core program

initiatives. In addition, planning will begin for the

acquisitions required to sustain the DoD HPC

infrastructure beyond fiscal year 2001. This planning

cycle will also involve a reassessment of long-term

requirements and various implementation options

that might uniquely meet the needs of the DoD

through the 2010 time frame. The description of

planned expenditures as documented in support to

the President’s budget includes the following.

HPC Centers Initiative. The program will

continue the modernization and sustainment of the

HPC centers, as follows:

Distributed Centers - The HPC Modernization

Program will again assess and prioritize HPC

requirements for distributed centers and will acquire

and deploy new systems or upgrades to existing

systems via service-sponsored white paper proposals
IMPLEMENTATION STRATEGY 31

to accomplish mission needs.

Major Shared Resource Centers - The program

will sustain the existing capability and continue the

modernization process by acquiring additional HPC

systems, storage, and scientific visualization capabilities; these are needed to populate and upgrade the

established major shared resource centers to meet

the projected HPC requirements of the laboratories

and research and development centers. Over the 2-

year period (i.e., fiscal years 1999 and 2000), major

shared resource center contract options will be

exercised to upgrade performance at the major

shared resource centers, minimally tripling the

capability at each site over the previous performance

level. Fiscal year 1999 marks the first year of this 2-

year effort.

Networking Initiative. Network connectivity

will be sustained for HPC Modernization Program

users. Options to increase bandwidth at selected sites

will be executed to meet the increased demands

resulting from upgraded computing capabilities at

the HPC centers. Additional sites will be added to

the Defense Research Engineering Network.

Software Support Initiative. Development

efforts in the software support initiative will continue.

The common HPC software support initiative will

continue, developing shared scalable applicationssupporting software to exploit scalable HPC assets.

Projects will be reviewed in accordance with established performance metrics and project specific

milestones. By the end of fiscal year 1999, we

anticipate being able to begin testing for those

projects not ready for alpha testing in the previous

year and complete testing and formal release for

broader use for the projects that entered alpha

testing in fiscal year 1998. By this time we also

anticipate being able to measure the impact of the

earliest codes on both user capability and on efficiency of use of program resources.

Table 13 - Funding Profile ($M)

F 7 Y96 F 8 Y9 F 9 Y9 F 0 Y9 F 1 Y0 F 2 Y0 F 3 Y0 FY0

S 3 ystem Acquisitions 1 0 29. 1 9 24. 9 5 1. 8 0 4. 6 6 7. 4 9 3. 5 2 4. 54.

N 8 etworking 1 0 7. 1 9 8. 2 7 6. 3 8 5. 3 6 7. 3 0 9. 3 4 8. 39.

S 6 ystem Applications Support 9 7 . 2 7 0. 2 7 2. 2 4 3. 2 6 4. 2 6 5. 2 5 4. 25.

S 3 ustainment and Operations 5 2 6. 8 6 4. 7 4 6. 8 7 9. 9 6 3. 9 6 8. 9 9 4. 97.

T 0 otal 2 9 13. 2 1 46. 2 3 18. 2 9 33. 2 4 22. 2 9 07. 2 0 11. 217.

Funding

The current program is projected to spend

$1310.7 million for fiscal years 1998 through 2003.

Table 13 shows the funding profile by year and major

spending category for fiscal years 1996 through

1997, including actual budgets for fiscal years 1996

and 1997 and 5-year projections from the President’s

fiscal year 1998 budget. This program funding

represents a reduction of approximately $500 million

from the budget profile shown in the 1994 HPC

Modernization Plan, assuming level funding projections for the fiscal years 2000 through 2003. While

there have been a number of budget adjustments

since the publication of the 1994 HPC Modernization

Plan, most of the difference resulted from a single

$269 million reduction sustained by the program as

part of the final resolution of the Defense Program

Objective Memorandum (POM) for fiscal years 1997

through 2001.

Comparison of the current budget profiles with

earlier program budget projections presented in the

1994 plan, industry briefings, and acquisition documents indicates that most of the major reductions

referenced above—approximately $100 million per

year beginning in fiscal year 2000—will be sustained

by the HPC Modernization Program system acquisition account. While the lower budget is sufficient to

exercise the base capability as currently programmed

for Performance Level 3 of the Major Shared Resource Center contracts, this capability will likely

address less than 30% of the projected validated

computing requirements for years 2000 and beyond.

As part of its strategic planning efforts, the HPC

Modernization Program staff is examining options to

assure that the highest priority warfighter needs are

met in light of these shortfalls.

The networking budget will increase to level

approximately twice that budgeted for fiscal year
32 HPC Modernization Plan

1997. While the DoD will realize substantial economic savings for comparable service in the transition from the Interim Defense Research and Engineering Network to the Defense Research and

Engineering Network, the demand for high performance networking will increase to reflect the dramatic increase in both shared resource capability and

HPC requirements across the user base. Another

important factor in this growth will be the increased

importance of remote collaboration across functional

and service boundaries, and with the broader research and development enterprises in academia and

industry. HPC-based collaborative methods—

important to cost-effective development of optimized

warfighter technologies for the future—will require

the transmission of extremely large image visualization objects, often in real time or near real time.

The continued strong funding in the software

support initiative and in the sustainment and operations budget lines reflects the importance that the

HPC Modernization Program places on software,

integration, and related shared services to the users

of HPC throughout DoD. The sustainment budget,

for example, includes the highly leveraged investments in training and collaboration with the academic community via the Programming Environment

and Training elements of the program. These

investments will continue to guarantee that the focus

of the program remains on enabling users to address

warfighter needs, not only by strengthening the

support infrastructure, but also by providing effective

mechanisms for continued leverage of both national

and industrial investments in HPC-based productivity.
Produced for the DoD HPC Modernization Office

by the Publications Branch (202) 767-2782

Technical Information Division

NAVAL RESEARCH LABORATORY

4555 Overlook Avenue, S.W.

Washington, DC 20375-5333

Coordinator: Patricia Staffieri

Design and layout: Jan Morrow
DoD HPC Modernization Office

1010 North Glebe Road, Suite 510

Arlington, VA 22210

Phone: (703) 812-8205

FAX: (703) 812-9701

http://www.hpcmo.hpc.mil