19981015 072

Department of Defense High Performance Computing

Ipprorvd for pwblte »toa««

DürtdbattsÄ ÖnllmKwi

IllllllölilfPKlliMfP^^

#mm

bfe^^S^
Front cover captions (clockwise starting from top right):

Computed wave pattern at an instant. Free surface is color-coded

with the velocity magnitude; side boundary surface is color-coded

with the fluid pressure.

Axial velocity contours after 10 propulsor revolutions

M113/BMP-2 OSV turret plastic equivalent strain contour plot

showing deformation after 50 ms of simulated rollover

Contours of electric field scattered from B-1B

Current image on GE404 nozzle (2 GHz, vertical polarization)

Back cover captions (clockwise starting from top right):

Mach field in stable level flight

LOCAAS geometry in stowed configuration

Simulation of solid para-hydrogen (small purple spheres) doped with

atomic lithium impurities (larger orange spheres) at 4 K. The doping

level is 3.3 mol percent. In the lower right, a section of the hydrogen

has been cut away so that the progress of the lithium recombination

can be seen more clearly.

Gridless large eddy simulation computation of the turbulent bow wave on

the DDG-51

Mean sea surface height in the Gulf Stream region from the 1/32-degree

Atlantic model compared with the Gulf Stream northern edge from

satellite infrared imagery (1982 to 1996) mean

Mach contours showing the interference flow field between the missile

and the BAT submunition at different radial locations

contents

Foreword

Executive Summary

Program Overview

Program Accomplishments and Future Plans
REPRODUCTION QUALITY NOTICE

This document is the best quality available. The copy furnished

to DTIC contained pages that may have the following quality

problems:

• Pages smaller or larger than normal.

• Pages with background color or light colored printing.

• Pages with small type or poor printing; and or

• Pages with continuous tone material or color

photographs.

Due to various output media available these conditions may or

may not cause poor legibility in the microfiche or hardcopy output

you receive.

I2£l If this block is checked, the copy furnished to DTIC

contained pages with color printing, that when reproduced in

Black and White, may change detail of the original copy.
HIGH PERFORMANCE

COMPUTING

MODERNIZATION PLAN

March 1998

OTIC 9BALWI maPBCTBD 4
DIRECTOR OF DEFENSE RESEARCH AND ENGINEERING

3030 DEFENSE PENTAGON

WASHINGTON, DC. 20301-3030

This is the fourth edition ofthe Department of Defense High Performance Computing

Modernization Plan. This plan has been prepared as required by Report H.R. 105-206 that

accompanied the Department of Defense Appropriation Bill, Fiscal Year 1998.

We continue to make significant progress towards our goal of improving the Department's

ability to exploit computation to provide the warfighter with the technological advantage. We

continued to strengthen the capability ofthe Department's high performance computing

resources for the science and technology and the developmental test and evaluation programs.

Many of our leading edge computationally intensive projects are making critical

contributions to high-priority warfighter systems; others are addressing research challenges

fundamental to assuring defense technology competitiveness on the 21st century battlefield. A

comprehensive presentation of project successes using high performance computing can be found

in the 1998 edition ofHigh Performance Computing Contributions to DoD Mission Success.

The High Performance Computing Modernization Program provides an essential enabling

technology that our leading scientists and engineers are using to assure our armed forces retain

technological advantage and force dominance on tomorrow's battlefield.

George T. Singley, III

Acting
EXECUTIVE SUMMARY

High performance computing (HPC) has historically

played a major role in the ability of the United States to

develop and deploy superior weapons, warfighting capabilities, and mission support systems. Under the auspices

of the Director, Defense Research and Engineering

(DDR&E), the Department of Defense (DoD) High Performance Computing Modernization Program (HPCMP)

is the focused modernization effort within the DoD to

acquire, manage, and sustain modern HPC resources in

support of defense science and technology, and developmental test and evaluation.

The HPCMP is rapidly evolving past its initial program

development and procurement start-up phase and is increasing focus on application of HPC technology and

resources to priority defense requirements. This plan

describes some of the accomplishments achieved by the

program in Fiscal Year (FY) 1997 and outlines the major

strategies and milestones we expect to achieve over the

next two years. The overriding goal of the program is to

exploit HPC technology for military advantage across

battlespace. The strategy to achieve this goal is to acquire

and sustain world-class high performance computing and

network capabilities for use by defense scientists and

engineers.

The use of HPC technology is reducing costs and time to

perform systems analysis, design, development, test, and

deployment; helping avoid environmental damage; and

improving the integration and effectiveness of complex

weapons systems. As the DoD continues to reform and

reengineer its acquisition processes, HPC assets, along

with high-fidelity scalable models and simulations, are

being used to reduce the number and cost of building

expensive prototypes. High-fidelity modeling and simulation is also being used to explore more design options

and identify important testing priorities at a fraction of

the cost and time than was possible with traditional theoretical, experimental, or operational methods. As a result

of these and similar mission impacts, HPC has become

recognized as a key ingredient to the successful implementation of major DoD acquisition programs. The technology is considered fundamental to the success of the

Simulation Based Acquisition and Simulation Based

Design reform initiatives.

During FY 1997, the HPCMP acquired and installed the

first phase of its Performance Level II capability. These

actions increased threefold the peak computational capacity of the program's major shared resource centers. At

the same time, several applications development teams

began releasing validated alpha and beta versions of their

new scalable software, further expanding the use of more

powerful scalable HPC technology into day-to-day research and development efforts of the DoD. In addition,

the first high-speed, high-bandwidth network connections, available under the new Defense Research and

Engineering Network (DREN) contract, were implemented as previous Interim DREN sites, and new sites

took advantage of enhanced network services.

Also in FY 1997, the Major Automated Information System Review Council (MAISRC), responsible for oversight of the HPCMP within the Office of the Secretary of

Defense, began an aggressive testing schedule of the

major initiatives of the program. Teams were sent out to

test and evaluate the performance capabilities of the various HPC major shared resource centers (MSRCs), distributed centers (DCs), and application development

teams.

Over the next three years, the HPCMP will complete its

Performance Level III capability upgrades, more than

doubling HPC capability within the DoD. At the same

time, demand for these resources is expected to rise at an

even greater rate, necessitating continued close management and prioritization of HPC assets across the many

users of these systems.

The success of the DoD HPCMP is ultimately measured

by what HPC resources enable defense scientists and engineers to accomplish for the warfighter. While the full impact of inserting HPC technology into the defense acquisition process will not be clearly visible until resulting

weapons, capabilities, and support systems have been

deployed and battle tested, an estimate of the value of

these systems is beginning to take shape. Major success

stories, technical papers, and scientific breakthroughs are

being released regularly; use of this technology by over

4,000 defense scientists and engineers has more than

tripled in the past two years. A small sampling of notable

successes using HPC technology can be seen on the next

two pages. A more in-depth review of these efforts and

other major HPC successes can be found in the 1998 edition of High Performance Computing Contributions to

DoD Mission Success.
HPC Modernization Plan

Aerodynamic Control of Theater

Defense Missiles Using Jet Interaction

Problem: Enemy theater ballistic missiles and cruise

missiles have emerged as a serious threat to both ground

forces and civilian populations. Our ability to track, acquire, and destroy these threats depends on our ability to

operate across a wide range of altitude and atmospheric

conditions that cannot always support the necessary maneuvering agility using traditional aerodynamic control surfaces such as fins or strakes. Wind tunnel experiments are insufficient to resolve the complex interactions

and control performance of the missile under various

conditions.

SINGLE DIVERT JET

Approach: Use advanced numerical algorithms coupled with high performance computational resources to attack the

three-dimensional problem of effective jet interaction control and autopilot design.

Results: Defense researchers have resolved several critical issues ofjet interaction control and generated the extensive

single and multiple jet data needed to understand different aspects ofjet interaction control during flight. These findings

are now being used to improve guidance and control accuracy, maneuverability, and reliability of the interceptors needed

to destroy enemy targets across multiple engagement conditions. With HPC resources, an efficient and robust autopilot

can be designed and tested for use across the entire battlespace.

Spatial Distribution of Water Vapor Using

Visible-Band Geostationary Satellite

Problem: Turbulent convection, water vapor, and cloud

formations can affect theater operations dramatically, creating poor line-of-sight visibility, increasing threats to pilot

safety, and decreasing weapon system performance and

communications.

Approach: Improve weather prediction models by increasing their ability to accurately simulate and predict horizontal variations in surface heating and precipitation. Model

predictions can also be improved greatly by feeding them

real-time, visible-band satellite imagery that precisely

identifies positions and movements of clouds and water

vapor observations.

Results: Using HPC technology, new higher fidelity

weather models, integrated with real-time satellite data, are

producing more accurate predictions of shelter height temperatures, moisture convergence patterns, and surface

winds. These positive effects are maintained throughout a

9-hour forecast, providing better predictions of clouds,

precipitation, and associated aviation and weapon system

impact variables (e.g., icing, downbursts). This technology

is also being transferred to the civilian aviation community

and other applications interested in the effects of weather.

Z 35 -—

LU

Q

D

85 80

LONGITUDE (°W)

85

85

LU

Q

85 80

LONGITUDE (°W)
EXECUTIVE SUMMARY

ON-LINE TEST DIRECTION

TEST

CONDUCTOR

/

On-Line Turbine Engine Test Support

Problem: Air-on-hours for turbine engine testing are

very expensive ($10,000/hour for small tests and up to

$27,000/hour for larger tests). Testing of modern turbine

engines also requires hundreds or thousands of sensors

collecting a wide variety of performance data simultaneously, several thousand times a second. Processing

test data on-line is required to make critical decisions

during the test, prevent expensive future retesting, and

support timely acquisition decisions.

Approach: Leading-edge high performance computing

resources are incorporated with array processors in the

data acquisition stream to convert raw test data to

engineering units (EU) data. EU data are sent to an HPC

system for real-time and near-real-time calculation of

engine performance parameters. Performance data are then sent over a high-speed communications network to local

workstations for on-line analysis and to a central archive for future access.

Results: With modern HPC technology, test data point turnaround time was decreased from an average of 10 minutes to

2 minutes. This concept was originally targeted for the F119 engine development tests, but because of its successful cost

reduction and improved turnaround time, it has been expanded to several turbine engine tests such as the F100, F110,

F414, F402, F404, Pratt & Whitney 4000, and Rolls Royce Trent engine.

HIGH

PERFORMANCE

COMPUTER

KNOWLEDGE

DATA CONVERSION

ENGINE/FACILITY PERFORMANCE

TREND ANALYSIS

PREDICTIVE PERFORMANCE

0.30

Broadband Electromagnetic Simulation 20 °

on Unstructured Grids

Problem: Electronic warfare requirements across many

of our critical defense systems call for rapid computing

of broadband radar scatter, advanced radar range design

and simulation, and determination of the effects of highpower microwaves, electromagnetic interference (EMI),

and electromagnetic pulse (EMP).

Approach: HPC technology is being used to

generate and analyze high-fidelity grids for

complex targets such as fighter aircraft and fit

them to target surfaces from a standard CAD

database. Accurate propagation of waves over

the target through variable geometry of unstructured cells is a major challenge requiring

very large computational capabilities.

Results: Computational electromagnetics (CEM) has rapidly become integral to advanced aerospace design, not only as

a stand-alone technology, but also as part of a multidisciplinary coupling that leads to well-optimized designs. HPC technology has advanced our knowledge and simulation of radar cross sections (RCS) for several very difficult targets. The

first pulse calculation of backscatter RCS for a fighter aircraft using an HPC-based unstructured grid CEM code is shown

above for a nose-on incidence. The figures illustrate the passage of a vertically polarized plane pulse over the target,

showing the total horizontal electric field at each instant. This information is being used to assess the vulnerability of

friendly and enemy warfighting platforms across a wide variety of electromagnetic frequencies. It also allows defense

scientists and engineers to design more effective electronic countermeasures to protect our warfighting platforms and

develop more effective weapons to exploit vulnerabilities in target platforms.
sj

BOs

ft,

•S3g

&o Qs

.00
PROGRAM OVERVIEW

History

The High Performance Computing Modernization Program (HPCMP) was initiated in FY 1993 in response to

congressional direction to modernize the Department of

Defense (DoD) HPC capabilities. Early on, senior leaders

recognized the unique potential of this emerging technology as critical to our nation's future defense. The High

Performance Computing Modernization Office

(HPCMO), staffed with representatives from each Service, was established in FY 1994 to perform life cycle

management and acquisition oversight needed to ensure

that the program supports the HPC needs of the defense

science and technology (S&T) and developmental test

and evaluation (DT&E) communities. The program has

evolved past an initial start-up phase of procurement and

program development activities. Since FY 1996, it has

fielded a world-class HPC infrastructure, available to the

full S&T and DT&E communities, that includes 4 major

shared resource centers and 13 smaller focused distributed centers across the country. Figure 1 illustrates the

DoD HPCMP integrated program strategy.

Program Mission

The HPCMP mission is to modernize the highend high performance computing capability

used by the DoD science and technology and

developmental test and evaluation communities,

in order to incorporate technological advantage

into superior weapons, warfighting capabilities,

and related support systems and to accomplish

this advantage more rapidly and qffordably with

reduced risks to human life and optimized

system performance.

Program Vision

The HPCMP vision is to enable the DoD to

maintain its technological supremacy over

adversaries ofthe United States in weapon

systems design and to foster the flow ofthis

technology into warfighting support systems by

providing world-class HPC capability to the

science and technology and developmental test

and evaluation communities.

Program Scope

The HPCMP scope is bounded both in terms of the user

community it serves and the technological capability it

delivers. By limiting the scope and by concentrating the

majority of resources at a small number of shared centers, the program has been able to provide world-class

computing capabilities that could not have efficiently

been obtained and sustained by individual Services or

Agencies. This sharing of resources reduces overall acquisition and sustainment costs and fosters collaboration

and cooperation across the DoD S&T and DT&E communities.

The scope of the user community is defined by Congress

(Public Law 104-61, December 1, 1995, 109 Statute 665,

Sec. 8073) to be

" (1) the DoD Science and Technology

(S&T) sites under the cognizance ofthe Director, Defense Research and Engineering, (2) the

DoD Test and Evaluation (DT&E) centers under

the Director, Test and Evaluation (Office ofthe

Under Secretary ofDefense, Acquisition and

Technology), and (3) the Ballistic Missile Defense Organization. ."

While the program serves the DT&E community, it is

important to note that its scope does not address operational test and evaluation requirements — including live

fire testing. Nor is it funded sufficiently to address many

of the unique applications that are not easily handled at

multiuser shared centers such as those maintained by this

program. Recognizing these limitations, and in response

to congressional reporting requirements,* the Director,

Developmental Test, Systems Engineering and Evaluation, and the Director, Operational Test and Evaluation,

are currently developing a separate plan for integrating

HPC fully into the production test and evaluation process

and enterprise.

From a technology capability perspective, the program

maintains a strict focus on providing high performance

computing. The definition of "high performance computing" changes as the technology continues to evolve. This

consistent evolution requires that the program continually

*National Defense Authorization Actfor Fiscal Year 1996, Conference Report 104-450, pp 701-702.
HPC Modernization Plan

reassess its acquisition plans to ensure that it is not acquiring systems that have become "departmental" class —

computational capability that remains the responsibility of

the local mission organization. The rapid evolution of high

performance computing also requires that the program

focus on delivering improved capability early in its commercial life cycle. This allows the DoD to maintain the

technological edge required to analyze, design, produce,

and deploy advanced weapons systems and capabilities to

the warfighter — before similar computational capabilities

are available to our adversaries and economic competitors.

Program User Requirements

From its inception, the HPCMP was designed to address

the continuing high-end computing needs of the DoD S&T

program. Congress later amended the program scope to its

current user base of over of 4,000 scientists and engineers

located at more than 100 DoD laboratories, test centers,

and contractor and academic sites. Each member of this

community continues to have evolving requirements for

high-end HPC technology. These requirements are identified and tracked via an extensive annual effort led by the

program office and a working group of representatives of

all of the major components of the user community.

The requirements process developed and used by the program includes a series of surveys sent to users and potential users throughout the DoD S&T and DT&E communities, coupled with visits to selected sites to discuss requirements. In addition, the program obtains feedback

through user satisfaction surveys, annual user group

meetings, and advisory panels that work with resource

managers to establish policies and procedures at the

shared resource centers.

A vigorous requirements gathering and validation process

was initiated in FY 1994 to establish a baseline of existing HPC resources and determine the present and future

needs for such resources. This on-going process includes

an annual analysis and revalidation of short-term and

longer term user needs to provide accurate, timely data

on which to base program decisions.

Figure 2 shows the HPC capabilities needed to resolve

important problems in six computational technology

areas (CTAs) with varying degrees of fidelity. Increasing

fidelity is required to solve critical problems such as

model-driven Integrated Modeling and Test Environments (IMT) and high-fidelity Forces Modeling and

Simulation (FMS). Other important HPC-related resource

requirements, such as on-line file storage, archival data,

and network bandwidth are not reflected in Figure 2.

However, these needs also require significant increases in

capability to resolve important defense problems across

the CTAs.

100 Ti10T -

1000 Atoms Armor Optimization

Advanced Materials Design (CSM)

1/64th° grid

w

LLI

I-

>- m

>

DC

o

DC

W

IQ.

o

Ü

Real-time

100 G

10G

1 G

Certification

(Minimum test)

Unsteady-Flow

1 GFIops 10GFIops 100GFIops

COMPUTER SPEED

1 TFIops 10TFIops

FY97 rve;

Figure 2 - HPC resource requirementsfor important computational technology area problems
PROGRAM OVERVIEW

Computational Technology Areas

The DoD HPCMP user community is organized around

10 CTAs. Each CTA has a designated leader who is a

prominent DoD scientist or engineer working within the

research disciplines included in his CTA. Table 1 provides a brief description of each CTA, and Table 2 lists

the CTA leaders. CTA leaders serve for a term of approximately three years, and, beginning in FY 1998, will

be rotated among the military Services.

Table 1 — Computational Technology Areas

Computational Structural Mechanics

CSM provides high-resolution, multidimensional modeling of

materials and structures subjected to a broad range of static,

dynamic, and impulsive loading conditions. Uses include effects

of explosions on various facilities, underwater explosion and ship

response, structural acoustics, structural analysis, propulsion

systems, lethality and survivability (aircraft, ships, submarines,

tanks), theater missile defense, and real-time, large-scale soldier

and hardware-in-the-loop vehicle dynamics.

Computational Fluid Dynamics

CFD provides the accurate numerical solution of the equations

describing fluid and gas motion and fluid-dynamics research.

Uses include the design of complex combustion and propulsion

systems that are inaccessible or too costly to prototype; the

dynamics of submarines, subsonic, transonic and supersonic air

vehicles, pipe flows, air circulation, missiles, projectiles; and

magnetohydrodynamics for advanced power systems and

weapons effects.

Computational Chemistry and Materials Science

CCM provides prediction of basic properties of new chemical

species and application of this molecular understanding to the

development of advanced materials. Uses include design of new

chemical compounds for fuels, lubricants, explosives, rocket

propellants, and chemical defense agents. In addition, advanced

modeling techniques are used to develop new high performance

materials for electronics, advanced sensors, aircraft engines,

semiconductor lasers, laser protection systems, advanced rocket

engines components, and biomedical applications.

Computational Electromagnetics and Acoustics

CEA provides high-resolution multidimensional solutions of

Maxwell's equations and acoustic wave equations. Uses include

calculating fields about antenna arrays; signatures of tactical

ground, air, sea, and space vehicles; the signature of buried

munitions; performance/design factors for electromagnetic gun

technology; high-power microwave performance; modeling of

acoustic fields for surveillance and communication; seismic fields

for mine detection; and acoustic shock waves of explosions for

antipersonnel weapons.

Climate/Weather/Ocean Modeling and Simulation

CWO provides the numerical simulation and forecast of the

Earth's climate as well as oceanic and atmospheric variability.

Uses include improved flight safety; search-and-rescue mission

planning; propagation of weapons; aircraft and ship routing;

antisubmarine and undersea warfare; enhanced capabilities in

adverse weather; and the capability to predict magnetic storminduced effects and outages on command, control,

communications (C3), surveillance, and navigation systems.

Signal/Image Processing

SIP provides for extraction and analysis of key information from

various sensor outputs in real time. Sensors include sonor, radar,

visible and infrared imagers, and signal intelligence and navigation assets. Uses include intelligence, surveillance and reconnaissance (ISR), avionics, communications, smart munitions, and

electronic warfare. Functions include detecting, tracking,

classifying, and recognizing targets in the midst of noise and

jamming; generating high-resolution low-noise imagery; and the

compression of imagery for communications and storage.

Forces Modeling and Simulation/C4I

FMS integrates high-speed command, control, communications,

computers, and intelligence (C4I) systems to manage a battle

space; provides large-scale simulations of complex military

engagements to facilitate mission rehearsal/training, mission

planning, and postmission analysis; and advances digital library

technology for support of FMS/C4I research and development

activities. Uses span the design, development, test, evaluation,

and deployment of a variety of warfighting and training systems.

Environmental Quality Modeling and Simulation

EQM provides high-resolution three-dimensional Navier-Stokes

modeling of hydrodynamics and contaminant transport through

air, ground, and aquatic ecosystems. Uses include stewardship

and conservation of natural and cultural resources; prediction of

chemical, and biochemical contaminant flows; design and

operation of installation restoration; integrated management in

support of environmental quality; noise evaluation and abatement;

and water quality models.

Computational Electronics and Nanoelectronics

CEN provides the design, modeling, and simulation of complex

electronic devices, integrated circuits, and small components.

Uses include lower costs, weights, size, and improved performance of electronics through predictive high-fidelity modeling and

simulation; analog/digital high-frequency circuit and device

simulation and optimization; modeling and simulation of microelectromechanical devices, micro-resonators, active and passive

millimeter-wave circuits and structures; electronic/photonic

interconnect and packaging analysis; and fault modeling.

Integrated Modeling and Test Environments

IMT applies high performance computing software tools and

techniques with live tests and hardware-in-the-loop simulations

for test and evaluation of DoD weapons, components, subsystems, and systems in virtual and composite virtual/real

environments. Uses include digital scene generation, six degreesof-freedom trajectory simulations, real-time test-data analysis and

display systems for test control and evaluation, and high-fidelity

engineering and closed-loop engagement models (one-on-one and

few-on-few).
HPC Modernization Plan

Table 2 — Computational Technology Areas Leaders

Computational Technology Area Area Leader Location

Computational Structural Mechanics Kent Kimsey Army Research Laboratory (Aberdeen)

Computational Fluid Dynamics Jay Boris Naval Research Laboratory (DC)

Computational Chemistry and Materials Science Scott Wierschke Air Force Research Laboratory (Edwards)

Computational Electromagnetics and Acoustics Joseph Shang Air Force Research Laboratory (Wright)

Climate/Weather/Ocean Modeling and Simulation Joseph McCaffrey Naval Research Laboratory (Stennis)

Signal/Image Processing Richard Linderman Air Force Research Laboratory (Rome)

Forces Modeling and Simulation/C4I Robert Wasilausky Naval Space and Warfare Systems Center, San

Diego

Environmental Quality Modeling and Simulation Jeffery Holland Army Waterways Experiment Station

Computational Electronics and Nanoelectronics Barry Perlman Army Research Laboratory (Fort Monmouth)

Integrated Modeling and Test Environments Andrew Mark Army Research Laboratory (Aberdeen)

Teams of leading defense scientists and engineers use

HPC resources to develop, advance, and maintain hundreds of scientific algorithms, codes, models, and simulations needed to analyze, design, develop, test, evaluate,

and deploy technologically superior weapons, warfighting capabilities, and related support systems. Their

efforts produce and support key enabling technologies,

capabilities, and demonstrations articulated by the various Defense Technology Objectives (DTOs). These enabling DTOs, in turn, support the 10 most critical Joint

Warfighting Capability Objectives (JWCOs) promulgated

by the Joint Requirements Oversight Council (JROC) of

the Joint Chiefs of Staff.*

While not all-inclusive, JWCOs provide focus, priority,

and a common reference point for much of the DoD's

research, development, test, and evaluation efforts.

They are described in detail in the annual Joint Warfighting Science and Technology Plan published by the

Director, Defense Research and Engineering and include

(1) information superiority, (2) precision force, (3) combat identification, (4) joint theater missile defense, (5)

military operations in urban terrain, (6) joint readiness

and logistics, (7) joint countermine, (8) electronic combat, (9) chemical/ biological warfare defense and protection, and (10) counter weapons of mass destruction.

Figure 3 shows the interrelationships between JWCOs,

DTOs, and CTAs and provides just one example of how

work being performed with DoD HPC resources is supporting high-priority warfighting capabilities. In most

cases, each CTA supports multiple DTOs and JWCOs.

Likewise, individual DTOs and JWCOs are supported by

multiple CTAs.

*A detailed description of the current Defense Technology Objectives and Joint Warfighting Capability Objectives can be found in various publications

updated and distributed annually by the Director, Defense Research and Engineering, including the Defense Technology Area Plan and the Joint

Waifighting Science and Technology Plan. These documents can also be found on the World Wide Web at http://www.dtic.mil/ddre.
PROGRAM OVERVIEW

1. Information Superiority

2. Precision Force

3. Combat Identification

4. Joint Theater Missile Defense

5. Military Operations in Urban Terrain

6. Joint Readiness and Logistics

7. Joint Countermine

8. Electronic Combat

9. Chemical/Biological Warfare Defense

and Protection

0. Counter Weapons of Mass Destruction

Defense Technology Objectives

Achieve JW

Computational Technology Areas

iT&E Efforts

monstrate D<

JO to Advance

lant Warfiqhtinq Systems

1. Air Platforms

2. Chemical/Biological Defense and Nuclear

3. Information Systems Technology

4. Ground and Sea Vehicles

5. Materials and Processes

6. Biomedical

7. Sensors, Electronics, and Battle Environments

8. Space Platforms

9. Human Systems

^10. Weapons

1. Computational Structural Mechanics

2. Computational Fluid Dynamics

3. Computational Chemistry and Materials

4. Computational Electromagnetics and Acoustics

5. Climate, Weather, and Ocean Modeling

6. Signal-Image Processing

7. Forces Modeling and Simulation

8. Environmental Quality Modeling and Simulation

9. Computational Electronics and Nanoelectronics

,10. Integrated Modeling and Test Environments

EXAMPLE OF HPC SUPPORT OF WARFIGHTING NEED: (COMANCHE)

Capability Need (JWCO) - Precision Force

"the capability to destroy selected high-value and

time-critical targets or to inflict damage with precision

while limiting collateral damage."

1. Information Superiority

Precision Force

3. Combat Identification

4. Joint Theater Missile Defense

5. Military Operations in Urban Terrain

6. Joint Readiness and Logistics

7. Joint Countermine

8. Electronic Combat

9. Chemicai/Biological Warfare Defense

and Protection

10. Counter Weapons of Mass Destruction

Enabling Technologies (DTO) - Air Platforms

includes: Aircraft Support/Sustainment Reduction -

"develop, demonstrate, and transition technologies to

extend the lives or reduce the cost of aging aircraft. The

ability to predict effects of unsteady aerodynamics loads

will be developed and used with advanced structural

concepts to increase fatigue life and reduce the support

cost of aircraft."

1. Air Platforms

2. Chemical/Biological Defense and Nuclear

3. Information Systems Technology

4. Ground and Sea Vehicles

5. Materials and Processes

6. Biomedical

7. Sensors, Electronics, and Battle Environments

8. Space Platforms

9. Human Systems

10. Weapons

Computational Technology Area (CTA) -

Computational Fluid Dynamics

includes: Comanche Helicopter and Missile Launch - HPC

used to model and simulate interactions and effects of

unsteady aerodynamic conditions and missile launch on

the Comanche Helicopter and its various systems. Predict

loading, handling, and host vehicle damage caused by

missile launch and off-body aerodynamics to support

maintenance, future design modification decisions, and

warfighting tactics."

1. Computational Structural Mechanics

Computational Fluid Dynamics

Computational Chemistry and Materials

Computational Electromagnetics and Acoustics

Climate, Weather, and Ocean Modeling

Signal-Image Processing

Forces Modeling and Simulation

Environmental Quality Modeling and Simulation

Computational Electronics and Nanoelectronics

Integrated Modeling and Test Environments

Figure 3 - HPC support to JWCOs
10 HPC Modernization Plan

The CTAs provide an effective framework for coordinating and focusing computation-based research efforts to

analyze, research, develop, and deploy advanced weapons, warfighting capabilities, and related support systems

in support of the critical DTOs and JWCOs. To maximize

the ability of the DoD S&T and DT&E community to

meet these mission needs, the program is providing and

sustaining an increasingly powerful high-end HPC

capability based on growing customer requirements. This

HPC environment must be broadly accessible to all

members of the community and includes

• a variety of well-balanced, high-end HPC computers;

a full complement of systems and applications

software selected to support the needs of the scientists and engineers in each of the CTAs;

• an aggressive technology insertion program that

continues the education, training, and expansion

of the DoD HPC community to accelerate the

exploitation of HPC technology;

• a reliable high-speed network that links DoD

scientists and engineers at their user sites with

each other and the computational resources located at the program's shared resource centers;

and

• software tools that help transition much of the

highest priority DoD computational workload to

scalable HPC systems.

Program Goals

The program goals established to provide an increasingly

powerful high-end HPC environment to the S&T and

DT&E community are to

(1) continually acquire the best commercially available high-end HPC computers,

(2) acquire and develop common-use software tools

and programming environments,

(3) expand and train the DoD HPC user base,

(4) link users and computer sites via high-speed networks (facilitating the creation of collaborative

work environments), and

(5) exploit the best ideas, algorithms, and software

tools emerging from the national HPC infrastructure.

Program Strategies

The program strategies to achieve these goals are

(1) Goal — Continually acquire the best commercially

available high-end HPC computers.

• Provide a balanced set of commercially available

heterogeneous computing platforms and HPC

systems to meet the full range of DoD requirements and permit optimum mapping of requirements to system types.

• Build complete HPC environments, including

large computing systems, software, and support

expertise at a few DoD laboratories and DT&E

centers designated as major shared resource centers (MSRCs) to support a wide community of

HPC users.

• Place modest-sized high performance computational systems at selected, distributed centers

(DCs) when that placement allows for either innovation or mission support not provided by

an MSRC.

• Support efforts to build complementary HPC

capabilities and technologies throughout government, academia, and industry that are applicable

to defense S&T and DT&E requirements.

• Continually acquire and upgrade equipment and

support services to assure that DoD HPC centers

provide world-class, state-of-the-art capabilities.

(2) Goal — Acquire and develop common-use software

tools and programming environments.

• Promote standards for DoD research, development, test, and evaluation software to ensure that

future transitions and advancements in software

technology are applied in an efficient and costeffective manner.

• Focus on applications software initiatives designed to overcome the technological inhibitors

that may delay the effective use of scalable high

performance computers.

• Support software components used in specific

CTAs or by a subset of HPC users across the

community.

• Identify software suites to be shared across selected MSRCs and focused DCs.
PROGRAM OVERVIEW 11

(3) Goal

base.

- Expand and train the DoD HPC user

• Provide user education in scalable computer

and computational sciences, software applications and optimization, and code conversion.

• Promote the formation of DoD-sponsored interdisciplinary teams and collaboration groups

to determine how best to support each CTA and

to leverage academia and industry expertise in

HPC for the solution of DoD problems.

• Develop shared application software in support

of high-priority and broadly needed CTAs.

• Aggressively transfer expertise and knowledge

among the DoD HPC user community.

• Expand the knowledge of the DoD HPC user

community as applications require.

(4) Goal — Link users and computer sites via highspeed networks (facilitating the creation of collaborative work environments).

• Deploy robust and cost-effective network connectivity consistent with HPC traffic requirements among and between DoD scientists and

engineers and HPC assets, and between the

DoD and external research and test and evaluation communities.

• Encourage remote usage where appropriate.

• Ensure that the highest priority applications

and end users are well supported.

(5) Goal — Exploit the best ideas, algorithms, and

software tools emerging from the national HPC

infrastructure.

• Ensure that DoD S&T and DT&E users remain

cognizant of, interact with, and leverage internal DoD HPC initiatives, such as the Defense

Advanced Research Projects Agency's technology development efforts and the software development initiatives of defense research offices (Air Force Office of Scientific Research,

Army Research Office, and Office of Naval

Research).

• Ensure that DoD S&T and DT&E users remain

cognizant of, collaborate with, and leverage

other government, academia, industry, and

each other's HPC efforts.

• Maintain cooperative contact with the appropriate committees, subcommittees, working

groups, and advisory panels associated with the

National Science and Technology Committee

(NSTC) structure.

• Maintain cooperative contact with the Department of Energy Accelerated Strategic Computing Initiative (ASCI).

• Provide HPC resources to support defense applications that directly relate to dual-use technologies and National Challenges, e.g., environment, medical, digital libraries, and manufacturing processes and products.

• Ensure the availability of HPC resources to

support defense applications that address DoD

Challenge Projects' needs, e.g., new materials;

meteorological, oceanographic, and environmental modeling; energy efficient vehicles and

airplanes; and the structure of biological and

chemical phenomena.

• Leverage the nation's HPC infrastructure to

benefit the warfighter.

Program Initiatives

The HPCMP pursues three initiatives to implement the

program strategies outlined above and achieve its goals.

Work began on each initiative in FY 1994. Initial assessments were provided in FY 1994 and were expanded in

FY 1996 and each year thereafter. Implementation of

program initiatives will continue as long as the DoD

needs to continue broadening and improving the ability

of its S&T and DT&E community to analyze, develop,

and deploy advanced weapons, warfighting capabilities,

and related support systems more rapidly and affordably

and with reduced risks to human life and system performance.

Goals Program Initiative

1,2,3,5

3,4,5

2,3,5

HPC centers

networking

software support
12 HPC Modernization Plan

HIGH PERFORMANCE COMPUTING CENTERS

Major Shared Resource Centers (MSRCs)

To enable the DoD S&T and DT&E community to effectively and efficiently use a full range of high-end HPC

resources, the HPCMP has established four large

MSRCs. Sustainment of resources at the MSRCs, along

with the needed support staff, are provided by a team of

government and contractor personnel. Table 3 lists the

four MSRCs.

Each MSRC includes various types of high-end high

performance computing and communications systems,

scientific visualization capabilities, peripheral and archival storage devices, and support staff providing expertise

in the use of these assets, with an emphasis on supporting

large computational requirements/problems. Further,

each center's hardware configurations, software, programming environments, and training efforts are focused

to best support the needs of a particular set of CTAs designated for that center. Table 4 shows which MSRCs are

tasked to provide emphasis support to which CTAs.

A key element of support at each MSRC is the Programming Environment and Training (PET) effort. This effort,

in addition to extensive training to meet a wide variety of

user needs, also provides system tools and knowledge

about state-of-the-art HPC technology to facilitate the

effective and efficient use of the various HPC systems.

This support is accomplished through collaborative partnerships established by each MSRC with several leading

civilian HPC centers and academic institutions in order to

rapidly capture innovation and draw needed civilian expertise into the DoD. Organized to provide emphasis on

particular CTAs, MSRCs focus on training the DoD user

base, identifying HPC technology opportunities, and

introducing these opportunities into the shared resource

centers' computing environment.

Table 3 — Major Shared Resource Centers

Center Location

Air Force Aeronautical Systems Center (ASC) Wright-Patterson Air Force Base, Ohio

Army Corps of Engineers Waterways Experiment Station (CEWES) Vicksburg, Mississippi

Army Research Laboratory (ARL) Aberdeen Proving Ground, Maryland

Naval Oceanographic Office (NAVO) Stennis Space Center, Mississippi

Table 4 — MSRC Focused Support of CTAs

Computational Technology Area

Major Shared Resource Center

CEWES NAVO ARL ASC

Computational Structural Mechanics X X X

Computational Fluid Dynamics X X X X

Computational Chemistry and Materials Science X X

Computational Electromagnetics and Acoustics X X

Climate/Weather/Ocean Modeling and Simulation X X

Signal/Image Processing X X

Forces Modeling and Simulation/C4I X X

Environmental Quality Modeling and Simulation X X

Computational Electronics and Nanoelectronics X

Integrated Modeling and Test Environments X
PROGRAM OVERVIEW 13

Distributed Centers (DCs)

DCs provide HPC capability to a specified local and remote portion of the HPCMP user community. Modestsized systems are deployed to DCs where there is a significant advantage to having a local HPC system and

where there is a unique potential for advancing DoD applications. DCs leverage HPC expertise or address problems that cannot be readily solved at the MSRCs, such as

real-time, near-real-time, embedded system applications,

and man-in-the-loop and hardware-in-the-loop testing and

evaluation. The centers are linked by high-speed communications to the MSRCs and remote users. Thus, they augment the MSRCs to form the total DoD HPCMP computational capability. Because DCs are typically established to

exploit a unique opportunity, a sunset policy has been

established to "retire" the center after completion of the

anticipated work.

Networking

The Defense Research and Engineering Network (DREN)

is a robust, high-speed network that provides connectivity

among the program's geographically dispersed user sites

and shared resource centers. Approximately 1,000 of the

4,000 HPC users reside at the MSRC sites, while the remaining 75% are at remote locations. The program's

networking initiative is supported by the Defense Information Systems Agency and the Defense Advanced Research Projects Agency. DREN provides gateways to

many existing military and civilian networks, allowing

access to the program's resource centers from contractor

and university facilities.

The networking services of DREN are provided by

means of a virtual wide-area network built on a public

communications grid supplied by American Telephone

and Telegraph (AT&T) as part of the DREN Intersite

Services Contract (Figure 4). Networking capability is

specified as a given level of service at each site, with no

definition of interconnecting pathways. Service levels at

each site can be adjusted as requirements change. Adding

and deleting sites from the network and increasing service levels can be easily accomplished. Because DREN

uses commercial services, the DREN Intersite Services

Contract can access the fastest available networking technology immediately upon entry to the commercial sector.

Growing problem sizes, remote interactive visualization

of computations, distance learning, virtual workshops,

remote training, interactive collaborations, and heterogeneous computing across sites all place increasingly heavy

demands on network bandwidth.

Major Shared

Resource Centers

# Distributed Centers

Remote Sites

Types of Service Delivery

Points (SDP)

• IP

•ATM

• Circuit Emulation

• Gateway

Figure 4 - DREN Intersite Services Contract
14 HPC Modernization Plan

Common HPC Software Support Initiative (CHSSI)

CHSSI is building efficient, scalable, portable software

codes, algorithms, tools, models, and simulations that run

on a variety of HPC platforms and affect a wide number

of S&T and DT&E scientists and engineers. CHSSI is

organized around the 10 CTAs and involves several hundred scientists and engineers working in close collaboration across government, industry, and academia. CHSSI

teams include algorithm developers, applications specialists, computational scientists, computer scientists and

engineers, and end users.

CHSSI is helping prepare the DoD to take advantage of

future computing and communications capabilities by

building software with an emphasis on reusability,

scalability, portability, and maintainability. In addition,

CHSSI is producing a new generation of world-class

scientists and engineers trained in scalable software techniques that will reduce the future costs of doing business

and increase our future defense capabilities. Table 5 lists

the current 43 CHSSI projects and principal investigators

across the CTAs.

Table 5 — Common HPC Software Support Initiative Projects and Principal Investigators

Computational

Technology Area Project Principal

Investigator Location

Computational

Structural

Mechanics

Small Deformation Structural Mechanics

Large Deformation Structural Dynamics

Scalable Algorithms for Shock Physics

Structure-Medium Interaction Model

Gordon Everstine

Raju Namburu

Kent Kimsey

Mark Emery

Naval Surface Warfare Center

Army Waterways Experiment Station

Army Research Laboratory

Naval Research Laboratory

Computational

Fluid Dynamics

FAST3D - Global Virtual-Cell Embedding Gridding

COBALT - Unstructured Gridding

FEFLO - Unstructured Gridding

OVERSET - Chimera Gridding

Zonal Navier Stokes - Block Structured Gridding

Scalable NPARC

Jay Boris

Bill Strang

Bill Sandberg

Bob Meakin

Walter Sturek

Jere Matty

Naval Research Laboratory

Air Force Research Laboratory (Wright)

Naval Research Laboratory

Army - NASA

Army Research Laboratory

Arnold Engineering Development Center

Computational

Chemistry and

Materials Science

Car-Parinello Methods for Solids

Quantum Chemistry

Tight-Binding Molecular Dynamics

Classical Molecular Dynamics

DJ. Singh

Jerry Boatz

D.A. Papaconstantopoulos

Ruth Pachter

Air Force Research Laboratory (Wright)

Air Force Research Laboratory (Edwards)

Naval Research Laboratory

Air Force Research Laboratory (Wright)

Computational

Electromagnetics

and Acoustics

Automatic Target Recognition and Scene Generation

Electromagnetic Interaction Code

Spectral Domain Low Observable Component Design

Synthetic Sensor Phenomenology for Fusion

Time Domain Numeric Methods for HPC Platforms

High Resolution Computational Electromagnetics and

Acoustics Code

Magnetohydrodynamic Code

Jeff Hughes

Jay Rockway

Kueichien Hill

Tom Blalock

Helen Wang

Miguel Visbal

Robert Peterkin

Air Force Research Laboratory (Wright)

Naval SPAWARSYSCEN, San Diego

Air Force Research Laboratory (Wright)

MSIC/Defense Intelligence Agency

Naval Air Weapons Center (China Lake)

Air Force Research Laboratory (Wright)

Air Force Research Laboratory (Phillips)

Climate/Weather/

Ocean Modeling and

Simulation

Ocean Models with Domain Decomposition

Scalable Global Weather Forecast System

Global and Regional Wind Wave Modeling

Steve Piacsek

Tom Rosmond

Robert Jensen

Naval Research Laboratory

Naval Research Laboratory

Army Waterways Experiment Station

Signal/Image

Processing

Radar Signal Processing

Scalable Algorithms for Sonar Beamforming

Synthetic Aperture Radar/Image Formation

HPC Embedded Applications for Target Recognition

Infrared/Optical Image Processing for Reconnaissance

and Surveillance

Rich Linderman

Bob Bernecky

Chris Yerkes

Cathy Deardorf

Lynda Graceffo

Air Force Research Laboratory (Rome)

Naval Undersea Weapons Center

Naval SPAWARSYSCEN, San Diego

Air Force Research Laboratory (Wright)

Army Night Vision Laboratory

Forces Modeling

and Simulation/C4I

Scalable HPC Environment for C4I (SHEC)

Simulation Leveraged Acquisition Test & Evaluation

Efficient Parallel Discrete Event Simulation for Analysis

HPC Frameworks for Warming and Training Simulations

Guy Leonard

Henry Ng

Bill Smith

Larry Peterson

Naval SPAWARSYSCEN, San Diego

Naval Research Laboratory

Naval Research Laboratory

Naval SPAWARSYSCEN, San Diego

Environmental

Quality Modeling

and Simulation

Structured-Unstructured Modeling

Resolved Transport Algorithm

Scalable Parallel Implementation of DoD Groundwater

Modeling System

Robert Bernard

Mark Dortch

Fred Tracy

Army Waterways Experiment Station

Army Waterways Experiment Station

Army Waterways Experiment Station

Computational

Electronics and

Nanoelectronics

Scale Algorithms for Dynamic Nonlinear Simulations

Power Electronics Simulation

Parallel Very High Speed Integrated Circuit Hardware

Description Language Simulation

Electromagnetic Solvers for High Frequency Design

Dave Rhodes

Robert Pastore

John Hines

Leo DiDominico

Army Comm and Electronics RDE Center

Army Intelligent Electronic Warfare Center

Air Force Research Laboratory (Wright)

Army Research Laboratory

Integrated

Modeling and Test

Environments

Real Time Synthetic Test Environment Representation

High Fidelity Physics Based Models for Testing

Simulation Based Design and Test Technology

Harry Heckathorn

Joel Mozer

Andrew Mark

Naval Research Laboratory

Air Force Research Laboratory (Phillips)

Army Research Laboratory
PROGRAM ACCOMPLISHMENTS AND FUTURE PLANS

Major Shared Resource Centers

Concurrent acquisitions to establish the four MSRCs

were completed in FY 1996, resulting in integration contracts to provide hardware, software, maintenance, operations, and support staffs. Each contract includes a base

computational capability, Performance Level (PL) I, with

two options for upgrade (PL II and PL III) as resources

become available.

Table 6 shows the HPC systems installed and planned to

be available to users effective June 1998. Since the first

systems were fielded by the program in FY 1994, the

total peak capability at the MSRCs has increased from 30

gigaflops to just over 1200 gigaflops at the end of FY

1997. Approximately 600 gigaflops have been added this

year as part of the PL II upgrade. Further upgrades to the

Army Research Laboratory MSRC are anticipated as the

final increment of the PL II upgrade is completed. During

this same period — FY 1994 to FY 1998 — utilization,

both in terms of the number of users and the size and

importance of the projects, has continued to rise sharply.

MSRCs FY1998

The program will sustain existing HPC capability at its

four MSRCs and continue to modernize these sites by

acquiring additional HPC systems, capacity, storage, and

scientific visualization capabilities. Contract options will

be exercised to upgrade capability, completing the second

year of PL II. The program will also focus on developing

a common "look and feel" for users across the centers.

This includes login accounts, file structure, web design

and user interfaces. Also, the Program Environment and

Table 6 — Major Shared Resource Center HPC Capability (as ofJune 1998)

Number of Total Memory Total Capability

Location DoD HPCMP System Processors (gigabytes) (Peak gigaflops)

Army Corps of Engineers Cray C90 16 8 16

Waterways Experiment Station IBMSP 256 256 138

Cray T3E 336 86 302

IBM SP P2SC 126 63 81

SGI Origin 2000 128 64 51

Naval Oceanographic Office Cray C90 16 8 16

SGI Origin 2000 128 64 51

Cray T90 24 8 43

SGI PowerChallenge 52 16 19

Cray J932SE (Classified) 12 4 2

SGI PowerChallenge (Classified) 8 4 3

Cray T3E 544 139 490

Army Research Laboratory SGI Origin 2000 288 160 115

Cray T90 12 8 22

SGI Origin 2000 (Secret) 64 32 26

Cray T90 (Secret) 8 4 14

Cray J932SE (Special Access) 16 32 3

Cray J932SE (Special Access) 16 8 3

SGI Origin 2000 (Special Access) 32 12 13

Aeronautical Systems Center Cray C90 16 8 16

IBMSP 256 264 138

SGI Origin 2000 512 320 205

SGI PowerChallenge 16 8 6

15
16 HPC Modernization Plan

Training effort will continue to provide HPC training and

technology transfer to the various DoD HPC users and

will incorporate and integrate common programming

tools and standards across the DoD HPC community.

MSRCs FY 1999

The program will sustain existing HPC capability and

continue to modernize by acquiring additional HPC systems, capacity, storage, and scientific visualization capabilities. Over the two-year period — FY 1999 and FY

2000 — MSRC contract options will be exercised to

upgrade performance at these centers, minimally resulting in a ten-fold increase over capability at each site in

FY 1996. Improved operating environments and supporting tools and software will be firmly in place and tested

across the CTAs. Additional training and interactions

with outside government agencies, industry, and academic HPC centers will continue as the nation's HPC

expertise, capabilities, standards, and infrastructure mature.

Distributed Centers

Augmenting the shared computational capabilities at the

four MSRCs, the HPCMP established or upgraded DCs

over the three-year period FY 1996 through FY 1998

(Table 7). In FY 1997, five existing centers were upgraded and one new center was created at the Redstone

Technical Test Center. In addition to the DCs listed in the

table, the Arctic Region Supercomputing Center, in

Fairbanks, Alaska, was funded by Congress in FY 1996

through 1998 and is providing computational HPC resources to the user community.

DCs FY 1998

The program approved the acquisition of new systems or

upgrades at five DCs as part of its FY 1998 implementation plan. Table 8 shows the HPC configurations at the

DCs funded by the HPCMP as of March 1998. Additionally, distributed center proposals from DoD Services and

Agencies have been solicited and will be evaluated late

this year for FY 1999 award and implementation.

DCs FY 1999

The HPCMP will announce selection of the FY 1999 DC

projects and will authorize acquisitions of the new or

upgraded capabilities at the selected centers. The program will also solicit DC proposals for FY 2000 award

and implementation.

Table 7 — Distributed Centers and Capability

Center Location Most Recent

Upgrade (FY)

Air Force Arnold Engineering Development Center (AEDC) Arnold Air Force Base, Tennessee 1997

Air Force Development Test Center (AFDTC) Eglin Air Force Base, Florida 1996

Air Force Maui High Performance Computing Center (MHPCC) Kihei, Hawaii 1998

Air Force Research Laboratory (Rome) Griffiss Technology Park, Rome, New York 1998

Army High Performance Computing Research Center (AHPCRC) Univ. of Minnesota, Minneapolis, Minnesota 1996

Army Redstone Technical Test Center (RTTC) Redstone Arsenal, Huntsville, Alabama 1997

Army Space and Missile Defense Command (SMDC) Huntsville, Alabama 1996

Army Tank-Automotive RD&E Center (TARDEC) Warren, Michigan 1996

Army White Sands Missile Range (WSMR) White Sands Missile Range, New Mexico 1997

Naval Air Warfare Center (NAWC) Patuxent River Naval Air Station, Maryland 1997

Naval Research Laboratory (NRL) Washington, District of Columbia 1998

Naval Undersea Warfare Center (NUWC) Newport, Rhode Island 1996

Naval SPAWAR Systems Center, San Diego San Diego, California 1998
PROGRAM ACCOMPLISHMENTS AND FUTURE PLANS 17

Table 8 — Distributed Center HPCMP Resources (as of March 1998)

Location DoD HPCMP System Processors Total Memory

(gigabytes)

Total Capability

(Peak gigaflops)

Air Force Arnold Convex C4640 4 4 3

Engineering Development Convex C4640 (Classified) 4 4 3

Center (AEDC) Convex Exemplar SPP-2000 64 16 46

HP Exemplar X (Classified) 48 11 34

SGI Origin 2000 64 32 26

Air Force Development Cray T3D 128 8 19

Test Center (AFDTC) SGI Onyx 32 8 12

Air Force Maui High IBM SP (Classified) 96 13 26

Performance Center (MHPCC) IBMSP 192 31 51

IBM P2SC 51 7 25

IBM SMP 64 8 14

1MB P2SC 192 96 123

Air Force Research Intel Paragon 321 21 30

Laboratory (Rome)*

Army High Performance Cray T3E 272 139 326

Computing Research Center

(AHPCRC)

Army Redstone Technical SGI Origin 2000 32 7 13

Test Center (RTTC) SGI Origin 2000 (Classified) 32 7 13

Army Space and Missile SGI Origin 2000 64 16 26

Defense Command (SMDC) SGI Origin 2000 32 10 13

SGI Origin 2000 32 10 13

SGI Origin 2000 32 10 13

SGI Origin 2000 128 32 52

Army Tank-Automotive SGI PowerChallenge 64 16 24

RD&E Center (TARDEC)

Army White Sands Missile TMC CM-500 128 16 20

Range (WSMR)* SGI Origin 2000 64 24 26

Naval Air Warfare Center SGI PCA (Classified) 40 12 15

(NAWC) SGI Onyx2 (Classified) 32 16 13

SGI Onyx2 (Classified) 32 8 13

Naval Research Laboratory TMC CM-500e 256 33 41

(NRL)* TMC CM-500e 32 4 5

Convex Exemplar SPP-2000 64 16 46

SGI Origin 2000 128 32 52

Naval Undersea Warfare Cray T3D 64 4 10

Center (NUWC)

Naval SPAWAR System Intel Paragon (Classified) 336 14 25

Center, San Diego (SSCSD)* Convex Exemplar SPP-1000 32 8 6

*Distributed centers funded for FY 1998 additional capability are not yet reflected in the table due to pending acquisition.
18 HPC Modernization Plan

Networking

The Defense Research and Engineering Network

(DREN) Intersite Service Contract (DISC) was awarded

to AT&T in July 1996. The contract allows the government to purchase high-speed network service to anywhere in the United States at bandwidths ranging from

1.5 megabits per second to 155 megabits per second

(OC3), with upgrade potential to 2.4 gigabits per second

(OC48) over the five-year life of the contract. Thirteen

Interim DREN sites were transitioned to DISC in Fiscal

Year 1997.

Networking FY 1998

The Interim DREN network, established when the program began by consolidating pre-existing Air Force,

Army, and Navy networks, will be phased out entirely as

all network services are shifted over to the DISC. An

additional 47 government sites will be implemented in

FY 1998, bringing the total to 60. Options to increase

bandwidth at selected sites will also be executed. Widearea classified networking will enable support to classified processing requirements over DREN using highspeed Fastlane encryption technology.

Networking FY 1999

Network connectivity will be sustained for HPCMP users. Options to increase bandwidth at selected sites will

be executed to meet increased demands resulting from

upgraded computing capabilities at the various HPC centers. Additional sites will also be added to the DREN.

Planning will begin to develop a follow-on DREN contract or transition follow-on high-speed support to standard Defense Information Systems Agency (DISA) network operations.

Common HPC Software Support Initiative (CHSSI)

The CHSSI Test and Evaluation Master Plan (TEMP)

was approved in FY 1997, and 43 software projects were

initiated to develop advanced scalable software codes,

algorithms, tools, models, and simulations. Several

CHSSI teams released alpha and beta software products

in FY 1997, and preliminary evaluations of these products show a significant improvement in capability to

efficiently exploit HPC scalable systems in support of a

large number of DoD user projects.

CHSSI FY 1998

CHSSI will continue to develop reusable, scalable, and

portable applications software to exploit massively parallel HPC systems. The focus of the year will be on software testing, and additional CHSSI projects will release

alpha and beta software throughout the year. Each CHSSI

project will be reviewed to assess the application of HPC

technology to solve the defense problems being addressed. Projects no longer deemed appropriate or high

priority, due to changing technology and/or requirements,

will be discontinued. As projects succeed to full software

release or are discontinued due to lack of progress or

applicability, other areas of opportunity will be sought

and new projects will begin.

Cross-collaboration with related technology efforts

throughout the DoD and other federal agencies, industry,

and academia will continue to strengthen. Emphasis will

be on developing standard computing interfaces, user

environments, visualization tools, and other capabilities

that assist nontechnical users working on critical defense

and national security problems to more easily exploit

HPC capabilities.

CHSSI FY 1999

CHSSI software will continue to be developed, tested,

and released to produce improvements in defense operations. Advances in high-speed computing and communications technology, especially in distributed computing,

metacomputing, massively parallel HPC architectures,

and remote desktop collaboration, will continue to

change the way the DoD performs its warfighting and

warfighting support missions. These changes will, in

turn, produce new areas of opportunity.

Emphasis on strong software engineering, reusability,

and maintainability will be expanded. Software and programming tools such as advanced compilers, debuggers,

and system evaluation tools from inter-Agency efforts

will be transferred and inserted into the DoD HPC community. These tools will greatly improve the accessibility

and usability of HPC resources to a broader user base.

PROGRAM OFFICE ACCOMPLISHMENTS

AND FUTURE PLANS

DoD Challenge Projects

To ensure that important computationally intensive problems receive priority resources, a set of DoD Challenge

Project proposals are requested, reviewed, and awarded

each year. These are then allocated approximately 20% of

the program's total HPC capability. Projects are chosen

annually, and recommendations are made by a selection

board that includes reviewers from outside the DoD.

Selection is based on priority military needs, scientific

merit, and potential for progress. The FY 1998 DoD

Challenge Projects are summarized in Table 9.
PROGRAM ACCOMPLISHMENTS AND FUTURE PLANS 19

Table 9 — Fiscal Year 1998 DoD Challenge Projects

Primary

CTA

Project Organization HPC

System(s)

CSM Development of Standards for Stand-Off Distance and

Blast Walls for Force Protection

Army Waterways Experiment Station IBMSP

Cray T3E

CSM Mine Plow Simulation by Smoothed Discrete

Element Modeling

Army Waterways Experiment Station Cray T3E

IBMSP

CSM Multistory Building Structural Response from Water

Tamped Explosions

Army Waterways Experiment Station IBMSP

Cray T90

Cray T3E

CSM Modeling of Complex Projectile-Target Interactions Army Research Laboratory Cray T90

Cray J90

Cray Origin 2000

CFD Analysis of Jet Interaction Phenomena for the

THAAD Interceptor

Army Space Missile Defense Command Cray T90

Cray Origin 2000

CFD Applied CFD in Support of Aircraft-Store Compatibility and

Weapons Integration

Air Force Development Test Center Cray Origin 2000

Cray T3E

CFD Automatic Aerodynamic Design for Complete Aircraft

Configurations Using an Adjoint Based Multiblock Method

Air Force Office of Scientific Research Cray Origin 2000

Cray T3E

CFD Parallel Simulations of Flow-Structure Interactions Office of Naval Research IBMSP

Cray T3E

CFD Parallel Simulations of Reacting Turbulent

Two-Phase Flows

Army Research Office Cray Origin 2000

Cray T3E

CFD Simulation of Explosions for Counter-Proliferation

and Counter-Terrorism Scenarios

Defense Special Weapons Agency Cray Origin 2000

Cray T3E

CFD Time-Domain Computational Ship Hydrodynamics Office of Naval Research TMC CM-5

Cray T90

IBMSP

Cray Origin 2000

Cray T3E

CFD Unsteady Hydrodynamics of the Maneuvering Submarine Office of Naval Research Origin 2000

Cray T3E

CCM Dynamical Response of Low Dimensional Materials Naval Research Laboratory TMC CM-5

CCM New Materials Design Air Force Phillips Laboratory

Air Force Office of Scientific Research

Cray T90

Cray Origin 2000

IBMSP

CCM Theoretical Investigation of Gun Tube Erosion

Related Requirement

Army Research Laboratory SGI PCA

Cray Origin 2000

Cray T3E

CEA Airborne Laser Laboratory Challenge Project Air Force Research Laboratory Cray T3E

IBMSP

CEA B-1B Radar Cross-Section Prediction Air Force Research Laboratory IBMSP

Cray Origin 2000

CEA Computational Assisted Development of High

Temperature Structural Materials

Air Force Research Laboratory

Air Force Office of Scientific Research

IBMSP

Cray Origin 2000

Cray T3E

CEA Virtual Prototyping of RF Weapons Air Force Research Laboratory Cray T90

IBMSP

CWO Global and Basin-Scale Ocean Modeling and Prediction Naval Research Laboratory Cray T3E

EQM Quantification of the Impacts of Subsurface Heterogeneity

on Military Site Cleanup

Army Waterways Experiment Station IBMSP

TMC CM-5

Cray T3E

CEN Atomistic Simulation of MEMS Devices via the Coupling

of Length Scales

Naval Research Laboratory IBMSP

CEN Quantum Simulation Defense Advanced Research Projects Agency Cray T3E
20 HPC Modernization Plan

Challenge Projects produce, in a timely fashion, significantly improved models and simulations to address important defense problems. Such problems include determination of blast damage on structures for counter proliferation and counter-terrorism scenarios, disposal of expired munitions, mine warfare in littoral waters, and finer

calculations of stealth signatures to improve invisibility

across the electromagnetic spectrum. Summaries of several FY 1997 DoD Challenge Projects are included in the

1998 edition of High Performance Computing Contributions to DoD Mission Success.

Customer Requirements and Resource Allocation —

Processes for gathering user requirements, upon which

the program is based, were expanded in FY 1997 to include on-line input via the World Wide Web. Further, the

HPC Modernization Office established a resource allocation policy to more effectively distribute the program's

HPC resources and ensure timely results for DoD's highest priority computational projects. This new policy is

being implemented at the MSRCs in FY 1998 and includes two parts: allocation of a specified portion of the

program's total computing resources (20%) to support

DoD Challenge Projects and allocation of the remaining

compute cycles (80%) to support Service/Agency requirements. Of these remaining cycles, 30% are allocated

to each of the Air Force, Army, and Navy, with 10% for

Defense Agencies. Each Service/Agency then distributes

its allocation according to its internal needs.

Security — Although the HPCMP retains overall security policy and oversight responsibilities, authority for

security certification and accreditation of individual systems has been delegated to the host Services. In response

to growing threats posed by the proliferation of computer

system intrusion software widely available commercially

and over the Internet, the HPCMP Security Working

Group (SWG) met during FY 1997 to develop ways to

strengthen existing safeguards to prevent unauthorized

access to DoD HPC resources. Over the next year,

SecurelD cards that provide one-time passwords will

be distributed to the entire DoD HPC user base and

Kerberos authentication protocols will be implemented

to enhance our security posture.

User Group Meetings — Annual user group meetings

provide an important forum for interaction between users, program office personnel, shared resource center

managers, and others involved in the program. The June

1997 meeting had the largest attendance to date, with

more than 300 participants. This event was sponsored by

the Naval Oceanographic Office MSRC and hosted by

the San Diego Supercomputer Center in San Diego, California. The 1998 meeting will be hosted by the Army

Waterways Experiment Station MSRC and Rice University in Houston, Texas. Beginning in 1998, the Shared

Resource Center Advisory Panel — the program's user

group leadership — has been designated as the "sponsor"

of the annual user group meeting.

PROGRAM OVERSIGHT AND REVIEW

In FY 1995, the General Services Administration (GSA)

issued a $1.1 billion Delegation of Procurement Authority (DPA) for the four MSRC contract awards. In FY

1996, GSA granted a $430 million DPA for the DREN

Intersite Services Contract. The HPCMP is designated as

a major automated information system program. As such,

the Major Automated Information Systems Review

Council (MAISRC) provides oversight to ensure that the

program meets its required life-cycle management milestones. In April 1994, the Program Office received Milestone 0 approval to initiate the HPCMP, and Milestone I

was granted in February 1995. The MAISRC conducted a

formal Milestone II review in December 1995 and issued

a favorable System Decision Memorandum to continue

with acquisition and modernization on 4 January 1996.

The Milestone 2 decision was key to the substantial

progress the program has made in the past several years.

Milestone 2 approval granted the program authority to

effect full and open centralized acquisitions for high performance computing system upgrades to the four MSRCs

and contracted for the program's wide area network services with the DREN Intersite Services Contract. The

approval further authorized high performance computing

system upgrades at six DCs and the start of CHSSI

projects. Subsequent MAISRC approvals in February and

October 1997 have allowed the program to continue

those software support projects and to provide upgrades

for additional DCs.

The HPCMP has matured sufficiently so that 1998 will

mark the first extensive testing cycle for the program.

The Joint Interoperability Test Command (JITC), the

DoD independent tester, is testing various facets of the

program: two MSRCs, two DCs, and several software

support projects. Preliminary evaluation of test results

shows that the program is on track — moving ever closer

to meeting the needs of our scientists and engineers.

BUDGET

Overall Program Budget

The DoD HPCMP is projected to spend $1,237 million

for fiscal years 1998 through 2003. Table 10 shows the

funding profile by year and major spending category for

fiscal years 1996 through 2003, including actual budgets
PROGRAM ACCOMPLISHMENTS AND FUTURE PLANS 21

Table 10 — Funding Profile ($M)

FY96 FY97 FY98 FY99 FY00 FY01 FY02 FY03

System Acquisitions 128.6 122.7 89.7 82.4 65.0 42.1 52.9 51.9

Networking 17.8 16.0 23.7 28.7 29.9 33.4 33.4 33.4

System Application Support 9.6 21.0 21.2 22.8 22.8 22.8 20.8 20.8

Sustainment and Operations 56.4 82.1 98.2 89.5 86.8 90.0 85.3 89.9

Total 212.3 241.7 232.8 223.3 204.5 188.3 192.4 196.0

for fiscal years 1996 through 1998 and five-year projections from the President's FY 1999 budget. This funding

profile represents a reduction of about $590 million from

the initial budget profile shown in the FY 1994 modernization plan, and a reduction of $88 million from the

profile presented in the FY 1997 plan. All of these reductions have been directed as a result of fiscal constraints

within the Department of Defense. Although there have

been a number of budget adjustments since the 1994

budget profile was presented, most of the reductions for

fiscal years 2000 through 2003 are the result of a single

$269 million reduction sustained by the program as part

of the final resolution of the Defense Program Objective

Memorandum (POM) for fiscal years 1997 through 2001.

Comparison of the current budget profiles with earlier

budget projections, industry briefings, and acquisition

documents indicates that over 75% of the cumulative

reductions referenced above — an average of almost

$100 million per year beginning in fiscal year 2000 —

will be sustained by the program's acquisition account as

a direct result of the $269 million POM reduction. As

stated in the FY 1997 modernization plan, the lower budget is sufficient to exercise the base capability as currently programmed for Performance Level III of the

MSRC contracts. Given the rapidly increasing user utilization of the resource installed through Performance Level II,

however, a majority of the projected user requirements will

not be addressed by even this capability. Therefore, as part

of its strategic planning efforts, the HPCMP staff continues

to examine options to assure that the highest priority warfighter needs are met in light of these shortfalls. Other significant results of the out-year budget reductions include

scaling back the growth rate for network capability and

phasing out of most funding for DCs beyond fiscal year

2000. Both of these reductions will have the greatest impact on new or just emerging applications requiring either

real-time or high-end interactive applications located remote to the MSRCs.

The continued strong funding in the software support initiative and the sustainment and operations budget lines reflects the importance that the program places on software,

integration, and related shared services to the DoD's HPC

user community. These investments will continue to guarantee that the focus of the program remains on enabling

users to address warfighter needs, not only by strengthening the support infrastructure, but also by providing effective mechanisms for continued leverage of both national

and industrial investments in HPC-based productivity.