a +++::+ +_++- _//_".I +.+::+

__+_+., X_++_+,++,';.++ _;{i___+'., _, _" _++ Ass°cillii° n 1°_llg'_'+eme[_In Snpri islver nega,tAiv°e+u 3011587..B202 ae?s20C9lu+tT_g00 M10aryland +a n _._._]_+ _ "'++_;++<;++" :_+++_..+. ___ _

+

+

"+' NIMNUFIMCTUREDTO I:IITN STI:INDIMRDS _ ++_:++_+++ +

+Y0++,++0 +.o++, +_+. ++ _?++++

+ +

r ,, GCW-0593

_o;0 F- q3oF3gf--

Workshop and Conference on

Grand Challenges Applications a_d

Software Technology

May 4-7, 1993-.

Pittsburgh, Pennsylvania
•f o° • ",,: ?.

Sponsoring Agencies

Advanced Research Projects Agency (ARPA)

Department of Energy (DOE)

Environmental Protection Agency (EPA)

National Aeronautics and Space Administration (NASA)

NationalScienceFoundation(NSF)

NationalInstituteosfHealth (NIII)

NationalInstitutoef Standardsand Technology (NIST)

NationalOceanicand Atmospheric Administration(NOAA)

NationalSecurityAgency (NSA)

Committees

Organizing Commit tee

Paul EI.Smith, Chair, NASA

Stephen M. Griffin, Executive Secretary, NSF

Norman Glick, NSA

Fred Johnson, NIST

Tom Kitchens, DOE

Jacob Maizel, NIH

Tom Pyke, NOAA

Joan Novak, EPA

Gil Weigand, ARPA

Robert Voigt, NSF

Program Commit tee

Rick Stevens, Program C'.hair, At'go)_tm National Laboratory

Fran Berman, University of (?aliforuia- San Diego

AI Brenner, Supercomput.ing ltoso.arch Center

Phillip Colella, University of Califoraia- Berkeley

Kevin Drogenmeir, Universit,y ofOklahoma

lan Foster, Argonne National Laborettory

Geoffrey Fox, Natioaal Parallel Arcllit.ec.ture Center, Syracuse

Mike Heath, University of Illinois and NCSA

Ken Kennedy, Rice Uuiw:r._ity

Tom Kitchens, DOE

Robert Martino, NIH

Paul Messina, Califoraia Instit.ute of Techuology

Reagaa Moore, Saa Diego Supcrcoluputer Ceater

Dan Reed, University of Illinois

Ralph Roskies, Pit,tsburgh Sttl_ercolnputiag Center

Manay Sale.s, NASA Lal_glcy fresh,at'oh Center

.Joel Saltz, Uuiversity of 3[arylatld

Ahmed Sameh, Ceater for Supcrcompttt, iug Research and Development

Frail Sullivaa, SupercompuLil}g Research Center

Andy White, Los Alamos Natioual l,ahoratory

Paul Woodward, University of _[iuucsota

Workshop on Grand Challetlge ApplicaLiolis alld Software Technology
GCW-0593

Workshop and Conference on

Grand Challenges Applications and

Software Technology

May 4-7, 1993

Pittsburgh, Pennsylvania
Q

Foreword

The need for multidisciplinary and interdisciplinary collaboration to solve large-scale computational

problems has been recognized since the beginning of the Federal HPCC Program. With this realization

came the awareness that necessary levels of interaction have not occurred in the normal course of events

in the past, and that it would be beneficial for the participating HPCC agencies to foster processes and

to provide occasions specifically for the purpose of stimulating cross-disciplinary communication and

interaction.

The motivation for convening the Grand Challenge Applications and Software Technology Workshop and

Conference came out of the continuing belief that collaboration, shared goals, and knowledge exchange

between applications scientists and software technologists are critical to meet the overall goals of the

HPCC Program. The gathering was designed to bring the HPCC agencies' grand challenge research

teams and leading software technologists from the academic and private sectors together with a few

of the key industrial leaders, and ask them to focus their expertise on identifying high-performance

computing needs and, more importantly, to anticipate those of the future.

Speaking for the Organizing Committee, we believe the workshop and conference was a valuable occasion

for all concerned and hope to see a continuation of such events in the future.

Paul H. Smith, NASA

Organizing Committee Chail;

Stephen M. Griffin, NSF

Organizing Committee Executive Secretary

ii Workshop on Grand Challenge Applications and Software Technology
il i ii ii ,i utin muni ........ __j - il i HI .... i

r

Contents

I Executive Summary 1

Program .................................................. 4

ApplicationSessions ........................................... 6

Working Groups ............................................. i0

2 Background and Objectives 13

3 Grand Challenge Applications Panel Sessions 15

PanelA - Environment and Earth Sciences .............................. 15

PanelB - Computational Physics.................................... 17

PanelC - Comp. Biology,Chemistry,MaterialsSciences....................... 18

PanelD - Computational Fluidand Plasma Dynamics ........................ 19

PanelE - ApplicationsofArtificiaIntl elligence............................ 21

4 Grand Challenges and Industrial Applications 23

ChemicalApplications:DuPont Corporation ............................. 23

Automotiveand ManufacturingApplications:Ford ......................... 24

FinancialApplications:PrudentialSecurities............................. 24

AeronauticsApplications:RockwellInternationa........................... l 24

Energy Resource ExtractionApplications:Western Atlas ...................... 24

TextileApplications:DuPont Fibers .................................. 25

HealthCare Applications:PhillipMsedicalSystems ......................... 25

5 Working Groups 27

Group A - I/O, Data Systems,and FileSystems ........................... 27

Status ............................................... 27

Targetsof Opportunity ..................................... 28

Relationships........................................... 29

Findings.............................................. 30

Group B - Parallel Programming Paradigms ............................. 31

Status .............................................. 31

Targets of Opportunity ..................................... 33

Relationships ........................................... 34

Impacts .............................................. 35

Findings .............................................. 35

Group C - Performance Characterizatiou and Ev_ltl_,ioll ...................... 36

',V;rkshop oa Grand Challenge Applications and Software Tcchl_ology iii
_

. iii i i ' =" ' ' ................. i ii | |1 i i i

Status .............................................. 36

Targets of Opportunity ..................................... 37

Relationships ........................................... 39

Impacts .............................................. 40

Findings .............................................. 41

Group D - Program Development Tools ................................ 41

Status ............................................... 42

Targets of Opportunity ..................................... 42

Relationships ........................................... 43

Impacts .............................................. 44

Findings .............................................. 44

Group E - Building Multidisciplinary Applications .......................... 46

Status ............................................... 46

Targets of Opportunity ..................................... 47

Relationships ........................................... 49

Impacts .............................................. 50

Findings .............................................. 50

Group F - Algorithms and Libraries I ................................. 51

Status ............................................... 51

Targets of Opportunity ..................................... 53

Relationships ........................................... 53

Impacts .............................................. 54

Findings .............................................. 54

Group G - Algorithms and Libraries II ................................. 55

Status ............................................... 55

Targets of Opportunity ..................................... 55

Relationships ........................................... 56

Impacts .............................................. 57

Findings .............................................. 58

Group H - Graphics and Visualization ................................. 59

Status ............................................... 59

Targets of Opportunity ..................................... 60

Relationships ........................................... 61

Impacts .............................................. 61

Findings .............................................. 62

Group I - National HPCC Infrastructure ............................... 63

Status .............................................. 63

Targets of Opportunity ..................................... 64

Findings .............................................. 67

6 Conclusions 69

7 Findings 71

A Grand Challenge Requirements Document 75

B Summary of Survey Responses 81

C Abstracts of Talks 85

D Workshop Invitees 109

iv Workshop on Grand ChallengeApplicationsand SoftwareTechnology
IN inn Ilill I illi I I I i I I J • --

SECTION Executive Summary

On May 4-7, 1993, nine federal agencies sponsored a four-day meeting on Grand

Challenge applications and software technology.

Over two hundred and fifty researchers from universities and national laboratories attended the meeting. The objective was to bring High-Performance Computing

and Communications (HPCC) Grand Challenge applications research groups supported under the federal HPCC program together with HPCC software technologists

to

• discuss multidisciplinary computational science research issues and approaches,

• identify major technology challenges facing users and providers, and

• refine software technology requirements for Grand Challenge applications research.

The first day and a half focused on applications. Presentations were given by

speakers from universities, national laboratories, and government agencies actively

The strategy is involved in Grand Challenge research. Five areas of research were covered: environto bT_ng together mental and earth sciences; computatiorJal physics; computational biology, chemistry,

applications and materials sciences; computational fluid and plasma dynamics; and applications

researchers of artificial intelligence.

The next day and a half was spent in working groups in which the applicaand software tions researchers were joined by software technologists. Nine breakout sessions

technologies took place: (1) I/O, Data, and File Systems; (2) Parallel Programming Paradigms;

to discuss (3) Performance Characterization and Evaluation of Massively Parallel Processhigh-performance ing (MPP) Applications; (4) Program Development Tools; (5) Building Multidiscicomputing needs, plinary Applications; (6) Algorithm and Libraries I; (7) Algorithms and Libraries

II; (8) Graphics and Visualization; and (9) National HPCC Infrastructure.

Workshop on Grand Challenge Applications and Software Technology 1
In I ill i -- -- i i ii i i i ii ill

On the final day, a representative from each of the applications areas presented a

ten-minute summary of the common themes, approaches, and obstacles in the specific Grand Challenge discipline. These were followed by thirty-minute presentations

reporting the findings of the nine breakout sessions and discussing significant new

opportunities for high-performance computing and communications research. In

addition, representatives from several U.S. industries reported on Grand Challenge

efforts under way in such areas e._ global finance and chemical production.

The researchers in the Grand ChalLenge community and in the Software Development community agreed that the federal ItPCC program is already having an

High-performance undeniable impact. Indeed, one might almost speak of a high-performance revolu.

computers tzon, with widespread interest in and use of scalable parallel systems by industry

are helping to and academia; increased focus ou parallelism by workstation vendors; growing emsolve previously phasis on the need for standards; and almost universal concern for developing better

products (machines, languages) to make parallel computing effective.

intractable And while the researchers agreed that it is probably too early to expect HPCC

problems, program activities to have had a major impact on science and industry, exciting

progress is being made. In particular, high-performance computing is enabling researchers to attack previously intractable problems, such as the recent 5123 simulation of compressible turbulence.

This increasing capability to handle complex problems .m.s already attracted

the attention of industry. Industry involvement was noted in numerous Grand

Challenge projects, including aerospace, environmental science, biophysics, computational fluid and plasma dynamics, computational chemistry, reservoir modeling,

and global computational finance.

Nevertheless, several outstanding issues were identified,

Programming • Software tools. A clear need exists for better parallel debugging tools,

tools are essential tools for multidisciplinary applications, performance-monitoring tools, and

for dramatically language support to allow users to write programs at a higher level than

increasing currently possible. The cause for the poor software support is the fact that

the Grand Challenge grants currently focus on the output of the applications

t scientific rather than oa the software to achieve that output. More effective mechanisms _ production on are needed for exchanging information oa tool availability and accessibility.

t l parallel computers. • Database management. The Grand Challenge studies are generating enormous databases (up to 8 terabytes, with predictions in the petabyte range).

Making this data useful will require not only high-density, high-throughput

storage devices but also improved aL'chive and directory services. Even further, specialized visualization systems will be essential for examining the vast

datasets expected from Grand Challenge research.

• Data rates. I/O is simply not, scaling fast enough to keep up with CPU speed

and is threatening to become a major bottleneck for teraflops applications.

Further research is needed t.o uuderstaad the I/O characteristics of massively

parallel computers. Parallel I/O her_chmarks are also needed that accurately

reflect applications to requiremeuts.

• Standards. Current effort.s such as the evolving Message Passing Interface

standard appear promising, but further work is needed, for example, to promote vendor standards for aL'chiving Large amounts of data.

2 Workshop on Grand Challenge Applicatious and Software Technology
!

, = . • I _ u|.|| ii | i., i , , i ii iii1_ i |

Execulive Summary

To address these problems, the workshop participants formulated severedspecific

Researchers suggestions:

have identified , Support a "Grand Challenge in Software Tools" and establish a Science and

priorities for Technology Center for HPCC Software Tools, to encourage collaborative deaccomplishing velopment of tools for Grand Challenge applications.

HPCC goals.

• Establish or encourage "software capitalization" programs to foster the development of research ideas and prototypes into production tools and to ensure

the maintenance of the software in the library.

• Promote the development of research tools into usable software aids (through

an existing organization such as OSF or a new organization or industrial

consortium).

• Develop a parallel software tools electronic journal/newsletter (complete with

an editorial board) to publicize tool availability and tool needs.

• Require computer vendors (through explicitly worded RFPs) to provide parallel debuggers, profilers, and compilers for MPP machines.

• Have HPCC projects explicitly promote the exchange of students and staff

between application and tool developers, to encourage computer scientists

and computational scientists to interact more closely on Grand Challenge

projects.

• Increase the current federal involvement, specifically by providing (1) stable

base-level support for ongoing projects and (2) stable long-term support for

new initiatives such as multidisciplinary programs and scalable I/O.

Workshop on Grand Challenge Applications and Software Technology 3
-- I i Illil I OI I I IIIIli I III III I III II |!' - '" " '' ' " "" m , ,i i | illl I I I I I I I II III II II I

Program

Program

Daily Schedule

Tuesday,May 4

8:00- 9:00 Welcome

9:00- 10:15 ApplicationSession

Panel A, Environmentaland Earth Sciences(3 presentations)

10:15-10:45 Break

10:45- 12:25 ApplicationSession

Panel A continued(4presentations)

12:25-1:30 Lunch

1:30- 3:35 ApplicationSession

PanelA continued(1 presentation)

Panel B, ComputationalPhysics(4 presentations)

3:35-4:00 Break

4:00- 6:05 ApplicationSession

Panel B continued(3presentations)

PanelC, ComputationalBiology,Chemistry and MaterialSciences

(3 presentations)

7:00 --8:00 Reception

8:00 Workshop Dinner

Wednesday, May 5

8:00- 10:05 ApplicationSession

PanelC continued(5presentations)

10:05- 10:30 Break

10:30- 12:35 ApplicationSessionPanel C continued(2presentations)

PanelD, Computational Fluidand Plasma Dynamics

(3presentations)

12:35-1:30 Lunch

1:30- 3:35 ApplicationSession

PanelD continued(3 presentations)

PanelE, Applicationofs Artificialintelligence

(2 presentations)

3:35- 4:00 Working Groups (lastructions)

4:00 - 5:30 Working Group Bleakout S_,ssion l

7:00- 7:30 Buffet Dinner

7:30 - 0:30 Working Group Breakout Session II

Thursday, May 6

7:30- 8:30 Breakfast Meeting

(Chairs and Co-chairs of Breakout Sessions)

8:30- 12:00 Working Group Breakout Session IlI

12:00- l:00 Lunch

1:00- 3:00 Working Group Breakout Session IV (Establish conclusions)

: 3:00 - 5:30 Working Group Breakout Session V (Writing)

4 Workshop on Grand Challenge Applications and Software Technology
E,xecu_ve Sumrn_Jy

Applicationt_ (Birds-of-a-Feather)

7:00 Conference Reception

Friday, May 7

8:00- 8:10 Welcome by Paul H. Smith,

NASA, Chairman of the Organizing Committee

8:10 - 9:10 Summary Reports from Application Panels

Moderator: David Nelson, DOE

Environment and Earth Sciences: Robin Dennis, £PA

Computational Physics: Ken Kliewer, Oak Ridge

Computational Biology, Chemistry and Material Sciences:

L. Ridgeway Scott, University of Houston

Computational Fluid Dynamics: Tom Zang, NASA Langley

Applications of Artificial Intelligence: Robert Berwick, MIT

9:10 - 10:00 Reports from Working Groups

Moderators: Lee Holcomb, NASA; Rick Stevens, Argonne

National HPCC Infrastructure: Maxine Brown, UIC

I/O, Data and File Systems: Cs.rlos Roberto Mechoso, UCLA

10:00- 10:25 Break

10:25 - 12:30 Reports from Working Groups Continued

P_rallel Programming Paradigms: Kim Mills, Syr_use U.

Performance Characterization and Evaluation of MPP

Applications:

Robert Harrison, Pacific Northwest Laboratory

Program Development Tools: L. Ridgway Scott, U. of Houston

Building Multidisciplinary Applications: Tom Zang, NASA

Langley Research Center

Algorithms and Libraries h Phil Colella, UC-Berkeley

12:30- 1:20 Lunch

1:20 - 2:10 Reports from Working Groups Continued

Algorithms aad Libraries If: Bob Sugar, University of California

at Santa Barbara

Graphics and Visualization: Paul Woodward, University of Minnesota

2:15 - 3:15 Conference Address: John H. Gibbons, Science and Technology

3:15- 3:30 Break

3:30 - 5:00 Industrial Applications Panel

Moderator: Donald Lindberg, HPCC National Coordination Office

Aeronautics: Jan Tulinius, Rockwell International Corporation

Chemical Products: David Dixon, Dupont

Automotive/Manufacturing: Howard Crabb, Ford Motor Co.

Energy Resource Extraction: James S. Nolen, Western Atlas Software

Health Care: Hartwig Blume, Phillips Medical Systems

Financial: David Audley, Prudential Securities

Textiles: James Plouffe, Dupont

5:00 Adjourn

Workshop on Grand Challenge Applications and Software Technology 5
iim i ii I I III I IIII I iiiIII iiii I I II I

Application Se_ionJ

ApplicationSessions

Panel A - Environmental and Earth Sciences

(Chair:Joan Novak, EPA)

Armiatead G. Russell

A DistributedComputationalSystem forLarge-ScaleEnvironmentalModeling

CarnegieMellon University

Kenneth Galluppiand Daewon W. Byun

Comprehensive EulerianAir QualityModelingof Multivollutant

Speciesin theAtmosphere

North CarolinaSupercomputer Centerand EPA

Thomas O. Barnwell,Jr.,and Robin L. Dennis

Linked Water and Air QualityManagement

EPA

Alexander MacDonald

Future ofHigh-ResolutionWeather Prediction

NOAA/Forecut Systems Laboratory

Mary Wheeler

ReservoirModeling forPorous Med_a: PartnershipinComputational Science

Rice University

Kenneth Eggertand ShiyiChen

ReservoirModeling forPorous Media: LatticeGas Computations

Los AlarnosNationalLaboratory

Dave Bader

Ten- and Hundred-YearClimate Prediction

PscilicNorthwestLaboratory

C. RobertoMechoso

Development ofan Eartk,System Model:

Atmosphere/Ocean Dynamics and Tracers Chemistry i

University of California- Los Angeles

Dan Kowalski

Development of Algorithms for Climate Models Scalable to Teraflops Performance

NASA/Goddard Space FlightCenter

Richard B. Rood (Presentedby PeterLyster)

High-PerformanceComputing and Four-DimensionalData Assimilation:

Impact on Futureand CurrentProblems NASA/Goddard Space FlightCenter

6 Workshop on Grand Challenge Applicatio.s and Software Technology
I I I iiii iii III1| I1| I I I

_

--"-- ExecutiveSummary

Panel B: Computational Physics

(Chair: Tom Kitchens, DOE)

John Gardner

Understanding Solar Activity sad Heliospheric Dynamics:

Naval Research Laboratory

George Lake (Presenter: Calvin Lin)

Large-Scale Structure and Galaxy Formation

University of Washington

Robert Roener

Convective Turbulence and Mixing in Astrophysics

The University of Chicago

Wojciech Zurek and M. Warren

Scalable Hierarchical Particle Algorithms for Galaxy Formation

and Accretion Astrophysics

Los Alsmoe National Laboratory

Bob Sugar

Lattice Gauge Theory

University of California- Santa Barbara

C. Grebogi, E. Oft, and _. A. Yorke

Controlling Chsos

University of Maryland

Richard Crutcher and Michael Heath

Radio Synthesis Imaging: An HPCC Application

National Center for Supercomputing Applications

Workshop on Grand Challenge Applications and Software technology 7
I IIii I I IIIIII I I I I II

Application Sessions

Panel C: Computational Biology, Chemistry and

Material Scmnces

(Chair: Bob Martino, NIH)

William Goddard

High-Capacity Atomic-Level Simulations for Design of Materials

California Institute of Technology

G. M. Stocks, B. N. Harmon, and J. W. D_venport

Computational Quantum Materials:

First Principles Simulation of Materials Properties

Oak Ridge National Laboratory, Ames Laboratory, Brookhaven National Laboratc

W. J. Camp, J. S. Nelson, S. J. Plimpton, and M. P. Sears

Computer-Aided Design of Biological and Biomimetic Materials:

Computational Design for Catalysis

Sandia National Laboratories

Jerry E. Solomon

High-Performance Computational Structural Biology

California Institute of Technology

T. H. Dunning, Jr., and A. Wagner

High-Performance Computational Chemistry:

Scalable Algorithms for Grand Challenge Applications

Pacific Northwest Laboratory and Argonne National Laboratory

Robert L. Martino

High-Performance Computing in Structural Biology and Medical Imaging

National Institutes of Health

D. Lansing Taylor and Scott E. Fahhnan

High-Performance Imaging in Biological Research

Carnegie Mellon University

T. Barnes, V. Overacker. M. R. Strayer, and S. Umar

Computational Quantum NIatcrials: Quantum Structure of Matter

Oak Ridge National [,aborntory ailtl Vanderbilt University

Mike Colvin

Computer-Aided Design of Biological and Biomimetic Materials:

Pharmaceuticals and Agrochemicais

Sandia National Laboratories

R. Glowinski, J. A. McCammon, B. M. Pettitt, and L. It. Scott

Texas Center for Advanced Molecular Computation

University of Houston

8 Workshop on Grand Challenge Applications and Software Technology
f

i m II ilillllll iiili ii ii nngul I I I I I I I II IN Jill

Execu¢ivegumm_y

Panel D: Computational Fluid and Plasma

Dynamics

(Chair: Lee Holcomb, NASA)

Carlos Felippa and Juri Toomre

High-Performance Computational Methods for Coupled Field Problems

and GAFD Turbulence

University of Colorado at boulder

Manny D. Salas

_( Multidiscipline Simulation of High-Speed Civil Transport

NASA/Langley Research Center

Kalpana Chawla

Multidiscipline Simulation of High-Performance Aircraft

NASA/Ames Research Center

Russell Claus

Multidiscipline Simulation of Propulsion Systems

NASA/Lewis Research Center

Phil Colella

Computational Fluid and Combustion Dynamics

University of California- Berkeley

Dan Barnes

Numerical Toksmak Project

Los Alamos National Laboratory

Panel E - Applications of Artificial Intelligence

(Chair: Y. T. Chen, NSF)

Robert C. Berwick and Tomaso Poggio

High-Performance Computing for Learning:

Super-Human Supercomputing vs. Super-Computing for Human Computations

Massachusetts Institute of Technology

RichardMuntz (PresenterCarlo Zaniolo)

Data Analysisand Knowledge Discoveryin GeophysicalDatabases

UniversityofCalifornia-Los Angeles

Workshop on Grand Challenge Applications and Software Technology 9
.... _

i II |1I III illll li i ii i • ,,,m, ,= ............ ,,, ,,, ,,,

Worldng Groupm

Working Groups

Group A - I/O, Data, and File Systems

Technical Chair" Reagan Moore, San Diego Supercomputer Center

Applications Co-Chair: Carlos lloberto Mechoso, University of California, LA

This session covered issues relating to mass storage, high-speed file systems,

solid state disks, parallel file systems for MPP machines, parallel I/O user libraries,

mass storage management, scientific databases, and other items relating to I/O.

Group B - Parallel Programming Paradigms

Technical Chair: Joel Saltz, University of Maryland

Applications Co-Chair: Geoffrey Fox, National Parallel Architecture Center

Choosing the right programming paradigm is important. This se_ion discussed

the limitations and advantages of data parallel programming (e.g., HPF) and other

programming models, such as message passing, Fortran D, and Fortran 90. Discussions focused on choosing the best programming system for the type of problem.

Group C - Performance Characterization and Evaluation

of MPP Applications

Technical Chair.' Dan Reed, University of Illinois

ii Applications Co-Chair: Robert Harrison, Pacific Northwest Laboratory

Unlike serial computers, understanding performance of MPP applications is s

sometimes complex vital process involving scalability, balance, and the proper use

of performance analysis tools. This session focused on the needs of applications

programmers for tools to assist in understanding the performance of MPP applications.

_! Group D - Program Development Tools

Technical Chair: Fran Berman, University of California, San Diego

Applications Co-Chalr: L. Ridgway Scott, University of Houston

A wide variety of software tools have been developed to aid in the construc-

. tion of parallel programs. This session covered compilers, parallel programming

libraries, and support for heterogeneous computing and distributed computing. It

also covered profilers, tracers, and logical debuggers.

Group E- Building Multidisiplinary Applications

Technical Chair: Inn Foster, Argonne National Laboratory

Applications Co-Chair: Tom Zang, NASA/Langley Research Center

Many modern simulation projects could greatly benefit from the combination

of simulation models that exist in many diverse computer codes. The goal of this

session was to explore the needs of applications developers for tools that will aid in

the construction of these multidiscipliaary applications.

10 Workshop on Grand ChaUenge Applications and Software Technology

II IIIIII I I ii i i Hi ,_ , ,,
Executire Summary

Group F - Algorithms and Libraries I

Technical Chair: Ahmed Sameh, Center for Supercomputing Research

sad Development

Applications Co-Chalr: Phillip Colella, University of California, Berkeley

The availability of quality numerical algorithms and libraries often limit the pace

of applications development. This session covered the state of the art in algorithms

sad libraries for numerical PDEs, linear algebra, and finite element methods.

Group G - Algorithms and Libraries II

Technical Chair: Frsa Sullivan, Supercomputing Research Center

Applications Co-Chair: Bob Sugar, University of California, Santa Barbara

Modern simulations often require new algorithms that have not been extensively used in scientific computing. This session covered areas like random number

generators (including parallel random numbers), multipole methods, Monte Carlo

methods, particle-in-cell algorithms, and geometric and symbolic algorithms.

Group H - Graphics and Visualization

Technical Chair: Paul Woodward, University of Minnesota

Applications Co-Chair: Tom DeF_mti, University of Illinois at Chicago

Graphics sad visualization are critical to modern scientific applications. This

session focused on the graphics needs for Grand Challenge-type applications, including the problems of data movement, parallel rendering, portable tools like AVS,

virtual reality, and other frontier issues in graphics and visualization.

" Group I- National High-Performance Computing and

i_ Communications Infrastructure

Technical Chair: Charlie Catlett, University of Illinois at Urbsa_-Chsmp_ign

Applications Co-Chalr: Greg McR_e, Massachusetts Institute of Technology

'_ Until recently the development of the national HPCC infrastructure has been

_' developed largely by individual institutions and funding agencies. Since the creation

• of the HPCC initiative more coordination is possible. This session discussed the

_i infrastructure needs and requirements of the applications teams. These include

_i high-performance networking, integration of workstations and MPP, methods of

access and allocations, support, problem reporting, and the best way to make the

national computing infrastructure responsive to the needs of applications developers

and users.

Workshop on Grand ChallengeApplicationsand SoftwareTechnology 11
-- Background and Objectives S'_ON 2

,r

In the past two years, the federal government has funded a number of research

teams to attack Grand Challenge problems in science and engineering. While these

problems include seemingly unrelated areas as materials science, biology, chemistry,

earth science, physics, and artificial intelligence, many of the obstacles faced by

the researchers are similar. A principal objective of the 1993 Workshop on Grand

, Challengesand SoftwareTechnology was to identifthy esecommon problems and

tosharerelatedexperiences.

Another objectiveofthe workshop was to encourageincreasedcommunication

Sharing related between computationalscientistsand computer scientistOsf.ten,softwaretechnolexperienceshelps ogistshave developedusefultoolsthattheapplicationtseams arenot aware of;and

identifysuccesses often,the applicationpseopleneed specialtoolsthat arestillunder development.

and future needs. By bringing together these two groups, the workshop sought to identify what enabling software is available ,and what tools are needed for successfully solving the

Grand Challenge problems.

This was tile second Grand Challenge workshop sponsored by the federal government. The specific agencies involved were the following:

. Advanced ResearchProjectsAgency (ARPA)

, Department of Energy (DOE)

, EnvironmentalProtectionAgency (EPA)

, NationalAeronauticsand Space Administration(NASA)

. NationalScienceFoundation(NSF)

• NationalInstituteosf Health (Nill)

Workshop on Grand ChallengeApplicationsand SoftwareTechnology 13

l
Q

III I ii I iiiIir-- _ iiiiii " i II i [ ii i _ iiiii I II

• National Institute of Standards and Technology (NIST)

• National Oceanic and Atmospheric Administration (NOAA)

• National Security Agency (NSA)

In the remainder of this report, we summarize the activities of the 1993 meeting on Grand Challenge Applications and Software Technology. These activities

included panel discussions, working group sessions, and industrial presentations.

We conclude with an evaluation of the four-day meeting and suggestions for future

meetings.

14 Workshop on Grand Challenge Applications and Software Technology
-- Grand Challenge Applications

Panel Sessions

On May 4-5, 1993, representatives from universitites, national laboratories, and government agencies gave presentations about their current HPCC applications. Five

paxtel sessions were held, focusing on Environmental and Earth Sciences; Computational Physics; Computational Biology, Chemistry, and Material Sciences; Computational Fluid and Plasma Dynamics; and Applications of Artificial Intelligence.

Panel A - Environment and Earth Sciences

Panel A focused on the processes occurring in our atmosphere, oceans, and earth.

Using advanced computational techniques and software tools, the researchers are

developing models that can be used for environmental management and prediction.

Nine topics were addressed:

Understanding • a distributed computational system for large-scale environmental modeling,

our environment • air quality management,

requires

understanding , linked water and air quality management,

the processes . high-resolution operational weather prediction,

occurring in

s reservoir modeling for porous media,

the atmosphere,

oceans, and earth. • Grand Challenge in climate change,

• development of an earth system model,

• development of algorithms for climate models scalable to teraflops performance, and

Workshop on Grand Challenge Applicatiot_s and Software Technology 15
, t

......

II I II llI fl III II III II iiiii IIIIll ] I ..........

Pue/A - Enviroamen__d Esrtb $c/eaces

L.__ II llll _ [ ] _ Illl

• high-performance computing and four-dimensional dataassimilation.

Despite the apparent diversity of topics, one can distinguish four main categories.

1. Environmental management. Whether the researchers are dealing with

air quality, water quality, or a combination of the two, the issue is the same: How

can we effectively use the models (after we have developed good models) to address

practical,problems such as ozone levels, acid deposition, or air pollution? The challenge is twofold. First, we must make high-performance computing understandable;

this may require development of expert systems and effective interfaces to existing

visualization packages for the average user, as well as design of a mets-computer

for the decision maker. Second, we must learn how to use different models for a

particular environmental situation; currently, major mismatches exist between the

spatial and temporal systems of models.

2. Weather forecasting. Despite great progress in long-range forecasting,

very little progress has been made in our ability to handle precipitation madcloud

formation. By the mid-1990s, we expect to have a thirty-fold increase in observsTeraflop8 tional data. To utilize these data effectively, researchers will need high-performance

computers will computing power of up to 30 gigaflops (and the associated standards for MPP tools

enable researchers and languages) to enable design of a 5-km-resolution model.

to consider 3. Reservoir and groundwater modeling. The need for more powerful

computers and standardized tools was also emphasized in the talks on reservoir and

atmospheric groundwater modeling. This Grand Challenge area deals with multiphase, multimodels component flows. Currently, the level of application is restricted to grid blocks of

' t" incorporatin 9 thousands of feet by thousands of feet. For practical apphca sons, researchers must

more realistic be able to deal with blocks on the order of centimeters. While parallel domain deconditions, composition solvers appear promising, experience with running these solvers on the

Intel DELTA parallel computer have underscored the need for efficient, standardized

parallel I/O servers.

4. Global and earth systems. Finer resolution was one of several recurrent

topics in the presentations on global and earth systems. Oceanic models in particular require fine resolution (say, in the tens of kilometers), which in turn depends

on the availability of teraflops computers and greatly increased storage. Coupled

oceanic/atmospheric models raise additional problems. Currently, researchers are

experimenting with sharing computer models at different institutions; issues such as

distributed heterogeneous computing and user traffic over a gigabit network must

be resolved, however, before such model sharing becomes practicable.

Regardless of category, the obstacles are similar:

• domain decomposition and load balancing,

• parallel I/O,

• speedup, and

• data management.

Not surprisingly, the approaches taken to address these obstacles are also similar

(numerical simulations of PDEs and ODEs, Navier-Stokes equations, and spatial

discretization of the PDEs), as are the numerical methods (finite difference, finite

element, spectral methods, and fast Fourier transforms). The result of this similarity is not--as one might fear--duplication of effort; rather, the result is value

16 Workshop on Grand Challenge Applications and Software Technology
ii iI _ II IIINI Inunnnnil I I I I II IIN .... IlUUllf I II NI IIN Ill I I

I

Grand ChaJlengeApplications P_nel Sessions

added:we areseeingteams of researcherstacklingmany facetsof thescienceand

computationalscience,

Joiningtheseteams area number ofU.S.industriesT.he activecollaborationof

utilities(includingthe ElectricPower ResearchInstituteand theTennesseeValley

Authority),the automotiveindustry(includingGM, Ford,and the Motor Vehicle

ManufacturersAssociation),theAmerican PetroleumInstituteand, theConsortium

forAdvanced Modeling of RegionalAir Qualityatteststo the importanceof this

Teamwork Grand Challengeresearch.

isenabling And thisteamwork ispaying offin the abilityto tacklelarger,more complex

researchers to problems(suchastheocean conveyorbelt).Indeed,thepanelconcludedthat"excitingprogressiscoming insciencee,nvironmentalmanagement, and h'.'gh-performance

tacklelarger,more computing."

complex problems.

Panel B - Computaionl Physics

The first panel focused on Ii_ldr'rstanding and managing our planet. The second panel session addressed the eve11 broader topic of understanding our universe.

Several areas of computational physics were considered:

• understanding solar activity and heliospheric dynamics,

s large-scalsetructureand galaxyformation,

• convectiveturbulenceand mixing inastrophysics,

• scalablehierarchicaplarticlalgorithm e s,

• quantum chromodynamics,

• controlling chaos,and

• radiosynthesisimaging.

Complez physics The computationalcomplexityoftheseproblemsisenormous. Phenomena ofwidely

problems require differentlengthscalesareintimatelyi,tertwined,vastnumbers ofparticleasnd vast

su,_ciently periodsoftime areinvolved,and thousal_dsofmultidimensionalintegralmus st be

enhanced modeling considered.

Forexample,a typicala.strophysicfluid al dynamics problem may involvetranscapabilitiesportationof heat energy,angularmomeutum, magnetic fieldsand , elementalsubtosimulate stances;mixingoftheturbuletttfluidsmay be drivenby thermalbuoyancy,semiconcomplezproblems vection,and gravitywaves. Facedwith theimpracticalityofdoinga realcalculation,

over enormous theresearchertypicallydealswith dimensionlespar s ameters,identifiesthesmallest

time periods, feasible scale, and regards the simulatiou as a type of controlled lab experiment.

Yet even such a severely restricted experiment may require a thousand hours on a

CRAY Y-MP.

Another typicalproblem foctlseson the evolutionof galaxiesand clustersof

galaxiesoverbillionofs years.Scientistmus s t considersuch questionsas,Are there

"Great Attractors"?Are therega._esor "'darkmatter" obscuringour view? To

answer such questionsrequiresarchivill_terabytesofdata.

From theevolutionofgalaxh'._tothedcvelol)mentoftheuniverseisanotherstep

incomputationalcomplexity.What were conditioIIslikeinitiallyH?ow have they

changed from the radiatioll-doll_hlaeratedtothe matter-dominatedera,and what

Workshop on Grand Challe_lge Applicationsand SoftwareTechnology 17
a

........ _ ........

L I ] ii iiiiiii i i imii, iiiii .__ iiiiii i i i iji mirai .m. =.. . .. II I

Pu_naCi - Comp. Bioiosy, Chemistry, MJcerials Sciences

are they likenow? Solvingthe physicalequationsrequiredto answer such questionscouldliterallytakeyearsusingtraditionamlethods. Researchersaretherefore

exploringnew paralleslchemes,includingtreecodesand fastmultipolemethods.

Common to alltheseexaml:lesare sixfundamental needs: (I) portablecode,

(2)standardsforlanguagesand messagepassing,(3)interactiveisualizationtools,

(4)increasedcomputer storage,(5)a "reasonable"debugger,and (6)improvednetworking.As a firststeptoward meetingtheseneeds,researcherare s experimenting

with remotelylocatedtelescopecsonnectedby high-speednetworkstoveryhighperformance computersand onlinedata archivesthatareaccessedovergigabit/second

networks.

Panel C- Comp. Biology, Chemistry, Materials Sciences

Panel C combined three areas of critical societal importance: biology, chemistry,

and materials science. In biology, new tools and advanced computer systems developed by the HPCC program areenablingresearchertsoanalyzedata from complex

moleculesinlivingcells.In chemistry,massivelyparalleclomputers arebeingused

to investigatoeutstandingproblems in environmentalremediation.And inmaterials science, high-performance computing is enabling more accurate simulation of

new materials--for example, for high-temperature superconductors.

The presentations focused on six area.s:

. high-capacity atomic-level simulations for design of materials,

. computationaiquantum materials,

o computer-aideddesignofbiologicaland biomimeticmaterials,

• high-performancecomputationalstructuralbiology,

• high-performancecomputationalchemistry,and

• high-performancecomputing instructurabliologyand medicalimaging.

Sincetheinitiationofthe HPCC program two yearsago,more than tenproduction

codeshave beendevelopedintheseareas,withfasterthroughputand increasecdosteffectivenessA .major achievementwas a 150,000,000-atomsimulationofthebiaxial

failureof ceramicmaterial.This simulationprovidesa macroscopicvalidationof

Realistirnod c eling the microscopicforcefield.

willbepossible Yet much more remainstobe done. Three exampleproblems illustratheecomonl_lwith orders plexityofthecomputationsbeingaddressed The firsetxample involvestudyofthe

of magnitude atmosphericdegradationofalt_'rllatitoveschlorinatefluo d rocarbons.Such alternativesarebeingexploredinaccor_lailcwith e federalrequirementsto replacecurrent

increases in refrigerants.Typicalconlptltatioli._ i_ivolv2e0 reactions,each requiring1,000-5,000

computational Cray hours,fora totalof20,000to 100,000CRAY Y-MP node-hours.The second

performance, example involvesdesignofductileord , eredintermetall(NiciAI)alloys.Thisproblem

involves20,000CRAY Y-MP hours per dislocationpathway. Sinceeach computation involves3-5 pathways, the totalexecutiontime is60,000-I00,000hours of

CRAY Y-MP node-hours.The finalexample involvesproteinengineeringforbioremediation.Nativemicrobesoftencannot use thecontaminantsforfood;therefore,

scientistsreatthe microbe'senzymes to produce a more effectivseet.The challengehereisthattheenzymes may have hundredsofthousandsofatoms. Quantum

18 Workshop on Grand ChallengeApplicationsand SoftwareTechnology
................

" __ inIN I I I II n I I

I

Gr_nd ChJllenge Applicationa Panel Sessions

effects must be included; an approach based on cl_ical mechanics is not enough.

Moreover, a huge amount of Cray time is required to compute even a very small

percentage of the enzyme's activity.

Researchers are confident that their goals can be reached. As computational

power increases, quantum chemistry physical models are improving; and new techniquee such as the force decomposition method are proving capable of handling problems with 10,000 atoms extremely rapidly on the Intel DELTA. To fully achieve their

goals, however, the researchers agree that they need portable, high-performance

U.S. companies math libraries. In particular, they need access to libraries of FFTs, random numare actively ber generators, and fully distributed dense linear algebra solvers with flexible data

involved in Grand structures.

One very encouraging note is the broad involvement of the chemical, metallurgiChallenge efforts, cal, petroleum, and pharmaceutical industries in addressing these Grand Challenge

problems. Industrial corporations are collaborating with computational chemists,

biologists, and materials scientists in universities and national laboratories. Already, in conjunction with the semiconductor industry, these scientists have engineered band-gaps for compounds with specified light-emission properties. And the

potential areas of application are enormous, including

• imaging- electron micrograph reconstruction, X-ray crystallography, and realtime visible light microscopy;

• macromolecular structure analysis and prediction - protein folding and sequence analysis; and

s simulation of materials at the atomic level - quantum mechanical methods,

classical molecular dynamics, and particle-continuum models.

i

Prone/D - Computational Fluid and Plasma Dynamics

Massively parallel computers and advanced software technology axe enabling

scientists to model fluid flow and to perform design optimization of complex systems.

These new capabilities have application in such areas as automobile manufacture,

Grid techniques aerospace vehicle design, and medicine.

and discretization The panel on computational fluid and plasma dynamics focused on four specific

methods appear topics:

especially/ , geophysical and astrophysical fluid dynamics,

promising.

• multidisciplinary analysis _nd optimization of aerospace vehicles,

Q compressible and reacting flows, and

• fusion power.

Within these topics, problem sizes vary widely. Computations involving controls

are insignificant. Problems involving structures may involve 10s degrees of freedom

and require up to 1 gigabyte of memory for direct solution. Problems involving

basic fluid physics may involve more than i0 r points with as many as 20 words per

point; and more memory may be needed for calculating reacting flows. Engineering

flows may involve lOS-lO _' points with up to .50 words per point; from 10 to 10a

Workshop on Grand ChallengeApplicationsand SoftwareTechnology 19
Q a

.... II I I i .......... ill| ii _ iillnll I I i

PanelD - ComputationadFluid and Plasml Dynamics

analyses are needed for optimization, where a typical analysis run includes as many

as 104 iterations.

For most of these problems, researchers use grid techniques or discretization.

Grid approaches are principally block structured (for fluids) or unstructured (for

structures). Discretization methods include second-order to fourth-order finite volume (for fluids), finite elements (for structures), PIC (for fusion), and pseudospectral and fnite difference (for geophysics).

These approaches have resulted in notable success, on a wide variety of highperformance computer systems. For example, researchers have carried out a 512s

simulation of compressible turbulence on the TMC CM-5, a time-dependent adaptive block-structured fluids computation on the CM-2, a simulation of an aircraft

engine on an IBM RS/6000 network, and aeroelssticity computations on the Kendall

Square KSR-1. Such successes have attracted the interest of several industries:

• The NASA computational aerosciences program has established interactions

with an industry consortium focused on HPCC technology and is expanding

its traditional technology transfer to the aerospace industry.

• The national Tokamak project LncludesGeneral Atomics and MicroeLectronics

Computer Corporation.

• The computational fluid dynamics and combustion project is working with

burner companies on the design of software for burner simulation.

At the same time, industrial interest has also focused attention on serious nee&

Graphics for these Grand Challenge projects. Mathematical and algorithmic needs are paztechniques can ticularly apparent, Interdisciplinary coupling in aerospace is not well founded, esaccelerate the pecially for ensuring stability. Aerospace design would benefit greatly from opmodeling process, timization methods that are more tightly coupled to the analysis and less "black

box." Engineering fluids simulations need improved turbulence models. And plasma

simulations would benefit from models that incorporate the essential physics more

economically.

From the viewpoint of the applications environment, all projects would like to

be able to run graphics code directly on the MPP--not on a graphics workstation.

In addition, most projects would benefit from the availability of object-oriented

scientific databases distributed across heterogeneous platforms.

A third major problem common to all these projects is dataset management.

' Thesestudiesgeneratene ormous,oftenmultidisciplindartasy etsTo. exploitht ese

datasetsr,esearchernseedrapidstorageaccess , a,nd perusal--preferainblyreal

time.Many scientists believetheywould alsobenefiftrom theincorporation of

scientific operationsina databasemanagementsystem.

Finally,physicistsand engineercsonsistently notedfouroutstandingproblems

withtoday'sMPPs: (I)peakvs sustaineped rformanceistoolow (lessthan10%

forproductiocnodesusingimplicimtethodson complexgrids),(2)theI/O bottlenecikssevere,(3)architecture-optimicodzeedcannotbe generaterad pidly,and

(4)littlheardwaresupportexistsfortiming/performanecevaluation.The solution

totheseproblemswillgreatliyncreasthe eusageofmassivelyparalleclomputersin

computationafll uidand plasmadynamicsapplications.

20 Workshopon Grand ChallengeApplicationsand SoftwareTechnology
Grand Chxllenge Applications P_uzelSessions

Panel E- Applications o[ Artificial Intelligence

This last Grand Challenge area was quite different from the preceding panels.

Applications in &rtificial intelligence do not involve a well-defined set of matrices

or equations. Rather, they deal with an environment that can understand human

speech and language.

Two presentations were given:

• high-performance computing for learning, and

• data analysis and knowledge discovery in geophysical databases.

Both presentations observed that Grand Challenge re.arch is overwhelmed by a

sea of data. To evaluate this data efficiently, one would like to have an intelligent

system that can handle English-like queries, can "learn," and can work quickly.

Two early successes were noted. The first involves development of a library of

"| _# radial basis function algorithms to adjust "templates" in an effort to train a system

"Intelligent" to evaluate "normalized" patterns. The second success involves construction of a

_ S_lstems may database system that can be used to examine the evolution of cyclones over time

,_ help evaluate and space. s data quickly. The researchers questioned whether a specially designed architecture will be needed for more extensive AI applications, A special VLSI architecture has been

developed, and there are special parallel machines with fewer than 1000 nodes that

use a database foundation. Serious questions about parallelism, scalsbility, and the

I/0 bottleneck remain to be answered, however.

l

Workshop on Grand ChallengeApplicationsand SoftwareTechnology 21
i ii I i i i i

SECTION 4 Grand Challenges and

Industrial Applications

One of the stated goals of the federal High Performance Computing and Communications program is to spur gains in U.S. productivity and industrial competitiveness.

Many of the Grand Challenge applications presented at the workshop included collaborative projects with industry.

On Friday, May 7, representatives from various U.S. corporations discussed examples of industrial involvement ill high-performance computing. Corporations represented included DuPont, Ford Motor Company, Prudential Securities, Rockwell

International, and Western Atlas International.

ChemicaJ Applications: DuPont Corporation

Experimentalists at DuPont Corporation rely on high-performance numerical

High-performance simulations for accurately calculating the properties of molecules. Such information

computing is used to predict binding energies, tertiary structure, enzymatic activity, operation

benefits major of diffusion-controlled processes, and inhibition and promotion of DNA expression--

industrial projects, information necessary in the design of environmentally safe new enzymes. Re- searchers are also interested in using accurate numerical predictions for the design

of chemical plants. Here tile Grand Challenges span several sciences, including

chemistry (thermodynamics, kinetics), chemical engineering (fluid dynamics), engineering (structural design), mathematics (optimization), and environmental control

(waste minimization). DuPont is also involved in ecosphere/geophysical modeling,

for example, to demonstrate compliance with environmental regulatory agency regulations or to identify favorable locations to drill for oil and gas.

Workshop on Grand Challenge Applications and Software Technology 23
ill Ill i i I

Financial Applications: Pr,dentia] Securities

Automotive and Manufacturing Applications: Ford

Ford Motor Company uses high-performance computers in diverse computersided engineering (CAE) and computer-sided design (CAD). By displaying CAE/CAD

data as realistic three-dimensional objects, researchers hope to dramatically reduce

the time required to construct, modify, and evaluate design concepts. In one case

(a styled wheel), Ford scientists have cut the process from 24 days to 16; in another

(a control arm), they reduced the process from 33 to 15 days. In both cases, the use

of high-performance computers and advanced software permitted the scientists to

triple the number of elements analyzed, thus providing a more accurate simulation.

Financial Applications: Prudential Securities

With an unpredictable number of diverse users throughout the world, and strict

time constraints imposed by global markets, Prudential Securities relies heavily on

high-performance computers (including IBM RS/6000 X-stations, a 72-node Intel

Paragon, and a 32-node Intel i860 hypercube). The company views scalable parallel

computers as a "computational opportunity" for gaining a competitive advantage

in selected strategic markets. Prudential is particularly interested in improved software and interfaces that will enable interactive, real-time computing on a globally

distributed network of heterogeneous parallel computers.

Aeronautics Applications: Rockwell International

Rockwell International scientists have turned to high-performance computing to

provide what they call "affordable system optimisation." The objective is to develop

an optimization process that best satisfies customer needs, at a given cost, through

ali stages--from specification of requirements, through design and manufacturing, to

Industrial operations and support. Rockwell is investigating Taguchi methods and concurrent

applications engineering techniques to improve design and production process.

include reservoir

modeling, design Energy Resource Extraction Applications: Western Atlas

optimization, The huge computational demands of reservoir simulation and seismic processing

textile processing, have made oil companies some of the earliest purchasers of advanced computers.

automotive Western Atlas International believes that numerical reservoir models are becoming

engineering, even larger, with more components, more unknowns, and a proportionate increase

global financing, in the number of runs required. Such applications will require high-capacity parmolecular allel computers. Effective use of these computers will, in turn, require significant

modeling, and advances in systems software--new algorithms for solving stiff systems of linear equations, improved load-balancing methods, better interprocedural analysis tools

simulation to handle very large applications (500,000 lines of code), interactive debuggers for

of the brain, each processor, and emulators capable of simulating parallel operation on a singleprocessor workstation.

24 Workshop on Grand Challenge Applications and Software Technology
................ I II I II IIII i II I iiiiii i i ii • II ii iii I i i I iiiiLi

Grand ChallenSem and lndumtrixl Application.

Textile Applications: DuPont Fibers

The textile industry and the DOE national laboratories have joined in a partnership called AMTEX. The objective is to develop the information technology necessary for "a demand-activated production system." According to DuPont Fibers,

such a system will require advanced systems integration and multidisciplinary design

capabilities. The result, it is expected, will be a step-change productivity increase.

Health Care Applications: Phillips Medical Systems

Phillips Medical Systems scientists are addressing three Grand Challenge applications: (1) generation of and access to very large databases with fundamental

clinical information, (2) performance modeling of imaging and information management systems, and (3) high-speed computations and communications for intelligent

information management. For example, to make effective use of an anatomical,

neuro-functional, physical-parameter four-dimensional atlas of the brain (covering

a scale from 1 micron to 20 centimeters and the entire human life span), researchers

need very fast, expert-driven database management tools.

Workshop on Grand Challenge Applications and Software Technology 25
I ii ii iiiii I i I i i i i - ii r i ii ii iii i iii i iii i ___

SECTION 5 Working Groups

Following the presentations by tile applications teams, the workshop broke up into

working groups comprising both computational scientists and software technologists. The focus was on the software technology requirements for Grand Challenge

applications research.

Group A - I/0, Data Systems, and File Systems

Chair: Reagan W. Moore: Co-chair: C. Roberto Mechoso

The currenttrendsincolnputerhardware have resultedinCPU speeds increasinglyoutstrippingl/O capabilityforsupportingboth datasetsizesand dataset

accessrates.Many applicatioasare limitedby the I/O capacityof currenthardware, and compromise by,worki,g with reduced problemsizes.Research efforts

areneeded forthe developnmldof high-l,erformancefilsye stemsthat integratlo- e

cal disk cache with archival.storagoa,s well as forthe development of database

Increasingly larger management systems supportiilg scientific data.sets.

datasets have

imposed enormous Status

demands

for e_cient The working group focused oal two i_articular issues: data size and data rates.

input/output

systems. Data Size of Applicatious. Typical current and near-term HPCC applications

involve data traffic of from I G I3 to 4 "['B per run. Several applications envision

writing out 10-GB files in the twar tcrtu and 100-GB or larger files in the longer term

per run. This projectiou has implicatioas for staging areas. In some applicationsa,

the amount of output data prodlJced by one run depends only on how long the

Workshop on Grand Challenge Applications and Software Techaology 27
illi ...... i I[i .... iiiii ii I . roll i, ...... iiii. I ii111111

Group A - [/0, Data Systems, and File Systems

simulation is run. In other words, the output size is directly influenced by the CPU

cycles available.

With increasingly larger data runs possible on HPCC platforms, archival volume

requirements are increasing at least as fast as processor performance. Current

archival sizes per Grand Challenge group typically range from 500 MB to 500 GB,

with a peak of I0 TB. Anticipated requirements go up to I0 PB per group.

This enormous amount of data imposes more than just storage requirements.

Several simulation-type HPCC applications generate large numbers of files (10 s)

and corresponding numbers of job submissions, requiring substantial human management time. Some groups are attempting to alleviate this problem by increasing

staging area storage and developing better data.management software.

Data Rates, Current and near..term HPCC applications involve I/O rates of 1-

40 MB/s to disk and 0.5-6 MB/s to archive. Users indicate that I GB/s to disk

and I00 MB/s to archive will be required in the near future. This disk rate is

achievable today on appropriately configured CIIAY C-90's, .but is not achievable

on most MPP platforms. The archival rate is not possible in any cost.effective way

today.

The choice of algorithm strategy and the necessity for the recomputation of data

are strongly influenced by the ratio of [/O to CPU power. For instance, current

limited I/O rates are forcing more codes to rely on "in-core" solution algorithms.

Targets of Opportunity

This section is organized top-down through the storage hierarchy: from primary

Fast and memory through secondary and tertiary storage. Requisite network communication

convenient access is implicit in all levels of the system. The main theme of recent I/O research is

to data--whether greater performance by increasing parallelism in all levels of the system, that is,

beyond the massively parallel processors. remote or

local--is needed.

Memory-Level I/O Research. Research is under way in compiler and runtime mechanisms to utilize the memory hierarchy more efficiently, for example, by

data blocking, data placement, prefetching, file caching, and run-time analysis of

I/0 access patterns. The goal is to give the illusion of a giant main memory by

static and dynamic optimizations that improve locality of accesses. One example is

memory servers, which expand the primary memory available to a given processor

by using memory in idle nodes within an MPP.

Disk-Level I/O Research, At the disk level the two major areas of research

are high performance and schemes for storing information found in MPPs. High

performance includes I/O benchmarks, high.performance file systems for MPPs,

and parallel disk organizations for MPPs. I/O benchmarks are needed to give

realistic metrics for evaluating alternatives in trying to accelerate Grand Challenge

applications. The lack of benchmarks has inhibited progress in many fields, while

good benchmarks have led to rapid innovation.

Research in storage schemes includes parallel file systems for MPPs, persistent

object stores, and scientific databases. The goal of parallel file systems is to remove

sequential bottlenecks between parallel processors and parallel disks. Persistent

28 Workshop on Grand Challenge Applications and Software Technology
IIII ImllI I II i ill iiii iiillllllll[ iillllll ........................ I II III II I I[ [ I]1 III II [ j IIII

Workin8Groups

................ ,,,, , ,H

object stores and scientific databMes are new paradigms to add type information

and structure, enabling researchers to access information more intelligently and

more efficiently.

An important consideration for all the topics at this level is convenient and fast

access over local and wide-area networks for distributed data. Grand Challenge

applications have huge data rel_ositories that must be made available to researchers

across the organization and the country.

"Tape'-Level I/O Research. "Tape"-level I/0 includes all storage media accessed via robots: tape archives, optical jukeboxes, and network-attached peripherals. At this level, migration strategies for network-attached peripherals are still

being developed. A key topic is extending automatic migration from local disk

to the next two levels of the storage hierarchy. Data management issues include

databases for large and numerous objects and data organization for different access

patterns. New latency-tolerant schemes must be invented to hide the inherent istencies of tape robots, measured in tens of seconds, so that this massive capacity

is efficiently used. A final challenge for this area is offering high-bandwidth data

transfers to the bottom of the memory hierarchy.

Relationships

The research topics identified in the preceding section raise pointed questions about

existing I/O policy and relationships between applications and software developers.

Software Sharing. Technological software-sharing mechanisms include anonymous FTP, Gopher servers, and the hypermedia MOSAIC software. These mechanisms are heavily used by the community. The near-term establishment of distributed file systems such as AFS and DFS will further facilitate software sharin&

Parallel [//0 The NASA HPCC software-sharing mechanism should be more strongly promoted,

benchmarks must and its use by all the Grand Challenge applications groups encouraged.

accurately reflect

applications Balance. The HPCC program has emphasized teraflops applications. This focus

requirements, has exacerbated the divergence of CPU speed and I/O capability: I/O is not scaling

asfast andis becoming a major bottleneck for teraflops applications. Storage of data

associated with real-time, I/O-intensive applications will require stronger funding

support to create easily used distributed systems. For many applications, both the

amount of data that must be stored and the rate at which the data needs to be

accessed will soon increase beyond the capability of current systems.

Interdisciplinary Interactions. The interdisciplinary Grand Challenge teams

are helping break down sociological barriers between computer scientists, applications scientists, and system integrators. Improved collaboration between these

groups can be facilitated through development of parallel I/0 benchmarks that

accurately reflect applications requirements. These interactions can be further enhanced by considering improvements in l/O and storage to be part of the deliverables for these projects.

Workshop on Grand Challenge Applications and Software Technology 29
i

[I IIIII I I I[ I IIlIi I IIIIIIIIIIII I [ /_ I -Z _ -_i [ I IIIHll [[ I J IH I .........

GroupA -I/0,DataSysJemsa,ndFileSystems

Vendor Software. Vendors are moving in the right direction for supporting

Grand Challenge application I/O requirements--but too slowly. The HPCC community needs to promote vendor standards for archiving large amounts of data. An

example is an industry effort to define standard interfaces for databases to make

use of tertiary storage. Commercial support for scientific databases also needs to

be encouraged.

][nduetrlsl Appllcations. The very large amounts of data generated by Grand

Chsdlenp applications is forcing systems software development for efBciently handlin| large objects. Scientific application projects have increased the understanding

of the importance of 1/0 among computer scientists and between them and application scientists. High-performance scientific databd can have a significant erect on

areu outside the current HPCC focus (e,g., medical image database, geographical

information systems).

Findings

The findings of the I/O working group are organized into two parts: (I) development

of secure, robust, high-performance, high-capacity, transparent file systems and (2)

Effective I/0 development of database management systems supporting scientific datuets. We

require_ robust feel that each of these areu is important in its own right, and we therefore rank

fi[e systems activities within each area rather than across both areas. We also recognize that

and database a great number of interdependencies exist between areas. In particular, for many

applications there is a need!for layered access to storage, which will require careful

management integration of many of the components discussed below.

systems. The performance parameters in the following discussions are based on requirements for the next few years.

Pile System Development, The group identified four findings in this section:

• Provide high-performance local file systems to keepjobs CPU bound (rate I0-

500 MB/s, size 300 GB-1000 GB). This effort comprises support for research

and development of parallel file systems (including the goal of standardisafish), standard parallel file system interfaces from programming languages,

parallel run-time library interfaces, and serialization methods for files stored

in parallel file systems.

• Develop high-performance archival storage systems (rate 1-50 MB/s, size I0

TB to I PB). No archival systems are available with high-speed parallel support. This should be an important priority. It will require support for staging

data before MPP execution begins, support for subsetting of large datssets,

development and standardization of archival storage interfaces to local parallel file systems, and support for third-party transfer to network attached

peripherals and systems.

• Ensure ubiquitous access to conventional distributed file systems (rate I-I0

MB/s). Such access includes client-level support of distributed file systems

(AFS and DFS) from within MPPs, interfaces and migration schemes for

integration of distributed file systems with archival storage systems, large file

li

30 Workshop on Grand Challenge Applications and Software Technology
.... I ,il - nn .) .... 111 III I I I, iim_ numm. - II/I " Iflll( Ha III _

i ,,v iiiii

Workmj Groups

i Ir -- : : :: ....... m , I _._t I II , " ,,,,,t ! ....... J _-- _ ,_ ,,.............. - ,,,,,/ , - _ Irr ,I ,I

capability in distributed file systems, and scaling for support of a very large

numberof files.

• Ensure ubiquitous access to high-performance distributed parallel file systems

(rate I0-600 MB/s). Research and development are needed in distributed parallel file systems, Also needed is support for third-party access in adistributed

parallel file system.

Database System Development. Five specific areas were suggested fordatabase

system development:

• Support for large object_

• Fast access to subsets of large datasets

• New indexing and acceu methodsfor scientificdata

• Interface to tertiary storage systems

• Language interfaces to parallel database systems

The group further suggested that a set of Grand Challenge benchmarks and

sample data be crested for all levels of computer systems. This set should be made

easily available to computer systems researchersso that they can evaluate new ideas

to improve performance of Grand Challenge applications.

,,,,Gr:- oupi ,, Ji,, B - P__ar........... allel Prog....r_ ................ amming Pairila[ di _ igms J iJ J J _ : r __.:

Chair: Joel Salts; Co-chair" Geoffrey Fox

Researcher# Choosing the right programming paradigm is important. The Working Group

must know how on Programming Paradigms considered the advantages and limitations of data parto choose the allel programming and other programming models such as message pusing, shared

memory, and data flow. The discussion focused on the fit between the various

right program paradigms and the HPCC applications.

paradigm for each

given problem. Status

The working group reviewed results of the written survey of the Grand Challenge

teams and discussed the general opinions of data parallel programming with al>-

plications researchers in the group. To supplement this information, the working

group constructed a set of questio.s with the intent of getting workshop attendees

to clarify their positions on some specific issues. Three other working groups with !

a high concentration of applicatio, researchers were approached with the followi'ag

questions:

s Which paradigms do you prefer: shared-memory (name space) process parallelism, message passing, data parallel, or none of the above (give counterexstaples)?

• Suppose that there is a "good data-parallel (Fortran)" compiler to handle

general unstructured and block-structured problems. Generalise this with

large-grain-sized functional parallelism support for such data-parallel modules.

Would this model be useful'? sufficient?

,,,, _ ,,, , , ,

Workshop on Grand Challenge Applications and Software Technology 31
_-_ - ii m..... U]IF I [ I ........................ iii1|11 I lIT I _-- ir iii ii ii .,re,l,,.,,,,,i n n II Ul---_

Group B -ParL/lel Programn,n8 Par4di|ms

........... - i, ,in ___ ,i,,i _ ,, - :

!

• How muchefficiencywouldyou be willing to giveup in exchangefor s major

improvementin productivity (e.g., automatic p_rsllelism)?

• The programming paradigm supported by KSR is am example of one that

supports quick ports (with poor initial performance) and incrementalcode

optimisation and parallelisation. How important to you is this sort of cap_.

bUity?

s How important is C++ functionality? Why are you using C++ (if you are)?

What particular features of C++ do you find useful? In particular, do you

make use of inheritance?

Based on all the input from the surveys and discussions, the working group identified the following items as summarising the current status of use of programming

paradigms,

i. No single paradigm is "the answer," even within a single program. The current

software provided by vendors represents different programming paradigms:

data parallel, basic shared memory, and message passing. There is no con.

sensus among Grand Challenge applications people on the best model to use.

I paradig No smingleia To appsomlication e extdomain ent the considerations, choice is based onon softwa taste, rebutavailabili it is alsoty, based and onondifferent specific

the answer, types of functionality found within a given program. The working group noted

that there is a hierarchy in the choice of programming paradigms. Users exprem s definite preference for general, portable, high-level languages such as

C++ for portions of programs associated with preproceseing, functional control, and poetproceseing. When it comes to computstionally intensive parts of

programs, however, applications researchers are more likely to use a language for which the compilers are more efficient, such as Fortran. There is a further

tendency among researchers working on heavily used algorithms (e.g., FFT or

linear equation solvers) to focus even more closely on efficiency with explicit

message passing and even _sembly language.

2. Currently, each vendor tends to promote only a single paradigm (matching its

architecture) and lacks the resources to give substantial support to multiple

paradigms. This situation is in contrast to item I above. Furthermore, the

J_ hardwar leading teo tceons chnology taalt pressure cycle isonfaster paradithangm tdheevelopment. robust tool development cycle,

i! Portability Lq 3. Portability of applications is an important consideration. It is not so much

vital, in view the case that users expect to be moving their codes daily, but rather that

of the diversit;/ the underlying hardware is still in a state of flux. Users are uncomfortable

of computer with committing substantial optimization efforts to vendor-specific language

extensions, memqe.pusing systems, or &uembly languages when they expect

architectures, hardware changes.

4. Informal standards are emerging. The need for portability across architectures has been noted, not only by the users, but also by the vendors. As a

result, informal vohonteer eft'errsh_ve emerged during the past year to address

this issue. The High Performance Fortran Forum (HPFF) has drafted a data

parallel programming interface which will be common across multiple vendor

32 Workshop on Grand Challenge Applications and Software Technology
architectures. This effort is expected to continue to extend the elm of also.

rithms and data structures addr_d. Following the HPFF model, s similar

effort is under way to provide a standard meuage-pusing interface (MPI).

These efforts have fairly wide participation and user interest.

5. Rich data and operation abstractions are getting increuing attention because

of the resulting code simplification and reuse. C++ is being used by several

projects, and there are reports of' substantial productivity gains due to the

higher level of abstraction.

Targets of Opportunity

The working group identified meversltargets for new reuarch directions.

Conducting Empirical Studies of Programming Paradigms. Many paradigms

suffer from the lack of critical evaluation. The studies proposed here should involve

the porting of realistic applications and should be carried out in clo4mcollaboration

with applications researchers. The results would answer many questions regarding

the suitability of programming paradigms for both general use and for specific topics, and could be used to drive the development of the next generation of computer

software.

It is important to note that some of these studies may fail to produce usable

codes; from the viewpoint of developing programming paradigms, such negative

results are as important as producing good codes. In order to come to a broad

understanding of the applicability of programming paradigms, the studies should

also include industrial and commercial applications.

Achieving Better Performance with Data Abstraction. A wide range of

Grand Chsllengeresearchersexpressedstrong interest in the use of data sbstracParadigm studies tion. The need for data abstraction is particularly pressing to researchers who

must include solve problems with complex hierarchically defined data structures such as strutrealistic industrial tured adaptive grids or irregular tree-type data structures found in fast multipole

and commercial algorithms. Most Grand Challenge applications scientists who employ data abstraction use C++. Unfortunately, current compilers typically do not generate efficient

applications, code when confronted with abstract data types and inheritance. (Fortran 90 _lso

supports many aspects of'dat_ abstraction, but there is little experience with this

language in the Grand Ci_allenge community, because of the poor availability of full

Fortran 90 compilers. Moreover, some of the applications researchers are exploiting

the C.. feature of inheritance, which is not supported in Fortran 90.)

Supporting Increme-tal Optimizations, The ability to port a code to a new

architecture quickly and optimize for performance later is very attractive to users.

For example, this is possible in shared-memory programming paradigms, such as

those supported by Kendall Square Ftesearch, software implementations of virtual

shared memory, or (future) compilation methods. For this approach to be successful,

users need to be able to run with realistic data sets and obtain feedback that they

can use tc determine how to optimize performance. Tools and techniques that

facilitate incremental performance optimization are potentially important.

Workshop on Grand Challe.ge Applications and Software Technology 33
ii nl _ i i Ir ] lmimrl_ n,Zln IIIII I II11 I

GroupB - P&ra//e/ProsrcmminifP_r_ism.

Providin I Language, Compiler, end Programming Paradigm Support for

Irveipzlar, Block-Structured, and Adaptive Problems. A large fraction of

Grand Challenge applications surveyed make use of sparse data structures or linked

lists (16 out of 28 teams). In addition, 12 teams employ block-structured gridm,

and 4 teams make use of nonuniform tree-type data structures for fast multipole

algorithms. Also, dynamic adaptivity is growing in importance. In order to prove

attractive to abroad range of Grand Challenge researchers,programming paradigms

sad associated compilers sad run-time support will have to be able to effectively

handle irregular problems. Continued research on programming paradigms, cornprier techniques, and run-time support aimed at such problems is needed.

Relationships

Many working group members from computer science emphasized the positive influence work with applications people has had on their research. This included

supplying practical problems and providing valuable feedback. The feeling of the

Working Group is that computer science and applications should be tightly intertwined. However, there is also a significant feeling that much of the current interaction is only token discussion. Instead of this, real interaction through day-to-day

New incentives contact is needed. For this to happen, both computer science researchers and appliers needed cations scientists must adapt. Computer science people must be willing to rapidly

to encourage prototype relatively prosaic (at least not leading-edge) systems, and applications

interdisciplinary scientists must be willing to tolerate the occasions] system error.

A fundamental question raised in the working group was, W/,at im#,e plrlPome

collaborationa, o! Grand CI,allsnge grant,? The Grand Challenge applications described at the

workshop focused on the output of the applications rather than on the software

used to obtain that output, In these cases, the computer science component was

often regarded as peripheral to the main goal of the grant. Applications specialists,

who are judged on the quality of their science, are understandably reluctant to

I

wait for computer scientists to develop general, robust systems. In HPCC projects

where computer science is more central, the opposite problem may occur--scientific

results may become peripheral. This reflects a more general problem: computer

scientists sad applications experts have different professional goals. Applications

scientists must develop new scientific results, while computer scientists must focus

on general computational principles. However, the funding agencies might improve

collaboration by clearly specifying the equal importance of computer science output

and applications science output in evaluating grant results. Also, new incentives

are needed to encourage interdisciplinary collaboration.

Another way to improve the effectiveness of computer science researchers in the

programming paradigm area would be to increase the visibility of programming Isaguage research outside the computer science community. This could lead to more

use of particular programming paradigms to exprm, advanced algorithms, resultThe federal ing in better evaluation of new languages and systems. It would also help ensure

HPCC' program that new language developments would be relevant to the needs of the applications

has heightened scientists.

It is worth noting that computational scientists developing numerical algorithms

interest in parallel face similar problems. One working group member reported that his group minicomputing, mized this problem by having a "captive" applications expert for doing evaluations.

This enabled them to attack hard problems all the way from the problem definition

34 Workshop on Grand Challenge Applications and Software Technology

I ........ I _. IIIII _f,,iIlSll......... IIII .. r j ....... _ I II II II
L

L iiin ii iii Ill i ] ITII IIIIm L _ IIII1_1 I111 I IIII I II II i [111[ ] " IIIIIII Ulillml Illllll II Illl I illll II

Wor_Jn8 Groups

,,,, .... ,,, ,,,, ,J,,,, , ,, ,,,,, L -- --

to its implementation. This may provide a model for workon parallel programming

paradigms.

i

Impacts

"_ Perhaps the overriding sense of this working group was that, at this point, it is

really too early to expect significant impact from the HPCC program outside of

the program itself. That said, however, it is important to make a few observations

about the potential impact of the HPCC program.

i It is clear that the HPCC work has heightened interest in parallel computing,

and, as a result, it has increased the pressure on hardware and software vendors

and the computer science research community to produce the products (machines,

languages, tools, etc.) that will make parallel computation effective. It has also led

to increased interest in standards, which will permit application developers to build

programs based on paradigtns and software infrastructure that will be supported

_I, Experience on a large number of different parallel computing environments. Examples include

in program High Performance Fortran (supporting the data parallel paradigm) and the evolving

paradigms must Message Passing Interface (supporting a process parallel paradigm) standards. In

be transferred the future, this sort of technology should have a major beneficial impact on non- HPCC computing.

i to aerospace, While the working group has no direct information concerning the amount of

petroleum, and technology transfer to date. it is clearly important to transfer the HPCC commupharmaceutical nity's experience with parallel paradigms to those performing high-end computation

industries, in industries such as aerospace, petroleum, and pharmaceuticals. Many of these in-

| dustries depend heavily on solving computational problems that are quite similar

to those attacked by HPCC researchers, and they can benefit from knowledge of

what has worked well for IIPCC researchers, even if they are not in a position to

make direct use of application software developed in the HPCC program.

Findings

The working group made the following suggestions:

1. Methods should be developed to allow mixing of differentprogramming paradigms

and languages in a sh_gle framework.

Examplesof suchmixingmightincludestandardways ofcallinmgessagepassingFortranfromC++, invokii_mesg sage-passing codefromHPF, orcouplinga heterogeneotslsetofdataparallepro l grams.At thecurrenttime,

vendorsdo notviewthesupportofmixed-programmingparadigmsasa parMixed ticularly high priority. Researchers interested in multidisciplinary problems

programming also expressed a strong interest in large-grained, process-level parallelism with

paradigms hsould subordinate data-parallel modules.

be integrated 2. "Templates," which capture key computational, communication, and I/O

into a unified characteristics of important application codes, should be written and made

framework, widely available.

Templatesarewell-structurdoed,cumentedalgorithmstosolvekeycomputationalproblems.These templatecsouldbe usedtocarryout researctoh

' r

Workshopon Grand ChallengeApplicationsand SoftwareTechnology 35
1

!

--- I I { III j II I 11111191 _ IIII Ill If Ill I III II. 11P!, IIl11|1 ......

Group C - Performance Characterization and Evaduation

develop and evaluate programming paradigms and for multiproce_or performance studies. Several efforts similar in spirit to this are already under way,

including the PBWG group collecting parallel benchmarks, but additional

efforts should be encouraged.

3. Widely available, modular components for software tools should be developed

to speed the implementation of new programming paradigms.

Measures are required to reduce the time and the costs associated with tool

and compiler development because of the rapid pace of hardware development.

Examples of such projects include the ARPA initiatives to develop a common

set of parallel compiler run-time support routines and to develop a compiler

infrastructure.

4. New incentives should be developed to encourage computer scientists and

application scientists to interact more closely.

These incentives could include new awards, sabbatical-in-place arrangements

to allow interaction with applications groups, or specific funding of software development for particular applL'ations. The model of having an "outside" member within the group should also be considered. Most important,

grants must recognize that computer science and application science results

are equally important (and the problems equally challenging).

Group C- Performance Characterization and Evaluation

Chair" Dan Reed', Co-chair' Robert Harrison

What sets performance tools apart from other software tools in the HPCC arena?

The answer is simply that performance is the ultimate reason for using massively

parallel architectures in the first place. Either absolute performance (solving larger

problems faster than previously possible) or price performance (solving the same

problems less expensively) is ultimately the driving rationale for using MPP sysTo understand terns. Thus, achieving a reasonable fraction of the potential peak performance

the per.for_nance of such s system is an esseatial part of the code development process. The deof massively velopment of stable performance analysis tools represents an enabling link in the

evolution of an application from a crude parallel model to an efficient research or

parallel processing industrial code. However, in order for such tools to be used with confidence by

applications a broad user community, they must move beyond research prototypes to become

requires tools robust, commercial-quality products.

focusing on

scalability, Status

load balance,

and program Many tools are available for evaluating performance on a single processor, a few

tools for a moderate number of processors, and almost no tools for a large number

performance, of processors. The problems plaguing Grand Challenge applications developers also

hinder the rapid development of robust performance tools:

s unreliable hardware,

• immature system software, and

36 Workshop on Grand Challenge Applications and Software Technology

|1
ii N II I IN nn nnl| ' I NI I I ill I IN I II Uli N

Work/rigGroups

s incremental delivery of software.

Many researchers are investigating research aspects of performance evaluation

for massively parallel systems. However, not all research groups have the expertise

or interest in producing distributable software, and few of the groups that do have

the interest and expertise have the necessary financial support and personnel to

produce complete, robust tools.

Vendors are announcing new architectures so rapidly that their internal performance tools group have difficulty producing robust, effective toolswboth from rapidity of development and from difficulty in securing the necessary resources. Some

of the vendors realize this problem and are reaching out to the research community,

but there have been few successful technology transfers of high-quality software

from academic or national laboratory research groups.

The result is a paucity of robust performance tools for popular parallel systems.

The perception of the (potential) user community is that this problem is even worse

than it is, with little awareness of what tools are available. For example, there are

several tools for balancing loads across processors, yet several users at the workshop

mentioned the need for such tools: This situation is exacerbated by difficulties in

disseminating performance analysis software:

1. No mechanism exists for exchanging information on tool availability and accessibility.

2. Significant restrictions (stemming from the current legal and funding environment) do inhibit the dissemination of software.

3. Even the researchgroups with the interest and staff to develop near-productionquality tools may not have the interest and staff to continue maintaining and

Information developing these tools, handling user comments and complaints, etc.

about new tools

must be dispersed 4. Many application groups and computer centers do not have the resources/staff to investigate what tools would be useful for the local user community, to

quickly, to enable install tools, and to support Localusers.

researchers to

ezploit new Targets of Opportunity

architectures.

The working group identified four areas meriting special attention.

Support for High-Level Languages. High-level programminglanguages present

special needs for performance measuren_ent tools to support source code optimizaLion by applications programmers. Also, the user interface to performance measurement tools should be accessible through high-level languages. An opportunity

exists for an enhanced relationship between measurement and languages to provide

the user with a more powerful program development environment. Requirements

include

• measurements correlated with high-level semantic constructs of emerging highperforwance computing languages,

s compiler optimizations traceable to source code and performance implications

provided to the programmer.

Workshop on Grand Challenge Applications and Software Technology 37

- I| I
I I I I IIII | I II ] I I I I

Group C- Performuce Characterisation and Evaluation

. automated instrumentation for new languages,

. languages/directives for explicit control of measurements, and

• hardware and software support for access to hardware performance instrumentation.

Examples of these facilities exist, but they shr 'J become standard within HPCC

languqes. The goal is to make measurement information on MPP performance as

accessible as symbolic debugging information is on conventional workstations.

I/O and Distributed Systems. The performance of HPCC systems is increasingly affected by disk I/O and network I/O to external systems such as mass storage, visualization devices, workstation clusters, and other heterogeneous components. Conventionally, consideration of these I/0 capabilities has been segregated

from performance evaluation of computer systems. The large datasets of Grand

Challenge problems and the trend towards application of distributed systems require evaluation of external support systems in combination with primary MPP

Needed are computers. Application programmers must be provided the means for measuring

performance performance parameters affecting their program execution, even when operation is

analysis tools on a remote subsystem. Both processes and data may be distributed among herthat co,sider erogeneous processing platforms. The resulting data accesses or the process communication can limit the overall performance. Access to I/O and remote subsystem

the effects of measurement tools is essential for application programmers to determine the causes

distributed of performance degradation. Additionally, distributed resources tend to be shared,

resources, and delays caused by the resulting contention must be included.

Open Performance Tools and Infrastructure. One possibility for increasing

the impact of current performance tools is to make them easier to use and more readily interoperable within an open performance system infrastructure. Higher-level

languages, more complex component modules, and heterogeneous systems are being

used in the construction of application codes; current performance tool technology

must be applicable to these new target areas. Furthermore, access to performance

data should be provided by language, module, and system developers for integration

into open performance measurement systems.

In contrast to performance tool standardization, we advocate an investment in

performance tool infrastructure development with an emphasis on open technology.

Good tools are It is better to have good tools than standard tools; but if tools are made interoperbetter than able through open interfaces, then good tools could be more easily applied to new

standard tools, target areas. Specifically, we suggest

• interfaces to performance measurements (developed by applic_ttion and system

software module developers);

• performance instrumentation and database components, integrated with compilation systems;

• development of scalable performance presentation technology, accessible to

performance tool builders; and

38 Workshop on Grand Challenge Applications and Software Technology
-- I i i iil|ii ill i jill i i __

WorkingGroup8

• specification of a default Levelof basic performance measurement sad analysis

functionality for parallel systems, providing a basis for transportable performance tools.

Performance Characterization, Modeling, and Experimentation. Performance analysis has both short-term and long-term goals. Typical short-term goals

are to help write and tune programs that run fast for current problems on current

computer systems, and to establish procurement criteria. One long-term goal is

to produce programs that will also perform well in the future (e.g., for larger or

longer problems on computer systems with more processors or different computation/communication tradeoffs). A complementary long-term goal is to help design

computer systems that will effectively support future applications.

All of these goals require determining the performance characteristics of application programs and computer systems. However, currently available tools and

techniques provide only a subset of the capabilities needed to meet these goals.

Current tools are These tools are generally restricted to empirical measurements of a particular comusually specialized, bination of code, system, and sample problems. Little or no support is provided

for developing predictive models of application performance, or even for acquiring

the information needed to construct such models. Additional work is needed characterize massively parallel systems and extract application requirements, as well as

to construct and validate models.

These gaps can be filled, in part by applying existing techniques to a wider

range of applications and systems, and in part by further research collaborations

between computer science researchers, application developers, and vendors. Our

major suggestion is that

• each major MPP class and application community should be supported by at

Leastone research group cl_arteredspecifically to characterize the performance

of important applications on that MPP class, both now and with an eye toward

future scalability, and to develop the tools and techniques necessary to support

such characterization.

Relationships

Are computer science people developing the right tools? The answer to this depends

on whom one asks. Performanco tools such as PICL and Paragraph from Oak Ridge

National Laboratory, PRISM from Thinking Machines Corp., or ATEXPERT from

Cray Research have many etlthusia.stic users in the HPCC community. On the

other hand, some respondents to the workshop survey believe that the state of

programming tools is dismal. Possible reasons for this attitude might be that

• parallel tools, like parallel programs, tend to be quite complex;

• tools are often geared to system developers rather than applications developUser-friendly ers;

tools are essential

• attention spans for learning or relearning tool use are notoriously short;

to increased

productivity. • scalability of performance analysis tools to large numbers of processors is an

open research question:

Workshop on Grand Challenge. Applications and Software Technology 39
t •

i ill iiiI • i i ii i 11 i j i • , i iiill I

Group C - Performance Characterization and Evaluation

• it is very hard to make a tool friendly to all users;

• users are quick to blame the tool (or tool maker) for any problem they encounter; and

• new tools usually lack adequate documentation.

The working group did suggest several new ways to improve collaborations:

• Funding from government agencies is needed to enable applications users, tool

developers, and representatives from the vendor community to work together

to develop and refine tools to make applications developers more productive.

• Technology transfer programs such as that fostered by the Software Capitalization Program from NSF to enable developers to transfer tools from a

research environment to the market place, and therefore to the applications

developer.

• Funding, such as ARPA fellowships, that would enable computer science students to spend 6-8 weeks with applications developers, or students in a scientific field to work with tool developers.

• Education projects for users, such as a catalogue that summarizes available

tools along with extensive help in the uses of the tools, possibly accompanied

by x videotape containing software demonstrations.

• Education projects for tool developers and performance tool users to determine real user needs that would drive development.

Impacts

The HPCC program has already had a dramatic impact on computer science research in general, and performance tools in particular. One measure of this impact

is the widespread interest in MPP systems among academic computer scientists,

which is in marked contrast to their general lack of interest in the previous generation of vector supercomputers. Applications code developers and users, on the other

hand, are also interested in exploring MPP systems, but many remain to be conFor the first time vinced of their practical utility. One benefit of this renewed interest in leading-edge,

in more than high-performance architectures is that for the first time in more than a generation,

a generation, computer scientists and applications scientists have a common interest--namely,

computer MPP systems--that could serve as a basis for substantial collaboration and cross fertilization. Performance analysts and tool developers can play an especially critical

scientists and role in broadening the acceptance and effectiveness of MPP systems for large-scale

applications applications. Because of the immaturity of algorithms and systems software for

scientists have MPP platforms, current users need all the help they can get in realizing the high

a common performance offered by such architectures. Moreover, many potential "converts" to

interest--MPP MPP in industry are reluctant to take the plunge until they know that a mature

systems, environment (including development tools and/or third-party applications codes) will be available to enable them to use these systems effectively.

Most potential industrial users are not inclined to invest time or financial resources in a parallel machine without some proof of principle that they will receive

40 Workshop on Grand Challenge Applications and Software Technology

.... ;_
t_l_ s

ii1|II Illllinl II u I Ill I I I INn I II I I I I

Working Groups

: adequate return through enhanced performance. One approach might be a coilabo-

" ration among a tool developer, a scientific research group, and industrial partners in

'i the co-development of both an application of interest and an emerging performance

analysis tool. Such a collaboration would provide the tool developer with vital feedback on the efficacy of the tool, while the application developers benefit from the

performance tuning that the tool enables. This symbiosis would address one of the

main problems noted in our discussions, namely, that applications developers simply either do not know about performance tools or do not know how to use them.

Remedying thissituationcould have a strongimpact on applicationsdevelopers

' within HPCC and on the competitiveness of U.S. industry as more companies be-

, come convinced that large-scale applications codes can be addressed effectively on

massively parallel machines. Performance characterization can also play s critical

role in the procurement process, particularly in facilitating an appropriate match

between applications requirements and machine characteristics.

Findings

We summarize the suggestions of the working group participants as follows:

• HPCC projects should explicitly fund the exchange of students and staff be-

- tween application and tool developers. Such exchanges will provide training

! for both groups, influence the development and use of performance analysis

Technology tools, and establish long-term research collaborations.

I transfer is a

key aspect of . The collection, maintenance, and distribution of a catalogue of performance tools and demonstrations should be a separate and explicitly funded activity.

developing This includes tutorials on basic concepts of performance evaluation, as well

programming tools as identification of actual tools for specific purposes.

for massively

' - • A permanent funding mechanism (similar to the NSF Software Capitalization

parallel computing. Program) should be established to bring software tools to the point where

technology transfer is both feasible and economically attractive.

• Collaborative development of tools to construct predictive performance models for scalability of Grand Challenge applications should be supported. Such

models are vital in reducing risk and development cost in achieving high performance on current and future architectures.

• In tool R&D, more emphasis should be placed on integrating support for

I/O and networked services, in anticipation of these activities becoming more

criticatol applications.

Group D- Program Development Tools

.I Chair: Fran Berman; Co-chair: L. Ridgway Scott

A vastarrayoftoolsexiststosupportparalleclomputing.These toolscome from

_._ three different sources_computer vendors, the research community, and commercial

|1 software companies. The existing tools fall into a number of categories:

• node compilers (e.g, aative compilers, AC);

Workshop on Grand Challenge Applications and Software Technology 41
IJ[ I I II I[lll _ I I I I II I millII I II I I I I IIIII £ I I I III I __

Group D- Prqr_rn Developmen¢Took

. parallelisation support tools, including data decomposition and mapping tools

(e.g., FORGE90, KAP, PYRROS, HENCE);

• debuggers (node and parallel debuggers);

• parallel programming languages (CM-Fortran, Fortran-M, Fortran-Linda, HPF,

PC N, pFortran, Strand, etc.);

• communication libraries (active messages, CMMD, Express, NX/2, PICL,

PVM, p4, TCGMSG, etc.);

• prograrn/performance analysis tools (e.g. IPS2, PABLO, Paragraph, Upshot,

etc.); and

• math libraries (e.g., ScaLAPACK).

Status

The tools frequently do not address the problems of interest to application programmere, do not function as advertised, and/or do not deliver a significant fraction of

the performance available from the computer. The availability, performance, and

spectrum of tools for parallel computers must be improved across the board: existing tools must work and be robust; new tools must be developed to support

parallelization, MIMD programming, etc. The poor quality or lack of these tools

inhibits the growth and extent of the high-performance computing community.

A uniform complaint voiced by the group was that the software tools provided by

We need the MPP computer vendors are grossly inadequate. The core set of tools available

enchanced on MPPs----operating systems, compilers, debuggers, performance analysis tools--is

mechan/sma to not up to the standards of robustness and performance expected for commercial

turn protoQ/pe computers. Tools are nonexistent or, even worse, do not work. Certain classes of tools were singled out as being in a particularly poor state. Parallel debuggers

tools into received the most criticism. Parallel math libraries were thought to be almost as

well-developed bad, though math libraries are essential for building a mature application software

tools with a base for parallel computing.

broad user base. Commercial and public domain sources have tried to fill the gaps left by the

computer vendors, but have had varying levels of success. Many active research

projects are also producing potentially useful tools, but these efforts are available

only in prototype, or fragmented and buried inside various application efforts. A

key issue for this working group was how to develop these technologies into effective

tools.

Targets of Opportunity

Many tools for parallel software development have been created; however, application programmers often do not know which tools might be applicable, how to get

the tools, or how good they are. We must develop and strengthen mechanisms to

turn research prototypes into well-developed tools with a broad user base.

42 Workshop on Grand ChallengeApplicationsand SoftwareTechnology
_ - .... . I II I Iii II i I i I ii I iii1|1 f I . - ............ [i I i II iflll iiiii__ I i[i i FII 1 I I

Wurking Groups

Basic Tools and Libraries. The primary target of opportunity is to develop

robust, effective versions of basic tools, many of which purportedly exist but are

not usable. This suggestion is in contrast to an approach that looks for sophisticated

"magic bullets" such as HPF---an approach greeted with considerable skepticism by

application programmers within this group.

Infrastructure. A mechanism for the exchange of high-performance software is

critical. This mechanism must be aggressively publicized, nurtured, and used. Support and maintenance for tools in this exchange must be addressed to make this an

effective mechanism for software sharing.

Distributed, Network, aud Heterogeneous Tools and Libraries. One area

in which new tools are seen as necessary by both users and computer scientists

is the burgeoning distributed, network, and heterogeneous (DNH) arena for highperformance computing applications. A clear target of opportunity is to develop

software tools based on experimental efforts in DNH. Homogeneous systems and

very heterogeneous systems represent distinct targets of opportunity because of the

fundamentally distinct research issues they raise.

Distributed Data Management Tools. A critical area of need voiced by almost

every HPCC applications group is tools to support distributed data management.

Significant attention must be devoted to the development of these tools.

Relationships

Applications programmers need parallel software tools that are robust, portable,

and easy to use. In addition, the tools must be very well supported, with clear value

apparent to the users within a short evaluation time. This level of sophistication

requires a substantial development effort.

Unfortunately, it has fallen to the computer science research community to deComputer velop these tools. We say "unfortunately" because academic computer scientists are

scientists must rewarded for basic research, not for software development. Furthermore, computer

be rewarded scientists are not always aware of the problems of interest to application programmers. As a result, a large gap exists between the invention and utility of applicable

for delivering software tools.

the software We believe that the currettt infrastructure for parallel software tool distribution

that makes is fundamentally flawed. Col_l_uter science researchers do not get credit or financial

state-of-the-art support for providing robust well-supported tools. The computer vendors are driven

computers usable, by high-margin hardwar,' sal,,s, with software lost in the shuffle. There are rewards

for delivering the latest at_d g;r,,at,,sr _llassively parallel computer, but not necessarily

in having the software to mak, r 111_., machine really useful. Furthermore, the small

software vendors are limited by having only a small HPCC market on which to

fOCUS.

The result is a gap betweml ram research arena and the marketplace. Therefore, one of our key suggestions is to facilitate technology transfer by providing or

strengthening a supporting infrastructure. Several ways exist to do this. The current approach supported by the H PCC program is to encourage interdisciplinary

teams', this activity should be expanded. In addition, there must be a mechanism for

Workshop on Grand Challenge Applications and Software Technology 43
......

..... JfiI I I Illl I " Ill Ii [i illIIII[INTIIII I Illllll I I ....... II | ] I Ill E I I filllllllIIIII

GroupD - ProgramDevelopmentToolJ

making tools widely available. Tile National Software Exchange is correct in spirit,

but it must be an active agent rather than a passive one. Moreover, it should provide adequate mechanisms for storing codes that have to be supported and updated

(tools) aa well ascodesthat.do not.

Impacts

Researchers have begun to think ill terms of very large problem sizes and data_ts

because of HPCC. In software tools development, researchers are attacking problems

inherent to large concurrent systems.

N_u'- and Long.Term Impact. Currently, individual application groups are

maintaining their own communications libraries and have been layering their own

environment-specific tools on top of them, Duplication of effort abounds. Effective

basic software tools will allow higher-level software to be developed and will expedite

applications research.

In the long term, effective software tools will facilitate the development of libraries serving as a solid base for technology transfer. Although not sufficient, this

is necessary for technology transfer outside of the HPCC program.

Applications' Impact on Computer Science. The computer science community believes that there has been a shift in the emphasis of research funding. Fundsmental research has lost ground to directly applicable research in today's zero-sum

funding environment, The question arises as to whether we really understand the

long-range impact of an implicit reduction of funding for basic research.

Industry's Impact on Applications. Vendors are driven to deliver hardware

before the software is usable. This situation results in unstable platforms, which

A stronger effort in turn have a negative impact on applications. Tool and applications developers

is needed to spend an inordinate amount of time and effort overcoming the hurdles presented by

transfer research an immature system.

results to both

computer vendors Applications' Impact ou Industry. Industry will be interested in high-performanc_

computers only when interesting applications are possible on them. Vendors have

and users, already used mature applications to market vector supercomputers. It is reasonable to assume that applications being developed now will be used in a similar and

effective manner.

Findings

i Our suggestions for improving _lPCC software tools fall into two broad groups. The

first group is concernecl with infrastructure and technology transfer mechanisms; the

second group consists of suggestions for specific tools.

Technology Transfer Infrastructure and Mechanisms. A theme throughout

il this meeting was the need to enhance the infrastructure and mechanisms to transfer

technology from the computer scientis '_to the applications programmer. A key

component of this is software sharing. For a truly effective software sharing system,

it must have the following features:

,r

44 Workshop on Grand Challenge Applications and Software Technology

II
I ...... I III1 I -- [ I II I11 I ' II fl ...... [I]] ,lit i ii ii nlfliil_lfflllm

Workin| Groupm

.,.,___ i T- : ill! : i LIE --- - -- ,lllll ii,llr i, ,,lln ,,,,,, i iilllr, II I [

i

I

• A software-sharing library must be funded to include people who maintain sad

support the software in the library. Hole that the HPCC Software Exchsase

as proposed does not provide such s solution.

• A parallel software tools electronic journal/newsletter (complete with edits.

risl board) should be developed to include codes, tool announcements, sad

publicity. The newsletter needs to be widely distributed within the community.

• Canned demonstrations of tools that are ready for use outside of their development community should be available in software-sharing libraries, so that

application users can evaluate tools quickly.

A great deal of discussion focused on the need for s smoother technology-transfer

from research to the computer vendors as well as to the users. A "bridge" process

is needed and should be funded by HPCC. It is critical that some organisation take

appropriate research community tools and develop them into robust prototypes,

providing support, maintenance, and documentation. Two possible options for the

"bridge organization" are to

• fund and expand the charge of an existing organization such as OSF, the

national laboratories, or the NSF supercompuler centers; or

• set up a new independent organization or industrisl consortium.

The vendors must be strongly encouraged to build sad support tools--especially

stable and adequate tools which would serve as a basis for effective high-level tools.

Vendors must We present two specific suggestions to facilitate this increased support:

be strongly • Word IIFPs to require computer vendors to provide high-performance cornencouraged pilers, and usable tools for MPP machines. Good tools should be made a

to provide requirement for the ve.dors, not an option.

adequate tools.

s Encourage the commercialization of tools that have hadsuccess in the research

community.

Other ideas to improve the availability and quality of high-performance software

tools include the following:

s Establish or fund existing "software capitalization" programs to foster the

development of research ideas and prototypes into production tools.

• Fund a "Grand Challenge in Software Tools." This would recognize that basic

tools for MPPs are needed across all applications. The whole life cycle of tools

is a difficult process and should be recognized as a Grand Challenge in and of

itself.

• Fund a Science and Technology Center for HPCC Software Tools.

s Change the rewardsystem for tools. For example, HPCC could sponsor a tools

competition that would have prestigious awards analogous to the Gordon Bell

prize. Applications users would be the judges, and the tool builders would

have an additional incentive to work on robust, usable tools.

.....

Workshop on Grand Challenge Applications and Software Technology 45
....... , ,nlrlr,f,,r, ...................... • .............. ............................................. I mill[I I

Oroup£ - Bui_BNIMultldi_:ipllnary AppBcstion.

...... __3:_ 3 _ - ne ,rlmllrlrllll I I I I III IIIIII II _ ...... _ I ]IIBI II II I[ H II __ ] ,, , ,,,, , ..............................

Software Tool Development. The utility of advancedsoftwaretools is seriously

undermined by the poor state ofthe basic software tools on MPP computers. This

includes node compilers, operatin| systems, and buic communication routines. Any

suuestions in this section are moot unless these issues are sddrmeed.

We have identified both short-term and Ionl-terms needs for HPCC software

tools. Therefore, we make the following surdKestions:

• Elevtte the qullity and performance of the buic HPCC tools such ss node

compilers, debuuers, and profilers.

• Encourqe st4ndsrds efforts where appropriate. For example, the efforts of

the Messqe Pusin| Interface forum will address sn importer and immediate

need for enhanced software portability.

• Fully fund heteroseneous computins in all of its aspects includins application heteroseneity0 resource m_slement, data msnssement, network management, and matching of tasks to machines. Some of this research needs to

be treated ss "bi| science," with mechanisms for lsrse projects that include

software, applications, and network research.

• Fully fund parallel debu_er efforts. Such efforts must be funded for &llphues

of the life cycle--fundamental research, prototypins, and development of robust tools and standard debuuer interfaces. MIMD and metscomputer debuuers need to be emphMized.

• Develop a compiler infrastructure for ruskin| visible the compiler's internal

,:epresentstions and transformations to other software tools.

Finally, further research is required to develop distributed dsts msnasement tools;

parallel application libraries; data abstractions for parallel architectures; easily layered, object-oriented data abstractions for parallel _rchitectures; multimedia tools

for parallel systems; and operating systems for parallel architectures.

.Group E - Building Multidisciplinary Applications

Ch_h's ILl Foster; Co-chalr: Tom Z_ng

This workin8 group &ddreued the systems integrations issues aA_ociated with

"multidisciplinsay applications," in which either _ver_l different "disciplines" _re

coupled in a sinsle modeling system (e.g., fluids snd structures in sn _ronsutics

code) or several difl'ere.t "levels" of analysis are combined within a single "disci-

/V/an_l modern pline" (e.g., linear, Euler, and Navier.Stokes aerodynamics). Issues concerning the

simulation internals of individual discipline modules were not considered germane to the group.

projects could

benefit greatIF Status

from the use of Various applications were represented in the working group, including aeronautics,

integrated codes geophysics (environmental and global climate modeling), biological systems, and

such aa the earth plasma physics. Chemistry, materials, astrophysics, and quantum chromodynamics

system models, were not represented.

There was a surprising commonality among the different fields in terms of both

status and obstacles to progress. In both aeronautics and geophysics, integrated

46 Workshop on Grand Challe.ge Applications and Software Technology
_LI I I I I ....... IHI I Ill ITIIII|IIH -- II III I III .......... I IIII]11--1 _ I II I II III II1H I Ill[li I ......

Workm| GroupJ

_ i _ ± i. ill [ . ill, iiil i i i [ _rl i _lil[i, i _ ii ii . _ i i i i! ]ll --- i - i i[1 - i ii ii

codes have been operational o. Cray ,upercomputers for several years. Typically,

these are limited to relatively simplified linkages among s few components, with a

,trong fluids component, Codes have also been developed in heterogeneous, networked environments. Examples of such intellrated codes include the following',

• Simulation of aircraft, both a high-performance aircraft and a high-speed civil

transport together with their propulsion systems, incorporating fluid dynamics, structural mechanics, surface heating, and controls models.

• Environmental model, incorporating atmospheric dynamics, surface water,

and groundwater models.

; • Earth system models incorporatinlt atmospheric dynamics, atmospheric them-

_ istry, biosphere, and ocean dynamics models.

• Predictive computational models of cellular organelles, coupling fluid dynamics, rigid body mechanics, molecular dynamics models, etc.

In all these arems, there is strong interest in developing increasingly sophisticated systems that to.pie ever snore advanced simulations of more diverse physical

A common data systems... However, progress is limited by two factors: computational limitations

mode/would of current ,upercomputers, and limited understanding of the phyiics and numeric;

. capture data of the interfaces betwee, components, especially with respect to stability. Mr P

from indiv4dua[ computers are seen msa potential solution to the first problem, but prolpres_ in this area will necessarily incremse the pressure on applications scientists to improve

discipline8 in a understanding of coupling issues.

unified framework. Multidisciplinary applications on MPP computers are for the most part underdeveloped, primarily because of the difficulties inherent in adapting not just one but

a suite of existing codes for efficient parallel execution. This is a severe problem.

Indeed, ma.y of the group members felt that this may well be _Aepacing item in

demonstrations of HPCCP tech.ology for real-world Grand Challenge problems.

Targets of Opportunity

The consensus was that opport..ities exist for enhanced activities in four areas:

common data models a.d ,latab_qe systems, languages and tools for developing

muir*disciplinary applicatio.s, application design optimization, and education.

Data Models a.d Databa.o Systelll,. Data management is very much an

i- enabling technology for mutt*disciplinary design and analysis, The current state of

_, the art i.volves a series of custom-designed data "translators" that map information

between various applicatio.s. These are expensive to generate and require extensive

:_ validation. The opportunity for errors entering the multidisciplinary design process

are greatly amplified across each application interface.

A preferred data manag_me.t approach would involve a "common data model"

; representation. This commo, model would capture individual discipline data fields

within a u.ifled representatio, that provides consistent information at the boundaries of each discipli.e. Disciplil_ary data is "owned" by each discipline and available

as limited subsets of the unified representation. This information can be distributed

f across a variety of computi.lg platforms. Data access requirements, as determined

Workshop on Grand Challe.ge Applications and Software Technology 4T

' _ • t ", -Wlr_".I.maL -- _ IIII Illlll _ IIII III HI
_ _ L illlillll ...... tillIII I I TIlillll]liltII ] I il[mlfl]l 1] It ................. IIIIHIi " - Illilll III - 11 IlIlII ] li . ....

Group E - Build,n| Mu|tid_:._ip]inaryApp_cat_ons

_ -- .... _i_ ..... • ............................... ,, ,,, ,i, ,,,,,

I

by the "physics"of disciplineinterfaces, define data protocols. Where it is approprialm,object-oriented data hierarchiesare used to organiseand minimisedata

s_r&ge while providinglinkagesinto disciplineapplications.This approachis facilitated by the useof geometrymodelingstandardsanddata standards.Forexample,

in aeronautics,the Non-Uniform Rational B-Splinesstandard(msembodiedin the

imminent NASA-IGES formal standard) enables linkagesbetween CAD systems,

grid generators,visualisationtools, and application codes. Finally, the database

managermust organiseinformation acrossa variety of hardware platforms while

maintaining "virtual" ownershipon perhaps a single workstation. Efficient access

to a heterogeneousmixture of computingplatforms (MPP to workstation)isneeded.

In summary, key pointsin this area are

. useof a commondata modelamongdisciplines,

. useof an object-orienteddata hierarchyto organisedata,

, standard geometryand data representations,and

. platform independel_ce.

Languages and Tools. Better languagesand tools are urgently required to support developmentof multidisciplinaryapplications. Currently, the vast majority of

componentprogramsare written in Fortran77. Couplingisachievedin a varietyof

ad hoc ways,including (a) creationof a singleexecutable,(b) flies, (c) specialized

shared-memorydatabaseroutines,(d) message.pmmingin a network, (e) AVS, and

(f) intercube communicationon a hypercube. In the future, an increasingnumber

of componentswill be written in C, C++, and Fortran 90.

There was a strong consensuswithin the workinggroup that languageswere

urgently requiredthat addressedissuesor'encapsulationm, odularity, and portability

in MPP and networkenvironments. Current meuage-pmmingsystemsare seenas

tooclumsy,and eitherspecializedlanguagesorgraphicalprogramming systemsare

To,sk-par'allel required. The needto support integration of both _._k- and data-parallel modules

and data.parallel wu emphasized,as was the need for tools that permit MPPs to be partitioned

modules should amongdisciplines,with simpleand efficientcommunicationbetweenpartitions.

be integrated. There was considerable debate within the working group as to the importance of object-oriented programming techniques for multidisciplinary applications. Several

argued that the abstraction capabilities of C++ were an important technology

that could be used to structure complex multidisciplinary applications codes and

encourage code reuse; others disagreed. One member argued for object-oriented

extensions to Fortran 90 that supported inheritance, by means of a preprocessor.

Two major topics of discussion were language interoperability and standards.

For example, it is increasingly common for software developers to use C++ wrappers

to call Fortran 77 or Fortran 90 routines. A standard procedural C+-I-/Fortran

interface would be very helpful. The reverse interface---from Fortran 90 to C++--

would also be useful for databa._e access and graphics libraries. Standard messagepassing systems were also seen as important.

In summary, key points in this area are the need for

. languages and tools for systems integration aspects of multidisciplinary applications on MPP and networked computers,

4S Workshop on Grand Challenge Applications and Software Technology
w

!i; ' °

|

!

in ......

I I I I II iini i m " - __1 iii _ ii ilmiiii iii .......... iiI_iT ill II IIIH I -

WorkingGroups

s languages and tools for integrating task and data parallelism, and

s standard interfaces between different languages.

Applications Design Optimization. Multidisciplinary applications in both Grand

Input from Challenge projects and ind,stry possess significant complexity. They may involve

industry is needed hundreds or thousands of design variables and tens or hundreds of modules, In

about ways to manufacturing, the move towards concurrent engineering introduces many other

considerations, since manufacturing or marketing considerations may not be deimprove design scribabh by continuum models.

• optimization A strong need exists for improved optimization methodologies and software sup

methodologies, port for such complex problems. The working group did not feel that its members

represented a sufficiently wide range of expertise to make detailed suggestions in

this area. Input from industry, universities, and government labs is urgently required to define goals in this area, and this input should be taken into account in

defining HPCC program directions.

Relationships

Communication difficulties, both between applications scientists and computer scientists and between application scientists working in different disciplines, were seen

Multidiaciplinary as hindering progress in multidisciplinary applications. In addition, several speak-

, approaches are ers alluded to lack of experience in modern software engineering techniques among

likely to become scientist_ developing components of multidisciplinary models. It was also felt that

new graduates, while often more exposed to new techniques, lack experience in

d dominant in more multidisciplinary research and industrial software engineering problems. Several

fields of science solutions were discussed, including (a) an increased emphasis on multidisciplinary

and engineering, degree programs (Georgia Tech and Virginia Polytechnic Institute were cited as

I examples of successful programs), (b) improving training in software engineering

for science majors, and (c) programs that involve students in software development

projects at labs and companies.

The working group expressed concern about the apparent lack of interest among

computer scientists in the problems facing developers of multidisciplinary applications. The current focus on getting single-discipline modules running on MPP computers is understandable and justified. However, multidisciplinary approaches are

likely to become dominant in most fields of science and engineering, and better tools

are urgently required. This situation in turn requires increased dialogue between

computer scientists and applications developers. The working group emphasizes

that good tools will be developed only in a multidisciplinary technical environment,

in which groups having l_ighly diverse combinations of computer scientists, applications developers, etc., tackle the needed research in this area.

The working group was also concerned with the difficulties facing multidisciplinary applications developers who wish to learn about and evaluate new languages

and tools. Currently, information is primarily disseminated by word of mouth; betTemplates are ter mechanisms are urgently needed. One approach would be to prepare a set of

t the key to reuse "template" codes representative of multidisciplinary applications concerns and to

of parallel code. encourage software tool developers to use these templates when evaluating their

systems.

. Workshop on Grand C'hMlengc Applications and Software Technology 49

i

_t _ 0 • _ III
I I I _ IIIIIII II I I I [ III I III_II III I II III II [I

Gro.p E -B.Vdl.| M.It_d_p_nJty App_cJtioas

Impacts

The working group agreed that it is too early to expect the activities of the HPCC

program to have had significant impact on science and industry. However, there

was broad consensus that by allowing simulation of entire systems rather than

individual components, the combination of high-performance MPP computers and

improved techniques for coupling difl.erent disciplines can be expected to enable

significant progress in many areas of science and engineering. We give just three

examples of where such "whole system" simulations are planned; each is dependent

on availability of the greatly increased computational capabilities that are being

developed in the HPCC program.

. The aerospace industry is implementing integrated product/protein development (IPPD). A key element of IPPD is the development of total system

simulations that allow the substitution of computational prototypes for physical prototypes. This appro_h could substantially reduce design cycle time

and development cost, and improve product q.ality.

. In the environmental sciences, integrated earth system models could greatly

improve the timeliness and scientific foundation of data, motivating important

environmental decisions such as mitigation strategies for global change.

. In biophysics, models that integrate multiple levels of knowledge about protein structure would allow new approaches to the problem of predicting the three-

' The availability dimensional shapes adopted by complex proteins. Solution of this proteinof high.quality folding problem would have great significance, both for science end for drug

numerical design and manufacture.

algorith and

iibrarie_ can Findings

determine Zhe pace The working group reached consensus on the following suggestions, which are given

of applications equal priority.

development.

1. Steps should be taken to strengthen connections between multidi_iplinary

applications and database communities. New research in this area should be

initiated if necessary technology is not already available.

2. The HPCC program should encourage both research on languages and tools

for the systems integration aspects of multidisciplinary applications, and the

timely dissemination of the results of this research to the multidisciplinary

applications community. In particular,

. languages and tools for coordinating execution of sequential, parallel,

and data-parallel components on MPP and networked computers should

be made a research priority;

. a framework should be established for the evaluation of such tools in

multidisciplinary applications; and

. standard interfaces should be defined between programming languages

commonly used for development of multidisciplinary components (e.g.,

Fortran 90 and C++).

50 Workshop on Grand Challe,ge Applications and Software Technology

'11..... II _ I _ I _ III II _ IIII _ A__ i
r •

IIIlIlI I I II I I I

Working Groups

3. Industry and university input should be sought to define HPCC program

directions in applications design optimization.

4. A program should be established for retraining current scientists and engineers

in critical RPCC mult.idisciplinary technologies, and the university community

should be encouraged to establish or strengthen multidisciplinary programs

and curricula.

In addition, several group members suggested that researchers working on loadbalancing techniques, visualization tools, and parallel debuggers be encouraged to

consider the particular requirements of multidisciplinary applications.

Group F- Algorithms and Libraries I

Chair: Ahmed Sameh; Co.chair: Phillip Colella

The lack of high-quality numerical and symbolic algorithms and libraries often

limits the pace of applications development. This situation is particularly true on

massively parallel machines. Group F considered the state of the art in algorithms,

libraries, and software for the numerical solution of partial differential equations

that arise in various applications, and the associated linear algebra algorithms and

Flezible parallel kernels. The majority of the members of this break-out session were application

languages will scientists who are not only experts in their applications domains but also seasoned

allow users to computational scientists quite familiar with computer architecture and system softmatch their ware, as well as with the design and implementation of numerical and nonnumerical

algorithms. Furthermore, ahnost all of them are associated with interdisciplinary !

algorithms activities that involve computer scientists and applied mathematicians. In spite of

i to a specific this, our deliberations considered the needs of applications scientists who do not

architecture, have the benefit of such expertise or the benefit of the involvement of the computer

science and the applied mathematics communities.

: Status

The working group focused on three areas: research activities, software-sharing

mechanisms, and federal support.

Major Research Activities. Grand Challenge teams assembled at national lab-

, oratories, universities, and a few industries represent a small number of computa,-

tionalscientistswho aresophisticatedusersofadvanced computer architectureands

who are willingto work hard at implementationsthat resultin high performance.

While theHPCC program isnotlimitedtoparallelcomputing,thedesignofparallel

algorithmsfora varietyofapplicationsisa vitalcomponent oftheprogram.

For some applicationdomains,parallelalgorithmscan be providedby generic

mathematicalsoftwarelibrariesthat are finelytuned fora few architectureand s

that containsome of the basicalgorithmsfor FFTs, linearalgebrakernels,and

algorithmsfordenseand sparsematrixcomputations.While such librariemay s be

sufficiefntorsome applicationsscientists, however,however,they arefarfrom being

satisfactory for sophisticated users or for users seeking high performance. Factors

that limit the usability of such generic libraries include limited functionality, inappropriate data structures, and lack of algorithm scalability. This situation shows

Workshop on Grand Challenge Applications and Software Technology 51

,, ,, , _, ........
i I ,i i i i i i i i i __

Group F - Algorithms and Libraries !

the need forparallellanguagesthatallowsophisticateusers d to match theiralgorithmstothearchitecturaethand,namely,by selectinagppropriatedata structures

and by matchingdata movements toarchitecturesFo. r lesssophisticateudsers,it

isadvantageousto incorporatesuch featuresin the compileras well.In addition

to the need forparallellanguageswith the above flexibility, the group pointed

out the need for (i)application-specific librarieths at may includemesh generationtools,discretizatitool on s,solutionand update tools,and mesh refinementand

load-rebalancintool g s,and (ii)application-specific languagessuch as FIDIL for

computationalfluiddynamics. Applicationsscientistwsho arereluctantor unable

tomake theinvestmentrequiredtocreatelarge-scaleapplicationcsodesthatachieve

the highestperformancepossibleforeach new architecturethat comes alongmust

have availablgeenericorapplication-specilificbrariesofvalueforrapidlyproducing

prototypecodesthat testor verifycomputationalmodels. This rapidprototyping

may then be followedby incrementalrefinementtoenhance performancefora particulararchitectureSu. ch refinementsshouldentailfurtheralgorithm/architecture

matching,includingparallelcommunication utilitiesforchangingdata structures

and implementingothercommon communication tasks.

Software-Sharing Mecha,isms. Softwaresharingoccursmainly between cooperatinggroups.Applicationscientistsare reluctanttoexchangesoftwarefortwo

major reasons:

s The softwareisexperimentaland oftenpoorlydocumented. Ifsuch software

isnot used properlyby others,itmay reflecbtadlyon the investigator.

• Distributionand maintenanceof large-scalaepplicationcodes isa very expensiveactivity.Researchersatuniversitiesand nationallaboratoriesarenot

fundedexplicitlyforprovidinghigh-qualitysharablesoftwarethatcan be used

by othersor forprovidingconsultation itsuse.

One approach isto fund activitiesat some institutionwshere applicationscientistsareworking closelyenough with systemssoftwaregroups thatare capable

ofsupportingand maintainingtheresultinsog ftwarefora much widercommunity.

Softwarethat has a largeenough market should be transferredto an interested

Software with industrial partner.

a large market Sharing mechanisms may not be limited only to complete software packages;

should be templates (or pseudo-code representations) of general algorithms that are languageand architecture-independent may be of great value as well. transferred to an

industrial partner. Federal Support. The federal HPCC program has concentrated mainly on Grand

Challenge problems and teams to solve them. Concern has been expressed that

support for the underlying software infrastructure may be inadequate. By software

infrastructure, the group means (i) system software for MOP's (i.e., languages, compilers, and communication primitives); (ii) research in new application-specific algorithms and the resulting software on a variety of architectures; and (iii) applicationspecific languages. Also, as mentioned above, additional funding is needed for supporting the maintenance, distribution, and consulting activities for applications

software of value to a large group of users in the national laboratoriec and universities. Industry involvement is highly desirable in the distribution of higher-quality

versions of such software.

52 Workshop on Grand Challenge Applications and Software Technology
• -- I I II Iill I i I i i I

Working Groups

Targets of Opportunity

Several areas of research need to be strengthened, while others need to be initiated:

* Algorithm and software developments in application-specific domains, including parallel computation and communication algorithms.

• System software that allows easier and more efficient use of parallel architectures. Such software should include language notation for data mapping, good

Computer compilers, and effective debugging tools.

vendors can help • Application-specific languages that increase the level of abstraction sufficiently

interdisciplinary so as to respond to architectural changes quickly and, if this language can be

teams, compiled efficiently, without loss of performance.

Relationships

The group believes that the HPCC initiative deserves credit in fostering interaction

between the computer science community and the applications scientists. While

most members of this break-out session belong to both communities, roughly half

expressed the need for more cooperation between these two communities in their

own institutions. The HPCC program has started to change the culture in several

science and engineering disciplines by demonstrating that computation is just as

important as theory and experiments in advancing these fields.

The most important barrier to interaction between computer and applications

scientists is the cost of entry. Most computer scientists have no background in any

of the classical science and engineering disciplines, and the majority of engineers

and scientists are not aware of the most advanced parallel computer architectures or

state-of-the-art algorithms for these computers. In fact, it has been demonstrated

that application experts can more successfully bridge this gap than computer scientists. One approach is to prepare graduate students in the interdisciplinary field

of computational science and engineering, which is the interface between computer

science and the classical disciplines of science and engineering. Resistance to establishing such new programs in various universities may be overcome by federal

funding of the most promising programs. Also, computational scientists (who are

amply represented in the Grand Challenge teams at national laboratories) need to

gain the confidence of their experimental colleagues by seeking experimental validation of their computational results.

Interaction with industry (including computer manufacturers) is vital to the

goals of the HPCC initiative. It is through industrial interaction that the knowlStaff ezchange edge from the Grand Challenge teams call be disseminated. Members of the group

programs should also pointed that computer manufacturers can be of help to interdisciplinary teams

be endorsed, involved in the research activities outlined above by providing access to dedicated

machines and assistance in implementing Grand Challenge software on their machines. Access to dedicated machines is essential, as the development of finely tuned

' algorithms and whole applications codes require performance-monitoring and measurement tools that are intrusive in multiuser environments. Expansion of existing

approaches, or development of new ones, for working with industry is vital if the

HPCC program is to have an impact on industrial competitiveness. With computer

manufacturers, these approaches could facilitate more effective computer architectures and systems software, as well as the development of applications software and

Workshop on Grand Challenge Applications and Software Technology 53
-- II II I I I iliglli Ill II I II i illl __

Croup F - A_otithmm and £ibr_i_ I

underlying software libraries on new machines. Collaborations with other industries

will facilitate technology transfer to those industries that can substantially benefit

from the technology produced in the HPCC program. Such technology transfer

can be significantly enhanced through exchange of researchers among universities,

national laboratories, computer manufacturers, and industry, in general.

Impacts

The group identified several ways in which the HPCC program has had a major

impact on algorithm and library development:

• The HPCC program has provided a mechanism for funding state-of-the-art

applications software. Industry involvement in this program needs to be

strengthened.

• The program has dramatically increased the number of scientists and computstional scientists with hands-on experience with parallel computers. This

represents sn advantage to U.S. research and development similar to that

provided by early experience with compilers and operating systems several

decades ago. It is imperative, however, that continued and increased support

be provided for educational programs in computational science sad engineering so as to ensure a permanent supply of researchers who combine both

application and computer science expertise.

• Parallel algorithms have been beneficial to most of the Grand Challenge applications represented in this session. In fact, several application scientists

observed that developing parallel algorithms for their applications often leads

to faster execution on uniprocessors. Since massively parallel systems are not

mature, however, more time and resources are needed before the impact of

parallel algorithms on applications is fully realized on such architectures.

• Grand Challenge problems have led to increased understanding and use of

broad-band communication structures and large distributed databases.

Findings

With respect to technical issues, we suggest that application-specific systems software and libraries be developed so as to provide a stable platform on which Grand

Applications. Challenge applications can be built. Such platforms should address

specific libraries

• the development and implementation of allforithms in support of applications,

must provide a including parallel communication algorithIlas;

stable platform for

Grand Challenge • high-level language support for data mapping and data communication; and

applications. • increasing programming productivity, for example, by application-specific languages.

With respect to institutional issues, we make four suggestions:

• enhance and expand technology transfer through scientists exchange and extended collaborative efforts on specific end-user applications;

54 Workshopon Grand ChallengeApplicationands SoftwareTechnology
I iiiiili I I i i il i ili iiii i iiii "

Work/ng Groups

• promote interagency and intermission activities in application-specific algorithms (e.g., PDE algorithms across several disciplines);

• redress the HPCC program imbalance: specifically, fund manufacturer, applications, and systems collaborations for developing software infrastructure;

and

• support educational programs in computational science and engineering or in

scientific computing.

Group G - Algorithms and Libraries II

Chair: _an Sullivan; Co-chalr: Bob Sugar

The development and implementation of algorithms for massively parallel computers are important components of the HPCC program and should receive greater

emphasis than they currently do. This is an area in which collaborations between

computer scientists and applications scientists can yield major dividends. Such dividends have already begun to be realized within the Grand Challenge applications

groups. As indicated below, however, we believe that there are a variety of problems on which new collaborative efforts are needed. New approaches are also needed

for sharing algorithmic ideas and software. In addition, vendors must pay greater

attention to the needs of the applications teams if the use of ma_ively parallel

machines is to progress.

Status

Grand Challenge applications groups are developing algorithms for specific machines

l

and writing optimized code to implement them. A number of groups are working

Modern on closely related problems and might benefit from greater communication.

simulations Existing software-sharing mechanisms are not working well and not rapidly

often require enough to be of real use to the individual Grand Challenge participants. There was

new algorithms a strong consensus that conventional libraries were not at all appropriate to the

i rapidlyevolvingneedsofthe Grand Challengecommunity. Codes aredevelopedby thathavenot the teams themselves,sad theteams cannot findtime toproducewell-documented,

been extensivel_l distribution-quality programs. The usual vendors such as IMSL cannot contemplate

used incomputing scientific , eontrycodeintooptimization a very thin for market individual with famachi st-evolving nes makes algorport ithmasbl. e Thlibraries e strongimpractical emphasis

at this time.

Targets of Opportunity

In the contextofthe HPC.C,program and itsassociatedGrand Challengeapplications,the followinspg ecificareasofresearchwere identified:

• Algorithmsfordecompositionofregularand irregulardata structuresarising

inPDE discretizationto obtainefficientloadbalancing,and fastalgorithms

forsolvingellipticPDE on thesetypesofstructures.Of particularneed are

efficientmethods thatcan be used with adaptivemethods where thesetypes

ofload-balancinissu g esmust be performed dynamicallyasthe discretization

adapts tothesolutionstructure.

Workshop on Grand Challenge Applications and Software Technology 55
Q •

I UlIIIiiI1 iii iii _ ii i i i

Group G - Algorithms and Libraries II

• Parallel random number generators with appropriate speed and correlation

properties. Correlation of random numbers across processors has also been a

major source of concern. Software to test these programs is also needed. Randomized algorithms are a major technique in algorithms research and often are

the only effective way to overcome NP-hard problems. For such algorithms,

the availability of parallel random number generators is fundamental.

• Symbolic and combinatorial (as opposed to floating point) algorithms. Symbolic algorithms are currently useful in a variety of local computations within

Grand Challenge problems. We expect combinatorial algorithms, numeric

(but not necessarily floating-point) computations, and other symbolic methods to gain in importance in such fields as biotechnology in the coming years.

• New algorithms for irregular problems. Sparse systems and irregular patterns

of data are not being addressed by the algorithms that are currently available.

In order for these systems to be useful, good parallel algorithms are needed

to handle these structures.

• General multidimensional pattern recognition and pattern-matching algorithms.

These will provide informational structures to analyze informational structures in higher-dimensional spaces, which are useful, for example, in threedimensional molecular structure databases.

Robust, correct implementations of these algorithms would be extremely useful

and, on the whole, are not available. However, work on the algorithms themselves

is as important as work on the implementations.

The HPCC should provide strong encouragement and incentives to vendors to

develop robust software environments for their MPP hardware (concurrent with

hardware delivery). Lack of this support has significantly hampered algorithm

development and implementation on these machines.

Relationships

Although most researchers have the feeling that the Grand Challenge community

(GCC) is not reinventing techniques already known to the computer science community (CSC), the true situation is the opposite: the GCC is, in fact, duplicating work

It is imperative several times over. Two factors lead to this situation. First, applications scientists

that the Grand believe that they require specialized algorithms for each application, so that general

Challenge methods cannot be used. Second, and more distressing, large segments of the CSC

community not have decided that their discipline can and should be isolated from the messy, real- world problems of Grand Challenges. While the GCC is making an effort to learn

reinvent computer what is already known, parochial issues from both sides hinder such efforts. The

science techniques, recent NRC report "Computing the Future" strongly suggests that computer science should, to a much greater extent than is now the case, derive inspiration from

real-world applications such as the Grand Challenges. We repeat that suggestion

and strongly support the collaborative interdisciplinary requirements of the HPCC

Grand Challenge program. These have the salutary effects of allowing the GCC to

become aware of the work of computer scientists and of bringing that additional

talent in computation to bear on Grand Challenge problems.

56 Workshop on Grand Challenge Applications and Software Technology
IIIII iiii I I ii I I I

ii I I iiii I i I i II I II I iiii iii I II

Workins Groups i i i. IH .i i

We note with caution that there are significant problems in the universities for

junior faculty in computer science who choose to work on applications. It is not always possible for them to be rewarded for doing "computer science" for such work.

Part of this is due to tile departmental structure of the universities--a problem

not generally shared by the laboratories. Part is clue to a tendency of alsorithmists in computer science to work on theoretically elegant rather than real-world

problems. We approve of current efforts to remediate the problem. There have

been several multidiscipiinary programs recently begun in computational science at

several universities, and there is a small but growing fraction of the CSC working

with Grand Challenge-like problems. We encourage increased interactions between

the national and industrial labs and the faculty and graduate students in computer

science. Another useful activity would be to organize open research conferences on

topics related to Grand Challenge problems.

Impacts

A short-term impact of the HPCC program has been in providing massively parallel

computers to Grand Challenge research teams. This has had the effect of promoting

parallel computing as an industry. In addition, the Grand Challenge teams are quite

sophisticated and have been able to affect the usability of these computer systems.

By striving for performance on immature systems, they have uncovered software and

hardware inadequacies in these systems, and vendors have used this information to

improve their systems. Thus the research teams benefit the entire community by

informing the vendors of mistakes and pitfalls at early stages of the development

HPCC research cycle. One hopes that the teams will also encourage the vendors to provide a stable

provides valuable and understandable set of development and optimization tools. Such tools, in turn,

feedback to would smooth the way for the rest of the scientific computing community as they

vendors early in try to enter the massively parallel computing field.

Another important impact has been on high-technology industries. It is interestthe design cycle, ing to note that workstation vendors are designing multiprocessor machines. IBM,

Sun, HP, and SGI have all started to provide high-end multiprocessor workstations.

In addition, some other industries have begun to use parallel computing in their own

research and development. These include the oil industry, where multiprocessors

are used for exploration and analysis of seismic data; the aerospace industry, in collaboration with NASA, for the design of new aircraft; and the materials industry, in

collaboration with DOE, for the computational analysis of materials properties. It

is hoped that the continued success of these efforts will lead to the desired long-term

impact of HPCC: improved industrial competitiveness.

Another objective of HPCC is to foster collaboration between the computer

and application science communities. To date, increased collaboration has occurred

among members of Grand Challenge teams, but the impact of HPCC directly on

The trend is computer science has been small. There does seem to be a trend in computer

clearly toward science towards the experimental, and the theoretical community is considering

more realistic more realistic models of computation than the PRAM. Nevertheless, the computer

science community is still perceived as too theoretical for the immediate needs of

models of HPCC.

computation. Finally, while most of the participaats believe that MPPs are the future trend,

they noted that some short-tet'm goals of the Grand Challenge projects still require

the more mature techtmiogy o["vector machines. Some researchers feel that it is too

Workshop on Grand Challenge Applications and Software Technology 57
II IIIIIII I I I!1 I I I I I I [ 1 I III III i i ii i i|llllr i iiiiiii1[i ............ lUl i ii i

Group G - Algorithms sad Libruies 1!

soon to assume that the continued health of the computer industry will come only

from massive parallelism.

Findings

To facilitate work on software development, vendors should

1. balance memory bandwidth with CPU speed;

2. balance communication speed with CPU speed;

3. provide good node compilers to allow programmers to make more effective use

of machines without resorting to assembly language coding;

4. provide fast multiple-precision arithmetic (the predominant techniques are

still numeric and greater size of problems demands greater precision. It should

be noted that some participants disagreed on the importance of this su_estion, while others find multiple-precision arithmetic essential to doing serious

work on these machines);

5. pay greater attention to symbolic and combinatorial aspects of computing,

such as fixed-point arithmetic;

6. provide language constructs that allow the easy definition of complex data 1

structures (C++ is an example of a high level language that allows this);

7. provide high-level standardized message passing routines', and

8. provide a random number generator with their machines.

The problem of sharing of information among scientists is very important. How- i

ever, we do not think that the massively parallel field is mature enough yet to

[n/orma[ libraries warrant portable standardized libraries. Portable libraries would not be efficient

should be set up enough to be used widely and so are not reasonable at the moment. However, we

at the national strongly surest establishing some system of informal libraries with a different set

supercomputer for each machine class. These informal libraries would include algorithms, tem- plates, and code. The library servers could be set up at different servers in different

centers, places (perhaps for different machines). The national supercomputer centers would

be logical choices for locating such servers.

Finally, we suggest that the current practice of requiring computer scientists on

Grand Challenge teams be continued. This practice has stimulated interactions between these groups. Efforts are also needed to increase interactions among scientists

in the academic, laboratory, and industrial communities. Although the HPCC has

helped to bring the first two closer together, greater efforts are needed to increase

the participation of the industrial community. This is essential if the program is to

succeed in its important goal of improving U.S. competitiveness.

58 Workshop on Grand Challenge Applications and Software Technology i
11 IIII I _ ................. iiliiililil _TIi ]LJ_ Ilii I I il I i II iililiiiiiiill 11

Workin_ Groups

. .H., i, - _j, ,n.l L

Group H- Graphics and Visualization

Chair: Paul Woodward; Co-chair: Tom DeFanti

Status

The scientific visualization working group reviewed the current status of software

and hardware development activities designed to support the high-performance

Craphics computing community, One of the principal trends in this area is the development

and other of specialized systems and environments for the interactive exploration of very large

visualizations datasets. Grand Challenge computations are often characterized by either the use of

or the generation of datasets of a terabyte or more. These datasets are so large that arc crucial to

visual representations are often the only means available to examine the data. Even

modern scientific with image representations, the data_sts are still so large that many views must be

applications, generated. Several groups are engaged in developing tools that enable a researcher

to explore such vast dataaets, bringing into the computer or workstation memory

only small subsets of tile data, which are specified interactively by the researcher,

and then rendering them in any of a number of ways in real time. Developing such

data exploration hardware and software systems places extreme demands upon the

computer network, [/0 subsystems, irr,age-rendering engines, and even the display

subsystem if the data is to be displayed at full resolution.

The working group noted efforts at several centers to develop image.renderinf

8oflware that can be executed or massively parallel processing systems. The need for

this software is driven by the immense size of the datasets to be visualized. Often,

the dataset can be mounted and dealt with only on the MPP system that created

it. Even when other systems can access the data_et over the network, rendering

of images on the MPP may be required if only this system has sufficient rendering

speed to produce an image of the data in a reasonable time.

The development of software to support scientific visualization in distributed,

heterogeneous computing enviro_Jmenis is proceeding rapidly. This activity is vital

if visualization requirements of researchers remote from the Grand Challenge computing hardware are to be met. However, the working group noted that much of

this effort is currently aimed at visualization of significantly much smaller datasets

than are common in the Grand Challenge applications community. Special efforts

of this sort targeted at support for Grand Challenge applications are needed in the

future.

' The working group also discussed research activities in developing virtual realzty environments in relation to the Grand Challenge applications. Both immersive

[ Visualization displays and large flat-screen displays were discussed. The principal advantage of

i

can produce these efforts for Grand Challenge applications was agreed to be their potential help

comprehensible in dealing with the visualizations of extremely complex systems. Both approaches

1 views of real offer high-resolution displays tltat can, if the accompanying I/O and data-rendering

I systems are large and fast enough, put onto a single display much more than the

t assysauttems omobilessuch smtomealld. windo In ewitherof c,, thtese woa frkamistlaiatironmechanism screen to iswhsupp ichorrese ted arcfohrersnavihav gatine gbecomthroeughaccuthesand airplanes, dataset, that of walking for the immersive systems, and that of driving (or flying)

i for the large-screen systems.

A brief discussion of fundamental research in visualization techniques noted

i that the NSF Science arid Technology Center for Computer Graphics and Scientific

I

i

I Workshop on Grand Challenge Applications and Software Technology 59

i
............. I lllmll I I HINNIInrlNIIIll Illll|lll I I I I I I[ II IIIIIIII1 Illl _ ...............

Group H - C;raphicsand VisuJJisaiion

Visualisationis developingnew renderingtechniquesthat are moresoundlybs_d "_

upon the physics of radiative Irans/er and scatier_ng. These new techniques should

be capable of producing much more photorealistic images of real systems, such as

automobiles and airplanes. The usefulness of these techniques in enhancing the _

perception of physical systems that cannot normally be seen by the eye, such as _'_

complex molecules or the structure of the earth's magnetosphere, remains to be

seen. Presumably, success will depend upon the extent to which familiar visual

cues, such as surface textures, can be employed to convey significant scientific data

in these unfamiliar settings.

Use of scientific visualization for the remote steering of instruments such as microscopes and telescopes was also noted as an important current activity. Remote

steering of supercomputer simulations, although possible using visualisation techniques to develop the user interface, is not yet common in this community because

of the enormous amount of computer time required for a single Grand Challenge

run.

Finally, the workinggroup notedthe tremendoususefulnessof scientificvisualization in education, tr_inin 9, aud tecAnology transfer. Not only are visualizations

of complex calculations indispensable in explaining the science involved to nonexperts, but also visual user interfaces to the associated simulation software serve to

reduce barriers to use of the software by nonexperts.

Software.Sharing Mechanisms. Present software-sharing mechanisms consist

of FTP servers at various centers and of informal software sharing among collaborators. These mechanisms are effective but inadequate. Clearly, some mechanism

is needed for locating software that is available over the network. In addition, more

incentives to share software are needed. There is a reluctance to share software

among competing research groups and centers, which is in part a result of the system that sets these groups up as competitors rather than as collaborators. Direct

and visible rewards for the sharing of software could change this situation. This

problem is not universal in the community, but it needs to be addressed by the

fundin$ agencies, who are the only members of the community with a means to

change the present system of incentives.

Balance of HPCC Funding. The working group concluded that present HPCC

funding is too strongly skewed toward the purchase of MPP hardware and that

more support is needed for the disk and I/O subsystems, networks, software tool

development, and people who are required to make these MPP systems useful in

Innovative addressing Grand Challenge applications. As developments in compilers and appligraphics include cation programs allow us to extract more and more of the potential performance

volume rendering from these MPP systems, this support issue becomes increasingly critical.

and _,'tual reality/.

Targets of Opportunity

The primary system needs of tIPCC scientific visualization that computer system

vendors should address are as follows:

. higher I/0 bandwidth,

. effective interactive operating systems,

60 Workshop on Grand Challense Applications and Software Technology
_._._1J II .... I[/I I III I [ I IIHI ....................... " .............. ..... . ' _i/ .......

Workinl Oroul_

- _ ' ii i, _ _ ,, ii i _ :.................. i,,,,,, _ ......... _

, high-speed network interfaces, and

, better compiler and debugging tools.

The working group identified several areas that need enhanced federal funding

support:

• development of scalable parallel visualization algorithms,

• tools and environments for interactive exploration of massive dstssets,

• development of new visualization/analysis paradigms,

• graduate student support for work in this area, and

• conferences and workshops that bring this interdisciplinary community together.

Relationships

The working group felt that not _et:ough computer science research in scientific

visualization is addressing Grand Challenge applications. These applications require

visualization tools with

• embedded quantitative analysis capabilities, such as the generation of individual numbers, various statistical measures of the data, spectra, and other

operations that may be application specific;

• support for complex data types and structures, such u those that arise from

overlapping, adaptively refined, or unstructured grids, staggered grids, mad

binary tree structures;

• high efficiency for use with large data,sets.

To improve collaborations between computational and computer scientists in

this area, the working group felt that programs aimed at interdisciplinary collabo-

/In eus_/.to-use, rations and workshops could be enhanced. These collaborations would also benefit

interactive too[ to greatly from better network connectivity and associated network-based collaboraguide large-scale tive software tools.

computations

at run time will Impacts

dramatically The impact of scientific visualizatio, ttpoJl the HPCC program is chiefly based

improve scientists' upon the role of v'muahzat,o' n'as an enabling technology, which allows a researcher to

productivity, understand extremely complex physical systems. Not only does visualization enable

the primary researcher's understanding of his simulation data, but it also enables

him to communicate that understanding to other members of an interdisciplinary

cupon ommunit to support y of experts, the rese anda,rcehq.uaSecondary lly importaimpa nt, toctsthofe pub scientific lic thatvisualization ultimately isocan lltheed

HPCC program include its role in driving requirements for network bandwidth and

for the interactive use of MPP computer systems,

The impact of the IIPCC program on scientific visualization research is chiefly

to drive the development ot"scalable visualization algorithms. Related to this is the

development of multidisciplinary tools for the visualization of massive datasets.

Workshop on Grand Challenge Al)plicatiolas and Software Technology 61
, t

............................. - .. ,....

.... III I Bill [ IIII IIIIII._ I nlrlnlrllnI II J [I - _ lit ]lllll Ill I Ill

Group H - Graphicsand ViJu_lisatdon

- iiii -- i ]1 i, ; _ [ ii ii 11irrll __ • jiiiui __ - .

The impact of all these developments on industry is to reduce the barriers to the

use of HPCC application codes in industry. Graphical user interfaces are perhaps

the moat effective means of promoting the u_e of new and complex software packqes by scientists not trained in the details of the inner workings of tho_ codes. A

common difference between commercial and research code packages is the lack of

flexible and powerful graphical user interfaces in the latter codes, for handling both

data fed into a simulation model and data produced ss code output. With accelerated development of modular visualization software targeted at Grand Challenge

applications, the development of such user interfaces will represent a much smaller

fraction of the overall code development effort and therefore make technoiosy transfer to industry both easier and more common.

Findings i

The visualisation working group focused on the following flndinp:

• Implementation of visualisation techniques on massively parallel

computers. Most Grand Challenge computations will be performed on powerful parallel computing platforms. In many cases, only these systems will

have enough memory, computing power, and peripheral storage to visualize

the simulation data. Therefore, visualization algorithms must be developed

that run efficiently on such systems. These visualisation programs must be

i

able to run in a highly interactive mode, in order to permit researchers to explore dataseta dynamically. Visualization techniques of particular interest to

the Grand Challenge applications community for implementation on MPP systems include three-dimensional perspective volume rendering, polygonal surface rendering, and combinations of both these techniques. The tremendous

computing power offered by MPP systems will also enable more sophisticated,

physically based rendering methods that can capture more of the information

inherent in the simulation. Such advanced rendering techniques might include,

for example, a detailed treatment of light scattering from textured surfaces.

• Very large datasets. Visualization packages like AVS and Explorer, which

allow many groups to write their own specialized modules and then to incorporate them into a larger, unified whole, are an excellent means of software sharing and reuse. However, a number of flaws in such packages were

noted that relate to the essential Grand Challenge need to visualize very large

Manl/Grand datasets. If visualization software sharing and reuse are to become the rule

CTtallenge.type in the HPCC community, research is needed on modular, high-performance

application, s are visualization environments that are scalable to handle very large Grand Challenge applications sad that are capable of utilizing powerful MPP systems.

too enormous These environments should allow quantitative information to be extracted

to be viewed from a dataset using the visualization tools, and they should allow a variety

in real time. of application,.specific analysis modules to be incorporated into the visualization environment. These enviro,ments should also support a broad range of

data types and structures. In particular, effective means of visualizing vector fields in three dimensions are ueeded, as are mechanisms to handle data

specified on overlapping or unstructured meshes or on staE_ered or adaptively

refined grids. Many of the Grand Challenge projects involve interdisciplinary

62 Workshop on Grand Challenge Applications and Software Technology
a #

A ..... I III ] Illl!llll II II II iiiii .,,F]_1 II IIII] I .............. IIII I I I I IIIII .... 1

Workin8 Groups

.......... : = i ifffl

collaborations, and the visualizationenvironment must be ableto handlesimultaneouslythe preferreddata strllctures for each discipline.

* Highly interactive response. The Grand Challenge applications, because

they often involve computations oil extremely fine grids, demand large data

volumes (and very high image resolution), The size of the datasets from these

applications canexceed a terabyte, making it impossible to render all the data

into any single image. Consequently, researchers often wish to explore their

datasets interactively, with only a fraction of the data displayed at any single

instant. This drives a need to get as much data onto the display as possible in

Complez each image, and hence very high resolution display is essential. In addition,

computations the researcher needs an effective means of navigating through his darner,

require a highly keeping at all timu a sense of where he is with respect to the whole. Virtual

interactive reality systems offer one means of creating a natural sense of this navigation

through the data. Flat-screen-based displays allow one to navigate through

response, the data as if in a virtual vehicle. There are tradeoffs in these approaches

l involving resolution and display speed, as well as other factors, and research

in this area is needed. Any such approach, however, has a funds_nental need

for very rapid image rendering aad display under interactive user control, and

with a special emphasis on very large datasets. A high degree of parallelism in

both the rendering engine and in the I/0 subsystem is likely to be a common

feature of these interactive visualization systems,

. 1Vlas_ive data storage, retrieval, and management in a heterogeneous

netwc, rking environment. These data storage, I/O, and networking issues

have been separated out for detailed consideration by other working groups.

_1 Nevertheless, our summary of the needs of high-performance scientific visualization could not be complete without stressiag the fundamental dependence

of this activity upon the infrastructure, both hardware and software, for data

storage and for data movement from one device to another.

. Increased support of visualization infrastructure. This infrastructure includes visualization hardware and software both at high-performance

computing centers and at the researcher's home institution, both wide area

and local networks, and supportpeople both at centers and working locally

with researchers. Dcvclopment of this visualization infrastructure for high-

: performance computing is a natural follow-on to the nation's successful program of installation of high-I_erformance computing infrastructure, The new

HPCC computing system._ have unprecedented power to perform scientific

computations, but unle._q the cal)ability to visualize the results of these corn-

• putations is adequately SUl>ported.many of the potential scientific discoveries

of this program may elude us.

[ Group I- National HPC'C' Infrastructure

t

! Chair: Charlie Catlett; Co-chair: ._laxiae Brown

The global economy is rapidly _hiftilmgtoward information tecilnologies. Advanced information tech.otogics (iJwl_l,lis_ supercomputers, new paradigms for immersive data analysis, digital lih,'ari,'s, col_ctlrrent engineering, and the ability of

i

Workshop on Grand Challenge Applications and Software Technology 63
_ ,,,I , llltmf Ji I . tl ,I llli I, - L_

GroupI- NationaltlPCC lafr_lructure

--- - ,,u , ,,,,i, ,,, ,,, j --- , ,,,,,

workers to use these technologies) form the basis for economic competitiveness during the next fifty years. The present U.S. infrastructure provides access to goods

and services, natural resources, manufacturinll centers, and other components of

an industrial society. The HPCC infrastructure must provide access to information

technologies, goods, and services that will be necessary in order to compete in the

new, information-bMed global economy.

Status

The temptation exists to think of the HPCC infrastructure in terms of capital investments such as supercomputers, databases, and networks. These are the most tangible components of the infrastructure; however, they are only the building blocks

The/edera[ HPCC from which an infrastructure can be constructed. We define the national HPCC

program requires infrastructure in terms of two distinct but interdependent areas: technology infrascoordinated efforts tructure and human infrastructure. The HPCC technology infrastructure consists

between individual of three layers:

institutions and • Hardware and networks, the bottom layer, includes computing hardware (sufunding agencie& percomputers, workstations), nc,#orks, and storage archives. Here, we focus

on systems that are available to the national community, including nations]

networks and computing and storage systems available to the national community (as opposed to private computing resources).

• Resources and services, the middle layer, include such resources as national

HPCC centers (providing computing services) and digital libraries (providing

access to information) as well u support services for mobile and wireless hosts

and workstation-based video conferencing. The resources emdservices provide

a bridge between data and information.

'. The user environment, the top layer, includes software systems that allow

the user to locate, access, and effectively utilize services and resources and

to collaborate with other users. This is the environment in which the user

"navigates" through information and through the lower two layers of the technology infrastructure. The user environment is a bridge between information

and knowledge.

Each layer builds on the layer below and provides infrastructure for the layer above.

Cutting across these three layers is a human infrastructure component consisting

of teams of professionals, centers, and various communities. These human structures

provide services and are themselves resources in terms of consulting, training, and

development and advancement of the infrastructure itself.

Targets of Opportunity

The working group identified targets of opportunity for each of the layers outlined

above.

Technology Infrastructure: Hardware. The need for teraflops computing

power is well established for Grand Challenges. Today a number of supercomputers

are available that have ideal peak processiag power of over L00 Gflops. The current set of large, scalable machi,,es will rapidly be outgrown by the current Grand

64 Workshop on Grand Challenge Applicatiotls slid Software Technology
I IIIII I I I I I III

Working Groups

Challenge teams, and so the infrastructure of large-scale systems needs to continue

to expand both in terms of the number of moderately sized machines available for

training and code development and in terms of the size of the major Grand ChalPeak performance lenge systems. At least as important as the amount of memory are the processoris not irnportant-- processor and processor-memory I/0 bandwidth and latency and the external I/O

teraflops bandwidth. We stress that teraflops ideal peak or even "benchmark" rates are not

application important to Grand Challenge applications--teraflops application performance is. The Grand Challenge teams are projecting single computational experiments

performance is. that create 10_ to 1012 bytes and archives (storing results from multiple experiments) of 1013 to 1015 bytes for a single Grand Challenge team archive. With

teraflops computers, these experiments will create 109 to 1012-byte data sets in a

matter of hours. Revolutionary approaches to data storage are essential. Moreover,

data must be readily accessible. Today's high-performance computing environments

can routinely handle datasets up to several gigabytes with effective transfer rates

at between 1 and 10 Mbytes/s. A balanced environment, according to the Grand

Challenge team surveys, should allow single data files to be transmitted in 1 to 10

seconds and datasets (of a thousand such files) in 15 minutes to 2 hours. Coupling

this with the data file and dataset sizes above results in the need for effective data

transmission rates of l0 s to 109 bytes per second--the higher rate being ten times

the peak throughput of high-end local area networks in 1993 and twenty times the

highest observed throughput of the fastest tertiary storage devices available in 1993.

To meet these increasingly higher data rates will require networks that can support multiple gigabits per second. In addition, the network capabilities requirements

of both the high-end and the average user community continue to grow.

Technology Infrastructure: Resources and Services Current network-accessible

resources and services are divided into three classes: those that are experimental

(MOSIS, WhitePages), those supported as unique national facilities (supercomputer centers, NIC, library search capabilities), and those supported by community

and/or volunteer efforts (on-line FTP archives, WAIS, World Wide Web, Gopher

servers, USENET news). New services are outlined below, with an analysis of their

projected impact on the infrastructure.

• Information services. These are becoming increasingly richer, with digital

libraries, yellow-pages, white-pages, and data archives providing more intuThe volume of itive user interfaces and thus growing in popularity.

scientific data

• Data archives. The amount of data being generated is growing at an expobeing generated nential rate. Instruments are becoming more commonplace on the network,

is growing and remote control of these instruments as well as real-time arc.hiving of data

exponentially, will produce data streams between 10 and 100 MBytes/s. Making this data

useful will require far more than high-density, high-throughput storage devices. The traditional directory services based on file names will yield to

archive services coupled with digital libraries that can search based on content.

• Speclal-purpose services. Special-purpose services such as networked in-

-_!r_,ments, production facilities for image/video/CD-ROM output, and access

_6 _"turnkey" computing services will allow novice users to take advantage of

popular and proven computational science codes. The community will need to

Workshop on Grand Challenge Applications and Software Technology 65
II I I III I I I IIIIIII I I I I I I II I II I III I

Group I - NJtio._/HPCC lnfr_tructure

focus on providing high performance per application, as well as on maximizing

access for training and software development. In addition, tale-classrooms and

workshop facilities will allow groups of users to congregate in one or severed

locations for conferences, training events, symposia, etc. These tale-centers

will provide for the distribution of meetings among major tale-centers as well

as participation by individuals from their home workstations.

• Access services. Access services are emerging that will have a profound

Emphasis must impact on the national infrastructure. In particular, packet video capabilbe placed on ities will be bundled in workstations over the next several years, with an

the security of estimated 0(104) systems capable of generating data streams of roughly 1

computer data. MB/s and 0(102) systems capable of generating up to tens of megabytes per

second at high resolution. These will produce an aggregate national network

requirement that may become the driving force of the network infrastructure.

Additionally, within three years, O(103) laptop systems will be capable of

generating 1 MB/s data streams, and O(104) systems will be generating 10

kbits/s streams; both sets will take advantage of wireless connections. The

infrastructure must be able to accommodate these mobile systems in terms of

both capacity and straightforward access.

All these resources and services must be demonstrable secure and trustworthy

enough to be heavily used by U.S. industry, government entities, and even many

educational institutions. Strong emphasis must be placed not only on security of

sites and data but on robust authentication of communication and access. Advanced

capabilities such as policy-based and type-of-service data routing and access will be

required. Research is needed on how to instrument the HPCC infrastructure to

support security (e.g., auditing technology) and on how to better understand (and

minimize) the corresponding tradeoffs in speed and ease of use.

Technology Infrastructure: Software and User Environment. Even for

very advanced scientific researchers such as those involved in Grand Challenge

teams, the user interface to the infrastructure must allow them to focu_ on the

scientific problems and not on the logistics of using the infrastructure. Research is

therefore needed in several key areas:

s National file system. A fundamental first step toward integration of the

infrastructure is to provide a national file systemmallowing users to access

their data, regardless of location, with consistent identifiers, or "pathnames."

A fundamental The implementation of such a, infrastructure will require considerable coorfinal step toward dination.

integration of the • Better operating systems, compilers, and code development, opinfrastructure is to timizat:ion, and execution environments for scalable systems. The

provide a national Grand Challenge teams are spending much effort on porting and optimizing

file system, applications for sealable systems. This experience should be captured through

software sharing and through involving outside computer scientists and software developers in the projects.

• Intelligent interfaces. The many separate display standards, image and

file formats, directory/indexing mechanisms, and other information storage

techniques in use today must be hidden from the user by intelligent interfaces.

66 Workshop on Grand Challenge Applications and Software Technology
a

i II Ill II II [IIIllIl I I I r |, i

Wotidng Groups

At the same time, coordination among information providers is necessary to

ensure that the legion of itlformation sources within the national infrastructure

does not become the digital Tower of Babel.

• Support for loading and selecting computer resources. Services that

allow the user to select the optimum computing resource at a particular moment, as well as services for allowing experiments to span several noncollocated

processing resources, will be required.

• Data analysis tools with intuitive user interfaces. Visualization and

other data analysis systems designed over the past several years for today's

datasets will not scale to the data sets being envisioned by the Grand Challenge teams. The sheer quantity of data to be produced by these Grand

Challenge experiments will require an entirely new data analysis paradigm.

• Virtual environments. Immersive technologies such as head-mounted or

"walk-in" displays have the potential for both data analysis and collaboration, perhaps enabling a new generation of teleconferencing. The impact on

the infrastructure of multiple, syJtchronized high-definition images will be significant compared even with single high-definition image streams, much less

with today's relatively loworesolution image streams.

• "Team" software. Systems that will allow multiple team members to cooperatively analyze large data sets will be needed in order to allow these teams

to work together effectively despite their geographic separation.

Human Infrastructure. The HPCC human infrastructure includes supercomputer centers, science and technology centers, Grand Challenge teams, the gigabit

Yeraflops network testbed eommuniry, and the lnternet community. These groups provide

computers require more than supercomputer access, also furnishing the expertise needed to "harness"

a well-balanced scalable computers, effective peer°review mechanisms for computational science, extechnolo#ical tensive training and documentatiot_, software, information, and data repositories, and development activities in u._er iflterfaces and computing environments themand human selves. These orgal_aLioiJ,_ _d._o ,_,,rv_, n._ iJ_tellectual centers, providing environinfrastructure, ments with long track records o1' <'atatyzi,g interdisciplinary computational science

teams.

Findings

The working group proferred several suggestions to support and enhance both the

technology infrastructure and the human infrastructure,

Technology Infrastructm, o. It is important to focus not only on the visible

components, such a.s teraflops con_putcr systems, but also on a well-balanced structure of networks, storage systems, atl,1 ._oftware environments that allow users to

effectively access and utilize these t.eraflol)s systems. To maintain this balance, we

make the following suggestions:

• Interconnect major _atio_al t'c_sottrccs such as supercomputers, archives, and

data analysis envirozlmelits with l_et,work rates of at least 622 Mbits/s by

Workshop on Grand Challe_lge Applications and Software Technology 67
t

I I I I II Ill I II I I I IllUlll n I ,ll|l Imll II ----_

Group [ - NationaJ HPCC ln[r_tructure

FY95 and at least 2.4 Gbits/s by FY96. Major universities and regional and

corporate laboratories should have network access mechanisms that allow for

peak rates of several hundred megabits per second by FY95 and up to one

gigabit per second by FY96. The typical university sites should have peak

capabilities of no less than tens of megabits per second by FY96.

• Fund coordinated efforts among major data and information providers in areas such as transparent interconnection of digital libraries and standards for

information indexing, storage, and display.

• Integrate capabilities such as desktop video conferencing, virtual environments, and mobile and wireless hosts, and coordinate these capabilities with

major national testbeds.

Human Infrastructure. The human infrastructure for HPCC provides fundamental reservoirs of expertise. These have a gestation period of at least two generations of technology, and their continuity assures smoother transitions from one

generation of technology to another. We suggest that the federal HPCC program

provide stable and long-term funding for new initiatives as well as stable base-level

funding for ongoing operational support.

68 Workshop on Grand Challenge Applications and Software Technology
__ II I I i I I I I ii i nllll II I II I III I II

SECTXO6 N Conclusions

By bringing together applications scientists and software developers--groups that

ordinarily simply "pass in the night"Nthe 1993 Grand Challenge workshop encouraged the sharing of ideas and problems in high-performance computing.

In addition to providing a status report on current Grand Challenge research and

on recent software developments, the workshop provided a forum for identifying new

mechanisms for improved software sharing and for exchanging information about

software and hardware limitations impeding the High-Performance Computing and

Communications program.

i The researchers in the Grand Challenge community and in the Software Development community agreed that the federal HPCC program is already having an unThe increasing deniable impact. Indeed, one might almost speak of a high.performance revolution,

capability to with widespread interest in and use of MPP systems by industry and academia;

handle complex increased focus on parallelism by workstation vendors; growing emphasis on the

need forstandards;and almostuniversalconcernfordevelopinbgetterproducts

problems (machines,languages)tomake paralleclomputingeffective.

has already/ And while the researchers agreed that it is probably too early to expect HPCC

attracted industry program activities to have had a major impact on science and industry, ezciting

involvement, progress is being made. In particular, high-performance computing is enabling researchers to attack previously intractable problems, such as the recent 5128 simulation of compressible turbulence.

This increasing capability to handle complex problems has already attracted

the attention of industry. Industry involvement was noted in numerous Grand

Challenge projects, including aerospace, environmental science, biophysics, computational fluid and plasma dynamics, computational chemistry, reservoir modeling,

and global computational finance.

Nevertheless, several outstandin9 issues were identified. Foremost was dissatisWorkshopon Grand Challenge Applicationansd SoftwareTechnology 69
IIIII i I ii | I III iii L I[ i i iiili iiii I i

faction with the current state of software tools. A frequent lament was that computer science researchers are not meeting the needs of the applications researchers,

that those software tools that have been developed for high-performance systems

usually are left in the prototyping stage. Indeed, some called the state of software

tools "dismal." A clear need exists for better parallel debugging tools, tools for multidisciplinary applications, performance-monitoring tools, and language support to

allow users to write programs at a higher level than currently possible. The cause

for the poor software support is clear: a lack of incentive. The Grand Challenge

grants currently focus on the output of the applications rather than on the software

to achieve that output. Academia generally does not encourage computer scientists simply to "help out on a project"; nor do research institutions fund long-term

projects to bring a software package up to commercial quality. Commercial vendors also are not rewarded for providing robust software tools: the emphasis of the

HPCC program has been on more powerful machines, even when the software is not

available to make the machines truly usable. The resulting paucity of tools is made

worse by difficulties in sharing those tools that do exist. More effective mechanisms

are needed for exchanging information on tool availability and accessibility.

Full use of A second major problem is dataset management. The Grand Challenge studmassively ies are generating enormous datasets (up to 8 terabytes, with predictions in the

parallel machines petabyte range). Making this data useful will require not only high-density, highthroughput storage devices but also improved archive and directory services. Enwill require tirely new data analysis paradigms will also be needed to add type information and

advances in tool structure, enabling researchers to access information more intelligently and more

design, dataset efficiently.

management, Even further, specialized visualization systems will be essential for examining

and _sualization the vast datasets expected from Grand Challenge research. Currently, several efforts

systems, are under way to develop image-rendering software that can be executed directly

" on massively parallel processing systems. Others are developing tools for graphics

display offiine. Moving the data to workstations results in high storage and I/O

bandwidth requirements; moreover, if the data is to be displayed at full resolution,

new techniques are needed to address the extreme demands placed on the imagerendering engines and the display subsystem.

Related to the growing data size of applications is the question of data rates.

Current HPCC applications involve I/O rates of up to 6 MB/s to archive; this rate

will increase to 100 MB/s in tile near future. I/O is simply not scaling fast enough

to keep up with CPU speed, and is threatening to become a major bottleneck for

teraflops applications. Further research is needed to understand the I/O characteristics of massively parallel computers. Parallel I/O benchmarks are also needed

that accurately reflect application I/O requirements. The Scalable I/O Initiative

sponsored by the Concurrent Supercomputing Consortium will provide a first step

toward meeting these needs, if it is funded by the various federal agencies.

Finally, a significant number of researchers felt that increased emphasis should

be placed on standards. Current standards efforts include High Performance Fortran

(which met with mixed reviews from the researchers, some of whom were extremely

skeptical about its value), and the evolving Message Passing Interface standard;

both focus on ensuring that programs are supported on a large number of different

parallel computing environments. Further efforts are needed to promote vendor

standards for archiving large amounts of data. An example is an industry effort to

define standard interfaces for databases to make use of tertiary storage.

70 Workshop on Grand Challenge Applications and Software Technology
m

ii1[ i|1 ii iiiii II i 11111 i iiii ii iii iii ii ii I i ii i_ iiiii I

SECTION7 Findings

There is nothing quite so frustrating for the avid do-it-yourselfer than

to begin a project, suddenly need a particular tool, but have no idea

where in the house to look for it.

- Practical Homeowner's 1987 Do-It- Yourself Annual

Many of the researchers at the Grand Challenge workshop expressed this feeling of

frustration in discussing their attempts to find a suitable tool (whether a debugger

or a performance evaluation tool), to use a (not-well-documented) tool efficiently,

and to reuse it (i.e., have access to it through a software library) in other appliThe workshop cations. Indeed, a majority of the suggestions at the workshop focused on the

participants problem of development, maintenance, and distribution of tools. Here we summasuggested several rize three of the principal suggestions and--more important--propose a mechanism

new mechanisms for implementing each suggestion.

for improving the • Suggestion: Collaborative development of tools for Grand Challenge applicaeffectiveness of the tigris should be supported. Special attention should be given to parallel deHPCC program, buggers, performajlee evaluation tools, and interactive visualization software.

Such efforts must be supported for all phases of the life cycle--fundamental

research, prototyping, and development.

Mechanism: Support a "Grand Challenge in Software Tools," and establish a

Science and Technology Center for HPCC Software Tools.

• Suggestion: Tools that are ready for use outside of their development community should be made available in software-sharing libraries. Mechanism: Establish or fund existing "software capitalization" programs to

foster the development of research ideas and prototypes into production tools

i and to ensure the maintenance of the software in the library.

Workshop on Grand Challenge Applications and Software Technology T1
-- --- I1[I I - I I IIII II IIII iii I iiii iii iii ii i ii 111111111 I i ii 1111111 iii i i II

v

• Suggestion:Toolsthathave had successinthe researchcommunity shouldbe

commercialized.

Mechanism: Fund an existingorganizationsuch as OSF, the nationallaboratorieso,r the NSF supercomputer centers;or setup a new independent

organizationorindustriaclonsortium.

• Suggestion:Researchersneed to be kept informedofthenumerous toolsthat

are availableAt. thesame time,computer scientistmsust be made aware of

the needsof applicationsscientistfsorspecifictools.

Mechanism: Developa parallesloftwaretoolselectronicjournal/newsletter

(completewith an editoriabol ard)topubliciztool e availabiliatynd toolneeds.

That thecomputerscientisistcaughtbetweenCharybdisand Scylla---encouraged

to createprototypetoolsasresearchprojects,but givenlittlcre e ditfordeveloping

production-qualitytoolsforwidespreaduse--was clearlyrecognizedby the conference participantsM.any feltthatthe vendorsmust assume more responsibiliityn

providingusefultools.

• Suggestion:The qualityand performance of the basicHPCC toolsshould

be improved. In particularv,endorsshouldprovidegood node compilersto

allowprog:ammers to make more effectivuese of machines without resorting to assemblylanguagecoding;providefastmultiple-precision arithmetic;

pay greaterattentionto symbolicand combinatorialaspectsof computing,

such as fixed-pointarithmetic;providelanguage constructsthat allowthe

easy definitionofcomplex data structuresp;rovidehigh-levels,tandardized

message-passingroutinesp;rovidea random number generatorwith theirmachines;and developa compilerinfrastructureformaking visiblthe ecompiler's

internalrepresentationands transformationtos othersoftwaretools.

Mechanism: Requirecomputer vendors(throughexplicitlyworded RFPs) to

providegood,usabletoolsforMPP machines.Have federalagenciessupport

commercialdevelopmentofsoftwaretools.

Of course,providingappropriatetools(whetherfrom vendorsor from computer

scientistsre)quiresanawarenessofwhat isneeded.While some applicationpsrojects

explicitlymade provisionfor"computer sciencework,"most do not. The workshop

participantags reedthatmuch more interactionisneeded.

• Suggestion:New incentiveshould be developedtoencourage computer scientistsand applicationscientisttos interactmore closelony Grand Challenge

projects.

Mechanism: Have HPCC projectsexplicitlyfund the exchange of students

and staffbetweenapplicationand tooldevelopers.

Softwaretechnologyrequirementscertainlyextendbeyond toolsforGrand Challengeapplicationresearc s h.Severalareaswere identifi_.d attheworkshop asneeding

increasedattention.Here we firstlistthe varioussuggestionsa;singlemechanism

isproposed to addressthesesuggestions.

• Suggestion:High-performancefilseystemsmust be developedtoensureubiquitousaccessto both conventionaland high-performancedistributefid lseystems.

72 Workshop on Grand ChallengeApplicationsand SoftwareTechnology
...... .

I II I I I I II _ -- I_ i I Ill[ I I II III IIII I I I ] IIII IIIIIIII i ilnll

FindinRa

• Su¢.3estion: Database management systems must be developed that (1) support large objects, (2) provide fMt access to subsets of large datuets, (3)

interface to tertiary storage systems, and (4) incorporate new indexing and

access methods for sciea_tiflcdata.

, Suggestion: Methods should be developed to allow mixing of different languages in a single framework.

i

• SugOestion: "Templates" that capture key computational, communication,

and I/O characteristics of important application codes should be written and

made widely available.

• Suggestion: More emphasis should be placed on integrating support for I/O

and networked services.

• SugOesfion: Standards efforts such as the Message Passing Interface forum

should be encouraged.

• Suggestion" Heterogeneous computing must be fully funded through large

projects that include software, applications, and network research.

• Suggestion" Engineers and scientists must be retrained in multidisciplinary

technologies, and new researchers must be educated in this critical HPCC

area through the establishment of multidisciplinary curricula at universities.

MecAanism: Increase the current federal involvement, specifically by providing

(!) stable and long-term funding for new initiatives and C2) stable base-level

funding for ongoing projects.

Workshop on Grand Challenge Applications and Software Technology 73
i

__ [I II ]lll T ...................... " .............. _ IIzj _I IIIIIlLI I II IIIIU I [ iiiiiii IIII I .............

 scTiONA.............Grand Chaflenge Requirements

Document

.................. Priortoihethree-daGry and Challensemeeting in Pittsburgh,theProgram Committeeprepareda detailedlisotfquestionsthatweremailedouttotheparticipants

severalweeksbeforethemeeting.The four-parsut rveyincludedquestionons applicationGsr;and Challengerequirements(resourcesal,gorithmstools , I,/O);the

currentstatusoftools,librariesgr,aphicsfacilitianes,d networks;and themost

importantneedsinhlgh-performancocemputing.

The results of the survey, which were circulated to the conference participants

prior to the meeting, proved an ivvaluable resource for focusing the discussion

groups.

In this appendix, we reproduce the survey and then summarized the survey

responses. The individual responses are available electronically via anonymous ftp

to the address Into.sos. anl.$ov in the directory/pub/tech.reports/hpcc. Two

flies are included: survoy-mergsd.ps and sarvsy.mergsd, enscrlp_.

Part A' Application Overview Issues

Please cover the following issues in the summary applications presentation sessions.

I. Brief Description of Your Application Problem.

s description of the "scientific problem"

, algorithms used

• codes(size and number)

2.CharacterizatioonfTypicalComputationsPlannedby GC Team.

• describ"typ e ical"computations

Workshopon Grand ChallengeApplicationsandSoftwareTechnology 75

H l . lllllll lllll l
IIIII ]F I _ II II ..... II ] I] II _ ___. lILJ_ .............. - .................... I I 11 ........... i _

• current/planned compute resources

• i/0 requirements

3. Use of Visualization and Networking.

• graphics workstations

• framebu_'ers

• remote computing via networks

• distributed computing via networks

4. Languages and Programming Tools Currently Used.

• Fortran, C, C++, others

• PVM, p4, Express, PCN, others

5. Most Outstanding Problem You Wish Addressed at This Workshop.

DetailedGrand ChallengeRequirements

Part B' Problem, Algorithm Description and Resource Requirements

i. Pleaselisatnd briefldyescribethemajor codesyou plan toemploy inthenext

3-5 yearsforeach of your Grand ChallengeapplicationsHo. w long do you

estimatethateach code would take to execute(pleasespecifysome current

platformor estimatedcomputationalrate)?

2. Which oftilefollowinmag thematicalproblemsaresolvedinthe courseofrunning tilecode? For example: N-body forcecalculationN,avier-Stokesequations,Magnetohydrodynamics, Schroedingerequation,Maxwell'sequations,

Eigenvalue/eigenvectocarlculation,Directsimulationofphysicalphenomena,

Sequencecomparison,Fouriertransform,Other (pleasespecify)

3. Which algorithmsdo you currentlyuse tosolveyour problem? For example:

• TreatmentofN-body problems-FastMultipoleMethod (Rokhlin-Greengard,

Barnes-Hut,etc.),Explicitcalculationof N-body interactionrequiring

O(N**2) operationsExpl , icitcalculationofN-body interactionusingcutoff.

• Spatialcliscretizatio ofPDE- n Finitelement,Finitedifference,Spectral,

Finitevolume,

• Temporal int,:gratiofouPDE: Explicisot lutionoftime-dependentPDEs,

Operatorsplittiimethods lg forimplicistolutionoftime dependentPDEs,

Fullyimplicitime dependent PDE solvers.

•Molite Carlo algorithms

• PIC methods

• Geometric and symbolic algorithms

s Discrete eve,it simulations

• FastPoissonsolvers

7'6 Workshop oll Grand C'hall,:l_ge Applications and Software Technology
_J I ---- I [ III 1 IIII III II II III III ILII Jl ................. "'" ' .......................... .

Grand Cballen|e Requ/remenWDocument

Ill I [ _ I - ii11 I 1111111 II llll I _ : Ill III I iii 1! I _ __ --: _ _ ....... _lq N _ __ 1[ ' f I I II I J

, Eigenvector/eigenvalue calculations: QR, Jscobi, Bisection methods.

e Linear system solving: Direct methods, Itsrstive method (Preconditioner: none, diagonal, SOR, SSOR, ILU, Block ILU, other - please

specify)

, Fast Fourier transform

s Other (please specify)

4. Estimate the percentage of time amociated with each of the above algorithms.

5. How much computational work (time on a specified platform) is e_mocisted

with each algorithm?

6. How much memory is needed for typical computations?

7. What type of data structures do you employ?

8. If your problem uses a grid or mesh, which of the following do you use? Single

structured grid, Block structured grid (problem solved on multiple, interacting i

meshes), Unstructured grid.

9. If your problem is formulated using matrices, which of the following do you

use? Dense matrix representation; Banded, envelope, or skyline representstion; Sparse matrix representation.

I0. What are (or will be) your I/O requirements with regard to the following?

Data (in bytes) moved to and from secondary or tertiary storage during program execution; Minimum I/O bandwidths; Desired I/O bandwidths.

I I. Must you execute several different programs to solve your problems? if so, do

these execute as a single application? as a pipeline? on the same computer?

on different computers?

Part C: Needs Assessments and Opinions

I/O and Data Systems

' 1. What features would you most like to see in new I/0 and data systems?

2. How large is the archive (something other than flat files) for data management?

Parallel Programming Paradigms and Program Development Tools

3. What parallel tools or languages have members of the _roup used?

4. Has the use has been minimal, moderate, or extensive?

Ex,.mple software systems for next four questions: Parallel Languages/Compilers

(CM Fortran, C*, *Lisp, PCN, Strand, Forgeg0, C with message-passing calls,

Fortran with message-passing calls, etc.); Parallel Programming Libraries or

Tools (Express, PVM, p4, Linds, Patti); Performance-Monitoring Tools (e.g.,

Gauge, Pablo, ParaGraph, Upshot); Parallel Debuggers (specify machine if

the tool is supplied by the vendor); Other software tools that pertain to parallel machines (name the tools)

Workshop on Grand Challenge Applications and Software Technology 77'
#

.... - .... ...... _ IJ_LI] I II IIIIIIII ilnnnuna ..... - ' .......... I ...... I --

....... t, , i,, , ,

5. Which tools do you Rnd are most useful?

6. Which tools are conceptually poor but are well implemented (i.e., the tool

does not do anything you need to have done, but does its tub reliably)?

7. Which tools are conceptually good but poorly implemented?

8. What is the source of your parallel programming support software?

Performance Evaluation of MPP Applications

9. How do you characterize the performance of your application (MFlops, size

of problem attempted, CPU time per n_._deltimestep, other)?

I0. How do you know that your code is running at a reasonable performance level?

II. Which is more impc tant: faster turnaround on existing problem sizes, or

larger problem sizes with same turnaround time as today?

12. Do you expect to be able to use the same numerical methods on serial machines

and MPPs for your application areas?

Discussion of Libraries

13. Do you use numerical libraries or solvers (e.g., LINPACK, LAPACK, EISPACK, PCGPACK, SPABSEKIT)? If you do use such codes, please specify

which.

14. Do you use commercial or publicly distributed applications codes (e.g., GASP,

NASTRAN)?

Graphics and Visualisation

15. How do you (or will you) view your results? Small set of fixed or floating

point numbers, Set of 2-D pictures, Set of 3-D pictures, Animations

16. What fraction of your group's time is spent analyzing the results of runs vs

running them vs developing code?

17. What is the biggest problem in your postprocessing environment?

18. What capabilities do you want in a visualization system?

19. Is it important to be able to generate graphics directly from an MPP system?

National High-Performance Computing and Communications Infrastructure

20. How often do you use networks of workstations on a single application? If

you do use networks of workstations on a single application, please answer

the following:

21.What performancedo youachieve?

22.Do youwishtouseheterogeneounsetworkosfmachinesor, areyouusingthem

becauseyou do nothaveaccesstotrueMPP systemsofsumcientcapacity?

23.What softwartoo e lsdo you usefordistributceomput d ing?

78 Workshop on Grand Challenge Applications and Software Technology
J_mmmmmm_ ............

• 0 I

II III IlIllI It Ill _[ III -_ I _ IHII III I II I i_ _ I IlllllI I I Ill It IlnlI Ill ..... I It I L - I1[ I I

Cr_d ChaJ/ense.qequltement_/_cumen_

, , , ,,,, ....... _ . ....... ....... : - ,,.

24. Would you prefer to compute on a network of workstations, on a tightly

coupled centralised MPP, or on a network of MPPs?

Multidisciplinar_ Applications

25. If your problem involves execution of several different programs, which do you

expect to parallelise? How will you exchange data between programs (e.g.,

files, sockets, meuage-psasing)?

Part D: Most Important Needs

Indicate the areas where prolres, on your research can be &ccele:,atedif you (and

your colleagues) could be usisted by other professional communities, including those

of mathematics, computer science, and other computational science disciplines, and

also by the computer industry. In particular, are there factors that are currently

limiting your productivity (you may use the items on the following list as examples,

adding to them where appropriate)? In your response, be as specific as possible

concerning the character of the problem(s).

• Mathematical description of the problem(s) - theoretical or practical limitstions

• Numerical limitations- word size; fraction or exponent limitation

• Algorithmic limitations- require new approaches, more generality, higher performance, etc.

• Library inadequacies

• Programming model inadequacies

• Language/compiler limitations- inadequate languages: what do you need? -

inadequate compilers: in what way?

• Tool limitations- programming, debugging, performance monitoring, graphice, data management

• Turnkey packages that you would like to see developed

• Programming environment limitations- too cumbersome, not responsive, inadequate procedures, etc.

• Hardware limitations- performance, memory, I/0, connectivity

If you had one "software magic bullet," what wouldyou use it for? Please specify

one of the items listed above, if appropriate.

Workshopon Grand ChallengeApplicationands SoftwareTechnology 79

ii iiiii _
i ' i i i ii li i iii i i ii ii iii

SECTION B Summary of Survey

Responses

t

1

t

l

AI Brenner (Super¢omputing Research Center); Alo_ Choudhary (Syracuse); Tom

Morgan, Gail Pieper, and Rick Stevens (Argonne National Laboratory); Joel Saltz

(Ma_land)

1. Overview

We received 28 completed surveys. This report provides a summary of some of

the issaes that have a particularly strong potential bearing on directions that need

to be taken in systems software and hardware.

2. Issues

The teams identified a wide range of issues that they believed important:

• development of capabilities to allow asynchronous data transfer while processors are engaged in computations;

• fasterparallelI/O;

• developmentofrobustcompilersforparallearc l hitecturesthat abstractaway

low-le_/eldetailsof machines and algorithms,withoutcatastrophicperformance losses:

• extensionofHPF ._ocovernonstandarddatastructures(block-structuredata, d

unstructureddata.,etc.);

• compilersthat take betteradvantage of RISC or vectorprocessorarchitectures;

• decreasing communicationlatency;

Workshop on Grand ChallengeApplicationsand SoftwareTechnology 81

i II II1[
,s

ii iii i i iiii il ii

• support for collective communications;

• tools and techniques for heterogeneous environments; and ,,

• methods for load balancing. _

3. Languages, Programming Tools, and Libraries Currently :, Used

Although there are almost as many different languages and tools being used

by GC teams as the number of teams, the following briefly describes the most .

commonly used languages and tools. Fortran and C are both widely used: many

teams use both (20 teams use Fortran, and 20use C); 13 use C+4-, although no team

reported using C++ as their only programming language. Message-passing libraries ._

are widely used (22 teams reported using some kind of message-passing library, !

including libraries supplied by vendors, PVM, and PICL). Ten teams reported using

a vendor-supplied parallel Fortran (typically CM Fortran); 5 teams reported using

a vendor-supplied parallel C (typically C*); 5 teams reported using extensions for

programming shared-memory machines.

Other languages and tools used by some teams include PCN, CC+-_, and pFortran.

Other tools used include p4, Linda, DTM AVS, PARTI, and Smalltalk (one or

two users each).

CMSSL is extensively used by a number of teams. A few teams use BLAS,

LINPACK, LAPACK, and EISPACK. A number of groups have developed their

own C.. class libraries.

4. Data Structures Employed

The directions in Grand Challenge-related systems software research must be

motivated by carefully considering the nature of the algorithms being tackled by

the teams. Out of the 28 Grand Challenge teams responding to the survey, 26 reported that part of their work involves solving well-structured problems, including

structured mesh and dense matrix calculations. However, 16 of the teams also report

that some of their calculations involve problems with irregular indirection patterns

including unstructured mesh and general sparse matrix calculations, traversal of

linked lists, and calculations involving explicitly enumerated atom lists (in molecular dynamics problems). Approximately 12 teams reported that some of their

calculations involved block-structured grids, in some cases the intergrid data transfer patterns are static; in at least 4 cases, the patterns change during th__ course

of the problem. There were 4 teams that reported using tree-structured algorithms

such as fast multipole algorithms.

5. Software Support Requirements

In general, most teams felt that most of the available tools are primitive and

inadequate. There is very substantial interest in portable compilers able to give

"reasonable" performance. There is a need to provide language support to allow

users to write programs at a higher level than what is currently supported on parallel

architectures. Many users waat to be able to specify parallelism and data and task

decompositions at a high level, Debuggers and performance tools were requested by

almost all the teams. A large majority felt that software and architectural support

for high-performance I/O is inadequate.

82 Workshop on Grand Challenge Applications and Software Technology
IIIII I I iii I I iiii ! I I i ii ii II i I III i ii iii ii

Summary o[ Survey Responses

6. Compiler, Programming Tool, and Library Requirements

Portability of compilers across various machines was the most frequently requested feature. Fine tuning codes for different architectures was a major problem

faced by most of the teams. Many researchers complained of buggy eta compilers.

Quite a few of the responses criticized the quality of code produced by the compilers.

Several responses asked for better node compilers.

Most of the responses pointed out the need for a much larger set of tools. Debugging tools and performance-monitoring tools were frequently given the highest

priority. Quite a few responses mentioned the need to standardize tools across

different architectures. Graphics tools were also considered highly inadequate.

I/0 libraries were requested by many teams. Image processing and display

libraries were also requested by a few teams.

7. Networking and Visualization Requirements

Msot teams access MPP through what are perceived to be inadequate networks.

many plan to move to faster local networks. Graphics and visualization for examining output create particularly troublesome network problems. Most researchers

report that they carry out visualization off-line; this results in high storage and I/O

bandwidth requirements. Dataset sizes can range from 1 MB to 8 TB.

8. Algorithmic Requirements

A number of researchers felt that new algorithms need to be developed, for

example:

. efficient algorithms to find eigenvalues in parallel,

e fast Poisson solvers for massively parallel machines'

• methods for efficiently solving sparse linear systems,

• efficient domain decomposition strategies,

, algorithms to minimize communication time for traversal of distributed data

structures, and

• static and dynamic load-balancing methods.

What appear needed are truly parallel algorithms that are not directly connected

with the earlier sequential formulations.

9. Hardware Requirements

Lack of sufficient I/O bandwidth on parallel machines was cited as the biggest

single hardware limitation. Many users believe that the I/O bandwidth at each processor must at least equal that of sequential machines. Some users do not currently

face I/O performance bottlenecks but predict that the I/O capabilities will at some

point become a bottleneck. A number of users felt that parallel nLachines should

i offer a single I/O interface, Users felt that machines currently have a wide variety

i of I/O interfaceand s that thislackofstandardizationcausesproblems in porting

programs onto differentmachines.

Many usersfeelthat themachines they are usingshouldhave a highercommunicationsbandwidth and lowercommunication latency.Increasedmessage latency

Workshop on Grand ChallengeApplicationsand SoftwareTechnology 83
.,_,

• •

III I,III I III I II ii I illlli ii ilum I II III I I i I i I __

was cited as a particular bottleneck. Another frequently cited limitation was a

scarcity of usable memory at each processing node.

Some users felt the need for better floating-point performance and larger cache

size for the processors. Other users were unhappy that individual processors could

achieve only a fraction of the possible peak performance on most real problems.

10. Mathematical Problems Solved

The Grand Challenge teams are solving a very wide variety of problems drawn

from various scientific disciplines. In this brief summary of the survey results, we do

not attempt to give a comprehensive picture of what physical or biological problems

each team is attempting to solve and the mathematical formulations and numerical

algorithms they employ. We do present a partial list of the mathematical problems

being attacked by the Grand Challenge teams:

N-body force calculation

Solutions of the Schroedinger equation

Solutions of the Dira¢ equations

Flow solver: Navier-Stokes equations

Hydrodynamics

Magnetohydrodynamics

Maxwell's equations

Stochastic PDE (Brownian dynamics)

Poisson solution of implicit radiation transfer equations

Radiative transfer through absorbing and scattering media

Conductive solver: conduction equation

Integration of well-behaved, stiff., and beyond stiff.ODEs

Controls: 4th-order system of equations

Shapiro filter

Numerical integration of 2nd-order diffferentialequations

Integro-differential equations (aerosol physics)

Eigenvalue/eigenvector problems

Solution of linear systems

Nonlinear parameter optimization

Fourier transforms

Moving body dynamics: 6 degrees of freedom

Convolutions

Radiative transfer equations

Sequence comparison

Direct simulation of physical phenomena

Multidimensional pattern recognition

3D ray tracing through diffuse media

Coupled structural dynamics and Navier-Stokes equations

Coupled control, structural dynamics, and system identification

Coupled Maxwell and Ginsburg-Landau equations

84 Workshop on Grand Challenge Applications and Software Technology
I ii ii I ii i i i I IIii iii ii i I • ,,, ,,,,,m iii i iii ii ii . i ii i i ii

SECTIONC A bstractosfTalks

Panel A - Environmental and Earth Sciences

Chair: Yoa. Novak, EPA

A DISTRIBUTED COMPUTATIONAL SYSTEM FOR

LARGE-SCALE ENVIRONMENTAL MODELING

Armistead G. Russell

Carnegie Mellon University

Environmentalissuesare ofgreatsocietalconc,:rn.In the United Statesalone,

more than $30 billionper year isspent on airpollutioncontrol.A key premise

of this proposal is that high-performance computing environments and numerical

experimentation can play vital roles not only in unraveling the complex physics

and chemistry occurring in the environment but also in shaping the design of costeffective public policies. Most advanced environmental models are very computationally and data intensive. Distributed computer systems offer the potential to

revolutionize the current approach to environmental modeling.

The proposed enabling technologies--giga-performance heterogeneous computing systems, advanced software environments, parallel architectures, and Gbits/s

networks--offer the possibility not only of increasing the resolution but also of

providing the power needed to perform systematic sensitivity/uncertainty analyses, assimilateobservationadlata,and incorporatemore realistictreatmentof the

underlyingphysicaland chemicalprocesses.

Workshop on Grand ChallengeApplicationsand Software l,_chnology 85
I I II I I II i ] i i I ii ill iii I i I iii __

An initial application of a distributed computing environment is proposed that

exploits high-speed communications networks and special-purpose architectures.

Research will be carried out to improve the models and their mapping onto advanced architectures a_ad to create a flexible software environment that will enable

more widespread use of the tools. Extensive collaboration with industry, government, and other universities is planned.

COMPREHENSIVE EULERIAN AIR QUALITY MODELING

OF MULTIPOLLUTANT SPECIES IN THE ATMOSPHERE

Kenneth Galluppi and Daewon W. Byun

North Carolina Supercomputer Center and

Environmental Protection Agency

CurrentEPA comprehensiveairqualitymodels simulateeffectosf atmospheric

dynamics (transport,diffusionand , deposition),cloud processes(cloudmixing,

aqueouschemistry,and wet scavenging),photochemicaland thermalreactions(gas

phase reactionsof NO, NO2, CH4, CO2, formaldehydes,VolatileOrganic Compounds, nitratesSO_ , , sulfatesand , oxidants),reactionsof atmosphericparticulates,and anthropogenic and biogenic emissions on air quality. Many of the inputs

to the air quality models are provided by meteorological numerical models or objective analyses of observations. Because of strong dependency of air quality models

on meteorological models, both modeling activities are coordinated.

One of the objectives of the Models-3 project is to develop next-generation

air quality models with upcoming HPCC technology. The models will be used to

enhance progress in related science and algorithms as well as to provide easy-touse tools for environmental policy decision-making. EPA's Grand Challenge team

is assigned to d_ign a modeling system that is generalized to address multiple air

quality issues from urban to regional scales and that is modular to enable continuous

upgrading of science components. At a later stage, the models will be expanded to

handle multimedia pollutant issues.

LINKED WATER AND AIR QUALITY MANAGEMENT

Thomas O. Barnwell, Jr., and Robin L. Dennis

U.S. EPA

The objectiveosfthisprojectare to assistth the EPA's Chesapeake Bay Liaison Officeindevelopingnitrogencontrolstrategiefsormitigatingeutrophicationin

the ChesapeakeBay and tostudy thedevelopmentofmultimediapollutantcontrol

strategiesforairand water emissions.Tb_ projectwilllinkthe MM4 Mesoscale

MeteorologicalModel, the RADM RegionalAcid DepositionModel, theHSPF HydrologicaSlimulationModel - FORTRAN, a 3-DimensionalHydrodynamic Model

of the Bay, and a 3-DimensionalWater QualityModel ofthe Bay.

Initiaelmphasiswillbe on linkageofRADM and HSPF. Improvements inILADM

willfocuson developmentofa finergridscaletobetterresolvedepositionprocesses

86 Workshop on Grand ChallengeApplicationsand SoftwareTechnology
IIIL III II I IIIII I I I ii re|ill ii i i iiiii i i i I II I III I I I I I I I

Abatr_W of T_ks

and emission sources, making RADM very compute intensive and prompting its

movement to a massively parallel architecture. Additional work in HSPF will improve its ability to track nitrogen deposition in forests hrough the development of

process descriptions for nitrogen transport and transformation in forested ecosystems. These improvements are necessary to provide the proper linkage between

atmospheric deposition estimates and nitrogen inputs to the Bay throughout the

terrestrial pathway. These improved codes will be tested in the Chesapeake Bay

Watershed.

We will be exploring the use of the linked models in distributed and heterogeneous computing environments. Issues investigated will include transferring information between models both th rough simple data linkages and through more

dynamic coupling. Also being developed are more intuitive model interfaces and

decision support systems for generating control scenarios and analyzing model output, as well as exploring visualization techniques for model calibration, validation,

and assessment of control strategies.

FUTURE OF HIGH-RESOLUTION WEATHER PREDICTION

Alexander MacDonald

NOAA/Forecast Systems Laboratory

A review of the past 30 years of weather prediction shows that progress in largescale fields such as the 30-hour 500-millibar height has been quite remarkable. In

the same period, improvements of precipitation forecasts, such as the 0- to 24-

hour heavy-precipitation prediction issued by the National Meteorological Center,

has been surprisingly modest. It is well known in the meteorological community

that accurate prediction of cloud and precipitation is a difficult problem, both in the

short term (will it rain tomorrow?) and the long term (will the central United States

become much drier in the year 2050 as a result of greenhouse effects?) predictions.

NOAA's Forecast Systems Laboratory is taking a three-step approach to improving weather prediction. These steps are to greatly increase observations by

using (1) heterogeneous data, (2) four-dimensional assimilation, and (3) massively

parallel computers for high-resolution mesa)scale models. Each step will be reviewed,

with current results from sm_ !1and regional scales presented, and implications for

global models given.

High-performance computing and communications will improve weather prediction if a balanced program of increasing observations, meteorological models

knowledge, and computing speed is pursued nationally.

RESERVOIR MODELING FOR POROUS MEDIA:

PARTNERSHIP IN COMPUTATIONAL SCIENCE

Mary Wheeler

Rice University

Many of our groundwater resources are threatened by sources of pollution. The

challenge is to develop efficient and accurate computer models for predicting the

Workshop on C_'_.,_J Challenge Applications and Software Technology 87

i I I_ I I
- I I . I I I [I Ik_ _ I I ii ii ii [

distribution and movement of subsurface fluids in underground geological structures,

and to predict contaminant transport through underground heterogeneous porous

media. Important aspect_ of the problem include the interaction between subsurface

and surface water, the spatial locations of contaminant sources, chemical reaO,ions

that occur in the subsurface (e.g., aorption, precipitation, dissolution, complexation,

biological trm_formation), and the lack of meuured data to adequately characterize

the _luifer. Successful models should aid in the development mad monitoring of

remediation processes.

The first main problem is the numerical computation of fluid flow and transport

in porous media, which involves solution of a coupled system of neffirlyhyperbolic

and nearly elliptic parabolic equations for the fluid pressures and concentrations of

dissolved substances. Emphasis is on efficient, high-accuracy solutions, especially

in flows characterized by discontinuous or nearly discontinuous fluid and/or porous

material behavior. The second main problem is to quantify the uncertainty associated with simulation results as a function of the uncertainty of medium ard fluid

parameters, especially when important measurement data are unavailable.

We plan a two-phased approach. A typical flrst-phase computation will involve

a computation of single phase (i.e., in the saturated zone below the water table)

fluid motion involving possibly several fluid components which react and diffuse.

DOE sites suffering from groundwater contamination will be targeted for compuration. Second-phase computations will take into account multiphase flow both in

the vadose zone above the water table, and in the saturated zone. These problems

typically require the solution of hundreds of thousands to millions of nonlinearly

coupled unknowns, for simulation times from days to hundreds or even thousands

of years.

This is a collaborative effort with Oak Ridge National Laboratory, Brookhaven

National Laboratory, Rice University, SUNY Stony Brook, Texas A&M University,

and the University of South Carolina.

RESERVOIR MODELING FOR POROUS MEDIA:

LATTICE GAS COMPUTATIONS

Kenneth Eggert and Shiyi Chen

Los Alamos National Laboratory

We are developing enhanced oil recovery strategies. Our focus is on lattice gas

computations. We use two major codes to model pore-scale processes. The first

code, the lattice Boltzmann multiphase porous media model, has the capability

of modeling multiple pha_m of fluids flowing through complex porous media. It

includes variable contact angle and variable surface interface tension capabilities.

The second code, the percolation-network model, bases the description of both the

medium and the displacement process at a somewhat larger scale. Both codes are

run on the CM-200.

88 Workshop on Grand Challenge Applications and Software Technology
_. I _--- iiii inli ill In I I II I I I i i ii iii ii i ii | i il J|| i !l ii i i ii i illi iii i JllJlll i

Abstracts of T_lks

TEN. AND HUNDRED-YEAR CLIMATE PREDICTION

Dave Bader

Pacific Northwest Laboratory

The goal of our project is to predict climate change on ten- to one-hundred-year

time scales. To this end, we use models capable of estimating the range of possible

future climate variables on spatial scales that are relevant to human activities to

guide planning and policy development. The current approach is to couple models

describing the atmospheric circulation, the ocean circulation, land surface material

and energy exchange, and sea ice dynamics and thermodynamics. Algorithms and

codes for each of these climate subsystems are different. The largest and most

complex codes are the general circulation models (GCMs) of the atmosphere and

ocean. The atmospheric codes typically include submodels for the land surface,

and the ocean models can include sea-ice submodels. A variety of algorithms and

approaches is used for GCMs, which must be capable of simulating geophysical

fluid dynamics on appropriate scales. The approaches include finite-difference and

spectral formulations of the geophysical fluid systems in spherical coordinates. The

model codes typically run 10,000 to over 50,000 lines of Fortran code.

We are planning three types of computation: finite-difference atmospheric GCM,

spectral-formulation atmospheric GCM, and finite-difference ocean GCM. The enormity of these computations is illustrated by our effort in developing a finite difference atmospheric GCM. Currently, our model requires about 8 single-CPU-hours

per simulated year on a CRAY C90. The simulation we are planningawhich will involve a ten-year simulation of the global atmosphere and land-surface, with a threedimensional grid of about l0 s cells, and prescribed ocean and sea-ice distributions m

will require 140 single-CPU-hours.

DEVELOPMENT OF AN EARTH SYSTEM MODEL:

ATMOSPHERE/OCEAN DYNAMICS AND TRACERS CHEMISTRY

C. Roberto Mechoso

Universitofy California-Los Angeles

The goalof thisprojectisto developa coupledearthsystem model (ESM) of

atmosphericand oceaniccirculationsand chemicaltracers.The model development

uses the followingmajor components: (I)the UCLA generalcirculationmodel of

the atmosphere (AGCM), (2)the GFDL/Princeton Universitygeneralcirculation

model of theocean (OGCM), (3)the NASA Ames/UCLA chemical/aerosoltracer

model (CATM), and (4)theLawrence LivermoreNationalLaboratoryatmospheric

chemistrymodel (LACM). Versionsofthesecomponent models areoperational,and

preliminarycouplingofthecomponents has been carriedout.

The ESM willhave a highlymodular structure,in which components san be

interchangedor added with a minimum ofcomputationaldifficulty. The code will

be highlyoptimized,includinglocalparallelizatiwithin c each module as wellas

between modular components. Finally,thecoupledmodel willbe configuredtowork

Workshop on Grand ChallengeApplicationsand SoftwareTechnology 89
Q

III II I II - I 1 IIII II iiiii I111111 i I i I ii III i I II IIII I ii i ill i ii ii iiiii1(iii ii

in a _listributed (heterogeneous) computer environment across high-speed networks.

The AGCM, OGCM, CATM, and LACM are all grid-point models, which is an

adw mtage in parallel architectures. An important issue to be addressed in model

dev,:lopment is load balancing among processors, which may become more critical

as the parameterizations of physical and chemical processes grow in sophistication

and the computational load between model regions becomes more variable.

The ESM will be used to study the climate system, climate change, and climate/chemistry interactions. This work is divided into studies of the global chemistry of the atmosphere and the fundamental coupled dynamics of the global atmosphere and ocean system. The interannual variability of the coupled atmosphere

and oceans, including the seasonal cycle and transients up to decadal time scales,

will be simulated. We plan to couple the AGCM and CATM to study global ozone

depletion, including the effects of atmospheric aerosols on the chemistry of the upper atmosphere. The LACM will be used to develop and test photochemistry, and

heterogeneous chemistry algorithms will be used in the coupled AGCM/CATM.

The LACM will also be employed for tropospheric chemistry simulations using dynamical fields from the coupled AGCM/OGCM.

DEVELOPMENT OF ALGORITHMS FOR CLIMATE

MODELS SCALABLE TO TERAFLOPS PERFORMANCE

Dan Kowalski

NASA/Goddard Space Flight Center

Our project involves long-term climate modeling using a coupled ocean, atmosphere, land-surface, and sea-ice model. The atmosphere and ocean models use the

Arakawa C and B grid finite-difference schemes, respectively, for calculating fluid

dynamic evolution. The atmospheric model includes multiband short- and longwave radiation parameterizations; a stability-dependent, turbulent boundary layer

scheme; the Relaxed Arakawa Schubert (ILLS) scheme for penetrative convection;

and a mosaic land surface model. The ocean model is a reduced-gravity, layered

ocean model and includes a surface mixing parameterization at the ocean's surface.

Currently, a typical coupled ocean/atmosphere run of 30 simulated years requires

about 225 single-node CRAY Y-MP hours. We plan an initial implementation of

several prototype problems from our coupled climate model on several testbed massively parallel processors, including a six-node CRAY Y-MP, Ms.sPar MP-I, [ntel

DELTA, and CRAY C-90. We will then conduct a full implementation of an improved coupled model. Some of the prototypes are atmospheric dynamical core,

ocean dynamical core, shortwave radiation parameterization, cumulus parameterization, mosaic land surface model, and model diagnostic I/O.

90 Workshop on Grand ChallengeApplicationsand SoftwareTechnology
........

(I II'' ' ' ............ lilmlll T ii ii llV iii iiii]L_III ] I rlfl llllllm iiiiii L lJ

A _¢r_lsofTAlks

HIGH-PERFORMANCE COMPUTING AND 4D DATA ASSIMILATION,.

IMPACT ON FUTURE AND CURRENT PROBLEMS

Richard B. Rood; Presenter Peter Lyster

NASA/Goddard Space Flight Center

Our proposal focuses on the Grand Challenge of four-dimensional data assimilation to produce research-quality datasets for earth science studies. This involves

the collection of diverse earth-observational data sets and the incorporation of these

data into models of the ocean, land surface, and atmosphere, including chemical

processes. Ultimately the goal of data assimilation is the calculation of consistent,

uniform spatial and temporal representations of the earth's environment that can

be used for scientific analysis and synthesis.

The proposal has two major thrusts. First, we propose to apply leading research in parallel software and algorithms to a number of existing techniques for

assimilation that can benefit from greatly increased computer power. The operational global atmospheric data a_sirnilation system of the Data Assimilation Office

at the Goddard Space Flight Center, along with techniques for tracer advection and

chemical analysis, will be implemented on parallel software/hardware systems and

assessed both for performance and for development effort required. The product

of our work will be a new generation of integrated, operational models running on

heterogeneous parallel systems at orders of magnitude greater performance than

, the current operational system. The usimilation of reactive constituents is a high

priority in NASA environmental observation programs, including the Upper Atmosphere Research Satellite and the Earth Observing System, and this will be an

important step in making the assimilation of reactive constituents computationally

viable.

Second, we propose to explore new data assimilation methodologies that can be

implemented only with computers that far exceed the capacity of today's machines.

We will emphasize the development of the first three-dimensional full Kalman filter

scheme. We expect to exploit existing parallel system software as well as to develop

techniques that arise out of _he _9ecial requirements of data assimilation. Threedimensional Kalman fllteriwg schemes will challenge the limits of teraflops machines

for the foreseeable future.

The data assimilation effort at NASA requires the extension of atmospheric data

assimilation techniques to many disciplines of Earth Science. In order to develop a

successful assimilation system for Earth Science, there will be a continual need to

process and reprocess data sets with ever-improving and more complete assimilation

systems. There will also be a requirement to diagnose the quality of the data sets.

Data assimilation provides the most computer-intensive undertaking in NASA Earth

Science research. It is essential that assimilation algorithms and models be able to

exploit the future computing environment.

Workshop on Grand Challenge Applications and Software Technology 91

I IIII I I
" - i HBIIIIII]IIII I I IlillllII [ .................. Illllllll I I 11 .... ---_1

Panel B- Computational Physics

Chair: Tom Kitchens, DOE

UNDERSTANDING SOLAR ACTMTY AND HELIOSPHERIC DYNAMICS

i

John Gardner

Naval Research Laboratory

This project seeks to develop and implement advanced numerical algorithms

on parallel computers in order to solve Grand Challenge problems in solar and

heiiospheric physics. The effort will augment and be leveraged by existing programs

at the Naval Research Laboratory in both solar theory and computation. We have

proposed that NASA's HPCC initiative should have as one of its goals to understand

the solar driving engine: the mechanisms of solar activity and the dynamics of the

heliosphere.

Solar activity is the underlying driver for many important phenomena in space

physics. UV and X-ray radiation from the sun play dominant roles in the formation

of the earth's ionosphere and are thought to be partially responsible for the observed

variations in the south polar ozone hole. High-energy radiation from solar flares

strongly perturbs the structure of the earth's upper atmosphere and thus greatly

affects the lifetimes of NASA satellites in low earth orbit. Planetary magnetospheres

form and evolve u a result of the interaction between each planetary magnetic field

and the solar wind originating in the corona. Solar coronal mass ejections drive

geomagnetic storms and are responsible for electric power disruptions on earth.

The ejecta are also the origin of interplanetary shocks and energetic particles.

During the next decade, solar activity will be the focus of a number of NASA

space physics missions. The Gamma Ray Observatory is making observations of

7-ray emission from solar flares. YOHKOH is returning exciting data on X-ray

emission from flares and active regions for the first time giving us detailed information on the three-dimensional structure and magnetic topology of solar flares and

associated active regions, Ulysses is beginning its study of the solar wind at high

latitudes, promising to provide new insights into the wind's acceleration mechanism

and the three-dimeneional structure of the heliosphe_re. In the next few years, the

Solar and Heliospheric Observatory will begin study of the sun's inner and outer

corona and the heliosphere.

To realize fully the scientific return from these missions, we urgently need to expand our dedicated efl'ortin numerical and theoretical modeling into the massively

parallel generation of computation hardware. We propose a program of research to

develop new methods for numerical simulations on terafiops-range parallel computers and to apply this powerful new technology to outstanding problems in solar and

heliospheric physics.

92 Workshop on Grand Challenge Applications and Software Technology
.......... i11 ............... ii ][ [ i ..... , ..... _ rm I, , I, II Hill iiiii i ii IIIM ..... III " | I I .... I11

AbstrActs of T_lks

LARGE-SCALE STRUCTURE AND GALAXY FORMATION

George Lake; Presenter Calvin Lin

University of WMhinston

When we observe the universe, we use the dynamics and clustering of the luminous galaxies to infer the properties of the dark matter. Yet, even in our largest

simulations of large-scale structure (N -- 10s'a), a galaxy is represented by _., I00

particles. As a result, it diuolves by two-body relaxation in _,,10s years. Many scientiflc problems can be addressed only by following the luminous tracers for billions

of years. Testing models of the initial fluctuations by simulating their evolution

into present-day structures is paramount now that COBE has provided a normalization of the power spectrum on large scales, sad we have the correlation functions

of clusters and galaxies on smaller scales. In addition, we have increasing data on

the structure of the universe at earlier epochs. The Hubble Space Telescope has

enabled researchers to follow QSO absorption line systems to small enough redshifts

that galaxy association is unambiguous. Researchers are imaging the "anomalous

population of blue galaxies" in clusters at modest redshift, and recent investigations

promise to show us how clusters of galaxies have evolved. To understand these new

discGv,_ries,we must follow the evolution of the luminous tracers in the general sea

of cosmic structure.

Common problems emerge in the simulation of large-scale structure, the formstion of a cluster of galaxies, and the formation of a galaxy itself. For ,xample, the

rate of the dissolution of substructure on all three scales has be_om_ _ pc,pular way

of estimating the present value of the cosmological density paramet_, i_. Carlberg

used the merger rate of galaxies to argue for a value of fl near unity. Richstone,

Loeb, and Turner came to a similar conclusion using the substructure present in i

clusters of galaxies. Toth and Ostriker concluded that f] << 1 by considering the

accretion of satellites by disks. All of these ideas are exciting, but none are useful

without careful calibration against simulations with 10s or 109 particles.

Our goals are threefold: (1) to simulate the formation of a cluster in an appropriate cosmological context (fl = I only), total mass = 10Is solar masses; (2) to

simulate the formation of a galaxy in an appropriate cosmological context, total

mass = 1013 solar masses (_;-5 times the mass of the local group); and (3) to simulate large-scale structure (f} = l) and a Cold Dark Matter power spectrum, total

mass = 10is solar masses. These simulations will each have follow 3 × 107particles

for 3,000 iterations. We will run some of these simulations on the KSR (where we

are willing to perform calculations that take weeks) and also intend to do at least

one production run on the Intel Touchstone DELTA (where each simulation will

take roughly I day on the 512-node machine).

CONVECTIVE TURBULENCE AND MIXING IN ASTROPHYSICS

Robert Rosner

The University of Chicago

Turbulentconvectiopnlaysa centralroleinmany astrophysicacircu l mstances,

Workshop on Grand ChallengeApplicationsand SoftwareTechnology 93
.......... .- -- --_ _ ............... IL I1 I II]lJ Ill I I II I_11 IIII[I [ I [ IIMIII Ill[I Ill lll[ll _11

ranging from the transport of energy from stellar and planetary interiors to their

surfaces, to the mixing of specific angular momentum and dynamo activity in stars,

planets, and pcm,ibly accretion disks, to finally the highly dynamic mixing encountered in novae and supernovae. We propose to develop the next generation of multidimensional hydrodynamic codes to attack these problems. Specifically, we plan

to use massively parallel machines, establishing testbed and scalable codes from

several approaches. These simulations are unique in their simultaneous demands

for computing power_ data handling capabilities, and scientific visualisation. To address these demands, we have organised a team of researchers from the University

of Chicago, Argonne National Laboratory, the University of Colorado at Boulder,

Michigan State University, and the National Center for Atmospheric Research/High

Altitude Observatory.

SCALABLE HIERARCHICAL PARTICLE ALGORITHMS

FOR GALAXY FORMATION AND ACCRETION ASTROPHYSICS

Wojciech Zurek and M. Warren

Los Alamoe National Laboratory

Particle codes can be applied to a variety of astrophysical (and other) problems.

We are using a cosmological N.body treecode to study galaxy halo formation caused

by gravitational clustering. We are also developing a smooth particle hydrodynamic

code to study tidal disruption of stars in the vicinity of a black hole. These codes

have been implemented on parallel supercomputers (esF_cially on the Intel Touchstone DELTA). A major challenge is the development of techniques for analyzing

the tremendous amount of data.

This project is carried out in collaboration with researchers at the California

Institute of Technology, Syracuse University, the Pennsylvania State University,

and Mount Stromlo Observatory.

LATTICE GAUGE THEORY

Bob Sugar

University of California at Santa Barbara

We plan to carry out a broad program of research in lattice gauge theory. Lattice gauge theory addresses some of the most fundamental theoretical problems in

high energy physics and is directly relevant to the Department of Energy's experimental programs in high energy and nuclear physics. Our past work has included

studies of the mass spectrum of the strongly interacting particles, the behavior of

nuclear matter at high temperatures, the weak interactions of strongly ,nteracting

particles, and the Higgs sector of the electroweak interactions. A controlled calculation of the spectrum would provide a major test of quantum chromodynamics,

the theory of the strong interactions. It would also demonstrate that we are in

a position to make reliable calculations of physical quantities that are not as well

determined experimentally as the masses. An understanding of the transition between the state of ordinary matter and the high-temperature quark-gluon plasma

94 Workshop on Grand Challenge Applications and Software Technology
Centimeter

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 mm

i,,,,i,,,,i,,,,t,,,,i,,,,i,,,,i,,,,i,,,,i,,,,i,,,,i,,,, i,,,,i

1 2 3 4 5

Inches IIIII '.0 ,'__ _ 2.5 ,_ IIII1_II_._, 1_ rl 3 6

L. IIIliN

lilli"

• ,7._L

Iltl'_l

Illll '_siilli_,ill'._

HPlNUFRCTUREOTO PtTTIMSTRNDIqRDS --,_u.,,,q._'..:_i'_

o__ BY IqPPLTED THIqGE, ENC. _ _._%''::_

I Iii | i i I i[ i m i ii i i i ii i a ii i i i i iiiii i ii i iii i | ii i i i

A _tracta o[ Talks

and of the properties of the plasma is important for the interpretation of heavy-ion

collision experiments planned by nuclear physicists at facilities such as the Relativistic Heavy Ion Collider. It is also important for cosmological models. The study

of nonperturbative effects in the standard model of high energy physics is likely to

be a particularly fruitful area for lattice gauge theory, as it will bring the theory

into direct contact with ongoing experiments at the major DOE high energy physics

laboratories and with experiments planned at the Superconducting SuperCollider

Laboratory. In particular, lattice gauge theory provides the only known avenue

for making first-principle calculations of the effects of strong interactions on weak

processes. Therefore, it holds the promise of enabling us to make crucial tests of

the standard model. We propose to continue our work in all of these areu.

The study of lattice gauge theory requires enormous computing resources and

can benefit greatly from the new generation of massively parallel computers. In

anticipation of these high-performance computers, we have developed parallel code

that runs on a wide variety of platforms. At least initially, we plan to use the Intel

Paragon at Oak Ridge National Laboratory, for which the code has been specifically

tuned.

CONTROLLING CHAOS

C. Grebogi, E. Ott, and J. A. Yorke

University of Maryland

Our research is concerned with using small controlling perturbations to improve

the performance of a system that is in a chaotic state. Two closely related general

problems are being investigated: (1) control of long-term system behavior by location and stabilization of unstable periodic and steady-state orbits naturally embedded in the chaotic attractor, and (2) the rapid direction of an orbit to some target in

phase space. Our technique for the solution of the first problem can be implemented

without knowledge of a mathematical model of the system. The technique, which

involves the use of embedding methods to aJnalyze experimental data, has already

been carried out in simple experiments by other groups, including applications to

the control of a magnetoelastic buckling beam, arrhythmically beating heart tissue,

lasers, spin-wave systems, and thermally convecting fluid. Our research currently

concentrates on development of basic techniques for the solution of problems (1)

and (2) for low-dimensional attractors. In this case computational requirements

are not severe. However, we anticipate that the computational requirements will

rapidly become limiting as we try to move to higher-dimensional chaotic processes.

In particular, we foresee a need for large memory and rapid search capabilities.

RADIO SYNTHESIS IMAGING: AN HPCC APPLICATION

Richard Crutcher and Michael Heath

National Center for Supercomputing Applications

This project involves the direct implementation of three computing recommendations of the Astronomy and Astrophysics Survey Committee (chartered each decade

Workshop on Grand Challenge Applications and Software Technology 95
I i I i|1 111 iii ii i L i ill i

by the National Academy of Sciences and the National Research Council to survey

needs of astronomy). The project will implement a prototype of the next generation of astronomical telescope systems--remotely located telescopes connected by

high-speed networks to very high performance, scalable architecture computers and

on-line data archives, which are accessed by astronomers over Gbit/sec networks.

Specifically, a data link will be installed between the millimeter-wave synthesis array

at Hat Creek, California, operated by the Berkeley-Illinois-Maxyland Association

(BIMA) and the National Center for Supercomputing Applications (NCSA) at Urbana, Illinois, for real-time transmission of data to the high-performance, massively

parallel Thinking Machines Corporation CM-5. The very computationally intensive algorithms for calibration and imaging of radio synthesis array observations

will be optimized, and new algorithms that use the massively parallel architecture

will be developed. The project will experiment with and develop the techniques

for (1) connecting via HiPPI the MPP CM5 and a high-performance visualization

workstation (the SGI ONYX) at NCSA for distributed high-performance computing

and visualization, and (2) extending this distributed computing and visualization

environment by connecting a high performance visualization workstation at the

University of Wisconsin to the NCSA CM5 over the BLANCA gigabit/sec network.

Finally, the project will implement an on-line, digital archive of the BIMA radio

, telescope database and a digital library of processed images.

This research and development project will be carried out by a teen_nof experts

in radio astronomy, algorithm development for massively parallel architectures,

high-speed networking, and database management in conjunction with Thinking

Machines Corporation personnel. The development of this complete software, distributed computing, and data archive and library solution to the radio astronomy

computing problem will advance our expertise in high-performance computing and

communications technology and our knowledge of the astronomical universe.

Panel C- Computational Biology, Chemistry, and

Material Sciences

Chair: Bob Martino, N.[H

HIGH-CAPACITY ATOMIC-LEVEL SIMULATIONS FOR

DESIGN OF MATERIALS

WilliamGoddard

Caltech

We areworkingon two projectsone : inquantum chemistry,the otherinmolecular dynamics. In the quantum chemistryproject,we areconcernedwith systems

ofhundreds tothousandsofatoms. The pseudo-spectraalpproach isbeingused,as

wellas Hartree-Fock.The goal isto buildan algorithmthat willscaleas N 2. in

the moleculardynamics project,we aredealingwith 200,000to i millionatoms per

cell.Our approachisto dividethe system intoa hierarchyofcells.On a 30-node

KSR machine,thisapproach turnsout to be both fasterand more accuratethan

standard approaches.We are now convertingto the CM-5 and studyingways to

exploitdistributedmemory.

96 Workshop on Grand ChallengeApplicationsand SoftwareTechnology
- i i ii iiii i i i iiii i i ii i i

Abstracts of Talks

COMPUTATIONAL QUANTUM MATERIALS:

FIRST PRINCIPLES SIMULATION OF MATERIALS PROPERTIES

G. M. Stocks, B. N. Harmon, and J. W. Davenport

Oak Ridge National Laboratory, Ames Laboratory, and

Brookhaven National Laboratory

We combine developments in the first principles theory of the electronic structure and energetics of materials, atomistic molecular dynamics simulations, and

parallel processing to solve materials science problems of scientific and technological importance. Specifically, we are developing a range of methods for materials

properties simulation that are based on a quantum mechanical description of the interactions. The methods include tight binding molecular dynamics, Car-Parrinello

moleculardynamics,and locallyself-consistent multiplescatteringtheory.These

methods bridgethegap between traditionamol l eculardynamics and Monte Carlo

simulationmethods basedon classicailnteractionsand modern firsptrincipleselectronicstructuremethods based on densityfunctionaltheory.The new methods

willenable us to perform quantum-statisticaslimulationson systemscontaining

hundreds tothousandsofatoms. We use parallepro l cessingtechniquesinorderto

developefficientscalabletechniquesthatwillallowus tosolvematerialsproblems

currentlyout ofreach.

COMPUTER-AIDED DESIGN OF BIOLOGICAL

AND BIOMIMETIC MATERIALS:

COMPUTATIONAL DESIGN FOR CATALYSIS

W. J.Camp, J.S.Nelson,S.J.Plimpton,and M. P.Sears

Sandia NationalLaboratories

Our Grand Challengeprojectisthecreationofmassivelyparallelcomputational

toolsenablingthestudy ofcatalytiacnd biocatalytirec actionsfrom quantum mechanicaland atomisticviewpoints.The applicationareasincludemicrobiology(enzymaticreactionsinproteins)and energy(gasolincerackingviapolymericreactions,

catalyticreactionson crystaland clustersurfaces).We are developingsimulation

codes thatperformquantum mechanicalelectronistru c cturecalculationvia s local

densityfunctional,Hartree-Fock,and many-body formulationsand moleculardynamic and Monte CarlocomputationsusingempiricalforcefieldsMost . ofthework

is with codes we are writing ourselves; in other cases we are parallelizing existing

commercial codes (DMOL and Discover from Biosym).

Typical computations are diffusion of small polymers in zeolites, metalloporphyr

in trapping and reduction of carbon dioxide, active site analysis of metal clusters,

and charge density shifts in enzyme and cofactor reactions. More algorithm development is needed to carry out these computations effectively. For example, longer

timescale molecular dynamics simulations will need different algorithms that allow for much longer timesteps. Currently, performance is limited by the speed

of individual processors. We are exploring implicit methods and bond-constrained

techniques to overcome this barrier. Similarly, new serial and parallel algorithms are

Workshop on Grand ChMienge Applications and Software Technology 97
needed to improve the scaling characteristics of electronic structure calculations, to

enable larger atomic systems to be studied. We are exploring two promising linear

scaling methods: one new algorithm is being explored that exhibits explicitly linear

scaling (i.e., the computational work to simulate N atome is A. N, with A being

a modest-sized number). In another project we are developing a massively parallel

version of Yang's divide-and-conquer algorithm which also scales linearly.

In conjunction with the Grand Challenge, an optimized parallel library of densematrix linear algebra and finite-difference decomposition routines has been developed. The library includes MIMD routines with C and F77 interfaces for finitedifference operations, FFTs, many BLAS Level 1 and 2 operations, matrix transpose and multiply, Cholesky and LU decompositions, and Hermitian eigenvalue

problems, as well as parallel graphics.

HIGH-PERFORMANCE COMPUTATIONAL STRUCTURAL BIOLOGY

JerryE.Solomon

CalifornInst ia itutoefTechnology

The ComputationalStructuralBiologyprojectisfocusedon twospecificsience

problems:(1)de novoproteinthree-dimensional structurpre edictionfromprimary

aminoacidsequence;and(2)predictivceomputationalmodelsofcellularogandies.

Both oftheseproblems,inadditiontobeingcomputationallyintensivsehar , ethe

necessityofbuildinngew programmingand usertoolsforexplorinthg efulldetails

oftheproblem.

We havedevelopeda hierarchicalomputationalapproachtothedenovopredictionproblemwhichutilizesa novelstatiMont c e Carlomethodforgeneratinlar g ge

ensemblesofcandidatprot e eiban ckbonestructureson anfcclatticase thetop-level

(coarse-grain) step. Our lattice model is capable of naturally incorporating known

protein folding constraints into the chain generation process, and the algorithm is

easily parallelized for use on large-scale massively parallel processors (MPPs). The

lattice model, although a drastic simplification of the real conformational space of

proteins, is able to capture much of the fundamental physics and chemistry of the

folding process, including solvent accessibility effects. We illustrate the algorithm

and some early results used for validation on serial workstations. The predictive

computational model problem for cellular organelles focuses on building accurate

(testable) models of the flagellar axoneme and the tRNA transcription operations

of the ribosome. These models utilize the so-called teleological approach, wherein

the problem is abstracted in terms of the system behavior rather than explicitly in

terms of the problem constraints.

On the HPCC technology development side, we are concentrating on building

sophisticated tools that enable both applications programmers and science users

to take full advantage of the computational power afforded by large-scale MPPs

to explore alternative approaches to problem solutions. In particular, we are attempting to build what we term a "numerical laboratory," which allows users to

literally conduct experiments much in the same way as an experimental biologist

would carry out experiments in a wet lab. These tools, implemented in CC++ (a

parallel, object-oriented language) developed at Caltech, are being used to construct

a protein-folding workbench and a predictive computational modeling workbench.

98 Workshop on Grand Challenge Applications and Software Technology
i

,f •

............

iiii i iii i i i ii i iii i ii i i iiii | i

Abstract_ of Ttlks

Both oftheseworkbenches arecapableofoperatingatthreedistincltevels,dependingupon thecomputer literacofy the user,and aredesignedto be easilyextensible

infunctionalcapability.

HIGH-PERFORMANCE COMPUTATIONAL CHEMISTRY:

SCALABLE ALGORITHMS FOR GRAND CHALLENGE APPLICATIONS

T. H. Dunning, Jr., and A. Wagner

Pacific Northwest Laboratory and Argonne National Laboratory

The High-PerformanceComputational Chemistry projectisa multilaboratory

collaborationto developthe computationaltechnologyrequiredto solveGrand

Challenge-classproblems in computationalchemistry.This focusof thisproject

ison the developmentofthenew algorithms,software,and diagnosticsthatwillbe

requiredto fullyexploitmassivelyparalleclomputers capableof terafloppes rformance. In particulareff , ortces nteron the developmentof algorithmsthatscaleto

largernumbers of processorsu,se hierarchicalmemory much more effectively, and

make much smallerI/O demands than currentmethods. The goal isto develop

modeling systemsthat arecapableofsolvingproblemsordersofmagnitude larger

than currentlypossibleTh. e work isfocusedon a suiteofcomputationalchemistry

modeling methods appropriateto threespecifipcroblemareasofurgentinterestot

the U.S. Department of Energy and the chemicaland petroleum industries:the

incinerationofhalohydrocarbons,the interactionofsoilmineralswith pollutants,

and the redesignofbiodegradativenzymes.

Work isunder way on the development of performancemodels-and code for

a fullydistributedHartree-Fockprogram for computing the electronisctructure

of molecules,scalablemethods fordenseeigensystemsolutionand , improved data

sharingand data communications paradigms forcomputationalchemistrycodes.

In the futurethiswork willbe extendedto includescalabletechniquesfortreating

electroncorrelation(bothperturbationtheoryand configurationinteractiontechniques)and forcoupledsemi-empiricaellectronicstructurem-oleculardynamics.

The High PerformanceComputationalChemistry projectisa collaborationinvolvingthe PacificNorthwest Laboratoryand Argonne NationalLaboratory,with

participationby fiveindustriaclollaborator(Alli s edSignal,Amoco, DuPont, Exxon

and Phillips).

HIGH-PERFORMANCE COMPUTING IN STRUCTURAL BIOLOGY

AND MEDICAL IMAGING

Robert L. Martino

NationalInstituteofs Health

Scalableparallelcomputer architecturesprovidethe computationalratesdemanded by advanced biomedicalcomputing problems.NIH has developeda number of parallelalgorithmsand techniquesusefulindeterminingbiologicastru l cture

and function. These applicationins cludeprocessingof electronmicrographs to

Workshop on Grand Challenge Applications and Software Technology 99

i ! i i i! II I

i
determine the three-dimensional structure of viruses, using ab initio quantum mechanical methods to investigate the energetics of how anti-cancer drugs affect the

DNA in cancer cells, applying molecular dynamics to study the affects of hydration

on protein structure, calculating the solvent accessible surface area of proteins to

predict the three-dimensional conformation of these molecules from their primary

structure, and aligning two three-dimensional positron emission tomography images

for brain function studies. Timing results demonstrate substantial performance improvements with parallel implementations compared with conventional sequential

systems.

HIGH-PERFORMANCE IMAGING IN BIOLOGICAL RESEARCH

D. LansingTaylorand ScottE. Fahlman

CarnegieMellon University

The goal of the Automated Interactive Microscope project is to bring highperformance computing to bear on the image processing, analysis, storage, and

display problems associated with advanced fluorescence microscopy in the biological sciences. We will investigate a wide variety of image processing, display, and

, recognition algorithms. For example, we will perform nearest neighbor and minimum entropy deconvolutions to extract a clear three-dimensional image from a set

of image planes at different depths in the specimen. Part of our research will focus

on determining which algorithms are most useful. We plan to use graphics workstations (e.g., the SGI Onyx/2 with VTX graphics attached to the microscope) and,

via a high-speed link, run on the CRAY C90, T3D, and CM-2 at the Pittsburgh

Supercomputing Center.

We believe that this research will change the practice of microscopy in at least

four important ways: (1) we will be able to run current image-processing and display

ablegoaribthms le to iemp n real loy time mu,chprmore ovidingambitiou timelys feeprdboack cessingfor thaned experime display ntalger;orithms (2) wethawinll

are used currently, allowing the experimenter to see the data in new ways; (3) we

' will be able to automate some of the recognition tasks associated with biological

experiments, making it possible for the system itself to detect and record interesting

events, where "interesting" is defined by the experimenter; and (4) once the system

can recognize interesting events, it can perform certain experimental interventions

automatically.

COMPUTATIONAL QUANTUM MATERIALS:

QUANTUM STRUCTURE OF MATTER

T. Barnes,V. Overacker,M. R. Strayer,S. Umar

' Oak Ridge NationalLaboratory,VanderbiltUniversity

This proposalcenterson threecomputationalmethods to apply classicaland

quantum fieldtheory to atomic,nuclear,and condensed matter physics.These

100 Workshop on Grand ChallengeApplicationsand SoftwareTechnology

!
are (1) the time-dependent Hartree-Fock method applied to collisions and structure of atoms, molecules, and nuclei; (2) Monte Carlo perturbation theory applied

to processes in accelerator and detector physics, notably FtHIC; and (3) hydrodynamic_ applied to relativistic nuclear collisions and plasmas. Our objective is to

advance the study of matter under strongly interacting conditions, as encountered

in many branches of energy research, including fusion plasmas, lasers, and accelerators. While many of the proposed methods are transferable into applied areas, our

emphasis is on understanding systems simple enough that results of high reliability

are possible. Part of our proposed research deals with matter at energies below one

MeVman area that is also relevant to quantum chemistry and materials research.

The remainder of our investigations deal with matter at higher energiesman area

unique to the Grand Challenges.

COMPUTER-AIDED DESIGN OF BIOLOGICAL

AND BIOMIMETIC MATERIALS:

PHARMACEUTICALS AND AGROCHEMICALS

Mike Colvin

Sandia National Laboratories

The U.S. pharmaceutical and agricultural chemical industries are unequaled in

the world. Not only do U.S. firms hold the largest share of the worldwide market, they are the source of most innovative new products in the biomedical and

agrochemical markets. Both industries differ from most manufacturing industries

in that a much larger fraction of their revenue goes into research and development

of products for which they have only a few years of exclusive license protection. For

example, a typical pharmaceutical drug takes 12 years to develop and get approved,

leaving only 5 years of patent protection in which to recoup hundreds of millions of

dollars in development costs. Hence, there is an enormous competitive advantage

in shortening this development time. Moreover, traditional, exhaustive screening

methods of designing new drugs and pesticides are proving inadequate. Instead,

""rational" methods are being sought to design pharmaceuticals and pesticides with

predictable potencies.

The Center for Computational Engineering at Sandia is collaborating with U.S.

industry and universities to use new computational tools to solve problems is

biomedical and agricultural chemistry. Sandia has unique experience in using highly

accurate ab initio quantum chemical methods for large chemical systems. These

very methods are far more computationally intensive than traditional semi-empirical

methods and require extensive development of new algorithms and implementation

on massively parallel supercomputers. In addition, Sandia has extensive capabilities

in computational modeling of materials and molecular dynamics simulations.

Sandia is collaborating with researchers at a major U.S. pharmaceutical and

several universities and medical schools to apply computational chemistry to solve

problems in pharmaceutical and agrochemical design. The collaboration with the

pharmaceutical company has involved computer modeling of numerous pharmaceuticals including anti-viral drugs and diagnostic imaging agents. Sandia is also

working with the Johns Hopkins Medical School to develop more potent analogs

Workshop ou Grand Challenge Applications and Software Technology 101
of the widely used anticancer drug cyclophosphamide. Sandia performed a series

of ab initio quantum chemical calculations of cyclophosphamide metabolites bound

to DNA, which constituted some of the largest quantum chemical calculations ever

performed. Another collaboration is with the Department of Entomology at the

University of California at Davis to help develop a new type of highly potent and

environmentally save insecticide. Sandia is also collaborating with New York University and Lawrence Livermore National Laboratory to study how environmental

carcinogens bind to DNA.

TEXAS CENTER FOR ADVANCED MOLECULAR COMPUTATION

Roland Giowinski, J. Andrew McCammon, B. Montgomery Pettitt,

and L. Ridgway Scott

University of Houston

High-performance computing can solve critical problems in biomolecular design,

once several issues are resolved. The most obvious problem relates to the difficulty

! in using advanced computer architectures. In addition, new algorithms (mathematical and computer science-based) and methods (physical scienc(oased) are needed

to solve problems with reasonable efficiency. Finally, to guide and provoke the development of both algorithms and software, scientists need a detailed understanding

of what types of problems can be treated with vastly improved computing resources.

We address all of these research areas in this proposal.

We will utilize and enhance parallel programming languages developed both by

our group and by others to facilitate development and maintenance of complex algorithms for diverse high-performance architectures. One goal is to achieve supercomputer performance (and better) on low-cost parallel computers for commercial-grade

codes with minimal programmer effort.

Examples of algorithms that we will study include fast summation methods for

nonbonded forces in molecular dynamics, more efficient (implicit) time stepping

schemes in molecular dynamics (which preserve energy), the use of multigrid and

domain-decomposition techniques to resolve the electrostatic force distributions in

the Brownian dynamics model, and parallel algorithms and data structures with an

emphasis on load balancing.

The following Grand Challenge problems will be studied to improve our understanding of what can be resolved computationally (and what cannot): (1) thermodynamic cycle perturbation theory applied to ligand-binding proteins, (2) Poisson.

Boltzmann/Brownian dynamics applied to protein design, (3) non-Boltzmann sampling and transition-state trajectories ofligand binding, (4) quenched high-temperature

dynamic peptide mimetic design, and (5) grand canonical dynamics applied to salt

solutions of nucleic acid triple helices.

102 Workshop on Grand Challenge Applications and Software Technology

ii i _ iiii ii
- II I III u I n Ill I I I I II I I I J[lll I _

A batr_t_ of TLIi_

Panel D - Computational Fluid and Plasma Dynamics

Chair: Lee Hoicomb, NASA

HIGH-PERFORMANCE COMPUTATIONAL METHODS FOR

COUPLED FIELDS AND GAFD TURBULENCE

Carlos FeLippa and Juri Toomre

University of Colorado at Boulder

This project represents a merger of two HPCC Grand Challenge proposals:

"High-Performance Computational Methods for Coupled Field Problems" and "Coherent Structures and Dynamics of Geophysical and Astrophysical Turbulent Flows."

The merged project involves a team of researchers at the University of Colorado at

Boulder, the University of Colorado at Denver, the University of Minnesota, and the

National Center for Atmospheric Research. This team will (1) develop and implement algorithms and software on massively parallel computers for solving coupled

field problems in structural and fluid dynamics and (2) study turbulent flows that

arise in geophysical and astrophysical fluid dynamics (GAFD).

The first component considers three problems: three-dimensional aeroelasticity,

massively actuated "adaptive" structures, and electromagnetic-thermomechanical

interaction. The emphasis in this component is on investigating benefits of finiteelement-based partitioned analysis methods and heterogeneous parallel computation.

The second component considers four problems: geostrophic turbulence, oceanic

convection, deep convection in planetary atmospheres, and stellar compressible convection, all constrained by effects of rotation and stratification. The emphasis in

this component is on exploiting massively parallel architectures to increase the resolution of in three-dimensional simulations employing (variously) pseudo-spectral,

finite-difference, multigrid, and PPM approaches in studying the intense turbulence

encountered in planetary and stellar settings, including the large-scale coherent

structures and mean flows that can coexist with such turbulence.

Computer science challenges paced by these problems include performance modeling and analysis, performance evaluation tools, program transformation and automatic data distribution, heterogeneous computation, very large scientific databases,

and high-performance visualization.

MULTIDISCIPLINE SIMULATION OF HIGH-SPEED CIVIL TRANSPORT

M. D. Salas

NASA/Langley Research Center

In the past two decades, a foundation has been established for the advanced

design of aerospace vehicles in mathematics, computer software and hardware, and

engineering practices, leading to a new emerging technology called multidisciplinary

design and optimization (MDO). This technology requires the repetitive analysis of

Workshop on Grand Challenge Applications and Software Technology 103
the vehicle structure, flow field, propulsion system, and controls as design variables

are changed in the search for an optimum. The number of analyses required is

proportional to the number of design variables that are allowed to change. For

a typical high-speed civil transport configuration, the number of design variables

can reach several hundreds. Since each analysis could take several hours of CPU

time on a CRAY Y-MP computer and could require a Gigaword or more of memory, this Grand Challenge imposes severe requirements on computer resources. The

difference in requirements imposed by the various disciplines lends itself to a heterogeneous computer implementation. In addition, the problem requires access to

large, dynamic databases from different computer architectures and the rapid interrogation and visualization of this data. We explore the computer requirements

both in terms of hardware and software needed to develop a framework for MDO.

MULTIDISCIPLINE SIMULATION OF HIGH-PERFORMANCE AIRCRAFT

i K. Chawla

NASA/Ames Research Center

This project involves comparing the performance of high-performance aircraft

, computational fluid dynamics simulations on the parallel machines Intel iPSC-860

and TMC CM-2 and on the vector machine CRAY Y-MP. The problems under

consideration require the of multizone overset and solution of the

use grid topology

Navier-Stokes equations. Our focus is on issues such as performance degradation

on the Intel resulting from heavy message passing while solving all grids in parallel,

,

i strate naturegy

oftothemapCMgrids -2. to processors, and disadvantages arising because of the SIMD

We have compared the performance of various matrix solvers on different parallel

machines, focusing on a single discipline application--specifically, the simulation of

flow past a delta wing with thrust reverser jets in ground effect. We are also

! . exploring strategies to map multidiscipline problems to the parallel machines, in

particular, the Intel.

_" MULTIDISCIPLINE SIMULATION OF PROPULSION SYSTEMS

Russell W. Claus

NASA Lewis Research Center

Implementing new technology in aerospace propulsion systems is becoming prohibitively expensive. One of the major contributors to the high cost is the need

to perform many large-scale system tests. The traditional design analysis procei dure decomposes the engine into isolated components and focuses attention on each

single physical discipline (e.g., fluid or structural dynamics). Consequently, the in1 teractions that naturally occur between components and disciplines can be masked

by the limited interactions that occur between individuals or teams doing the design

and must be uncovered during expensive engine testing. This situation can pose

serious problems for a highly integrated propulsion system, where tight coupling

can produce unforeseen interactions having adverse effects on system performance.

104 Workshop on Grand Challenge Applications and Software Technology

..........

bl ill
- ii i i i[11ii i ]illil i ii ii ii iii RII II i i i iii iiii i [ iii i ii ill i iiii

Abstracts of Talks

For example, the installation of an improved efficiency turbine into an existing engine can result in a rebalancing of the engine cycle, with tile consequence that the

compressor is driven at a less efficient operating point, thereby reducing the total

system performance as measured by specific fuel consumption or some other system

parameter. As a result, the introduction of advanced technology into propulsion

systems takes many years and is prohibitively expensive. New design techniques

that combine concurrent engineering and numerical simulation may provide a system design solution.

We discuss a cooperative effort of NASA, industry, and universities to integrate disciplines, components, and high-performance computing into a Numerical Propulsion System Simulator (NPSS). NPSS is involved in the development

of propulsion models for the computational aerosciences Grand Challenges. Development of propulsion modules requires advances in the following technologies: (1)

interdisciplinary analysis to couple the relevant disciplines such as aerodynamics,

structures, heat transfer, chemistry, materials, controls; (2) integrated system analysis to couple d subsystems, and components at an appropriate level of detail; (3)

a high-performance computing platform composed of a variety of architectures, including massively parallel processors, to provide the required computing speed and

memory; and (4) a simulation environment that provides a user-friendly interface

between the analyst and the multitude of complex codes and computing systems

that will be required to perform the simulations.

COMPUTATIONAL FLUID AND COMBUSTION DYNAMICS

i

Phil Colella

University of California- Berkeley

We are studying computational fluid dynamics as applied to combustion problems. Specifically, our focus is on three-dimensional, time-dependent, turbulent

fluid-flow problems, in a variety of Mach number regimes ranging from extremely

subsonic to hypersonic, and in complex physical geometries. Our numerical methods

are based on finite difference methods on block--structured, dynamically adaptive

grids in which the total number of grid points and their distribution in space change

as function of time and the solution. We also use two techniques for representing

complicated boundary geometries: composite overlapping meshes (e.g., CMPGRD,

Chimera), and Cartesian grid/volume-of-fluid representation of geometry. We exploit visualization--in particular, animations generated directly from a massively

parallel processing system--to gain new insights into the phenomena. To date,

our principal limiting factor has been the capability to run very large simulations.

MPP systems promise to provide the necessary resources to address truly Grand

-_ Challenge-scale problems.

Workshop on Grand Challenge Applications and Software Technology 105
NUMERICAL TOKAMAK PROJECT

Dan Barnes

Los Alamos NationalLaboratory

The Numerical Tokamak Project (NTP) is applying high-performance computing and communication to the most elusive aspect of tokamak design and operation m

the problem of losses of plasma particles and energy. These losses always exceed the

rates calculated for a completely static plasma because of fluctuations in which a

large number of the confined particles move together (collectively). Such collective

fluctuations enhance transport of heat in much the same way that fluid fluctuations

(turbulence) lead to enhanced heat transport rates, for example, because of waves

in the ocean or eddies in the atmosphere. The understanding of this process and

its scaling as machine parameters are increased is critical to tokamak development.

While impressive progress has been made, a key difficulty that remains is the detailed calculation of the solution of plasma transport models. The recent increase in

the computing power available from massively parallel computers has made feasible

the direct numerical solution of these models.

Unique physical features of this problem include the representation of the toroidal

geometry, in which magnetic field lines travel on nested tori with a varying winding

number, nonlocal response associated with the almost free motion of plasma particles along this magnetic field, nonlocal particle interactions via the electromagnetic

field, and removal of fast time scales and/or small space scales by analytic or numerical approaches. The particular numerical difficulties that are introduced include

use of general coordinates with nonstandard periodicity conditions, nonlocal dependencies associated with the solution of nonconstant coefficient elliptic problems,

and nonlocal mapping of particle lists onto grid lists.

The NTP has developed both fluid and particle (kinetic) approaches for the

study of tokamak turbulent transport. The former offers the advantages of no

numerical noise and relatively few degrees of freedom per physical mesh point.

The latter faithfully represents the long-mean-free path effects in a much more

computationally intensive approach, and introduces the additional complication of

sampling noise. Typical fluid calculations include about half a million degrees of

freedom, while typical particle calculations use one to several million particles.

Unique numerical techniques developed to deal with the NTP problem include

domain decomposition as a means of partial localization of particle information,

fast parallel FFTs, convolutions, and linear system solvers, and conjugate gradient elliptic solvers. These approaches are described in terms of their numerical

implementations on various (massively) parallel platforms (CRAY, TMC, and Intel). Network and visualization requirements to support these algorithms and their

applications are also described.

106 Workshop on Grand ChallengeApplicationsand SoftwareTechnology
Panel E- Applications of Artificial Intelligence

Chair: Bob Voigt, NSF

HIGH-PERFORMANCE COMPUTING FOR LEARNING:

SUPER-HUMAN SUPERCOMPUTING vs SUPER-COMPUTING

FOR HUMAN COMPUTATIONS

Robert C. Berwick and Tomaso Poggio

Massachusetts Institute of Technology

Many supercomputer problems involve important scientific calculations carried

out at superhuman speeds: physical simulation of molecular structure through quantum mathematics, hydrodynamics, weather and atmospheric prediction, and many

others. Another set of Grand Challenge problems underscores a rather different

kind of high-performance calculations: those carried out by the human brain, especially involving learning and perception. From this perspective, these human-like

problems stretch the limits of sample and memory complexity, rather than the

computational complexity traditionally linked to supercomputing. This distinction

illuminates once again the difference between super-human supercomputing and

super-computing for human computations.

By envisioning a simple conversation with two people, we illustrate the pervasiveness of memory-intensive learning and brain-like supercomputing in everyday

human tasks, what computational progress has been made and what new computational demands are entailed, and how this work points the way to new supercomputer architectures suited to nonscientific, human-oriented Grand Challenge

problems.

DATA ANALYSIS AND KNOWLDEGE DISCOVERY

IN GEOPHYSICAL DATABASES

Richard Muntz; Presenter Carlo Zaniolo

University of California- Los Angeles

This project will demonstrate the technical feasibility of building sophisticated

information systems for geophysical databases and the benefits of such systems in

promoting both theoretical and experimental research in the earth sciences. The

testbed includes atmospheric model data, satellite stratospheric data, and climate

data. Characteristic of these applications are the identification and monitoring of

complex patterns and their evolution over space and time. We will study novel

indexing and abstraction techniques for efficient search and monitoring of massive data sets, and the optimization of complex spatial-temporal queries and rules.

An implementation based on a mass-storage system and an intelligent front-end is

planned. Supercomputer testbeds will be used for parallel search and computation

intensive functions.

Workshop on Grand Challenge Applications and Software Technology 107
j I j

i i ii iii iii I II r i i i iii

SECT,OND Workshop Invitees

I

Robert Allen, AT&T

Lou Auslander, CUNY

Dave Bader, DOE Office of Health sad Environment_ Research

Mike Bailey, San Diego Supercomputer Center

Ron Bailey, NASA Ames Research Center

Raymond Bsir, Pacific Northwest Laboratory

Dan Barnes, Los Alamos National Laboratory

Thomu Barnwell, Environmental Protection Agency

Alan Barr, Caltech

Forest Baskett, Silicon Graphics Corp.

Isabel Beichl, National Institute of Standards & Technology

John Bell, DOE

Marsha Berger, Courant Institute

Donna Bergmsrk, Cornell University

Fran Berman, University of California - San Diego

Robert C. Berwick, Massachusetts Institute of Technology

Kenneth Birman, Cornell University

Alan Blatecky,MCNC

MarjorieBlumenthal,NationalAcademy ofSciences/Nat.ResearchCouncil

RonsldBoisvert,NIST

Jay Boris,NavalResearch Laboratory

Lou Branscomb,John F.KennedySchoolforGovernment

JosephBredekamp,NASA Headquarters

AlfredBrenner,SupercomputingResearchCenter

FredericBkrooks,Jr.,UniversityofNorthCarolina

RobertBrown,SiliconGraphics,Inc.

Workshopon Grand ChallengAepplicationasndSoftwareTechnology 109
- ill i i i i i i i i a m.m I II I II ii I ii al i

J.C. Browne, UniversityofTexas

Duncan Buell,Supercomputing ResearchCenter

Henry Burkhardt,KendallSquare ResearchCorp.

Ralph Burnstein,ElectricaPol wer ResearchInstitute

James Burrows,NationalInstitutoefStandards & Technology

DaeWon Byun, EnvironmentalProtectionAgency

BillCamp, Sandia NationalLaboratories- Albuquerque)

David Cargo, NationalSecurityAgency

John Carlson,Cray Research,Inc.

WilliamCarlson,Supercomputing ResearchCenter

Pat Carson,NationalInstitutesofHealth

T. L.Casavant,UniversityofIowa/ParallelProcessingLab.

CharlieCatlett,UniversityofIllinoisNCSA

John Cavallini,Department ofEnergy

Kalpana Chawla, NASA Ames ResearchCenter

Y. T. Chien,NationalScienceFoundation

Alok Choudhary, SyracuseUniversity

Mel Ciment, NationalScienceFoundation

CharlesClark,NIST

RusselClaus,NASA Lewis ResearchCenter

CarlieCoats,North CarolinaSupercomputing Center

Torn Cole,U.S. Army EngineerWaterways Experiment Station

PhillipColellaUn, iversityofCalifornia-Berkeley

Stephen Colley,Ncube

MichaelColvin,DOE

George Cotter,NationalSecurityAgency

Thomas Crocker,NASA LangleyResearchCenter

RichardCrutcher,UniversityofIllinois

ErnieDaddio,EnvironmentalInformationService

James Davenport,Brookhaven NationalLaboratory

Anil Deane, NASA Goddard Space FlightCenter

ViktorDecyk, Jet PropulsionLaboratory

Torn DeFanti,UIC/EECS

Robin Dennis,EnvironmentalProtectionAgency

CarletonDeTar, UniversityofTennessee

John Dennis,Rice University

Dave DeWitt, UniversityofWisconsin

CharlesDickens,OSTP

Pete Dillingham,Cray ResearchInc.

Jack Dongarra,UniversityofTennessee

John Dorband, NASA Goddard Space FlightCenter

Shawn Downing, NOAA

Richard Draper,Supercomputing ResearchCenter

Kevin Drogenmeir,UniversityofOklahoma

Thorn Dunning, PacificNorthwest Laboratory

Doug Dwoyer, NASA LangleyResearchCenter

Roger Dyson, NASA Lewis ResearchCenter

Todd Elvins,San Diego Supercomputer Center

Robert Ewald, Cray Research,Inc.

tL0 Workshop on Grand ChallengeApplicationsand SoftwareTechnology
_ II I II I II II I III I II I I II III I I I i I

Workshop In vitees

Richard Ewing, Texas A&M University

Carlos Felippa, University of Colorado at Boulder

Jack Fellows, DOE Office of Management and Budget

Jeanne Ferrante, University of Colorado

Robert Ferraro, Jet Propulsion Laboratory

James Fischer, NASA Goddard Space Flight Center

Jon Flower, ParaSoft Corp.

Greg Follen, NASA Lewis Research Center

fan Foster, Argonne National Laboratory

Geoffrey Fox, Syracuse University

Joan Francioni, University of Southwestern Louisiana

RichardFreund,NCCOSC RDTE

RichardFriesnerColu , mbia University

Kenneth GaUuppi, North CarolinaSupercomputer Center

DennisGannon, IndianaUniversity

John H. Gardner,Naval ResearchLaboratory

AI Geist,Oak Ridge NationalLaboratory

LouisGerstner,IBM Corporation

John H. Gibbons,OfficeofScienceand TechnologyPolicy

Norman GIick,NationalSecurityAgency

William A. Goddard, CaliforniaInstitutoefTechnology

Robert Goldberg,Government Operations

Mel Goldstein,NASA Goddard Space FlightCenter

Gene Golub,StanfordUniversity

Howard Gordon, Supercomputing ResearchCenter

Al Gore,White House

David Gottlieb,Brown University

Steven Gottlieb,IndianaUniversity

LeslieGreengard,Courant InstituteN,ew York University

SteveGriffinN,ationalScienceFoundation

Nico Habermann, NationalScienceFoundation

Chuck Hansen, Los Alamos NationalLaboratoryACL

Anthony Hearn, RAND

Robert Harrison, Battelle-PNL

Michael Heath, University of Illinois

Kristin Hessenius, NASA Headquarters

Danny Hillis, Thinking Machines Corp.

Dan Hitchcock, Department of Energy

Jeff" Holand, U.S. Army Engineer Waterways Experiment Station

Lee Hoicomb, NASA Headquarters

Terry Hoist, NASA Ames Research Center

Sally Howe, National Institutes of Health

Fred Howes, DOE

Paul Hunter,NASA Headquarters

Mack Hyman, Los Alamos NationalLaboratory

SteveIsakowitz,DOE OfficeofManagement and Budget

Barry 2acobs,NASA Goddard Space FlightCenter

Van Jacobson,Lawrence BerkeleyLaboratory

CalvinJohnson,DCRT, NIH

Workshop on Grand Challenge Applications and Software Technology 111
4, '

I

Frederick Johnson, National Institute of Standards & Technology

Gary Johnson, Department of Energy

Lennart Johnsson, Thinking Machine Corp.

Anita Jones, University of Virginia

Bob Kahn, CNRI

Jeffrey Kalb, MasPar Computer Corp.

Charles Kalina, National Institutes of Health

Malvin Kalos, Cornell University

Ken Kennedy, Rice University

Christopher Kerr, Princeton University

Bill Kirchhoff, DOE

Thomas Kitchens, DOE

Kenneth Kliewer, DOE

Chuck Koelbel, Rice University

Sylvain G. Korzennik, Smithsonian Astrophysical Observatory

Norman Kreisman, Department of Energy

George Lake, University of Washington

Monica Lain, Stanford University

Meemong Lee, Jet Propulsion Laboratory

Creon Levitt, NASA Ames Research Center

Kai Li, Princeton University

Donald Lindberg, National Institutes of Health

Jerry Linn, National Institute of Standards & Technology

Rik Littlefield, Battelle-PNL

Fred Long, National Oceanic and Atmospheric Administration

John Lou, Jet Propulsion Laboratory

Steve Louis, Lawrence Livermore National Laboratory

Sandy MacDonald, National Oceanic and Atmospheric Administration

Jacob Maizel, National Institutes of Health

Allen Malony, University of Oregon

Vincent Marier, National Security Agency

George Marsaglia, Florida State University

Robert Martino, National Institutes of Health

Michael Mascagni, Supercomputing Research Center

Edward Masi, Intel Scientific Computers

Daniel Masys, National Institutes of Health

Edward McCracken, Silicon Graphics Computer Systems

Greg McRae, Lawrence Livermore National Laboratory

Scott McNealy, Sun Microsystems, Inc.

Carlos Roberto Mechoso, University of California, Los Angeles

Piyush Mehrotra, ICASE (USRA)

John Mellor-Crummey, Rice University

Paul Messina, California Institute of Technology

Barton Miller, University of Wisconsin

Ray Miller, National Security Agency

Clark Mobarry, NASA Goddard Space Flight Center

Reagan Moore, San Diego Supercomputer Center

Robert Morris, National Security Agency

Richard Robert Muntz, University of California, Los Angeles

112 Workshop on Grand Challenge Applications and Software Technology
i II I II II I ill I iiiii i I ii ii iiii rim| I II II II I I II I I IIIII I i

Workshop [nvitees

Dave Nelson, Department of Energy

Mike Nelson, Office of Science and Technology Policy

Lea Nichols, NASA Lewis Research Center

Mike Norman, University of Illinois NCSA

Joan Novak, Environmental Protection Agency

Ed Ott, University of Maryland

James Ousley, Control Data Systems, Inc.

Ross Overbeek, Argonne National Laboratory

Robert Palmer, Digital Equipment Corp.

V. Pankoni, National Science Foundation

Janice Patrick, IBM Federal Systems Co.

Merrel Patrick, National Science Foundation

David Patterson, University of California

Michael Pavloff, Silicon Graphics

Larry Peterson, University of Arizona

Eckhard Pfeiffer, Compaq Computer Corporation

Lawrence Picha, NASA

Lewis Platt, Hewlett-Packard Company

Gary Pope, University of Texas at Austin

Terrence Pratt, NASA Goddard Space Flight Center

Tom Pyke, National Oceanic and Atmospheric Administration

Irene Qualters, Cray Research, Inc.

Cal Ramos, National Institutes of Health

Daniel Reed, University of Illinois

David Robard, DCRT, NIH

John Rollwagen, U.S. Dept of Commerce

Richard Rood, NASA Goddard Space Flight Center

Joan R. Rosenblatt, National Institute of Standards & Technology

Ralph Koskies, Pittsburgh Supercomputer Center

Robert Rosner, University of Chicago

Bruce Ross, Geophysical Fluid Dynamics Laboratory

Armistead G. Russel, Carnegie Mellon University

Manuel Salas, NASA Langley Research Center

Joel Saltz, University of Maryland

Ahmed Sameh, University of Illinois CSRD

William Scherlis, DARPA/SISTO

Klaus Schulten, Beckman Institute, University of Illinois at Urbana

L. Kidgway Scott, University of Houston at University Park

Mary Anne Scott, DOE

John Sculley, Apple Computer, Inc.

Kamy Sepehrnoori, University of Texas at Austin

,lames Sethian, University of California

Peter Shames, Jet Propulsion Laboratory

Bruce Shapiro, FCRDC

Margaret Simmons, Los Alamos National Laboratory

Horst Simon, NASA Ames Research Center

Ronald Skates, Data General Corp.

Burton Smith,Tera Computer Co.

Lauren Smith,Supercomputing ResearchCenter

Workshop on Grand ChallengeApplicationsand SoftwareTechnology 113
A j _

iiiii II illil I I I I iiiiiiiii i i I [ I Ill Ill I I IIIIII iiiii I

Paul H. Smith, NASA Headquarters

David Smitley, Supercomputing Research Center

Marc Snir, IBM T.J. Watson Research Center

Jerry Solomon, California institute of Technology

Stephen Squires, Defense Advanced Research Projects Agency

Guy Steele, Thinking Machines Corp.

Thomas Sterling, NASA Goddard Space Flight Center

Kenneth Stevens, NASA Ames Research Center

Rick Stevens, Argonne National Laboratory

Walter Stevens, NIST

Malcom Stocks, Oak Ridge National Laboratory

James M. Stone, University of Maryland

Michael Strayer, DOE

Max Suarez, NASA Goddard Space Flight Center

Bob Sugar, University of California, Santa Barbara

Francis Sullivan, Supercomputing Research Center

D. Lansing Taylor, Carnegie Mellon University

Frank Tower, National Oceanic and Atmospheric Administration

J. G. Treybig, Tandem Computers

Lew Tucker, Thinking Machine Corp.

Salt Umar, Vanderbilt University i

James Unruh, Unisys Corporation

Anthony VillMenor, NASA

Robert Voigt, National Science Foundation

Al Wagner, Argonne National Laboratory

David Walker, Oak Ridge National Laboratory

Steven Wallach, Convex

Michael Warren, Los Aiamos National Laboratory

Harvey WaMerman, Los Alamos National Laboratory

Richard Watson, Lawrence Livermore National Laboratory

Howard Watts, Cray Computer

Gilbert Weigand, DARPA/CSTO

Jack Wells, Oak Ridge National Laboratory

Mary Wheeler, Rice University

Andy White, Los Alamos National Laboratory ACL

Elizabeth Williams, Supercomputing Research Center

Irving Wladawsky-Berger, IBM Corporation

Stephen Wolff, National Science Foundation

Paul Woodward, University of Minnesota AHPCRC

John Wooley, DOE

Patrick Worley, Oak Ridge National Laboratory

Chee Yap, New York University

Jeffrey Young, Environmental Protection Agency

Steven Zalesak, NASA Goddard Space Flight Center

Thomas Zang, NASA Langley Research Center

Mary Zosel, Lawrence Livermore National Laboratory

Wojciech H, Zurek, Los Alamos National Laboratory

114 Workshop on Grand Challenge Applications and Software Technology
m m