# Presentation: 098476

## Scientific Computing Group

- Ricky A. Kendall** **
**Scientific Computing**
**Center for Computational Sciences**

## Center for Computational Sciences/

**Cray **
**Supercomputing**
**Center for **
**Excellence**
- J. Levesque
- T. Darland
- L. DeRose4
- D.Kiefer4
- J. Larkin4
- N. Wichmann4

**Scientific Computing**
- R. Kendall
- L. Rael
- S. Ahern##	V. Lynch5
- S. Alam5 	B. Messer
- E. Apra5 	R. Mills
- R. Barrett5 	G.Ostrouchov5
- R. Barretto2 	R. Sankaran
- D. Banks3 	A. Tharrington
- J. Daniel 	R. Toedte
- M. Fahey 	T. White
- S. Klasky#
- J. Kuehn5
-   #End-to-End Solutions Lead
- ## Viz Task Lead

**User Assistance**
**and Outreach**
- J. White
- L. Rael
- Y. Ding
- D. Frederick
- C. Fuson
- C. Halloy3
- S. Hempfling
- M. Henley
- J. Hines
- S. Jones6
- B. Renaud
- B. Whitten
- L. Williams5
- K. Wong3

**High-Performance**
**Computing Operations**
- A. Baker
- L. Gregg*
- M. Bast	J. Lothian
- J. Becklehimer4	D. Maxwell##
- J. Breazeale6	M. McNamara4
- J. Brown6	J. Miller6
- A. Enger4	G. Phipps, Jr.6
- C. England	G. Pike
- J. Evanko4	V. Rothe#
- M. Griffith	S. Shpanskiy
- V. Hazlewood5	D. Vasil
- T. Jones	C. Willis4
- C. Leach6	T. Wilson6
- D. Londo4	S. White
-   #Team Lead, Computer Ops
- ##Technical Coordinator

**Technology **
**Integration**
- S. Canon
- L. Gregg*
- T. Barron
- S. Carter##
- K. Matney
- M. Minich
- S. Oral
- D. Steinert
- F. Wang
- V. White
- W. Yu5
- ## Networking Lead

*Site Preparation*
- K. Dempsey
*Hardware Acquisition*
- A. Bland*
*Test and Acceptance*
- Development
- S. Canon
*Commissioning*
- A. Baker
*Project Management*
- D. Hudson5
*Project R & D*
- A. Geist
*Cray Proj. Director*
- K. Kafka4

**Operations Council**
- A. Stewart, Quality Assurance
- T. Jones, Cyber Security
- M. Dobbs, Facility Mgmt.
- W. McCrosky, Finance Officer
- W. Besancenez, Procurement
- M. Palermo, HR Mgr.
- K. Johnson, Recruiting
- R. Toedte, Safety & Health
- N. Wright, Org. Mgmt. Specialist

- Leadership Computing Facility
**A. Bland, Director**
**L. Gregg, Division Secretary**

**Deputy Project Director**
- K. Boudwin

**Director for Science**
- D. Kothe

**Director of Operations**
- D. Kothe*

**Chief Technology Officer**
- A. Geist

**LCF System Architect**
- S. Poole

**Advisory Committee**
- J. Bernholc	S. Karin
- T. Dunning	 D. Keyes
- J. Dongarra	D. Reed
- K. Droegemeier	T. Sterling
- J. Hack	W. Washington

- 1Student
- 2Post Graduate
- 3JICS
- 4Cray, Inc.
- 5Matrixed
- 6Subcontract
- *Interim
- New position within ORNL
- New hire at ORNL

## Scientific computingTasks and deliverables

**Project partnerships in science, visualization, workflow, and data analytics**
**Level of integration is dependent upon the project**
**High-level user assistance**
**Main staff function.****	****Whatever it takes!**
**Algorithmic development at scale**
**For scientific codes, visualization and workflow**
**Application and library code optimization and scaling**
**How to use the LCF resources for science**
**User voice within LCF planning exercises**
**Users are the focus **
**Requirements gathering, documentation**
**What’s next for our users and our center**
**Exploiting parallel I/O & other technologies in applications**
**Specific focus for targeted applications**
**Acquiring & deploying viz and end-to-end systems**
**Infrastructure building**
**Benchmarking and evaluation exercises with Vendor interaction**
**Learning how to best use current and new LCF resources  **

## Partnership with projects on LCF resources

- Partnership with projects on LCF resources

**Porting/Tuning**
**Library utilization**
**Compiler flags**

**Optimizations**
**Choice of Tools**

**Code Design**
**and **
**algorithmic developments**
**@**
**scale**

**Shared R&D Staff**

**Pilot**

**End Stations**

**Projects**

**Level of integration**

## High-level user assistance

**Project**

**Liaison**

**End-to-End**

**Visualization**

- Whatever it Takes!

**Compute**

## Algorithmic and programming model development at scale

**Complex Tradeoffs for applications**
**Usually among storage, computation and communication**
**Redesign new methods**
**Fixed grid to AMR grids**
**Programming model changes**
**Message passing to one-sided model transformation**

## Application and library optimization and scaling

**Applications for the LCF systems**
**Catamount issues for Apps**
**Dual Core/Single core settings**
**Compiler flags for optimal performance (Application/Library specific)**
**Modifications for scaling**
**Libraries Built and kept up-to-date**
**HDF5, PETSc, FFTW, etc.**

- Number of Cores

**Before**

**After**

**Ideal**

## Requirements gathering and documentation

**Participate in Applications Requirements Council (ARC)**
**Determine requirements for applications on the machine**
**Algorithmic**
**Programming models**
**Architectural**
**Infrastructure**
**Develop ties to applications areas not currently at scale on the machine**
**Work with developers to understand application properties**
**Memory footprint**
**Algorithmic scalability**
**Potential for future resources**
**Collela’s 7 Dwarfs Anaylsis **
**The 7 algorithm types are scattered broadly among science domains, with no one particular algorithm being ubiquitous and no one algorithm going unused;**
**Structured grids and dense linear algebra algorithms are the most widely used algorithms (by over half of the representative codes), hence system attributes such as node peak flops and memory capacity, memory latency, and interconnect latency will be important;**
**Particle-based and Monte Carlo algorithms, which have similar properties from a system standpoint, are also broadly used, and can tax interconnect latency and in some cases node memory capacity, depending upon implementation and usage.**

## Collela’s 7 Dwarfs analysis

| Science
Domain | Code | Structured
Grid | Unstructured
Grids | FFT | Dense
Linear
Algebra | Sparse
Linear
Algebra | Particle | Monte
Carlo |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Accelerator Physics | T3P |  | X |  |  | X |  |  |
| Astrophysics | CHIMERA | X |  |  | X | X | X |  |
| Astrophysics | FLASH | X |  | X | X | X | X |  |
| Biology | LAMMPS |  |  | X |  |  | X |  |
| Chemistry | MADNESS |  | X |  | X |  |  |  |
| Chemistry | NWCHM |  |  | X | X |  |  |  |
| Chemistry | OReTran | X |  | X | X |  |  |  |
| Climate | CAM | X |  | X |  |  | X |  |
| Climate | POP/CICE | X |  |  |  | X | X |  |
| Combustion | S3D | X |  |  |  |  |  |  |
| Fusion | AORSA | X |  | X | X |  |  |  |
| Fusion | GTC | X |  |  |  | X | X | X |
| Geophysics | PFLOTRAN | X | X |  |  | X |  |  |
| Materials Science | QMC/DCA |  |  |  | X |  |  | X |
| Materials Science | QBOX |  |  | X | X |  | X |  |
| Nanoscience | CASINO |  |  |  | X |  | X | X |
| Nanoscience | LSMS | X |  |  | X |  |  |  |
| Nuclear Energy | NEWTRNX |  | X |  | X | X |  |  |
| Nuclear Physics | CCSD |  |  |  | X |  |  |  |
| QCD | MILC | X |  |  |  |  |  | X |

## System attribute priority per science domain

| System Attribute | Climate | Astrophysics | Fusion | Chemistry | Combustion | Accelerator Physics | Biology | Materials Science |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Node Peak Flops |  |  |  |  |  |  |  |  |
| Mean Time to Interrupt (MTTI) |  |  |  |  |  |  |  |  |
| WAN Network Bandwidth |  |  |  |  |  |  |  |  |
| Node Memory Capacity |  |  |  |  |  |  |  |  |
| Local Storage Capacity |  |  |  |  |  |  |  |  |
| Archival Storage Capacity |  |  |  |  |  |  |  |  |
| Memory Latency |  |  |  |  |  |  |  |  |
| Interconnect Bandwidth |  |  |  |  |  |  |  |  |
| Disk Latency |  |  |  |  |  |  |  |  |
| Interconnect Bandwidth |  |  |  |  |  |  |  |  |
| Memory Bandwidth |  |  |  |  |  |  |  |  |
| Disk Bandwidth |  |  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |  |  |

## Contact

- Ricky A. Kendall
- Scientific Computing
- Center for Computational Sciences
- (865) 576-6905
- kendallra@ornl.gov

-    Presenter_date