# Presentation: 153129

## Petaflops Applications:Chemistry

- Theresa Windus
- Pacific Northwest National Laboratory
- theresa.windus@pnl.gov

## NWChem Capabilities

- Quantum Mechanical Capabilities:
- Hartree-Fock & density functional theory at the local and nonlocal levels (with N3 and N4 formal scaling) energies, gradients, & second derivatives. Linear scaling quadrature and exchange.
- Multiconfiguration self consistent field (MCSCF) energies and gradients.
- Many-body perturbation theory energies and gradients.
- Effective core potential energies, gradients, and second derivatives.
- Coupled cluster [CCSD and CCSD(T)] and configuration interaction energies.
- Segmented and generally contracted basis sets including the correlation-consistent basis sets.
- Plane-wave pseudo-potential codes (periodic and free-space) with dynamics
- Classical Mechanical Capabilities:
- Energy minimization; molecular dynamics simulation; ab initio dynamics
- Free energy calculation
- Supports variations such as: multiconfiguration thermodynamic integration or multiple step thermodynamic perturbation, first order or self consistent electronic polarization, simple reaction field or particle mesh Ewald, and quantum force-field dynamics
- Mixed QM + MM models and ONIOM

## Maps to many algorithms

- Many body methods – uses DGEMM, I/O intensive, large data transformations
- Hartree-Fock, DFT methods – BLAS routines, large number of gets and puts, eigensolvers
- Molecular dynamics – domain decomposition, communication latency dependent, FFT
- Planewave codes – BLAS routines (DDOT and DAXPY), FFT

## Global Arrays 

**Global Arrays **

- Shared-memory-like model
- Fast local access
- NUMA aware and easy to use
- MIMD and data-parallel modes
- Inter-operates with MPI, ...
- BLAS and linear algebra interface
- Ported to major parallel machines
- IBM, Cray, SGI, clusters,...
- Originated in an HPCC project
- Used by several major chemistry codes, financial futures forecasting, astrophysics, computer graphics
- Supported by DOE 2000, ACTS, SciDAC

- Jarek Nieplocha and coworkers, PNNL

## Biggest challenges in architectures

- Diverse algorithms have diverse needs – some are communication latency limited, others are I/O latency and bandwidth limited, others are CPU intensive
- Latency and bandwidth of communication and memory
- Memory locality/hierarchies need to be apparent yet transparent
- Large I/O needs for out-of-core solutions
- New hardware requires new algorithms (or at least modified software)
- Need information as soon as possible to plan
- Fault tolerance

## Fastest time-to-solution for our science

**Notes:**
Improved efficiency since we have the same number of operations on the same number of processors in less time.

Preliminary results on larger systems show similar results.  We have not had access to more than 256 processors with Elan 4 to determine large scale improvements.  However, we anticipate that this trend will continue.

## What do we need from programming models?

- Automatically solve all of our problems
- Flexibility and adaptability and portability
- Performance
- Need good default behavior, but need to be able to “localize” or modify as needed
- Support for multi-level programming – but transparently!
- Better “prefetching” and latency hiding capability at all levels of the heirarchy (memory, disk, and communication)
- Fault tolerance

## Need maximum flexibility

- Hartree-Fock example
- If have enough memory
- Store all integrals in memory
- Replicate matrices
- If not enough memory, but fast disk
- Store integrals on disk
- Replicated, mirrored or distributed matrices
- If not enough memory and slow disk
- Recompute the integrals as needed
- Replicated, mirrored or distributed matrices

- Now map this to I/O and communication latency and bandwidth

## Abstractions

- Program models
- Adaptable shared memory constructs
- Mathematics
- Can we please have an FFT standard interface?
- Parallel
- Common or compatible abstractions among programming models – or at least a mapping between them
- Chemical
- Have already made some strides here
- Middleware – Data and Workflow

## Mirrored Array example

## When to try a new programming model

- Your existing model is not meeting your needs
- New model has what you need
- Low entry barrier
- Available on many platforms
- Relatively low language training cost
- Good working relation with supported PM developers
- Method/time/manpower to make the conversion

- You see a new functionality/capability that you can’t get in an existing model that you know you WILL need
- Can get in on ground floor
- Adequate “creative space” to explore new ideas
- People
- Time
- Access to others

- Forward thinking

- Practical

## Summary

- Build it and we may not come

- Build it with us and/or using our requirements, we may come