# Presentation: 803206

## Implementation of Density Functional Theory based Electronic Structure Codes on Advanced Computing Architectures

- W.A. Shelton, E. Aprá, G.I. Fann and R.J. Harrison

## Cray-X1E

- 1024 Multi-streaming vector processor (MSP)
- Each MSP has 2 MB of cache and a peak computation rate of 12.8 GF
- 4 single-streaming processors (SSPs) form a node with 16 Gbytes of shared memory
- Memory is physically distributed on individual modules
- all memory is directly addressable to and accessible by any MSP in the system through the use of load and store instructions

## Cray XT3

- 5294 nodes
- 4-processors per node connected through hypertransport 2.4-GHz AMD Opteron processor and 2 GB of memory
- SDRAM memory controller and function of Northbridge is pulled onto the Opteron die.  Memory latency reduced to <60 ns
- Interface off the chip is an open standard (HyperTransport)

## Applications

- Porphyrin Functionalized Nanotube
- 1532 atoms
- STO-3G basis set = 6380 basis functions
- Do the covalently attached porphyrins undergo facile absorption of visible light and transferelectrons to the nanotube
- What type of efficiencies does one obtain
- Time for LDA energy + gradient using 300 processors = less than 1 hour

## LSDA &Multiple Scattering Theory (MST)

- Multiple Scattering Theory (MST)
*J. Korringa, Physica 13, 392, (1947)*
*W. Kohn, N. Rostoker, PR, 94, 1111,(1954)*
- MST Green function methods
*B. Gyorffy, and M. J. Stott, “Band Structure Spectroscopy of Metals and Alloys”, Ed. D.J. Fabian and L. M. Watson (Academic 1972)*
*S.J. Faulkner and G.M. Stocks, PR B 21, 3222, (1980)*

## Algorithm Design for future generation architectures

-  More accurate
-  Spectral or pseudo-spectral accuracy
-  Wider range of applicability
-  Sparse representation
-   Memory requirements grow linearly
-   Each processor can treat thousands of atoms
-   Make use of large number of processors
-  Message-Passing
-   Each atom/node local message-passing is independent    of the size of the system
-  Time consuming step of model
-  Sparse linear solver
- Direct or preconditioned iterative approach

## Multiple Scattering Theory

**decay slowly with increasing distance**
**contain free-electron singularities**

- Generalization of t-matrix. Converts* *
- incoming wave at site n into outgoing wave
- at site m in the presence of all the other sites

## Complex Energy Plane

**The scattering properties at complex energy can be used to **
**develop highly efficient real-space and k-space methods**

**Semi-conductors and insulators could work **
**well since they have no states at ********f**

## Multiple Scattering Theory

- Representation is ideal for disorder systems since t-matrix
- is site diagonal !!
- Coherent Potential Approximation (CPA)
- Non-local CPA (based Dynamical Cluster Approximation to
- Dynamical mean-field theory

## t-matrix

- Solve for t-matrix inside Voroni polyhedron
- Using Calergo method

## Parallel Implementation 

-  **Green’s function**

-  Scattering path matrix: real space

*t : *scattering from single site
- G: structure constant matrix

- _M_=[_t_-1()-_G_(Rnm,]

- __=_M_-1

** **Once M is fixed increasing N does not
-    affect the local calculation of _M_-1

## Tight-Binding MST Representation

- Tight Binding Multiple Scattering Theory
- Embed a constant repulsive potential** **
- Shifts the energy zero allowing for calculations at negative energy
- Rapidly decaying interactions
- Free electron singularities are not a problem
- Sparse representation

## Screened Structure Constants

**Screened Structure Constants G****s**** ****on the left unscreened on the right**
**Screened structure constants rapidly go to zero, whereas the free space structure constants have hardly changed**

**Linear solve using m atom cluster that is less than the n atom system**
**Easy to perform Fourier transform**
**K-space method**

## Screened MST Methods

**Formulation produces a sparse matrix representation**
**2-D case has tridiagonal structure with a few distant elements due to periodicity**
**3-D case has scattered elements**
**Mainly due to mapping 3-D structure to a matrix (2-D)**
**A few elements due to periodic boundary conditions**
**Require block diagonals of the inverse of ******** matrix**
**Block diagonals represent the site ******** matrix and are needed to calculate the Green’s function for each atomic site**
**Sparse direct and preconditioned iterative methods are used to calculate ********ii****(********)**
**SuperLU**
**Transpose free Quasi-Minimal Residual Method (TFQMR)**

## Screened KKR Accuracy

## Timing and Scaling of Scr-K KKR-CPA

## Conclusion

**Initial benchmarking of the Screened KKR method**
**SuperLU N****1.8 ****for finding the inverse of the upper left block of ******
**TFQMR with block Jacobi preconditioner N****1.06 ****for finding the inverse of the upper left block of ******
**Extremely high sparsity (97%-99% zeros increases with increasing system size)**
**Large number of atoms on a single processor**
**Real-space/Scr-KKR hybrid may provide the most efficient parallel approach for new generation architectres**
**Single code contains**
**LSMS, KKR-CPA, Scr-LSMS and Scr-KKR-CPA**

## Local Poisson Equation

- Discontinuous Galerkin approach