Acceleration of Quantum Chemistry on GPGPU architecture
towards faster virtual screening
Yuki Furukawa, Ryota Koga : X-Ability Co.,Ltd. Koji Yasuda : Nagoya Univ
E-mail : rkoga@x-ability.jp

Results & Discussion

Purpose of this Research
1. To virtually screen biologically important molecules by using
Gaussian, Fragment MO (electronic-structure calculations) byusing GAMESS, Many-core processors such as Graphics
Processing Units will be used to drastically accelerate
heavy computations.
2. To analyze performance limiting factors of calculation and todevelop better algorithms suitable for many-core processors.
3. To develop easy-to-use and commercial quality software.

Method (electronic-structure calculation)
1. Total energy and the gradient calculation with s, p, d basis
functions are accelerated by GPUs.
2. Based on Yasuda [2008], most time-consuming steps (calculationof Coulomb, Hartree-Fock exchange, and exchange-correlation
matrices…) are executed on GPUs. Remaining parts are calculatedon CPUs. The Electron Repulsion Integrals (ERIs) and theCoulomb matrix in terms of primitive Hermite Gaussian basis areevaluated by GPU.
 log | ( p | q) Dq |3. Use single precision arithmetic
as much as possible for best
speedup, without degradation ofaccuracy. The Schwartz upperbound of each ERI is firstevaluated and ERIs which aresmall in magnitude are
Valinomycin
calculated on GPU with single(x108 , converged 1-RDM)
precision. The rests (10% ofs,sp,p,dERIs) are calculated on GPU
GPU(DP) GPU(SP) unnecessary
with double precision.
f or moreCPU

unnecessary

4. Take special care for task scheduling and host-GPU
communication time to keep as many CPUs and GPUs busy.
Currently communication time between CPU and GPU was only
about 10 % of GPU computation time. Multi-GPUs are used whenavailable. DDI/OpenMP parallel version available for GAMESS,
Linda parallel version available for GAUSSIAN.
5. Because of the strictly limited fast storage (e.g. registers) available
on GPUs, conventional algorithms to evaluate Hermite ERIs do not
run fast. We developed the special McMurchie-Davidson algorithm(submitted to JCTC).
6. We developed GPU computation library (XA-CUDA-QM) toaccelerate famous QM software (GAUSSIAN, GAMESS-US…). Allthe functionality of these software are kept while the most importantparts are accelerated by GPUs.

Summary
As the key technology of virtual screening of large molecules, DFT
calculation by GAUSSIAN and FMO calculation by GAMESS weresuccessfully accelerated by using GPUs. The calculated results
were essentially the same, indicating cost and power efficiency ofGPU-accelerated virtual screening.
Since the computational time of DFT was drastically reduced, thereduction of inter-node communication becomes necessary to run
on massively parallel GPU clusters.
Thanks to matured QM software we can combine QM/MM(ONIOM…) and solvation models (continuum model, energy
representation…) easily.
Practical application to catalytic reactions is now within the scope.

Table 1-1 : Performance of ERI calculation (% of theoretical peak)Angularmomentum
LP, LQGPU single
precision (NVIDIAGTX5801581 GFLOPS)

0,0
3,3
4,4
5,4

398 (25)930 (59)639 (40)652 (41)

CPU double precision(INTEL core i7 3930K
1 core 26.4 GFLOPS)
presentconventional
4.7 (18)
4.8 (18)
11.5 (45)
2.9 (11)12.8 (50)
2.8 (11)10.2 (40)
2.7 (10)

Our new ERI algorithm runs much faster on both CPU and GPU thanthe conventional (GAUSSIAN) one. It is four times faster for highangular basis functions even on the same CPU because of thecomplete AVX vectorization.
Figure 1-2 : DFT calculation time (sec) of Pt13(H2O)15 platinumclusters by GAUSSIAN
1500

4921s 2574s 1411s(x1/4) (x1/2)

1000
500

Fast Multipole
Coulomb (GPU)Ex-cor (GPU)

Coulomb (CPU)Ex-cor (CPU)Communication

269s

228s

217s

1240124# of CPUs# of CPUs / GPUs

Most time-consuming part (Coulomb) was drastically accelerated byGPUs and the communication time dominates even on four nodes.
Table 2-1 : FMO2 monomer energies of each residue ofInsulin(2HIU) by GAMESS (FMO-RHF/6-31G, a.u.)
ResidueE(CPU)
DE(GPU) The dipole moment wasexactly the same.
-91(GLY)-442.916776617 5×105(CYS)-1254.889732125 4×10-9 Total time using 2 nodes ofsystem1
was3279.9
sec-89(ILE)-347.549172133 1×10with CPUs, 790.5 sec with
-7Total-21635.448865252 3.3×10GPUs.
Table 2-2 : FMO environmental electrostatic potential and total
acceleration rate of Insulin (2HIU)
Basis set / method
6-31G / FMO2, Single Point
6-31G*/ FMO2, Single Point
6-31G / FMO3, Single Point
6-31G / FMO2, Gradient

System 1
System 2
ESP Total ESP Total
13.8
4.1
3.3
2.1
16.6
4.4
4.7
2.5
19.5
3.7
4.8
2.3
10.6
2.6
2.9
1.4

System 1: Intel Core i7-3930K @3.2GHz (6 core) , GTX580(cudacore 512) x 2, 32GB mem, CUDA 4.0
System 2: Intel Xeon E5-2650 @2.00GHz (32core) , a TESLA
K20m(cuda core 2496), 64GB mem, CUDA 5.0 x 2 nodes
The calculation of ESP consumes more than 75% time. GPU
calculates it 20 times faster than multi-core CPUs for system 1.
The direct SCF method is unfavorable for each fragment
calculation because they are so small that we can store all ERIs ina host memory.
Because of large number of CPUs in system2, the acceleration ratio
was lower than that in system 1. Each fragment molecule isassigned to a GPU and is calculated independently, so we canincrease performance by adding new GPUs.