Acceleration of Quantum Chemistry and Chemical Similarity
calculations on GPGPU architecture towards faster virtual screening
Yuki Furukawa, Ryota Koga, Naoki Nariai : X-Ability Co.,Ltd.

Results & Discussion

Background
1. Rare disease related macromolecules, such as proteins, arewithin the scope of electronic-structure calculation by using
FMO because of the improvement of computational resources.
2. LINGO [Vidal et al. 2005] is a computational method tomeasure chemical similarity by comparing SMILES
representations of two molecules.
3. GPGPU is successfully applied for many kinds of problems
in high performance computation.

Purpose of this ResearchFirst, based on Yasuda [2008], we develop XA-CUDA-QM toaccelerate famous QM softwares (GAMESS-US etc) by using NVIDIA
GPU. We show the excellent FMO performance by XA-CUDA-QM.
Second, we develop a LINGO similarity calculation method to utilize
multiple CPUs and GPUs at the same time, based on SIML [Haque etal. 2010] originally developed to accerelate LINGO similarity
calculations by using GPGPU.

Benchmark Environment : (1) Intel Core i7-3930K @3.2GHz (6 core) , GTX580
x 2, 32GB, CUDA 4.0 (2) Intel Core i7-3930K @3.2GHz (6 core), GTX580 x 4,32GB, CUDA 4.0, (3) Intel Quad Core XeonE5450 3.0GHz (many core), 32GB :Compiler : Intel Composer XE 12.0 + mkl

Table 1-1 : FMO2 monomer energies ofeach residue of Insulin(2HIU) by GAMESS2010 R3 (FMO-RHF/6-31G, a.u.) (1) + (2)Residue E(CPU)ΔE(GPGPU)

1(GLY)-442.916776617

0.000000005

1. The value of dipole moment isexactly the same.

2(VAL)-308.573606501

0.000000009

2. Total Monomer Energy3(GLU)-457.046042916

0.000000011

E(Original) : -21635.4488652520

4(GLN)-437.255799945

0.000000002

E(GPGPU) : -21635.4488649211

5(CYS)-1254.889732125

0.000000004

6(CYS)
7(THR)
8(SER)
9(ILE)-1254.930631919-344.327645726-305.393476307-347.549172133

0.000000003
0.000000007
0.000000004
0.000000010

ΔE: 3.31×10-7 a.u.

3. Calculation Time (sec) :Original : 3279.944
GPGPU : 790.527

Table 1-2 : Calculation time of FMO2 environmentalelectrostatic potential of Insulin(2HIU) (sec)Coding Policy
1. Total energy and the gradient of under FMO approximation.
2. Most time-consuming steps (the evaluation of Coulomb, HartreeFock exchange, and exchange-correlation matrices…) are
executed on GPUs. Remaining parts are calculated on CPUs.
3. Use single precision arithmetic as much as possible for the sakeof effective speedup, without degradation of accuracy.
4. Take special care for task scheduling and host-GPUcommunication time to keep as many CPUs and GPUs busy.
5. Modify LINGO original-code for multi-CPUs/multi-GPUs.

ESP(CUDA : XA-CUDA-QM)

Time(Original) Time(GPGPU)
2571.490
170.897

We achieved 4.15x acceleration of 44 residue protein calculation
for total time by GPGPU than the original GAMESS, and 15.0x
acceleration for ESP alone. The calculation of ESP consumes
predominant time (over 75%) so that FMO2 accelerates so much.
The direct SCF method is unfavorable for them because fragments
are so small that we can store all ERIs in a host memory.
Table 3 : LINGO Similarity calculation time (16,384 x 16,384
chemical compounds all-to-all) (2) or only 60CPU by (3)Time(msec)

Algorithms
Electrostatic potential (Coulomb or J-matrix) :The two-electron integrals and the J-matrix in terms of primitive
Hermite Gaussian basis are evaluated by GPU (Direct J Engine).
p
J ab   (ab | cd ) Dcd   Eab  ( p | q ) Dq
cdpq

Schwarz upper bounds are used to reduce the number of ERIs to becalculated, but we don’t use integral symmetry ( e.g. (p|q)=(q|p) ).
The McMurchie-Davidson algorism is used to evaluate Hermite ERIs.
The communication time between CPU and GPU to send / receivep, q, Dq , J p was only about 10 % of GPU computation time.
 log | ( p | q) Dq |The upper bound of each ERI is
first evaluated and ERIs which aresmall in magnitude are calculated
on GPU with single-precision. Therests (10% of total count of ERIs)
are calculated on GPU with double
precision.
Environmental Electrostatic
Potential (ESP):

Koji Yasuda : Nagoya University

Fig 1. Valinomycin(x 108 , converged D)s,sp,p,dJ-matrix algorithm is reused.
GPU(DP) GPU(SP) unnecessary
This is the mainly timef or moreconsuming step of FMO.
CPUunnecessary
LINGO algorithm on multi-CPUs and multi-GPUs:

Original algorithm implemented in SIML [Haque et al. 2010] is based on asingle-CPU and single-GPU architecture. We have modified the sourcecode so that LINGO algorithm can run parallel on multi-CPUs and multiGPUs at the same time. In our implementation, calculation workload isdivided into each set of CPU-GPU process with utilizing MPI.

Speed up

1CPU

53,488x1.00

4CPU14,097x3.79

60CPU (*)

2,640x20.26

1CPU+1GPU

1,980x27.01

2CPU+2GPU

1,010x52.96

3CPU+3GPU

713x75.02

4CPU+4GPU

529

x101.11When running on CPU+GPU, calculation is dominantly
conducted by GPUs.

Summary
The total energy of FMO and the dipole moments were essentiallythe same. GPGPU calculation of environmental electrostatic
potential becomes 15.6 times faster than multi-core CPUs
calculation.
LINGO calculation method that utilizes 4CPUs and 4GPUs
improves calculation speed by more than 100 times comparedto the speed calculated by one CPU. The calculation results
were all coincident.

Future Theme
1. GPGPU acceleration of ERI with higher basis (metal protein).
2. Parallel evaluation of target using RI-MP2 algorithm by GPGPU
3. LINGO similarity calculations on millions of compounds(*) The super-computing resource was provided by Human Genome Center, Institute ofMedical Science, University of Tokyo
http://sc.hgc.jp/shirokane.html