II-9. Instrumentation

RIKEN Accel. Prog. Rep. 54 (2021)

RIKEN Accel. Prog. Rep. 54 (2021)

GPU acceleration of SAMURAI particle tracking simulation
J. Gao∗1,∗2
The possibility of graphical processing unit(GPU)acceleration of the trajectory simulation of particles
passing through SAMURAI was evaluated. To obtain
A/Z in particle identiﬁcation plot, information such as
the ﬂight path and rigidity obtained from this type ofsimulation is necessary. Usually, this tracking simulation is the most time-consuming step in the analysis ofSAMURAI data.
As with most other steps in nuclear physics data
analysis, this particle tracking simulation is performed
event by event. Therefore the simulation could be accelerated by distributing the events to large amount of
threads of a GPU to process in parallel. In principle,
all the event-by-event analysis could be accelerated byparallel computing and could take advantage of a GPU.
Another advantage of a GPU is its special cachestructure. In contrast to the linear structured cachein a CPU, the cache in a GPU could have a higherdimensional layout.1) Therefore, when interpolating inthe magnet ﬁeld map, the GPU could fetch the neighboring mesh point with fewer cache misses.
In this work, a simpliﬁed task is designed to evaluate the performance of trajectory simulation on a CPUand GPU. The standard program used in data analysis takes the position and angle before and after themagnet and provides the rigidity as the output. It iterates several times to obtain a certain rigidity that
reproduces the position and angle measured in the experiment. This simpliﬁed version takes the position,angle, and rigidity before the magnet and outputs theposition and angle after the megnet without any iterations. The CPU version is modiﬁed from the code used
for the particle identiﬁcation of SAMURAI11 data.2)
The program uses a fourth-order Runge-Kutta
method to simulate the trajectory of a particle in theSAMURAI magnet. Both the CPU and GPU versions of the program were developed. The test wasperformed on a server with 4 Intel Xeon Gold 6136
CPUs (each has 12 cores/24 threads), 128 GB memory, and one Nvidia Titan V GPU. The CPU version
code is written in Go programming language and optimized using the AVX2 assembly.3,4) The GPU version
is written in C++ and CUDA. 1024000 events were fed
into the programs, and each version ran 3 times.
The results are summarized in Table 1. The timer
starts immediately after the ﬁeld map is loaded into themain or GPU memory and the input data are loaded
into the main memory, and it stops as soon as the calculation ﬁnishes, which means the hard disk I/O timeis excluded so that the time of calculation is isolated.
∗1∗2Table 1. Test results of CPU and GPU versions of the simplified simulation. The GPU could accelerate the simulation by a factor of 5 to 13 times approximately.
codeCPU 1CPU 2
GPU
configuration 32 coroutines 126 coroutines 320 × 32
test #1
13.292 s
5.277 s0.999 stest #2
13.309 s
5.319 s1.004 stest #3
13.317 s
5.378 s1.011 saverage13.306 s
5.324 s1.004 sIn Table 1, the CPU 1 code enabled 34 threads andallocated 32 coroutines for simulation. The CPU 2
code enabled all the 128 threads and allocated 126
coroutines for simulation. The GPU version dividesthe input data into 100 groups, each of which is passed
to a device function using a conﬁguration of 320 thread
blocks and 32 threads per block. Diﬀerent groups
are processed asynchronously to hide the time of datatransfer between the main memory and device memory.
For SAMURAI11 data, it takes 3.15 iterations on
average to obtain the rigidity and a maxmium of 5 iterations. For the CPU1 and CPU 2 conﬁgurations, thetime for simulation should be 13.306 × 3.15 = 41.91 s
and 5.324 × 3.15 = 16.77 s, respectively. If we consider
the worst case for the GPU, where each thread blockcontains an event that need 5 iterations, then the simulation time is 1.004 × 5 = 5.02 s, which is 8 or 3 times
faster than the CPU version, depending on the numberof threads used in the CPU code.
In conclusion, with a reasonable GPU cost and coding eﬀort, the simulation could be made signiﬁcantly
faster.
References

1) CUDA C++ Programming Guide,
docs.nvidia.com/cuda/cuda-c-programming-guide/index.html
2) J. Gao et al., in this report.
3) Intel 64 and IA-32 Architectures Software Developer’sManual,software.intel.com/content/www/us/en/develop/
articles/intel-sdm.html
4) A Quick Guide to Go’s Assembler,
golang.org/doc/asm

School of Physics, Peking University
RIKEN Nishina Center- 104 -1