Efﬁciency and Effectiveness Analysis of a
Scratchpad Memory on FPGA and GPU for
Diffuse Radiation Transfer Simulation
FURUKAWA Kazuki YAMAGUCHI Yoshiki YOSHIKAWA Kohji KOBAYASHI Ryohei
FUJITA Norihisa
BOKU Taisuke
UMEMURA Masayuki
Introduction
Radiation hydrodynamics is a fundamental scientiﬁc concept to unveil the cosmic physics
process in astrophysics. In this project, we target the acceleration of a set of Radiative Transfer(RT) simulation code, the ARGOT (Accelerated Radiative transfer on Grids using Oct-Tree) [1]program. The ARGOT framework consists of two algorithms (Fig. 1), and the concrete target
in this study is the ART (Authentic Radiation Transfer)[2] scheme, which computes the Diffuse
Radiation Transfer. We have implemented the ART scheme on GPUs and FPGAs, and analysethem in this poster.

Figure 1: Basic Concept of ARGOT [1]Diffuse RT Simulation

Application-Speciﬁc Buffering Scheme: PRISM

The ART scheme takes more than 90% of total
simulation, which is not not matrix calculation.
It has to calculate results of the different atoms.

PRISM (PRefetchable and Instantly accessible Scratchpad Memory) [3]
Key Concept:
Realised by:

Based on Ray Tracing algorithm:

1. To increase available bandwidth−→ Also increase memory access locality
2. To overlap computation and memory access• Rays go straight ahead without reﬂection.
• Rays can have 768 or 3,072 different angles(N_ANG = 768, 3,072).
• Each Ray is computationally and geometrically parallel and independent.
−→ Suitable for many-core architectures.
Algorithm 1: The most critical part of the ART
for ipix = 0 to N_ANG−1 do2:for ray = 0 to N_MESH_SIDE2 -1 do
3:
I_old ← read_initial_intensity;
4:
while Ray is in the current space do
5:
M ← read_initial_mesh_data;
6:
path_length ←7:
calc_pathlen(init_position[ray],angle[ipix]);
8:// Radiation intensity calculation
9:
tau ← M.absorption * optical_path_length;10:etau ← exp(-tau);11:
etaum1 ← 1.0 - etau;
12:
I_new ← I_old * etau + M.source_func * etaum1;
13:// Update intensity for next mesh14:
I_old ← I_new;15:// Atomic accumulation of mesh data
16:
M.accum_I ← M.accum_I + I_new * etaum1;
17:
M.accum_tau ← M.accum_tau + tau;
18:
Write_back(M.accum_I, M.accum_tau);
19:
end while // Loop for a single ray20:
end for // Loop for rays having the same angle
21: end for // Loop for angles
1:

PRISM ismade up by
a size of____ for ray data
mainly effective by• Combination of external & internal memory• Reusing mesh data many times on chip
• 2 of elongated prism shape subspace (Fig. 3)−→ To put many spaghetti (ray) bundles in it

Xilinx Alveo U280 FPGA / Verilog HDL
UltraRAMs (960x36KB/Device)
72KB (2x UltraRAM block)Possible to communicate with neighbours
Overlapping computation and mem accessNvidia A100 GPU / CUDA C++
Shared Memory (160KB/SM)
17KB / 160KB
Independent from others
Increasing cache hit rate

In PRISM-GPU, the parallelism is increased by dividing into the several Ray Groups.

Figure 3: Comparison of Control Flows between PRISM-FPGA (Left) and PRISM-GPU (Right)

Implementation Result

Memory Access is complex (Fig. 2):
When the ART deals with 512 meshes, it requires
more than 2GB for storage.
3Figure 4: Data Reuse Rate on Scratchpad
Memory (N_ANG is the number of ray an- Figure 5: Performance Comparisons on Various Implementations
gles. Data is theoretical rate. FPGA is (Result of PRISM is always the best. When the simulation size isalways higher than GPU.)small, FPGA is better because GPU cannot hide its overheads.)

Conclusion

Figure 2: Memory Access of the ART

Using our proposed method, we conclude that the original ART can be accelerated by both FPGAs andGPUs. An FPGA yields better when the simulation space is small, while a GPU is better when large
because of GPU’s high parallelism. We also prove that the PRISM reduces the memory access bottleneck
and contributes to a signiﬁcant increase in the utilisation of the arithmetic circuits on the accelerators.

References

Acknowledgements[1] T. Okamoto et al., “ARGOT: accelerated radiative transfer on grids using oct-tree,” Monthly Noticesof the Royal Astronomical Society, vol. 419, pp. 2855–2866, Feb. 2012.
[2] S. Tanaka et al., “A new ray-tracing scheme for 3D diffuse radiation transfer on highly parallelarchitectures,” Publications of the Astronomical Society of Japan, vol. 67, pp. 62(1–16), May 2015.
[3] K. Furukawa et al., “An efﬁcient RTL buffering scheme for an FPGA-accelerated simulation of diffuse
radiative transfer,” in 2021 International Conference on Field-Programmable Technology (ICFPT),
pp. 1–9, 2021.

This work was supported in part by the MEXT Next Generation HighPerformance Computing Infrastructures and Applications R&D Program, entitled “Development of Computing-Communication Uniﬁed Supercomputerin Next Generation”, and in part by the JSPS KAKENHI #21H04869. We wouldalso acknowledge the support of Xilinx Inc. through the Xilinx Univ. Program.