GRAPE クラスターを利用する並列 SPH コード
TreeConstruction
東大天文 中里直人 (GRAPE プロジェクト g02a6)
Original PDF file at

http://pucca.astron.s.u-tokyo.ac.jp/research/MUV2002.pdf

1. Introduction
The Smoothed Particle Hydrodynamics (SPH) method, which has been introduced in late '70s, isthe most popular and sccessful method when we construct a N-Body model of galaxy formation
including hydrodynamics. Despite the great success of theSPH method, it is not clear that the number of particle(numerical resolution) usually used in galaxy models is sufficient. A example of this problem is presented in the workby Sommar-Larsen etal (2001). They try to solve the problem known as ``angular momentum problem''. This problem
is that when we numerically model the collapse of gas withina dark halo (cosmological SPH simulation), the formed gas
disk has much less angular momentum than observed disk
galaxies (see left figure). One of thier conclusions is thatthe resulted specific angular momentum of the disk highly
depends on the numerical resolution. To address this problem with a much higher resolution SPH
model, we are developing own Parallel Tree-SPH code.

2. Implementation
The tree method consists of three stages: (1) tree construction, (2) tree walking and (3) summation. The details about each stage are explained in the right panel. A highlight of our code isto take a full advantage of the GRAPE cluster in NAOJ, Tokyo. There are many possible choiceof algorithms when we implement a parallel tree code, however, all choices are made to maximise
the performance with this GRAPE cluster. The SPH part of the code is easily implemeted if wealready implement the parallel tree code. Also, an indivisual time step scheme is incorporated tofurther obtain a greater performance. The elapsed time of the largest model (Model 3 below)
ever is 140 hour (total ~ 52000 steps) from z ~ 24 to 1.6 (~ 3 Gyr).

3. Early Results

In this stage, a tree data structure is created according to the position of particles.
The particles are divided into boxes (8boxes in 3-D case) recusively and this division is represented by the tree structure.
For the parallel implementation, we adopt
the locally essential tree (LET) method.
Specifically, first the local tree is createdin each proccesors, and then the LET is computed by exchanging tree data. On eachprocessor, another LET is constructed for
the SPH computation.

Tree Walking and Summation
Here, we define sink particles as the particles which are need to compute the gravity
and source particles as the particles wihch exert the gravity. In our parallel versionof code, a list of source particles (interaction list) are computed by walking the LET
with apripiate conditions. In the tree method with GRAPE, the interaction lists areshared within a group of particles. First things to do is that we divide the sink particles into the sink groups (typical number in a sink group is a few thousand). For eachgroup, we compute the shared interaction list. To obtain the maximum performance,
better the interaction list is longer. Once the shared interaction list is computed fora group, the gravity is computed with GRAPE according to the particles in the interaction list. We repeat this procedure until there are no sink particles. For the simulations in this presentation, we use 4 processors in parallel. Each computer has one
GRAPE-6 board.

Figure 2

Model 3Using our serial and parallel version of our Tree-GRAPE-SPH code, we have computed theevolution of an isolated halo. According to Sommar-Larsen etal, we generate a initial model withthe warm dark matter (WDM) spectrum by dumping the small scale fluctuation. We adopt atransfer function as Pwdm = exp(-0.5*a1 - 0.5 *a1*a1) Pcdm, where a1 is the cutoff radius. Ouradopted size corresponds to the cutoff mass Mc = 1.9 x 1010 Mo/h, where h is the Hubble
parameter. Namely, the fluctuations smaller than Mc are completely reduced. We use theGRAFIC2 software to generate initial models. With the GRAFIC2, we easily generate multiscale initial models. In this panel, we show example results of the CDM comological simulation

Model 4

with the GRAFIC2. Both panels show the projected density of dark matSPH Mass
NSPH
SPH eps DM Mass
NDM
DM epsratio
ter particles where the model in the right panel is 8 times higher mass
Model 1 5.1 x 106 Mo
40778
1.0 Kpc 4.6 x 107 Mo
40769
2.0 Kpc
0.1
resolution than the left.
Model 2 5.1 x 106 Mo
40778
1.0 Kpc 5.7 x 106 Mo 326866
1.0 Kpc
0.9
We have done four WDM cosmological SPH simulations. The adopted
Model 3 6.3 x 105 Mo326842
0.5 Kpc 5.7 x 106 Mo 326866
1.0 Kpc
0.9
cosmological parameters are h = 0.5 and the flat universe without thecosmological constant and the baryon fraction is 0.1. Using the GRAFIC2,
Model 4 6.3 x 105 Mo
326842
1.0 Kpc 5.7 x 106 Mo 326866
1.0 Kpc
0.9
we generate two levels of a grid, i.e., Level-1 (L1) grid and Level-2 (L2)grid. L2 grid is 8 times higher mass resolution model than L1 grid. Each SPH particles and dark
The resulted angular momentum content of baryon (SPH particles) after 3 Gyr of evolution ismatter (DM) particles, there are its own L2 and L1 grid. To summarise, we have four grids ascompared in the Figure 2. Model 1, 2, 3, and 4 correspond to the lines with black, red, green, andL1SPH, L1DM, L2SPH, and L2DM. With the adopted value of the baryon fraction, the mass ratio
blue, respectively. The higher resolution models tend to have higher angular momentum contentof particles between L1SPH and L1DM is ~ 0.1 and L1SPH and L2DM is ~ 1.0. By combining theinside 10 Kpc. The difference between Model-1 and Model-3 is not as large as 2 times noted infour grids, we generate four initla models. We summarize the four models in the table. 4-th andSommar-Larsen etal. Model-2 has highest angular momentum. Note that the mass ratio between
7-th column show the gravitational softening length for SPH particles and DM particles, respeca SPH and darkmatter particle in Model 2 is most desirable in terms of artificial numericaltively. Here, we adopt the Plummer softening length. The last column shows the mass ratioeffects. Another intersting point to be noted is that the force resolution (gravitational softenbetween SPH particles and DM particles. Model-1 is the standard model with a modest numbering length) significantly affects the evolution of SPH particles. We compare the face on densityof particles. Model-3 is the high resolution model and both mass and force resolutions are 8of baryon disk for Model-3 and Model-4. The projected density for Model-3 (high force resotimes higher than Model-1 (also the number of particles is 8 times larger). Model-2 is the mixedlution) clearly shows more clumpy density distribution. From these results, it is shown that themodel using L1SPH grid and L2DM grid. Model-1 and 2 are computed with the serial code, andinplementation of our parallel Tree-GRAPE-SPH code works well. We will do higher resolution
Model-3 and 4 are computed with the parallel code. Radiative cooling is always effectve in all
simulation in the coming year.
models.