II-9. Instrumentation

RIKEN Accel. Prog. Rep. 54 (2021)

RIKEN Accel. Prog. Rep. 54 (2021)

CCJ operations in 2020
S. Yokkaichi,∗1 H. En’yo,∗1 T. Ichihara,∗1 W. Nakai,∗1 and Y. Watanabe∗1Overview
The RIKEN Computing Center in Japan (CCJ)1)
commenced operations in June 2000 as the largest oﬀsite computing center for the PHENIX2) experimentbeing conducted at RHIC. Since then, CCJ has been
providing numerous services as a regional computing
center in Asia. We have transferred several hundred
terabytes of raw data ﬁles and nDSTa) ﬁles from theRHIC Computing Facility (RCF)3) to CCJ.
Many analysis and simulation projects are being conducted at CCJ, which are listed on the web page
http://ccjsun.riken.go.jp/ccj/proposals/. As of December 2020, CCJ has contributed to 44 published papers and 45 doctoral theses.

Table 1. Number of malfunctioning HDDs in HP servers
during 2011–2020.

user

total

Computing hardware and software
The network conﬁguration and the computing hardware (nodes) and software (OS, batch queuing systems,
database engine, etc.) are almost the same as describedin the previous APR.4) We have two login servers, onemain server (users’ home directory, NIS, DNS, andNTP), and two disk servers, the disk sizes of whichare 13 and 26 TB. The main server has an external
SAS RAID (21 TB) for the home and work regions ofusers as well as system usage. Moreover, the server has
a RAID with built-in disks (13 TB) that can be used
temporarily by users and the system.
We operate 25 computing nodes, of which 16 nodes
were purchased in March 2009 and 9 nodes were purchased in March 2011. Thus, in total, 344 (= 8 × 16nodes + 24×9 nodes ) jobs can be processed simultaneously by these computing nodes using a batch queuingsystem, LSF 9.1.3.5) Upgrade to LSF 10 is planned in
JFY 2021. Table 1 lists the number of malfunctioning
SATA or SAS disks in the HP servers, namely, computing nodes and NFS/AFS servers.
One database (postgreSQL6) ) server and one AFS7)
server are operated in order to share the PHENIX computing environment. It should be noted that only theSL58) environment is shared by the computing nodes,
which have approximately 0.9 TB of library ﬁles. We
operated two data-transfer servers, which have a 12 TB
SATA RAID with built-in disks. This year, anotherdata-transfer server, which has a capacity of 39 TB,
was deployed. Data transfer from J-PARC was performed in June during an experiment over 16 days, and33 TB of raw data were transferred to CCJ. The data
were archived in a tape device in Hokusai.9) In addition, we operate two dedicated servers for the RHICfgroup10) and two servers for the J-PARC E16 group11)

size (TB)

921

∗1a)

RIKEN Nishina Center
Term for a type of summary data files in PHENIX

Type (Size) total 20 19 18 17 16 15 14 13 12 11
SATA (1 TB) 192
SATA (2 TB) 120
SATA (4 TB) 10SAS (146 GB) 38
SAS (300 GB) 26

9 8 16 18
5 10 2 100 0 0 −3 6 3 11 2 0 18 14 11 16 20 9
2 10 0 2 5 4− − − − − −5 3 2 0 1 10 1 1 0 0 1Table 2. Tape usage in Hokusai as of December 2020.
PHENIX KEK/RHICfoﬃcial
J-PARC
749

60

3user-levelarchive
109in order to keep their dedicated compilation and libraryenvironments along with some data.
A server for the test of Docker12) was deployed tosupport the sPHENIX/EIC users as well as to develop
an OS-independent job submission scheme.
Joint operation with ACCC/HOKUSAI
CCJ and the RIKEN Integrated Cluster of Clusters(RICC) have been jointly operated since July 2009. InApril 2015, a new system named “HOKUSAI Greatwave” 13) was launched by RIKEN ACCC,9) and thejoint operation with CCJ continued, with the inclusionof a new hierarchical archive system in which approximately 900 TB of CCJ data are stored. A breakdownof the data is presented in Table 2.
In autumn 2020, ACCC started a system of charging
users for their usage of CPU time and storage capacity.
Tape usage is not yet charged as of 2020, and a charging
policy for the future has not yet been ﬁxed. We are indiscussiions with ACCC to set a reasonable fee.
References

1) http://ccjsun.riken.jp/ccj/ .
2) http://www.phenix.bnl.gov/ .
3) https://www.racf.bnl.gov/ .
4) S. Yokkaichi et al., RIKEN Accel. Prog. Rep. 53, 137(2020).
5) https://www.ibm.com/docs/en/spectrum-lsf/ .
6) http://www.postgresql.org/ .
7) http://www.openafs.org/ .
8) http://www.scientificlinux.org/ .
9) http://accc.riken.jp .
10) Y. Itow et al., arXiv:1409.4860 (Proposal).
11) S. Yokkaichi, in this report.
12) https://www.docker.com/ .
13) https://i.riken.jp/supercom/ .

- 121 -1