Ⅱ-9. Instrumentation

RIKEN Accel. Prog. Rep. 53

CCJ operations in 2019
S. Yokkaichi,∗1 H. En’yo,∗1 T. Ichihara,∗1 W. Nakai,∗1 and Y. Watanabe∗1Table 1. Number of malfunctioning HDDs in HP servers
during 2011–2019.

Overview
The RIKEN Computing Center in Japan (CCJ)1)
commenced operations in June 2000 as the largest oﬀsite computing center for the PHENIX2) experimentbeing conducted at RHIC. Since then, CCJ has been
providing numerous services as a regional computing
center in Asia. We have transferred several hundred
TBs of raw data ﬁles and nDST a) ﬁles from the RHIC
Computing Facility (RCF)3) to CCJ.
Many analysis and simulation projects are being
conducted at CCJ, which are listed on the web page
http://ccjsun.riken.go.jp/ccj/proposals/. As of December 2019, CCJ has contributed to 43 published papers and 44 doctoral theses.

for the RHICf group8) and two servers for the J-PARC
E16 group,9) in order to keep their dedicated compilation and library environments along with some data.
Three 10-KVA UPSs are operated as power supply forthese CCJ nodes, should be replaced in 2020.

Computing hardware and software

Joint operation with ACCC/HOKUSAI

The network conﬁguration and the computing hardware (nodes) and software (OS, batch queuing systems, database engine, etc.) are almost the same as
described in the previous APR.1) We have two login
servers, one main server (users’ home directory, NIS,
DNS, and NTP), and two disk servers the disk sizesof which are 13 and 26 TB. The main server has anexternal SAS RAID (21 TB) for the home and work
regions of users as well as system usage. Moreover, theserver has a RAID with built-in disks (13 TB) that canbe used temporarily by users and the system.
We operate 26 computing nodes, of which 16 nodes
were purchased in Mar. 2009 and 10 nodes were purchased in Mar. 2011. Thus, in total, 368 (= 8 × 16nodes + 24 × 10 nodes ) jobs can be processed simultaneously by these computing nodes using a batch queuing system, LSF 9.1.3.4) The LSF license contract in
CCJ is for 358 in total in JFY 2019, after the transferof 80 licenses from the RIBF cluster. Table 1 lists thenumber of malfunctioning SATA or SAS disks in theHP servers, namely, computing nodes and NFS/AFSservers.
One database (postgreSQL5) ) server and one AFS6)
server are operated in order to share the PHENIX computing environment. It should be noted that only theSL57) environment is shared by the computing nodes,
which have approximately 0.9 TB of library ﬁles. We
started two new data-transfer servers, which have a10 G Ethernet I/F and 12 TB SATA RAID with builtin disks. Data transfer of the order of 100 TB fromJ-PARC will start in 2020 and also from BNL in thefuture. In addition, we operate two dedicated servers

CCJ and the RIKEN Integrated Cluster of Clusters(RICC) have been jointly operated since July 2009. InApril 2015, a new system named “HOKUSAI Greatwave” was launched by RIKEN ACCC,10) and the joint
operation with CCJ continued, with the inclusion ofa new hierarchical archive system in which approximately 900 TB of CCJ data are stored. As of Nov.
2019, 886 TB is used, with the inclusion of 749 TB ofPHENIX oﬃcial data, 25 TB of KEK/J-PARC data,
3 TB of RHICf data, and 109 TB of user-level archive
data. Subsequently, the “HOKUSAI BigWaterFall” IAcluster, which has 840 nodes/33600 CPU cores, waslaunched in 2017 by ACCC. CCJ has not started touse the cluster, because it lacks support for NFS toshare the computing environment of PHENIX, unlike
RICC. In autumn in 2020, ACCC will start a charging
system to users for thier usage of CPU-time and storage capacity. Thus we are discussing the optimal ratioof purchasing new machines to paying to ACCC.

∗1a)

RIKEN Nishina Centerterm for a type of summary data files in PHENIX- 137 -Type(Size) total
192
SATA (2 TB) 120
SATA (4 TB)
10SAS (146 GB)
38SAS (300 GB)
26
SATA (1 TB)

19 18 17 16 15 14 13 12 11
8 16 18 8 14 11 16 20 9
10 2 10 2 10 0 2 5 40 0
6 3 1 5 3 2 0 1 1
2 0 1 0 1 1 0 0 1

References
1) S. Yokkaichi et al., RIKEN Accel. Prog. Rep. 51, 188(2018).
2) http://www.phenix.bnl.gov/ .
3) https://www.racf.bnl.gov/ .
4) https://www.ibm.com/support/knowledgecenter/en/
SSETD4_9.1.3/lsf_welcome.html .
5) http://www.postgresql.org/ .
6) http://www.openafs.org/ .
7) http://www.scientificlinux.org/ .
8) Y. Itow et al., arXiv:1409.4860 (Proposal).
9) S. Yokkaichi, in this report.
10) http://accc.riken.jp/ .