RIKEN Accel. Prog. Rep. 52 (2019)

CCJ operations in 2018
S. Yokkaichi,∗1 H. En’yo,∗1 T. Ichihara,∗1 W. Nakai,∗1 and Y. Watanabe∗11Overview

The RIKEN Computing Center in Japan (CCJ)1)
commenced operations in June 2000 as the largest oﬀsite computing center for the PHENIX2) experimentbeing conducted at RHIC. Since then, CCJ has been
providing numerous services as a regional computing
center in Asia. We have transferred several hundred
TBs of raw data ﬁles and nDST a) ﬁles from the RHIC
Computing Facility (RCF)3) to CCJ.
Many analysis and simulation projects are being
conducted at CCJ, which are listed on the web page
http://ccjsun.riken.go.jp/ccj/proposals/. As of December 2018, CCJ has contributed to 43 published papers and 43 doctoral theses.
2

Computing hardware and software

The network conﬁguration and the computing hardware (nodes) and software (OS, batch queuing systems,database engine, etc.) are almost same as describedin the previous APR.1) We have two login servers,
one main server (users’ home directory, NIS, DNS andNTP), and two disk servers whose disk sizes are 13 and26 TB. The main server has an external SAS RAID (21TB) for the home and work regions of users, and system usage. Also, the server has a RAID with built-indisks (13 TB) which can be used temporarily by users
and the system.
We operate 26 computing nodes, of which 16 nodes
were purchased in Mar. 2009 and 10 nodes were purchased in Mar. 2011. Thus, in total, 368 (=8×16 nodes+ 24×10 nodes ) jobs can be processed simultaneously
by these computing nodes using a batch queuing system LSF 9.1.3.4) Table 1 lists the number of malfunctioning SATA or SAS disks in the HP servers, namely,computing nodes and NFS/AFS servers.
One database (postgreSQL5) ) server and one AFS6)
server are operated in order to share the PHENIX computing environment. It should be noted that only theSL57) environment is shared by the computing nodes,
which have approximately 0.9 TB of library ﬁles. We
have two data-transfer servers, on which the grid environment8) is installed for data transfer to/from RCF.
Two new data-transfer servers will be in operationin early 2019 and replace two servers being currently
used. Data transfer of the order of 100 TB from JPARC and BNL will be performed in the future. Inaddition, we operate a dedicated server for the RHICfgroup9) and two servers for the J-PARC E16 group10) ,∗1a)

RIKEN Nishina Centerterm for a type of summary data files in PHENIXin order to keep their dedicated compilation and library environments along with some data.
A 10-KVA UPS (supplied by NTT Facilities) wasdiscarded without battery exchange owing to its highcost of exchange. The power wiring was relocated inNov. 2018. Subsequently, a total power in our UPS
system is 30 KVA and a typical average load factor is51% without jobs and 59% with maximum number ofjobs in the calculation nodes.
3Joint operation with ACCC/HOKUSAI

CCJ and the RIKEN Integrated Cluster of Clusters(RICC) have been jointly operated since July 2009. InApril 2015, a new system named “HOKUSAI Greatwave” was launched by RIKEN ACCC11) and the joint
operation with CCJ continued, including a new hierarchical archive system in which approximately 900 TBof CCJ data are stored. Subsequently, the “HOKUSAI
BigWaterFall” IA cluster, which has 840 nodes/33600
CPU cores, was launched in 2017 by ACCC. Thena dedicated usage of legacy RICC 10 nodes by CCJ
was ended and the direct connection between CCJ andHOKUSAI with the two 10G Ethernet links was also
ended. CCJ has not started to use the cluster because
it does not support the NFS to share the computing
environment of PHENIX unlike RICC. Instead, we arein preparation to use one of the “container technologies” (such as “Docker”12) ) to share the environment.
Type(Size)
SATA(1TB)
SATA(2TB)
SATA(4TB)
SAS(146GB)
SAS(300GB)

total
192
120
1038262018
162030’1718
10
11’16
82
50’15
141031’14
110
21’13
16
200’1220510

Table 1. Number of malfunctioning HDDs in HP servers
during 2011-2018.

References
1) S. Yokkaichi et al., RIKEN Accel. Prog. Rep. 51, 188(2018).
2) http://www.phenix.bnl.gov/3) https://www.racf.bnl.gov/4) https://www.ibm.com/support/knowledgecenter/en/
SSETD4_9.1.3/lsf_welcome.html
5) http://www.postgresql.org/
6) http://www.openafs.org/
7) http://www.scientificlinux.org/
8) http://www.globus.org/toolkit/docs/latest-stable/
gridftp/9) Y. Itow et al., arXiv:1409.4860 (Proposal).
10) S. Yokkaichi, RIKEN Accel. Prog. Rep. in this volume.
11) http://accc.riken.jp/
12) https://www.docker.com/

1’11
9411