RIKEN Accel. Prog. Rep. 51 (2018)

CCJ operations in 2017
S. Yokkaichi,∗1 H. En’yo,∗1 T. Ichihara,∗1 and Y. Watanabe∗11Overview

The RIKEN Computing Center in Japan (CCJ)1)
commenced operations in June 2000 as the largest oﬀsite computing center for the PHENIX2) experimentbeing conducted at RHIC. Since then, CCJ has been
providing numerous services as a regional computing
center in Asia. We have transferred several hundred
TBs of raw data ﬁles and nDST a) ﬁles from the RHIC
Computing Facility (RCF)3) to CCJ.
Many analysis and simulation projects are being carried out at CCJ, and these projects are listed on theweb page http://ccjsun.riken.go.jp/ccj/proposals/.
As of December 2017, CCJ has contributed to 43 published papers and 42 doctoral theses.
2Computing hardware and software

The network conﬁguration and the computing hardware (nodes) and software (OS, batch queuing systems,database engine, etc.) are almost same as describedin the previous APR.1) We had 28 computing nodes,of which 18 nodes were purchased in Mar. 2009 and
10 nodes were purchased in Mar. 2011. In 2017, two
old nodes out of these were retired. Thus, in total,
368 (=8×16 nodes + 24×10 nodes ) jobs can be processed simultaneously by these computing nodes using
a batch queuing system LSF.4) The version of LSF is9.1, which was upgraded from 8.0.0 in Mar. 2016. b)
One database (postgreSQL5) ) server and one AFS6)
server are operated in order to share the PHENIX computing environment. Now, only the SL57) environmentis shared by the computing nodes, which have approximately 0.9 TB of library ﬁles. We have two datatransfer servers, on which the grid environment8) is
installed for the data transfer to/from RCF. Two new
data-transfer servers will be delivered in Jan. and Mar.
in 2018. After the deployment of the two, the current
two servers will be retired. Data transfer of the orderof 100 TB from J-PARC and BNL will be performedin the future.
Moreover, two login servers, one main server (users’
home directory, NIS, DNS, NTP), and two disk servers
are operated. In July 2017, a disk server machine wasreplaced by HP DL180 Gen9, which has 25.5 TB ofSATA RAID6. In August 2017, a login server machine
was replaced by HP DL20 server with SL7.4.
Table 1 lists the number of malfunctioning SATA or∗1a)b)

RIKEN Nishina Centerterm for a type of summary data files in PHENIX
It was performed in 2016 and should be written in the previous APR1) .

SAS disks in the HP servers, namely, computing nodes
and NFS/AFS servers.
The current total power of the four UPSs is 40 KVA,
in which one 10-KVA UPS will require an exchangeof batteries soon. Downsizing of the machine rooms(258/260 in Main Bldg.) was planned in 2017, but ithas been postponed.
3Joint operation with ACCC/HOKUSAI

CCJ and the RIKEN Integrated Cluster of Clusters(RICC) have been jointly operated since July 2009. InApril 2015, a new system named “HOKUSAI Greatwave” was launched by RIKEN ACCC9) and the joint
operation with CCJ has been continued, including
a new hierarchical archive system in which approximately 900 TB of CCJ data are stored. The dedicated usage of 10 nodes, a legacy of old RICC, wasalso continued and ended in June 2017, at the timeof deployment of the new cluster system “HOKUSAIBigWaterFall” by ACCC.
The usage of BigWaterFall, which has 840
nodes/33600 CPU cores, by CCJ has not been started
yet. It is planned to use one of the “container technologies” (such as “Docker”10) ) to share the computing environment of PHENIX, in place of NFS, which
was used in the old “dedicated” system. When NFS isnot used, the nodes can also be used by other users ofHokusai, which results in more eﬃcient usage of Hokusai’s CPUs.
Type(Size)
SATA(1 TB)
SATA(2 TB)
SATA(4 TB)
SAS(146 GB)
SAS(300 GB)

total192
1201038262017181011’16
8250’15141031’14
110
21’13
16
200’12205
10’119411

Table 1. Number of malfunctioning HDDs in HP servers(calculation nodes and NFS/AFS servers) in 2011-2017,
including nodes retired in 2017.

References
1) S. Yokkaichi et al.: RIKEN Accel. Prog. Rep. 50, 225(2017).
2) http://www.phenix.bnl.gov/3) https://www.racf.bnl.gov/4) http://www-03.ibm.com/systems/technicalcomputing/platformcomputing/products/lsf/index.html
5) http://www.postgresql.org/
6) http://www.openafs.org/
7) http://www.scientificlinux.org/
8) http://www.globus.org/toolkit/docs/latest-stable/
gridftp/9) http://accc.riken.jp/
10) https://www.docker.com/

1