Ⅱ-9. Instrumentation

RIKEN Accel. Prog. Rep. 56 (2023)
RIKEN
Accel. Prog. Rep. 56 (2023)

CCJ operations in 2022
S. Yokkaichi,∗1 Y. Akiba,∗1 T. Ichihara,∗1 and Y. Watanabe∗1Overview
The RIKEN Computing Center in Japan (CCJ)1)
commenced operations in June 2000 as the largest oﬀsite computing center for the PHENIX2) experimentbeing conducted at RHIC. Since then, CCJ has been
providing numerous services as a regional computing
center in Asia. We have transferred several hundred
terabytes of raw data ﬁles and nDSTa) ﬁles from theUSA.
Many analysis and simulation projects are being conducted at CCJ, which are listed on the web page
http://ccjsun.riken.jp/ccj/proposals/. As of December 2022, CCJ has contributed to 47 published papers
and 45 doctoral theses.

queuing system, LSF 9.1.3.6) Table 2 lists the numberof malfunctioning SATA or SAS disks in the HP servers,
namely, computing nodes and NFS/AFS servers. The
OS of nine calculation nodes and of two interactive
servers operated in SL7.97) and the rest are still old.
Two old interactive servers will be shutdown following
the upgrade of the computing nodes.
Table 2. Number of malfunctioning HDDs in HP servers
during 2011–2022.
Type (TB) total
SATA (1.0)
192
SATA (2.0)
120
SATA (4.0)
26
SATA (6.0)
20
SAS (0.15)
38
SAS (0.3)
26

Computing hardware and software
The network conﬁguration and computing hardware(nodes) and software (OS, batch queuing systems,database engine, etc.) are nearly the same as describedin the previous APR,3) and the number of servers aresummarized in Table 1. The main server (users’ home
directory, NIS, DNS, and NTP), SAS RAID for theserver (15.5 TB), and one login server were replaced,
and a new interactive server was deployed in Oct. 2022.
Interactive servers are used for the compilation and test
runs for the jobs before the submission to the computing nodes. Two spare machines for interactive and login
servers were prepared.
Table 1. Number of servers and disk sizes, as of 2022 Dec.
Dagger(†) shows replacement and newly deploying inthis year (detail is in the text).

main server

number1login serverinteractive server

24

disk size(TB/node)6(built-in)+ 15.5(RAID)-calculation node 1
calculation node 2
work disk server
DB server
library(AFS) servertransfer server
docker test server

16821121102026 / 39
1912 / 39-typeDL360G10†
DL20G9/DL20G10†-/DL320G6/
DL160G9/DL20G10†
DL180G6
DL180G6
DL180G9/DL385G10
DL145G3
DL180G6
DL180G9/DL380G10
DL20G9a)

RIKEN Nishina Centerterm for a type of summary data ﬁles in PHENIX

1220
5101316
2001411021151410311682501718101118162003019810006220950031211092052Three 10-KVA UPSs are operated as power supplies
for these CCJ nodes. The next replacement is plannedin Mar. 2026 considering the 5-year battery life. Rearrangement of AC power lines for the nodes from these
UPSs was performed for the re-balancing of load, immediately after the deployment of new servers, during
the planned power outage in Wako Campus.
The replacement of the main switch is planned in2023, to support the 10 GBASE-T network for servers,
while 10 GBASE-LR and SR are used now only for theuplink to RIKEN-LAN and for the downlink to edge
switches for the calculation nodes, respectively.
Joint operation with ACCC/HOKUSAI
CCJ and the RIKEN IT division have been jointly
operated since July 2009. In April 2015, “HOKUSAIGreatwave”8) system was launched and the joint operation with CCJ continued, with the inclusion of a
new hierarchical archive system wherein approximately
1070 TB of CCJ data were stored as of December 2022.
A breakdown of the data is presented in Table 3.
Table 3. Tape usage in Hokusai as of December 2022.

In addition, we operate one dedicated server for theRHICf group4) and two servers for the J-PARC E16group5) in order to maintain their dedicated compilation and library environments along with some data.
We operate 26 computing nodes, and 352 (= 8 ×17 nodes + 24 × 9 nodes) jobs can be processed simultaneously via these computing nodes using a batch∗11194
1
1

user

total

size (TB)

1068

PHENIXoﬃcial749

KEK/J-PARC173

RHICf8user-levelarchive137

References
1) http://ccjsun.riken.jp/ccj/ .
2) http://www.phenix.bnl.gov/ .
3) S. Yokkaichi et al., RIKEN Accel. Prog. Rep. 55, 103(2022).
4) Y. Itow et al., arXiv:1409.4860.
5) S. Yokkaichi, in this report.
6) https://www.ibm.com/docs/en/spectrum-lsf/ .
7) http://www.scientificlinux.org/ .
8) https://i.riken.jp/supercom/ .

- 122 -22570021