II-9. Instrumentation

RIKEN Accel. Prog. Rep. 55 (2022)

RIKEN Accel. Prog. Rep. 55 (2022)

CCJ operations in 2021
S. Yokkaichi,∗1 H. En’yo,∗1 T. Ichihara,∗1 W. Nakai,∗1 and Y. Watanabe∗1Overview
The RIKEN Computing Center in Japan (CCJ)1)
commenced operations in June 2000 as the largest oﬀsite computing center for the PHENIX2) experimentbeing conducted at RHIC. Since then, CCJ has been
providing numerous services as a regional computing
center in Asia. We have transferred several hundred
terabytes of raw data ﬁles and nDST a) ﬁles from theUSA.
Many analysis and simulation projects are being
conducted at CCJ, which are listed on the web page
http://ccjsun.riken.jp/ccj/proposals/. As of December 2021, CCJ has contributed to 46 published papers and 45 doctoral theses.
Computing hardware and software
The network conﬁguration and computing hardware(nodes) and software (OS, batch queuing systems,database engine, etc.) are almost the same as describedin the previous APR3) and the number of servers aresummarized in the Table 1. One of disk servers, which
had a 13-TB built-in RAID, was replaced by one with
39 TB in November. The main server (users’ home directory, NIS, DNS, and NTP) and SAS RAID will bereplaced in March 2022. In 4Q 2021, Hewlett-Packerd
hardware was in short supply, owing to the “semiconductor crisis.” Thus, we gave up to purchase built-in
RAID as the previous server had. In addition, we operate two dedicated servers for the RHICf group4) andtwo servers for the J-PARC E16 group5) in order tomaintain their dedicated compilation and library environments along with some data.
Table 1. Number of servers and disk sizes.
main server
login serverinteractive server
calculation server 1calculation server 2work disk server(replaced in Nov.)
DB server
library(AFS) servertransfer server
docker test server

number disk (TB)
(type)
121 + 13
DL380eG82
DL145G3/DL20G93-/DL320G6/DL160G91610DL180G6820DL180G6
226 / 13
DL180G9/DL180G6→39→DL385G1011DL145G3
19DL180G62
12 / 39 DL180G9/DL380G10
1DL20G9a)

Table 2. Number of malfunctioning HDDs in HP servers
during 2011–2021.
Type (TB) total 21 20 19 18 17 16 15 14 13 12 11SATA (1.0) 192 10 9 8 16 18 8 14 11 16 20 9SATA (2.0) 120
9 5 10 2 10 2 10 0 2 5 4SATA (4.0)
26
2 0 0 0 - - - - - - SATA (6.0)
200 0 0 0 - - - - - - SAS (0.15)
38
5 3 6 3 1 5 3 2 0 1 1SAS (0.3)
26
2 1 2 0 1 0 1 1 0 0 1

Joint operation with ACCC/HOKUSAI
CCJ and the RIKEN Integrated Cluster of Clusters(RICC) have been jointly operated since July 2009. InApril 2015, a new system named “HOKUSAI Greatwave”8) was launched by RIKEN Advanced Centerfor Computing and Communication (ACCC), and thejoint operation with CCJ continued, with the inclusionof a new hierarchical archive system in which approximately 1060 TB of CCJ data were stored as of December 2021. A breakdown of the data is presented inTable 3. Data transfer from J-PARC was performedin February and June during an experiment of over 18and 12 days, and 28 and 46 TB of raw data were transferred to CCJ, respectively. The data were archived ina tape device in Hokusai.8)
Table 3. Tape usage in Hokusai as of December 2021.
user

total PHENIX KEK/ RHICf user-leveloﬃcial J-PARC
archivesize (TB) 1059
749
1718130

References

We operate 25 computing nodes, and 328 (= 8 × 17nodes + 24 × 8 nodes ) jobs can be processed simultaneously by these computing nodes using a batch queu∗1ing system, LSF 9.1.3.6) Table 2 lists the number ofmalfunctioning SATA or SAS disks in the HP servers,
namely, computing nodes and NFS/AFS servers. The
OS of nine calculation nodes and of one interactive
server were upgraded to SL7.9 from SL5.37) in October, and the remainder will be upgraded in March
2022.
Three 10-KVA UPSs are operated as power supplies
for these CCJ nodes and their batteries were replacedin March.

RIKEN Nishina Centerterm for a type of summary data ﬁles in PHENIX- 103 -1) http://ccjsun.riken.jp/ccj/ .
2) http://www.phenix.bnl.gov/ .
3) S. Yokkaichi et al., RIKEN Accel. Prog. Rep. 54, 121(2021).
4) Y. Itow et al., arXiv:1409.4860 (Proposal).
5) S. Yokkaichi, in this report.
6) https://www.ibm.com/docs/en/spectrum-lsf/ .
7) http://www.scientificlinux.org/ .
8) https://i.riken.jp/en/supercom/ .

1