RIKEN Accel. Prog. Rep. 48 (2015)

Ⅱ-9. Instrumentation

CCJ operations in 2014
S. Yokkaichi,∗1 H. En’yo,∗1 T. Ichihara,∗1 and Y. Watanabe∗11Overview

The RIKEN Computing Center in Japan (CCJ)1)
commenced operations in June 2000 as the largest oﬀsite computing center for the PHENIX2) experimentbeing conducted at the RHIC3) . Since then, the CCJ
has been providing numerous services as a regional
computing center in Asia. We have transferred several
hundred TBs of raw data ﬁles and nDST a) ﬁles fromthe RHIC Computing Facility (RCF)4) to the CCJ.
The transferred data ﬁles are stored in a High Performance Storage System (HPSS)5) . A joint operation of
the CCJ with the RIKEN Integrated Cluster of Clusters (RICC)6) is continued since July 2009. HPSS anddedicated 20 PC nodes are maintained by them.
Many analysis and simulation projects are being carried out at the CCJ, and these projects are listed on theweb page http://ccjsun.riken.go.jp/ccj/proposals/.
As of December 2014, CCJ has contributed 31 published papers and 40 doctoral theses.
2

Computing hardware and software

In 2014, computing hardware (nodes and RAIDs)
and software (OS, batch queuing systems, databaseengine, and so on) were changed slightly from thosedescribed in the previous APR1) . In summary, we
have 28 computing nodes, two login servers, one main
server (users home directory, NIS, DNS, NTP), two
disk servers and HPSS machines in our machine room,and 20 computing nodes in the RICC room. In total, 524 jobs can be processed simultaneously by thesecomputing nodes.
One database (postgreSQL7) ) server and one AFS8)
server are operated in order to share the PHENIX computing environment (the other database server retiredin 2014). Sharing of the Scientiﬁc Linux9) 4 (SL4) environment was stopped in April 2014, and now only theSL5 environment is shared by the computing nodes,
which have approximately 0.9 TB of library ﬁles. We
have four data-transfer servers on which the grid environment10) is installed for the data transfer to/from
RCF. Two servers out of the four retired in Jan. 2015.
Disk failure occurred in March 2014, during a reboot after a planned power outage. File system of anusers work disk became corrupt and about 1.5 TB ofdata were lost in spite of adopting VxFS, a journalingﬁle system. Another trouble, namely, power outage atthe RAID due to the power failure on Wako Campus
in February, is thought to be an underlying cause of∗1a)

RIKEN Nishina Centerterm for a type of summary data files in PHENIX

the corruption of the ﬁle system. This has been themost serious accident at the CCJ so far, although it isdeclared that there are no backup of the disk.
Table 1 lists the numbers of malfunctioning SATA orSAS disks in the HP servers, namely, computing nodes
and NFS/AFS servers.
Table 1. Number of malfunctioning HDDs in 2011-2014
TypeSATASASSize
1 TB
2 TB
146 GB300 GBTotal
192
12038242014
110212013
1620020122051020119411

2.1 HPSS
The amount of data archived in the HPSS were approximately 1.73 PB (1.53 PiB) in 2.15 million ﬁles,as of December 2014, not so changed in 2014. No newdata ﬁle has been transferred in 2014, and therefore,
the data list in the previous APR1) is still eﬀective.
RICC will be upgraded to the “HOKUSAI” system,a new RIKEN computing cluster launched in April
2015, and HPSS will retire around August 2015. Datamigration to the new archiving system was performed
in Jan. 19 - Mar. 16 2015. The 863 TB (785 TiB) ofdata in 1.66 million ﬁles out of 1.73 PB were migrated
to the new archiving system, because the raw data ofPHENIX, which were transferred for DST production
and already served out, were not migrated.
3Prospect

The replacement of the main RAID and server ison-going, and should be completed in May 2015. Thebatteries of the four UPSs have expired and should be
replaced in 2015. Regarding to the upgrade from RICC
to HOKUSAI, conﬁguration changes in the networkand job submitting environment are planned in theﬁrst half of 2015.
References
1) Y. Ikeda et al.: RIKEN Accel. Prog. Rep. 47, 237(2014).
2) http://www.phenix.bnl.gov/3) http://www.bnl.gov/rhic/4) https://www.racf.bnl.gov/5) http://www.hpss-collaboration.org/
6) http://accc.riken.jp/ricc/7) http://www.postgresql.org/
8) http://www.openafs.org/
9) http://www.scientificlinux.org/
10) http://www.globus.org/toolkit/docs/latest-stable/
gridftp/- 248 -

完全版2014_本文.indd 248

15/10/16 17:57