Introduction to Super-Kamiokande
for geeks(?)(rev1.5)

Yoshinari Hayato
Kamioka obs., ICRR / Kavli IPMU
The Univ. of Tokyo1© HiggsTan 2012-2016

Elementary particles

2Elementary particles

Neutrino ( )3Elementary particles

Two types : Quark and Leptons
Both have three generations.

4Proton decay
Proton is known to be very stable.

© HiggsTan 2012-2016

Late ’70s, proton decay was predicted.

5Search for the proton decay
Predicted decay mode :→+Ring imaging water Cherenkov detector is suitable.
• Excellent efficiency• Relatively easy to realize gigantic detector.
Technically and cost wise.

6

Search for the proton decay
Predicted decay mode :→Simulation+

Ring imaging water Cherenkov detector is suitable.
• Excellent efficiency• Relatively easy to realize gigantic detector.
Technically and cost wise.

7

Search for the proton decay
Predicted life time of proton ~ 1032 years
1 proton is expected to decay
if we have 1032 protons in a detector.

1032 protons ~ 160 tons
KamiokaNDE (1983 ~ 1996)
Kamioka
Nucleon
Decay
Experiment
Water Cherenkov Detector
with 3000 tons of water.

16m
8Location of the detector
Underground in the mine
• Cosmic rays

~1Hz / 10cm2(~size of your palm)on the ground surface.
•emits Cherenkov photons.
• Locate detector under the groundto suppress by 0.00001 (10-5).
• Necessary to haveplenty of pure water.

©Yuki A. higgstan .com

~1000m9

Background (enemy)
Atmospheric neutrinose+(Positron)©Yuki A. higgstan .com

Careful studies of neutrinos are essential.

10Atmospheric neutrino anomaly
Observed number of upward going
was much smaller than expected.
Observation @ Super-Kamiokande

Initial expectation

Data

Seems

becomes

Downward
going

Upwardgoing? .

©Yuki A. higgstan11.com

Discovery of neutrino oscillations
and non-zero mass

The 2015 Nobel Prize in Physics was awarded
to Professor Takaaki Kajita.

12Neutrino ( )One of the elementary particles (素粒子)Most ordinary particle
Average density
Electron~0.1/m3
Proton~0.1/m3
Neutron~0.01/m3© NASA/WMAP Science Team

Neutrino
3x108/m133

Neutrino ( )One of the elementary particles (素粒子)Most ordinary particle

6.6 x 1010 (66G) neutrinos /cm2/s
are coming from the Sun.

14

StarsSupernovae

Neutrino ( )Neutrinos sources
Sun

Reactors,
Accelerators(c) Anglo-Australian Observatory/
David Malin

Atmosphere

Earth core15

Neutrino ( )Neutrinos from Supernova
Kamiokande detected 11 neutrinos from SN1987a.
Feb. 23, 1987 16:35:35 (JST)
Neutrinosfrom SN1987a(c) Anglo-Australian Observatory/
David Malin

The 2002 Nobel Prize in Physics was awarded toProfessor Masatoshi Koshiba.
16

Neutrino ( )One of the elementary particles (素粒子)One of the “leptons”
Leptonswith electric charge

Leptons
without electric charge

electron (e)electronneutrino

muon ( )neutrinotau ( )

neutrino17

Neutrino ( )One of the elementary particles (素粒子)One of the “leptons”• Masses of neutrinos are “almost” 0.
• Interaction probability is so small.
Typical solar neutrino can penetrate~1015km (1Pm) of soil.

18

Super-Kamiokande
Kamioka Neutrino Detection Experiment
Water Cherenkov detector
50,000 tons of pure water

50cm Photo Multiplier Tube
~11,000 PMTs for ID

Dual layer
19

The Super-Kamiokande collaboration

About 180 collaboratorsfrom 10 countries and ~40 institutes.
Japan, Canada, China, France, Italy, Korea, Poland, Spain,
United Kingdom and the United states of America
20Super-Kamiokande
Kamioka Neutrino Detection Experiment
41.4m

SK-I started in Apr. 1st, 1996.
SK-IV finished on May 31st, 2018.
SK-V started in Jan. 29th, 2019.
39m

1996
SK-I

2002
SK-II

2006 2008
SK-III

SK-IV

2018 2019
SK-V

40%
20%
40%
Photo coverage 40%
Accident Full reconstruction Replace electronics & DAQ system

40%

Preparationfor SK-Gd

21

Super-Kamiokande
Ring imaging Water Cherenkov detector
Cherenkov light• Emitted when the speed of the charged particle
is faster than the speed of the light in the medium.
• Relativistic charged particle
in water~ 42.

• Color (wave length)Blue ~ Ultra violet

Neutron Radiography Reactor, Hot Fuels Examination
22
Facility, Idaho National Laboratory.

Super-Kamiokande
Ring imaging Water Cherenkov detector• # of the charged particles & photons~ # of rings• Momentum (~energy) of a particle∝ # of photons• Generated position of the particles• Direction of a particle• Type of the particle
Reconstructed using thearrival timings andgeometrical distributionsof the Cherenkov light.
23

How to detect neutrinos
Electron neutrinos
from the Sun

electronelectrons
Oxygen atom
Atmospheric neutrinos

muon

Example) muon neutrinos

mp protonn neutron
Oxygen atom

n

p
24

Super-Kamiokande
Ring imaging Water Cherenkov detector
Signal : Cherenkov light emission occurs
in a short period of time, a few ~ 200 ns.
Noise : Continuous (~5kHz/PMT)# of PMTs
with signal

Record as signal

ThresholdTime

Signal could be identified
when multiple PMTs
produce signals within ~200ns.

25

Atmospheric neutrino events~ 8 events / dayElectron and muon produces
different patterns

Electron
(e)

Muon
( )

26

Atmospheric neutrino events~ 8 events / daySometimes, multiple rings (particles) observed.

27

Solar neutrino events~ 10 events / day

25 ~ 100 PMTs
detect light.

28

Supernova neutrino events©NASA©NASA/CXC/M.Weiss~99% of energy from Supernova is released as neutrinos.
From a supernova at the galactic center (~10kpC ~ 32,600 ly)~10,000 events are expected in just 10 seconds.
29
If Betelgeuse becomes supernova, 10M ~ 100M events are expected.

Super-Kamiokande
Background events :radio activities

Radon in the water etc..

~12,000 events / second(with current setting.)

30

Super-Kamiokande
Background events :Cosmic ray muons
2 ~ 3 events / second31Super-Kamiokande
Ring imaging Water Cherenkov detector
What we want to measure? ( necessary functionalities )Measure charge from photo sensors
Measure arrival timing of photons
What we need?
• Self gating signal digitizer (Charge + Timing)• Accurate clock synchronization system( Synchronization Clock + Counter )• GPS
Digitizer

Readout

Data processor

Storage

We don’t know when neutrinos are detected.
24 hours 7 days operation
Stable hardware and software.

32

Photo multiplier tube (PMT)
光電子増倍管Able to detect single “photon”.

©Higgstan

Apply high voltage (~2000V) to amplify the signal.
(= Increase the number of electrons.)
PMT is so sensitive to the photons.
Need to be operated in the dark environment.

33

Photo multiplier tube (PMT)
光電子増倍管-50050Time (ns)• Amplify signal by 107
Pulse heighta few mV (for single photon)
to a few V (~1000 photons)• Timing resolution is ~3ns.
• But there are noises (“dark” noise) at ~5kHz…

34

Charge and timing digitizer module
SiTCP : TCP/IP implemented
on FPGA (without CPU)
Fast
Ethernet

PMTs
x24

Charge to time converter (QTC)(ASIC developed for SK
by ICRR and IWATSU)

Nominal ~750kB/s/board
35
Maximum ~10MB/s/board

Charge and timing digitizer module

All the cable lengths
are same: ~70m.

36Super-Kamiokande Data acquisition system
A/D conversion board

A/D conversion board
Ethernet

Readout computers

Ethernet~25MB/s/1com.
(x 24 computers)

Software triggerzcomputers~500MB/s
Temporary storage
data server~500MB/s
Special “software trigger”~85MB/s/com.
(x6 computers)
3~4MB/s/com.,~20MB/s in total.

Readout ~ 30 boards by
one readout computer.
“Software trigger” computers
select “events”.
Reduce noise & keep signal
from ~500MB/s
to ~20 MB/s.

zData recorder computer

Mass Storage(outside of the mine)

37Super-Kamiokande Data acquisition system
Difference from the usual “network” traffic pattern.
Usual case

Data acquisition systemclientFront-end

Distribute data from the server.

clientclientCollect data fromfront-end computers.

Front-end

Front-end

Network switch

Network switch

Data server

Server

Opposite direction38Super-Kamiokande Data acquisition system
100Mbps x 30

1Gbps x4 (trunk)

2960G(X)
24 PMTs30 boards(20 boards for OD)

CPU
2.1GHz x2(Xeon E5-2620v4 8C16T)
Memory 64GB
HDD
1TB (mirror)

Selection of the switch : Requirements• FIFO-like behavior is preferred.
• Large and independent “port buffers” are preferred.
• Unexpected packet discards
including the intelligent packet shaping and QoS
make the data transmission uncontrollable.

39Computers for the SK data acquisition system
1996~
Data collector (readout)
SPARC Station Classic x 9
CPU
50MHz
( micro SPARC)
Memory
64MB
HDD
1.5GB

Data recorder (Event builder)
SPARC Station 10
CPU
50MHz x 2
(Super SPARC )Memory
112MB
HDD
14GB

2017~
Data collector
IA32(64bit) server x 25
CPU
2.1GHz x2(Xeon E5-2620v4 8C16T)
Memory64GBHDD
1TB (mirror)

Data selection, Data recorder
IA32(64bit) server x 12
CPU
3.5GHz x2(Xeon E5-2637v4 4C8T)
Memory96GB
HDD
24TB（集積用）

40

Super-Kamiokande Data acquisition system(1996~2008)

120 boards

Classic
8 computers

SS10“Trigger” was issuedby hardware.
Trigger
GPS

SS10

Network was FDDI, initially.
Later it was replaced with
Fast Ethernet andGigabit Ethernet.41

Super-Kamiokande Data acquisition system(1996~2008)
40MHz micro/Super SPARC -> 600MHz ~1GHz Ultra SPARC
FDDI + Ethernet-> Gigabit Ethernet(See Appendix 2)
Energy (MeV)~12KHZ@ SK IV

42

T2K : Accelerator neutrino experiment
Produce neutrino beam @ J-PARC, Tokai, Ibaraki
Aiming at the direction of Super-Kamiokande.

Detect neutrinos @ Super-Kamiokande
43

T2K : Accelerator neutrino experiment©Higgstan

To Kamioka

Accelerated
proton44

T2K : Accelerator neutrino experiment©Higgstan

Neutrino production target for T2K
Old FEM simulation(beam config. has been changed.)

30GeV 750kW
Graphite(carbon）
Diameter 26 mm
Air He cooled.

Maximum temp. >700oC45

Super-Kamiokande Data acquisition system
for the accelerator neutrinos
Accelerator produces neutrino every 2 sec.
( beam width ~4 micro seconds.)
Record the neutrino production time with GPS.
Send information via L2VPN. (Typically ~10ms.)
Distance = 295km
Neutrino arrives at SK in ~1ms.

J-PARC
Kamioka

L2VPN (SINET)

Tokai

46

Super-Kamiokande Data acquisition system
for the accelerator neutrinos
In SK, keep all the data for several seconds.
GPS data is always recorded @ SK, also.
Record the data around the beam arrival timing,from −
to +.

Distance = 295km
Neutrino arrives at SK in ~1ms.

J-PARC
Kamioka

L2VPN (SINET)

Tokai

47

Super-Kamiokande Data acquisition system
for the accelerator neutrinos(T2K Preliminary)(ns)
Kamioka

L2VPN (SINET)

J-PARC
Tokai

48

Super-Kamiokande Data acquisition system
for the accelerator neutrinos
Are the 2 GPS receivers providing “correct” time?
Usual GPS receivers use multiple satellites
and calculates the “most probable” time.
Special GPS receiver (common-view receiver)records information from each satellites.
NICT provides JST-GPS satellite difference
on the next day.

(See Appendix 3)49

Super-Kamiokande offline computer system
Offline computer system
1) Storage ~ Keep all the data.
All the data files since 1996 are available
on the disk.
2) CPU
Simulation, reconstruction and analysis

50Super-Kamiokande offline computer system
Offline computer system
1) Storage ~ Keep all the data.
All the data files since 1996 are available
on the disk.
Large volume distributed file system
FEFS (enhanced version of Lustre)
Capacity 9PB (4.5 PB x 2)
Meta data servers
2x2
Object storage servers 4 x 2
Home directory server
NR1000 F8020 (NFS server)
Capacity 23TB (+snapshot 4.5TB)
Tape
4.2PB (LTO7, 8drives, 688 tapes)(Backup management software is TSM.)

51

Super-Kamiokande offline computer system
Offline computer system
2) CPU
Simulation, reconstruction and analysis
Total 2212 CPU cores
Number of nodes

79

CPU
Xeon E5-2680v4(2.4GHz, 14coers) x 2
Memory128GB
OS
CentOS 7
Job scheduler
Technical Computing Suite
52

Super-Kamiokande offline computer system
Tape system
Tape sever : VPX210/10S
Total 12TB (8000 tapes)
Computers for analysis
SS10 (Super SPARC)x10
Connected withULTRANET 1000 and FDDI

53

Super-Kamiokande offline computer system
Need large amount of simulated data.
Several to 10 times larger statistics is necessary.
Atmospheric neutrino~ 10,000 neutrino interactions / year
We have ~20 years of collected data.
Need to generate and analyze
2,000 years equivalent simulated data.
(~20M neutrino interactions)
It takes ~ 1 month with 500 CPU coresto process simulated data with“traditional” event reconstruction tool.
Another weeks to fit neutrino oscillation params.

54

Super-Kamiokande event reconstruction
What we want to measure:••••

Neutrino interaction position
Number of particles
Type of the particle
Energy of the particle• Identify neutrino type
• Estimate neutrino energy
and• Check whether the event is causedby the decay of proton or a neutrino.

Image processing using the charge and the timing.

55

Super-Kamiokande event reconstruction
Reconstruct interaction position (#1)
Use observed timing.
(relative time difference)
T1

Tn

T2
X

56

Super-Kamiokande event reconstruction
Search for the rings and identify # of rings.
Project ( , ) plane anduse Hough transformto search for the rings.
Cherenkov ring

Possible ring center

Reconstructed
CenterPMTwith signal
57

Super-Kamiokande event reconstruction
Identify particle type
Rings with fuzzy edges
e-likeRing with sharp edge
-like

Use the particle type information,
re-evaluate interaction position
andestimate the energy of each particle.

58Super-Kamiokande event reconstruction
New method
Fit everything simultaneously!
(neutrino interaction position,
particle types, directions
and energies.)
Pre-calculate expectedcharge distributions of
all the particles withvarious energies.
Generate various combinations andsearch for the best one by comparing
the expectations and observed charge and timing.

59

Super-Kamiokande software• Neutrino interaction simulation program
Fortran with C++ (mixed)• Detector simulation program
Fortran (GEANT3 based)• Event reconstruction
Fortran (Traditional)
C++ (New method)• Data handling softwareand analysis frameworks
Fortran (CERNLIB/PAW)
C++ (ROOT)• Software version control
Subversion

60Remaining questions
At the beginning of the universe,same numbers of particles
and anti-particles are
believed to be produced.
But now, only particles are remaining.
There must be some differences
between particles
and anti-particles.
Possible to study
by comparing neutrino andanti-neutrino oscillations.
But we need to havemuch larger # of events.

electronpositron61

Remaining questions
So far, there is no indication of proton decay…
Simulation

We need much larger exposure~ much larger detector.

62

Next generation : Hyper-Kamiokande
Need more statistics!
Fiducial mass of Hyper-Kamiokande is 190kton,
which is ~ 8 times larger compared to SK.

https://www.youtube.com/watch?v=JFOE3D2z7LM

Expected to start observation in 2027!

63

スーパーカミオカンデ 一般公開のお知らせ
スーパーカミオカンデ実験エリアの一般公開を2019年11月2日（土）に開催致します。
2019年9月2日に申し込み方法などの募集要項を発表します。
申し込み期間は9月9日～24日を予定しています。

でも検出器の中はhttp://www-sk.icrr.u-tokyo.ac.jp/pr/event/2019/07/skopen.html

64

Fin.

65

Appendix 1
スーパーカミオカンデ実験施設の web site

http://www-sk.icrr.u-tokyo.ac.jp/sk/index.html
今年の一般公開について
http://www-sk.icrr.u-tokyo.ac.jp/pr/event/2019/07/skopen.html

スーパーカミオカンデ実験施設のパノラマ画像

http://www-sk.icrr.u-tokyo.ac.jp/panorama/SKXMASS/
検出器内の写真（２０１８年改修工事）
http://www-sk.icrr.u-tokyo.ac.jp/sk/tankopen2018/photo.html

66Appendix 2

Solar neutrino fluxandsurvival probability

The reason why we want to observe “lower” energy solar neutrinos.
Transition from the matter effect dominant region
to the vacuum oscillation dominant region
can be observed by lowering the energy threshold.

F. L. Villante@Neutrino2016

Expected
oscillation(survival)
probability.
67Appendix 3
JST and common view method (in Japanese)
https://www.nict.go.jp/pamphlet/TS_j-panf.pdf

68Appendix 4
Related to the question about the delay of the lightand real-time supernova monitoring system.
Observation of neutrinos from SN1987a is~ 3 hrs. earlier than the first optical observation.

69

Appendix 4
Related to the question about the delay of the lightand real-time supernova monitoring system.
Neutrino observation from SN1987a is~ 3 hrs. earlier than the first optical observation.

Illustration by R.J. Hall. Redrawn by Magasjukur2

Just after the core collapse, neutrinos are emitted.
Photons are emitted in the later stages.
70Appendix 4
Related to the question about the delay of the lightand the real-time supernova monitoring system.
超新星のリアルタイム検出システムについて（論文）

Real-time supernova neutrino burst monitor at Super-Kamiokande
Astropart. Phys. 81 (2016) 39-48 arXiv: 1601.04778“In summary, the SNmonitor reformats the data and determines
the vertex position, direction, and energy of events within a few
minutes of the data being collected.
It then searches for bursts of events – a cluster occurring within
20s – with energies above 7MeV and whose vertices fall within
SK’s 22.5-kton fiducial volume.
Within an hour of a warning being issued, the experts gather andhold a meeting to determine the appropriate public announcement
to make, if any, based on the quality and nature of the detected burst.”

SuperNova Early Warning System (SNEWS)https://snews.bnl.gov/whatissnews.html
71Appendix 5© HiggsTan 2012-2014

72

Appendix 5© HiggsTan 2012-2014

73