INSTALLATION AND COMMISSIONING OF NEW EVENT TIMINGSYSTEM FOR SuperKEKB
H. Kaji, K. Furukawa, M. Iwasaki, E. Kikutani, T. Kobayashi, F. Miyahara, T.T. Nakamura,
M. Satoh, M. Suetake, M. Tobiyama, KEK, Ibaraki, Japan
Y. Iitsuka, T. Okazaki, East Japan Institute of Technology, Ibaraki, JapanT. Kudou. S. Kusano, Mitsubishi Electric System & Service Co. Ltd., Ibaraki, Japan
Abstract
The new Event Timing System was installed in 2014. The
parameters of new system to operate LINAC were optimized
through the commissioning which was based on the triggermonitoring and the experimental beam operation. The necessary functions for the Main Ring injection were additionallyintegrated in 2015. Finally we performed the study of longterm stability and confirmed the soundness of new system.
The new Event Timing System is ready for the injection into
Main Ring.

INTRODUCTION
The Event Timing System is one of the most importantcomponents to succeed the SuperKEKB project [1, 2]. Thissystem has already been installed at LINAC [3] for implementing Pulse-to-Pulse Modulation (PPM), which madethe operations of both KEKB [4, 5] and the light sources
compatible.
The new Event Timing System was designed to satisfy
the new requirements for SuperKEKB [6]. It was installedin the 2014 spring.
The commissioning of new system was performed throughthe trigger-monitoring and the experimental beam operationsin the 2014 autumn. We successfully operated LINAC in
October and injected beam-pulses into two light source ringsin November.
We increased the system in the first half of 2015. The
time-to-digital converter (TDC) was installed for “roughly”
synchronizing the injection trigger of Main Ring (MR) withpower-supply frequency of 50 Hz (AC50). The Bucket Selection system [7, 8] was integrated with the installation of
distributed shared memory.
We introduce the above activities in the following sections.

EVENT TIMING SYSTEM FOR
SuperKEKB
The module configuration in the Main Timing Station1
part of Event Timing System is upgraded for the SuperKEKB
project. The new configuration is reported elsewhere [6, 9].
In this section we explain more details which cannot be
described in the previous documents and updates which werecarried out during the installation and the commissioning.
1 It was called as Main Trigger Station during the KEKB era.

Figure 1: Picture of Event IOC: the system is based on
VME64x sub-rack. From left, the CPU, RAS, upper-EVG,
TDC, middle-EVR, two lower-EVGs, two fanout modules,and distributed shared memory are installed.

Configuration of EPICS IOC
Figure 1 is a picture of the Main Timing Station part of
Event Timing System. The system is constructed as the
VME64x based EPICS IOC [10] since it is the standard forthe accelerator components at KEK. Three Event Generators(EVGs) and one Event Receiver (EVR) [11] are installed
into the new Event IOC.
Two layers are configured with three EVGs. Two EVGs
(lower-EVG) are placed in the downstream of another EVG(upper-EVG). The EVR (middle-EVR) is placed for connecting two EVG layers. It receives Events from upper-EVG anddelivers TTL level triggers to lower-EVGs. The parallel configuration of two lower-EVGs is required from the Damping
Ring (DR) operation. One lower-EVG manage the LINAC
beamline before DR while the other lower-EVG manages
that after DR. These two beamlines are operated separately
when we inject e+ beam-pulse into MR.

In addition to the modules of Event Timing System, thetime-to-digital converter (TDC) and the distributed sharedmemory are installed in the IOC. The roles of these modules
are explained the later section.
For the CPU module, we adopt MVME6100 [12] not
MVME5500 [13], which is the standard CPU module forSuperKEKB. The faster bus clock of new CPU makes PPM
more stable. We consider it is necessary also for future
upgrades.

Firmware upgrade
The firmware version of EVG and EVR is upgraded fromversion-3 to version-5 even though we use same types of
hardware as those of KEKB. The new functions of version-5
firmware are necessary to satisfy the requirements of PPM
at SuperKEKB.
The other reason for firmware upgrade is the upgrade of
EPICS version. We adopt the EPICS version of 3.14.12 forthe accelerator components of SuperKEKB. The new Event
Timing System is operated with this EPICS version and thedevice/driver version of mrfioc2 [14]. It causes the Event
modules the firmware upgrade.

Figure 2: Trigger lines of Event Timing System: the sequence of upper-EVG is triggered by the division of MR
revolution. Lower-EVGs have three input channels. The
revolutions of light source rings are put into two of them.
The signal from middle-EVR is put into the last channel.
The input channels of lower-EVGs can be switched in 50 Hz.
The TDC also receives signal from middle-EVR.

lower-EVG which delivers Events towards the EVRs on thebeamline before DR. Other two fanouts are connected under the other lower-EVG. This arrangement is necessary tooperate the beamlines before and after DR, separately.

COMMISSIONING

Trigger Line
Figure 2 is the schematic view of the injection trigger lines
at the Event IOC. In the configuration for SuperKEKB, lowerEVG receives three kinds of inputs to launch the Events.
They are the injection triggers for MR and two light source
rings (PF [15], and PF-AR [16]).
The ACIN channel of lower-EVG receives the trigger
from middle-EVR. Upper-EVG and middle-EVR are functioned with the division of MR revolution. Therefore, thistrigger line can be used to launch Events for the MR injection. The revolutions of PF and PF-AR are put into extra
input channels, IN0 and IN1 of lower-EVG, respectively.
For the “simultaneous” top-up filling operations into morethan one rings, the PPM operation of SuperKEKB needs
switch of trigger lines among MR, PF, and PF-AR. It isrealized by just switching input channels of lower-EVGs
among ACIN, IN0, and IN1, on pulse-by-pulse. This is oneof the benefits of firmware upgrade.

INSTALLATION
The installation of new Event IOC was implemented in
the spring of 2014. The new IOC was installed without
removing the currently operating Event IOC.
The Event Timing System of LINAC becomes redundant.
This is important since even during the installation and commissioning of new IOC, LINAC must deliver e− beam-pulses
into PF and PF-AR. Note following works are performed
without disturbing any LINAC operations.
During the installation, the optical network for Event Timing System were bundled up to be two by changing theconfiguration of Event fanout modules [9]. Totally 4 fanout
modules were installed in the optical network for Event Timing System. Two of these fanouts are connected under one

The machine commissioning of new Event Timing System was performed through the trigger-monitoring and theexperimental beam operation. The detailed parameters for
operation of this system were optimized. Besides, the function to synchronize MR trigger with AC50 and the Bucket
Selection system were integrated on this IOC. We introduce
these activities in the following subsections.

Trigger-Monitoring
The trigger-monitoring was performed on each time during the second half of 2014 and the first half of 2015 forchecking the soundness of new Event Timing System.
During the summer shutdown of KEK accelerators in
2014, we connected the new Event IOC with all EVRs which
belong to the LINAC beamline. Then we studied the longterm stability of triggers with the trigger-monitoring system,
which is equipped at the LINAC beamline. This triggermonitoring system measures the interval of triggers which
are delivered to the LINAC components and counts up when
the interval is not to be “20 ± 1 ms”2 . We did not observe
any loss of triggers with this system.
The same study was repeated after integrating additional
functions on Event IOC. We introduce its results in the later
subsection.

Experimental Beam Operation
The further tuning of operation parameters was performed
through the experimental beam operation in October andNovember of 2014. We confirmed whether “Events delivered from lower-EVGs did not have a miss”. Then we optimized the delivery timing of all Events.
2 This condition is required from hardware at the LINAC beamline.

Figure 4: Timing relation among 8 injections, 9 injections,and rev/2254: the relation when the sequence is triggered ontime “0” is shown in cases of 8 injections and 9 injections.
The interval between the end of sequence and the next trigger
are different between these cases.

Figure 3: Storage current of PF during experimental beam
injection: the increase of current which is caused by injecting
e− beam-pulses with the new Event Timing System are seen.

During the LINAC operation, we successfully produced
e− beam-pulses for both PF and PF-AR. Especially, it is theimportant achievement that these two kinds of beam-pulses
are produced in turn. The productions of beam-pulses forboth PF and PF-AR in the same LINAC run were realized forthe first time while those for PF and MR have already beenrealized in the same LINAC run during the KEKB operation.
It is one of important mile stones for the top-up filling into
PF-AR, which has not been performed in the KEKB era.
After the commissioning at LINAC, the produced e−beam-pulses were injected into PF and PF-AR in November
2014. The injections into PF-AR and PF were performedon November 13 and November 20, respectively. Thesedays were machine study days. We successfully carried out
the injections with only a few hours of allocated time. We
achieved this important mile stone with the careful preparation and the redundancy of Event IOC. Figure 3 shows
the storage current at PF during the PF injection. The e−
beam-pulses were injected definitely and accumulated up to
450 mA, which is the normal current in the PF operation.

Synchronization with Power-Supply Frequency
The TDC module [17] was installed on the new Event IOC
for monitoring AC50 and synchronizing the MR injectiontrigger with it. To stabilize the performance of LINAC, theinjections are implemented on the same phase of AC50.
Therefore, all injection-triggers from Event Timing System
must be launched with the coincidence between AC50 andthe revolution of ring to be injected.
For the injection into light sources, we make the coincidence between AC50 and the revolution of PF (PF-AR) in
advance, and put those signals into the channel IN0 (IN1).
For the injection into MRs, we must use the 2254 times
division of revolution (rev/2254) as the input of upper-EVG.
The period of this signal is longer and to be 22.68 ms, sothat we cannot coincide it with AC50 in every time.
We developed the “rough” synchronization system for
MR injection with TDC. As shown in Figure 2, upper-EVG
is triggered by rev/2254. Then it launches Events for more

Figure 5: TDC measurement of AC50: the measurements
are performed with the configuration shown in Figure 2.
The timing jump at the boundary of sequences can be seen.
Totally 150 measurements are shown in turn. Note, we put
the signal from middle-EVR into the start channel of TDC
and monitoring AC40. Therefore the direction of timing
jump becomes opposite way from the discussion in the text.
than one injections with the longer sequence. All triggers
generated in this sequence precisely relate with the inputtedrevolution and are synchronized with MRs.
We manage upper-EVG with two different lengths of sequences. They are 8 injections sequence and 9 injections
sequence. By switching these sequence, MR triggers can“roughly” be synchronized with AC50.
Figure 4 explains the relation of these sequence andrev/2254. When upper-EVG processes an 8 injections sequence, it receives the next trigger on 18.748 ms after theend of sequence. It means the next sequence is launched∼1.25 ms earlier than the usual 20 ms periods. The opposite thing is happen on a 9 injections sequence. The next
sequence is launched 1.4 ms later than usual periods.
We manage the length of sequence for keeping the TDC
measurement of AC50. Then, the MR injection-triggers aresynchronized with AC50. Figure 5 shows the TDC measurements during the operation of new Event Timing Systemwith the management of sequence length. By switching thelength of sequence, we keeps the TDC measurement to be
10 ms. Therefore we can deliver the MR injection-triggers
on the same phase of AC50. Note these triggers are, ofcourse, synchronized with rev/2254.
In Ref [6, 9], we described the sequence length of upperEVG is to be ∼100 injections. However we decided more
shorter sequence length. This is because we understandFigure 6: Schematic view of three IOCs for Bucket Selection: the one of nodes is integrated in the Event IOC whileremaining two nodes are placed in the MR side. They areconnected via dedicated optical network.

Figure 8: Difference of TDC measurements, ∆T: the distribution of timing difference between AC50 and outputs fromthe temporal EVR are shown. The TDC measurements arecollected in 50 Hz for 242 hours. The results of PF (green),
PF-AR (blue), and MRs (black) modes are shown separately.
Figure 7: Schematic diagram of communication between
the Event IOC node and the CCB node: the processes of two
nodes are in parallel. However they are consolidated withthe data synchronization and the network interruptions.

from the trigger-monitoring that the fluctuation of AC50
is larger than that we expect and we should apply more
frequent correction. After several tests and considerations,we optimized them to be 8 injections and 9 injections frommany possible combinations.

Bucket Selection System
The new Event IOC works also for Bucket Selection [7,8].
The distributed shared memory module, VMIVME-5565[18], was installed for this purpose.
The VMIVME-5565 module is connected with those at
KEKB Central Control Building (CCB) and the D7 hall
via the dedicated optical network. The Event IOC receives
the delay value based on the MR revolution from the CCB
node and adjusts injection-timing into the RF-Bucket to
be injected beam-pulse (injection-bucket). This timingadjustment is managed at two lower-EVGs.
Figures 6 and 7 are the schematic pictures of hardware
and those processes in the PPM operation. The LINAC
node uploads the necessary information to decide the next
injection-bucket on the memory and launches the network
interruption to the CCB node. The CCB node decides thenext injection-bucket with the necessary information. Then
this node calculates the delay value, upload them on thememory, and returns the network interruption.

The entire process of Bucket Selection is managed at theMain Timing Station node with the network interruption
while the decision on the injection-bucket is made at CCB
node. They well represent the authorities of both LINAC
and MR.

Study of Long Term Stability
We studied the long-term stability after integrating thesynchronization system and the Bucket Selection system. We
put the temporal EVR at Main Timing Station and connected
it with one of lower-EVGs. Then, the output NIM signals
from this EVR were measured.
During this study, we set the schedule of injections into
PF, PF-AR, and two MRs to be equally 12.5 Hz. Therefore
two lower-EVGs were switching inputs in the PPM operation.
Actually, the switch of trigger-line makes timing fluctuationof LINAC operation and it is one of the concerns for new
Event Timing System. Therefore, we studied the size of thisfluctuation.
For Bucket Selection of MRs, we chose the provisionalmode. In this mode, the CCB node always returns the nominal timing, which is always matched with the AC50 measured by TDC. Note, in real operation, Bucket Selection is
implemented by adjusting timing within ±1 ms from thisnominal timing.
We put the outputs from temporal EVR into the TDC andcompare timings between it and AC50. Figure 8 shows the
distribution of difference between two TDC measurements.
We can confirm the stability of injection-triggers from the
new Event Timing System as a reference of AC50. Even

though the new Event IOC delivers triggers with three kindsof inputs, like the revolutions of PF/PF-AR and rev/2254,
the triggers were stable and periodically delivered within the
400 µs fluctuation. The fluctuation satisfies the hardware
requirement of <2 ms. Also it indicate we have the enoughtiming-window which we utilize for Bucket Selection.
The long-term stability was studied also with the triggermonitoring system which we described before. The monitoring was performed at the same time as the TDC measurements and we did not observe any loss of triggers during thetest.

CONCLUSIONThe Event Timing System at Main Timing Station was
upgraded for the SuperKEKB project. The two-layers of
EVGs are configured for the new Event IOC. The new system
was installed redundantly with the current operating system.
The activities after installation were smooth because of thisredundancy.
The parameters of new system to operate LINAC were
optimized through the commissioning. During this commissioning, the study of long-term stability and the experimental
beam operations were performed. As the experimental beam
operation, the injections into PF and PF-AR were carriedout successfully.
After the experimental beam operation in 2014, we integrated the synchronization system and Bucket Selection
system. All system for the MR injection were actualized at
the new Event Timing System. The soundness of system
was confirmed in the further study of long-term stability.
The fluctuation of injection-triggers delivered from the new
Event Timing System satisfies the requirement from the
LINAC hardware.
In conclusion the Event Timing System was successfully
upgraded for SuperKEKB. The new system is ready for theoperation of MR injections.

REFERENCES[1] Y. Ohnishi, et al., “Accelerator design at SuperKEKB”,
Prog. Theor. Exp. Phys., 2013, 03A011.
[2] K. Abe, et al., “Letter of Intent for KEK Super B Factory”,
KEK Report 20014-4.
[3] M. Akemoto, et al., “The KEKB Injector Linac”,
Prog. Theor. Exp. Phys., 2013, 03A002.

[4] K. Oide, et al., “KEKB B-Factory”,
Prog. Theor. Phys. 122, 2009, pp.69-80.
[5] S. Kurokawa, et al., “Overview of the KEKB Accelerators”,
Nucl. Instrum. Meth. A 499, 2003, pp.1-7.
[6] H. Kaji et al., “Upgrade of Event Timing System at SuperKEKB”,
Proceedings of ICALEPCS’13, San Francisco, CA, USA(2013).
[7] E. Kikutani et al., “KEKB Bucket Selection System, Present
and Future”,
Proceedings of 7th Annual Meeting of PASJ, Himeji, Japan.
[8] H. Kaji et al., “Bucket Selection System for SuperKEKB”,
THP100, These Proceedings,
Proceedings of 12th Annual Meeting of PASJ, Fukui, Japan.
[9] H. Kaji et al., “Construction and Commissioning of Event
Timing System at SuperKEKB”,
Proceedings of IPAC’14, Dresden, Germany (2014).
[10] EPICS website:
http://www.aps.anl.gov/epics/[11] MRF website:
http://www.mrf.fi/index.php/vme-products[12] https://www.slac.stanford.edu/grp/ssrl/spear/
epics/vme/6806800D58E_V6100_IU.pdf[13] http://www.mvme.com/manuals/MVME5500e-SPEC.
pdf[14] MRFIOC2 website:
http://epics.sourceforge.net/mrfioc2/[15] PF website:
http://www.kek.jp/en/Facility/IMSS/PF/PFRing/[16] PF-AR website:
http://www.kek.jp/en/Facility/IMSS/PF/PFAR/[17] T. Suwada et al., “Wide dynamic range FPGA-based TDC
for monitoring a trigger timing distribution system in linear
accelerators”,
Nucl. Instrum. Meth. A 786, 2015, pp.83-90.
[18] VMIVME-5565 website:
http://www.geautomation.com/products/
vme-5565-reflective-memory