DEVELOPMENT OF FPGA-BASED TDC WITH WIDE DYNAMIC RANGEFOR MONITORING THE TRIGGER TIMING DISTRIBUTION SYSTEM
AT THE KEKB INJECTOR LINAC
Tsuyoshi Suwada∗ , Kazuro Furukawa, Fusashi Miyahara, KEK, Tsukuba, Japan
Abstract
A new field-programmable gate array (FPGA)-based
time-to-digital converter (TDC) with a wide dynamic range
greater than 20 ms has been developed to monitor the timing of various pulsed devices in the trigger timing distribution system of the KEKB injector linac. The pulsed devices
are driven by feeding regular as well as any irregular (orevent-based) timing pulses. For monitoring the timing asprecisely as possible, a 16-ch FPGA-based TDC has beendeveloped on a Xilinx Spartan-6 FPGA equipped on VME
board with a time resolution of 1 ns. The resolution wasachieved by applying a multisampling technique, and theaccuracies were 2.6 ns (rms) and less than 1 ns (rms) within
the dynamic ranges of 20 ms and 7.5 ms, respectively. The
various nonlinear eﬀects were improved by implementing
a high-precision external clock with a built-in temperaturecompensated crystal oscillator.

INTRODUCTION
Recent advances in FPGA technology have made it possible to apply them to TDCs in which the FPGA is embedded(called FPGA-based TDCs) in high-energy particle and nuclear physics [1] and also in materials science [2]. FPGAbased TDCs are very attractive because of their high performance, high-speed data transfer, and customizability andflexibility in precision timing measurements without any
external complex hardware. These excellent features enable high-precision time-duration measurements down toa few picoseconds (see references in [1]). These features
also enable the possibility of new applications to precisely
monitor various timing pulses with a wide dynamic range
in large accelerator complexes, where an accuracy of a few
nanoseconds is suﬃcient.
We have developed a new FPGA-based TDC required for
application to the KEKB injector linac. The required specifications of the TDCs are a wide dynamic range greater than
20 ms and an accuracy of 1 nanosecond level, where lowcost fabrication should be also required. The basic design,development, and experimental results of the new FPGAbased TDC for applications to large accelerator complexes
are reported in detail.

by-pulse modulation scheme at 50 Hz, which allows the injector linac to perform virtually simultaneous injections [4].
For the virtually simultaneous injections, a new trigger timing distribution system is under development on the basisof the event-based timing and control system [5]. A new
additional system is also under development for monitoring
all timing pulses generated in the trigger timing distribution
system as precisely as possible [6]. The purpose of the introduction of a timing monitoring system is to increase thereliability of the event-based timing and control system. Because the injection beam charges need to be increased to befour to five times greater than those of the previous KEKB,
such a timing monitoring system may serve not only the stable operation and complex simultaneous injection but alsoreliable radiation safety and machine protection of the injector linac.
The fiducial repetition frequency of the injector linac isbased on 50 Hz. All pulsed devices are driven by the trigger
timing pulses appropriately delayed from the fiducial timing. These trigger timing pulses are fundamentally generated with a frequency of 50 Hz at maximum by obeying thebeam injection timing sequences programmed in the control
system. However, in a strict sense, the fiducial time duration
does not need to be determined exactly to be 20 ms owing
to the synchronization condition of the linac and ring RF
frequencies on the injection bucket position in the ring.
The allowable width of the fiducial time duration is restricted to the trigger timing condition for the modulatorsof the high-power klystrons in the injector linac because themodulators should be driven within 20 ± 1 ms to stabilize
the applied voltage and to suﬃciently allow for stable charging. This specification means that the allowable time duration of the fiducial timing should be monitored to maintainthem within 20 ± 1 ms.
The trigger timing for most of other pulsed devices is lessthan 1 ms delayed from the fiducial timing, and more especially, the trigger timing for the beam-position monitors
should be maintained at an accuracy of 1-ns level. Therefore, a resolution of 1 ns and a dynamic range greater than
20 ms are required for the time-duration measurements in
the TDC along with a higher accuracy.

Basic design
FPGA-BASED TDC
Required specifications
All downstream storage rings of the KEKB injector
linac [3] will be filled in top-up injections based on a pulse∗ tsuyoshi.suwada@kek.jp

The present event-based trigger timing distribution system was constructed in a VME-based system, and accordingly, it was necessary to construct new TDCs (6U
VME64x module) in the same VME-based system. The basic specifications required for the VME/FPGA-based TDC
are listed in Table 1.

Table 1:
Basic Specifications Designed for the
VME/FPGA-based TDC

Start pulse

T

Stop pulse
Tclk

Basic parameter

Value

Number of common starts
Number of stops
Number of multistops
Number of bits
Dynamic range (max.)
Resolution
Clock frequency
Gigabit Ethernet

1
16
4
32
4.3
1
250
available

Units

CLK1 (φ = 0)
T0
CLK2 (φ = π/2)
CLK3 (φ = π)
CLK4 (φ = 3π/2)

s
ns
MHz

Interpolator 1

Interpolator 2

Figure 1: Developed VME/FPGA-based TDC module.

Basic principle for the time-duration measurement
A fundamental principle of the TDC implementation is
to count the number of the reference clock pulses generated
during the time duration (T) between the leading edges ofthe start and stop pulses with a coarse counter, as shown infig. 2.

∆T2
NTclk

Coarse Counter

The TDC has a common start and sixteen stop input channels. The input voltage level is based on a Nuclear Instrumentation Module (NIM) standard with a 50-Ω input
impedance. The fiducial pulse is fed into the common start
input channel, and the delayed trigger pulses to the pulsed
devices are fed into the stop input channels. Each delay
time from the fiducial pulse and the time duration between
the fiducial pulses are synchronously measured with 32-bit
counters with a resolution of 1 ns. A wide dynamic range of
4.3 s can be implemented at maximum on a Xilinx Spartan6 (XC6SLX75) FPGA [7]. A multistop function in eachstop input channel is available by which the delay timingsof successive trigger pulses (up to four at maximum) during one fiducial period generated on one trigger line can besynchronously measured. This is an important specification
required for two-bunch acceleration in an RF pulse delivered from the A1 electron gun, in which the time duration
between two bunches is exactly 96.3 ns [5]. The content of
the buﬀer data in the FPGA can be accessed via an A32/D32
VME64x interface. The developed TDC module is shownin fig. 1.

∆T1

0

1

2

N-1

N

Figure 2: Timing diagram of the fundamental time-duration
measurement of the TDC based on the multisampling technique.
In such implementation schemes, the time resolution of
the TDC is limited to Tcl k at maximum, where Tcl k is thetime duration for a period of the reference clock when thestart and stop pulses generated in the trigger timing distribution system are not synchronized at all with the reference
clock of the TDC. There are several smart methods (sometimes called interpolation methods [8]) to improve the time
resolution.
The time duration to be measured may be expressed asT=

NTcl k + (∆T1 + ∆Tϕ1 ) − (∆T2 + ∆Tϕ2 ),(1)

where ∆T1 (∆T2 ) is the fractional time duration measured
between the leading edge of the start (stop) pulse and thatof the next nearest pulse of the valid reference clock (CLK4(CLK2) in this case), and ∆Tϕ1 (∆Tϕ2 ) is the phase delay
time of the valid reference clock with respect to the mainreference clock (CLK1) (∆Tϕ1 = Tcl k /4 (∆Tϕ2 = 3Tcl k /4)
in this case), although most of the time duration may bemeasured as NTcl k synchronously with the main reference
clock (see fig. 2). If the fractional time durations can bemeasured with a higher resolution on the basis of any interpolation method, the time resolution of the TDC may becomprehensively improved.
The multisampling (sometimes called 4×oversampling)technique [1] is one of many smart interpolation methods. In the multisampling technique, four similar reference
clocks are generated in the FPGA with a period of Tcl k , forwhich each relative phase diﬀerence is π/2, which corresponds to a delay time of Tcl k /4, shown as CLK1–CLK4(see fig. 2). The fractional time, ∆T1 (∆T2 ), for the start(stop) pulse can be measured by detecting the leading edgeof the temporally closest reference clock by a first (second) interpolator with a four-times improved resolution of
T0 = Tcl k /4, whereas most of the time duration (NTcl k ) canbe measured with a coarse counter driven by the main reference clock. The total time duration can be simply calculated
based on eq. (1) by decoding and adding these results witha resolution of T0 .

Table 2: Specifications of the External Clock

Circuit architecture
Here, the generation of the reference clocks along with
their relationships used in the TDC is described in detail.
A schematic block diagram of the circuit architecture of the
TDC along with a block diagram of the phase-locked loop
(PLL) are shown in fig. 3.

Parameter

Value

Te Range

Frequency
Stability
Property

50 MHz< ±1.0 × 10 −6
25 ± 2o C−6o−1< ±0.28 × 10
C
– 40 to 85o C

Spartan-6 FPGAPLL Primitive

External CLK
PFD
50 MHz

CP

LF

VCO

125 MHz

÷8
1 GHz

200 MHz÷5

250 MHz (CLK1) φ = 0

÷450 MHz÷ 20

250 MHz (CLK2) φ = π/2

÷4

250 MHz (CLK3) φ = π

÷4250 MHz

250-MHz three-phase CLKs
16 channels

17 channels (1 Start + 16 Stops)

Start

SET

3bit

4ns Coarse
counter

Common Input

D

CLR

SET

D

CLR

SET

D

CLR

SET

D

CLR

SET

D

LevelConverter
Converter
Level
Converter
Level
Level
Converter

CLR

Level
Converter
Level
Converter
VLTTL
FastNIM
Level
Converter
VLTTL
FastNIM
Level
Converter
VLTTL
FastNIMVLTTLFastNIMVLTTLFastNIMVLTTLFastNIMVLTTLFastNIM
LVTTL
←
NIM

Start time
SET

16 Stops
Decrementer

Stop-Start

16ch×3bit
1 input/channel

Edge detect&
Encode

D

CLR

SET

SET

D

CLR

D

SET

D

CLR

CLR

SET

D

CLR

SET

SET

D

CLR

D

SET

D

CLR

CLR

SET

D

Start
Stops

CLR

SET

D

CLR

1ns Stop time
SET

Stop register file

D

CLR

SET

CLR

D

SET

D

CLR

SET

CLR

D

SET

D

CLR

250 MHzTCP
Clock Interface FIFO
Stop register file

SiTCP Registers

RBCP

VME Interface Control

PHY

SiTCP Core

RJ45
Connector

125 MHz
200 MHzVME Interface

Figure 3: Schematic block diagram of the circuit architecture of the developed TDC. The meanings of the labels aredetailed in text.
Three reference clocks, CLK1, CLK2 (delayed by
Tcl k /4), and CLK3 (delayed by Tcl k /2), with each clock
frequency of 250 MHz (Tcl k = 4 ns) and a phase diﬀerence of π/2 are generated via a voltage-controlled oscillator (VCO) with a frequency of 1 GHz in the PLL primitive,
where the frequency of the reference clock is divided byfour through a frequency demultiplier. Two other frequencies, 200 MHz and 125 MHz, for the SiTCP and a physical layer transmitter and receiver of Ethernet (PHY) are
generated by frequency-demultiplying the VCO frequency
by five and eight, respectively. A frequency of 50 MHz is
generated by frequency-demultiplying the VCO frequency
by twenty, and it is directly phase-locked with the external clock via a standard digital PLL with a bandwidth of
4 MHz, which comprises a phase frequency detector (PFD),a charge pump (CP), and a loop filter (LF). Here, it is important to choose an external clock from a viewpoint of frequency stability because it dominantly influences the characteristics and performance of the TDC in terms of the frequency stability, jitter, skew, and temperature eﬀects. An
external 50-MHz temperature compensated crystal oscillator (TCXO, EPSON TG-5501CA [9]) has been selectedduring TDC development because its frequency stability is
guaranteed within ±1 ppm in a temperature (Te ) range of– 40 to 85o C, for which the frequency stability is substantially an order smaller than that of standard crystal oscillators. The basic specifications of the external clock are listed
in Table 2.
The fractional time durations (∆T1 and ∆T2 ) are measured
in the 1-ns interpolator as shown in fig. 3. In the 1-ns interpolator, the start and stop pulses of the input signal arebasically sampled and latched at respective D input via a

flip-flop array with the use of both the leading and trailingedges of the two reference clocks with a phase diﬀerence of
π/2 (CLK2 and CLK3, called sampling clocks), while another reference clock (CLK1, called system clock) is used
as a sampling clock for the 4-ns coarse counter, and however, it works as the system clock in the 1-ns interpolator.
In such a sampling scheme, the fractional time durations are
determined by data sampled at the first row (4 flip-flops) of
the flip-flop array. Thus, this sampling scheme is equivalent
to that with the use of four diﬀerent reference clocks witha phase diﬀerence of π/2 as shown in fig. 2. It should be
noted, however, that the system clock (CLK1) is not used at
all for determining the fractional time durations in the 1-ns
interpolator, while it is used only for determining the coarse
time duration (NTcl k ). This scheme is of advantage to saveclock resources in the FPGA as much as possible and it isalso due to increase the stability and reliability of the sampling clocks as much as possible by separating them fromthe system clock. A subsequent 4 × 4 flip-flop array is required for the two sampling clocks to be synchronized withthe system clock domain and also to avoid any metastablestates. Thus, the input signal is sampled five times eachclock period in total.
In the decision logic (shown as “Edge detect & Encode”),
the timing measurement result is converted to a 2-bit information along with an additional bit, which is used for an
acknowledgement of both the valid edge detection and thevalid phase with respect to the system clock. Then, a 3bit and sixteen 3-bit information for the start and sixteen
stop pulses, respectively, are sent to another logic where
the measurement results are decoded. The fractional time
duration measured in the 1-ns interpolator is decoded with
its sign in a register (shown as “Stop-Start”). On the other
hand, the coarse time duration is decoded in the 4-ns coarse
counter. Both the coarse time and fractional time durations are simply calculated in a register (shown as “Decrementer”) with taking into account the signs of the fractionaltime durations based on eq. (1). The final result is stored ina register (shown as “1-ns Stop time”). Finally, the samedata are moved to another register (shown as “Stop registerfile”) that can be accessed via a VME interface. It shouldbe mentioned that in this decoding scheme the fiducial duration itself can be easily measured on the basis of the nominal start-stop implementation scheme even without any additional stop pulse, in which the next nearest start pulse isapplied to the stop pulse.

EXPERIMENTAL TESTS
Characteristic nonlinearity measurements
The characteristic nonlinearity measurements were carried out with a high-precision rubidium time-based delay generator (SRS, Model DG645) for investigating thesesystematic-error factors. Because the timing jitter of this
delay generator is 10 ps/s (rms), the measurement errors expected from the delay generator are negligibly small. The
start and stop pulses were generated with the delay generator, and they were fed into the TDC in an atmospheric environment. The time duration between the leading edges ofthe start and stop pulses was measured by changing the delay time to be set, where for simplicity, the asynchronous
measurements were performed using the delay generator todrive the TDC. The results are shown in fig. 4.

Figure 5: Variations in the accuracy as a function of the set
delay time. The solid line is an exponential function fitted
to the data.
a given dynamic range; that is, the measurement errors are
dominated by statistical errors. If in such measurement conditions the decimal part (c) of the ratio (T/T0 ) is given by
c = Frc[T/T0 ],(2)

the measurement precision (σT ) is expressed by the following equation [8]:√σT = T0 c(1 − c).
(3)

Figure 4: Variations in the time diﬀerence between the measured and set delay time with a dynamic range of 20 ms.
Each solid point represents the average value over 5000
measurements and only the statistical error bars are plotted.
The solid lines show a guide for the eyes only in a straightline fitting.
Figure 4 shows the variations in the time diﬀerence between the measured and set delay time with a dynamic rangeof 20 ms, which is the required dynamic range. The nonlinear point of the time diﬀerence curve is expected to be∼3 ms as shown in fig. 4, and the absolute time diﬀerence
at 20 ms is 2.6 ns. An 1-ns accuracy is attainable within
the region less than 7.5 ms, and the quantization errors are
dominant in this region.
The accuracy is generally given by the absolute time difference divided by the set delay time. The results are shownin fig. 5 and show that the accuracy reaches a saturation
point at which the set delay time is Tset ∼3 ms, where theattained accuracy is 0.13 ppm, which may originate fromthe frequency stability of the external clock.

Precision and integral nonlinearity
The precision of the TDC is generally defined by a standard deviation of repetitive measurements of the time duration in which the quantization errors are dominant [10]if the systematic nonlinearity is suﬃciently smaller within

The obtained precisions depending on the set delay time areshown in fig. 6(a).
Here, the solid line shows calculations based on eq. (3),where the average values are applied to the parameter (T).
The variations in the precision measurement are in good
agreement with calculations. The obtained result meansthat the precision is mainly caused by the quantization errors within a dynamic range of 20 ms. The maximum (minimum) diﬀerential is 0.52 (0.13) ns. This result means thatthe least significant bit (LSB) due to quantization errors corresponds to 0.52 bits at maximum. The variations in thediﬀerential time diﬀerence between the measured and set
delay time with a straight-line fitting within the same dynamic range are shown in fig. 6(b).
The integral nonlinearity is defined by the maximum difference within a given dynamic range. The maximum integral nonlinearity is 0.39 ns at the set delay time of 20 ns.
This result means LSB = 0.39 bits. Thus, the precision in
the time-duration measurement may be expected to be lessthan 1 ns in total within a dynamic range of 20 ms. The obtained result is suﬃciently acceptable for the trigger timing
distribution system.

Temperature stability
The temperature stability of the TDC was measured
within a temperature region of 5–35o C by changing theenvironmental temperature, where the TDC along with a
VME crate without any CPUs were placed in a thermoregulated bath while the delay generator was outside. The results are shown in fig. 7.

Figure 6: Variations (a) in the precision measurement depending on the set delay time within a dynamic range of
20 ms and (b) in the diﬀerential time diﬀerence between
the measured and set delay time from a straight-line fitting
within the same dynamic range.
Figure 7(a) shows the variations in the diﬀerence (∆⟨T⟩)
between the measured time duration and the set delay time
as a function of temperature, and fig. 7(b) shows the diﬀerential coeﬃcient with temperature as a function of the set
delay time. In fig. 7(a), it should be mentioned that the data
points are shown as the diﬀerentials from the reference datameasured at 25o C (room temperature).
The variations in the temperature stability cannot be expressed by any simple monotonic functions because thetemperature eﬀect is automatically compensated by feedback controls embedded in the external clock, and theircharacteristic feedback patterns may originate from its intrinsic temperature compensation scheme. The maximum
measured time duration is |∆⟨T⟩| ∼1 ns at 20o C and a set delay time of 19.5 ms. The variations in the diﬀerential coefficient (CT [ns/o C]) with temperature at 25o C are shown infig. 7(b). The magnitude of the coeﬃcient starts to slightly
increase at a set delay time greater than 2 ms. The obtained
result is suﬃciently acceptable for the trigger timing distribution system.

CONCLUSIONS
We have successfully fabricated and tested a new
VME/FPGA-based TDC with a wide dynamic range greater

Figure 7: Variations (a) in the diﬀerence between the measured time duration and the set delay as a function of temperature, and (b) in the diﬀerential coeﬃcient with temperature as a function of the set delay time. The lines indicate a
guide for the eyes only in (a) and a second-order polynomial
fit to the experimental data in (b).
than 20 ms and a resolution of 1 ns. The required specifications of the TDC were realized on the basis of the suitable
design with a high-precision temperature-compensated external clock with an accuracy of 0.13 ppm. The results are
fully suﬃcient for monitoring the trigger timing distribution
system of the injector linac. The developed TDC could beapplied to further complex timing systems in large accelerator complexes.

REFERENCES[1] A. Balla, et al., Nucl. Instr. and Meth. A 739 (2014) 75.
[2] S. Kishimoto, et al., Nucl. Instr. and Meth. A731 (2013) 53.
[3] M. Akemoto, et al., Prog. Theor. Exp. Phys. (2013) 03A002.
[4] K. Furukawa, et al., ICALEPCS’13, San Francisco, CA,
USA, 2013, pp. 1497 – 1500.
[5] K. Furukawa, et al., PAC’09, Vancouver, British Columbia,
Canada, 2009, pp. 4797 – 4799.
[6] T. Suwada, et al., Nucl. Instr. and Meth. A786 (2014) 83.
[7] Xilinx Inc., http://www.xilinx.com/.
[8] J. Kalisz, Metrologia 41 (2014) 17.
[9] SEIKO EPSON Corp., http://www.epson.com/.
[10] L. Zaworski, et al., Metrol. Meas. XIX (2012) 115.