IBIC2023, Saskatoon, Canada
JACoW Publishing
ISSN: 2 6 7 3 - 5 3 5 0
d o i : 1 0 . 1 8 4 2 9 / J A C o W - I B I C 2 0 2 3 - T H 1 I 0 1

LCLS-II TIMING SYSTEM AND
SYNCHRONOUS BUNCH DATA ACQUISITION∗
C.B. Mattison, K. Kim, P. Krejcik, M. Weaver, M. Zelazny
SLAC National Laboratory, CA USA
Abstract
The new timing system for the LCLS-II SC linac and FEL
meets the challenging requirements for delivering multiple
interleaved timing patterns to several different destinations
at rates up to 1 MHz. The timing patterns also carry information on bunch charge and beam energy to prevent inadvertent
selection of beam dumps beyond their rated beam power.
Beamline instruments are equipped with a timing receiverthat performs bunch-by-bunch synchronous data acquisition
based on the timing pattern for that location. Data is buffered
in on-board memory for up to 106 machine pulses (1 s at1 MHz). The large data volume can be locally processed
and analysed before transmission to clients on the network.
Commissioning and experience with the new system will bepresented

INTRODUCTION
The LCLS-II Timing System developed at SLAC is thefirst timing system able to provide programmable timing
patterns with variable rates from 1 Hz to 1 MHz with interleaved bunches to multiple destinations. The LCLS-II
Timing System provides the triggers for the timing receivers
along the 4 km beamline that are used to trigger hardware
pulsed devices (kickers, Klystrons) and diagnostic devices(BPMS, cameras, wire scanners etc) as well as providing
soft triggers for the data acquisition.
The LCLS-I and LCLS-II timing systems are physicallydecoupled: two separate dedicated fiber networks. The Timing pattern is generated upstream in the LINAC with a Timing Pattern Generator (TPG) and is then distributed to theTiming Pattern Receivers (TPR) at the device level. In shared
areas, between LCLS-I and LCLS-II, the TPRs are capable
to select the timing source to trigger on.
The SLAC accelerator complex provides multiple sourcesand destinations options to the machine operators.
The complexity of programming the LCLS-II timing pattern is ultimately resolved in the Timing Pattern Generator
Graphical User Interface (TPGGUI). The LCLS-II projectis currently under commissioning and it is the scope of thispaper to describe the components that made the LCLS-II
Timing System successful as well as discuss the commissioning experience with this new cutting-edge system.

TIMING PATTERN GENERATOR
The LCLS-II Timing System is composed of the mastersource, phase reference distribution, timing pattern generator, and timing pattern receiver units.
∗ Work supported by the U.S. Department of Energy under contract number

DE-AC02-76SF00515.

The LCLS-II master source generates the 1300 MHz and185.7 MHz (1300 MHz/7) phase reference signals with small
jitters, phase locked together. The transmission of the1300 MHz phase reference to LLRF has tight requirementson the stability and jitter of the timing signal over a large
distance.
The LCLS-II Requirements parameters are listed in Table 1 below:
Table 1: LCLS-II Timing System Parameters
Timing Attribute

Value

Phase Reference (Linac RF)
1300 MHz
Clock (Gun RF = Linac RF/7)
185.71 MHz
Nominal Beam Rate (Clock/200)
0.92857 MHz
Fiducial Resync Freq (Clock/2600)
0.07143 MHz
Fiducial (Power Line Phase)360 HzStability - Standard (fiber)
0.4 nsStability – Phase reference line (PRL) 1 fs/sec, 1 ps/day
Jitter – Standard (fiber)30 ps
Jitter – PRL (50 Hz to 5 kHz)0.005° or 10.7 fs
The Timing Pattern Generator (TPG) runs on an FPGA in
an ATCA crate. The TPG publishes the timing pattern overa dedicated fiber network. A timing pattern is a collectionof timing frames and contains unique information about abeam pulse (Table 2). The timing frame is time stamped by
the TPG and tagged with a unique pulse ID for post-dataacquisition processing, allowing cross-correlation of signals
between multiple devices.
The timing pattern frame in Fig. 1 shows how the timing
pattern frame is encapsulated within control characters thatallow additional data streams to be serialized onto the link.
This configuration allows control transmissions of differing domains that firmware outside of that domain are notrequired to parse. The entire 1 MHz frame is validated by
32-bit CRC calculation [1].

Figure 1: Timing pattern frame (visual).
The timing frame is manipulated by the firmware andsoftware. The software, hosted on the EPICS control system,
can send information to the firmware by programming thesequence engines, which can then update the pulse timing
frame content providing useful information such as the beam
destination.
TH1I01

09 Data Acquisition and Processing Platforms

453

Content from this work may be used under the terms of the CC-BY-4.0 licence (© 2023). Any distribution of this work must maintain attribution to the author(s), title of the work, publisher, and DOI

12th Int. Beam Instrum. Conf.
ISBN: 9 7 8 - 3 - 9 5 4 5 0 - 2 3 6 - 3

12th Int. Beam Instrum. Conf.
ISBN: 9 7 8 - 3 - 9 5 4 5 0 - 2 3 6 - 3

IBIC2023, Saskatoon, Canada
JACoW Publishing
ISSN: 2 6 7 3 - 5 3 5 0d o i : 1 0 . 1 8 4 2 9 / J A C o W - I B I C 2 0 2 3 - T H 1 I 0 1

Content from this work may be used under the terms of the CC-BY-4.0 licence (© 2023). Any distribution of this work must maintain attribution to the author(s), title of the work, publisher, and DOI

Table 2: Timing Pattern Frame
Field

Size Description

Pulse ID

64

Unique, monotonic., increments
at base rate

Time Stamp64

Time since 1990 epoch., increments at programmed step size

Fixed Rates

10Fixed rate markers 0-9; one bitfor eachACRates

6Power line synchronized markers
0-5, one bit for eachTime Slot

3360 Hz time 1-6 persistent. Computed from TS1 input

Beam Request

1

Beam is requested from theinjector

Destination

4Beam destinations

Charge Inj.

16 Bunch charge

Beam Energy

4×16 Beam energy at 4 locations

BSA Control

4×64

For each buffer, initialize, average, acquire, finalize

ControlSeq[0:17] 288

16b control step data for each of
18 sequences+ others
The main sequence engines used for generating the timing
patterns in the firmware were developed at SLAC:• Beam-Engines: used to schedule the beam patterns torequest beam to a specific destination published on thetiming frame.
• Allow-Engines: hosts the allowed sequences given thepattern running on the beam engine. And through theAllow-Table, which selects the appropriate allowed sequence to run, provides the handshake between Machine Protection System (MPS)/Timing.
• Experimental/Control Engines: Manipulates the tail,288 control bits, of the timing frame to support multidestination beam, beam synchronous acquisitions, experiment requirements.

The AC Rate Compatibility Requirement
The Event Generator, used on the LCLS-I accelerator
operates at a maximum frequency of 120 Hz.
The LCLS-II project had to embrace the challenge to increase the trigger rate by up to 104 times higher frequency
but still provide the capability to provide AC rates synchronized to the power line. This requirement is essential forsome of the pulsed-power devices like Klystrons (XTCAVand STCAV). The triggering system for these devices needs
to be synchronized to the AC Power line phase zero-crossings(timeslots) up to 120 Hz in order to minimize power supply
ripple and jitter to the beam.

TIMING FIBER DISTRIBUTION
The LCLS-II Timing System hardware is completely decoupled from LCLS-I Timing System. The timing PatternGenerator (TPG) sends Timing Frames to each Timing Pattern Receiver (TPR) over a dedicated fiber network at 1 MHz.
The timing fiber distribution starts in Sector 2 where theTPG is located and is patched through passive and active
fanouts. The active fanouts, developed at SLAC, provide
the required drift compensation to resolve the challenge ofsignal stability over this long fiber network. The timing fiber
distribution extends through the 4 km accelerator complex
which is subject to large variations in temperature (Fig. 2).

Figure 2: SLAC Accelerator Complex in meters.
The Active Fanouts are placed at three strategic locations
along the linac to provide drift compensation. The first
Active Fanout is in Sector 2 where it receives a signal directlyfrom the TPG and distributes it to nearby passive fanouts
plus the next Active Fanout in Sector 20. The Active Fanout
in Sector 20 is like the one in Sector 2 where it feeds nearby
passive fanouts plus the next Active Fanout in B005 (Fig. 3).
The passive fanouts distribute the timing pattern to the finalreceivers.

The Fixed Rate Markers
The Fixed Rate markers are set by the firmware and are
the base rate reference used to define all beam patterns.
The Fixed Rate markers are computed by dividing the185 MHz reference clock by 1, 13, 91, 910, 9 100, 91 000,910 000, LSB to MSB.
The frames are then tagged accordingly to the appropriate
spacing between fixed rate markers, but all timing frames
are tagged as 1 MHz frames. This results in overlap of all
the markers at 1 Hz.

Figure 3: Timing fiber distribution.
Figure 4 shows the signal drift on 6 channels measured atthe Sector 20 Active Fanout.

TH1I01454

09 Data Acquisition and Processing Platforms

IBIC2023, Saskatoon, Canada
JACoW Publishing
ISSN: 2 6 7 3 - 5 3 5 0
d o i : 1 0 . 1 8 4 2 9 / J A C o W - I B I C 2 0 2 3 - T H 1 I 0 1
The patterns that satisfy these requirements are definedas Standard Patterns and are listed in Fig. 5.

Figure 4: Signal drift on 6 channels measured at the Sector 20 Active Fanout. On the y axes the signal drift (1 ns)
over time (about 12 h).

BEAM SYNCHRONOUS ACQUISITION
LCLS-II is a single-pass machine which requires that diagnostic devices be able to read out the same pulse throughout
the machine on a bunch-by-bunch basis. The Superconducting Beam Synchronous Acquisition (SC BSA) provides databuffers time stamped by the SC Timing System (TimingPattern Generator – TPG ). An acquisition can be set withdata filtering to satisfy user needs for each beam destination. The data is released in data buffers containing up to20,000 data points. Each data point can be averaged overup to 8191 beam pulses. This feature is very useful foracquisition running at a very high rate such as 1 MHz.
There are five regions, or beam destinations, of interestsin the LINAC: the diagnostic line (SC_DIAG0), the Beam
Switch Yard Dump (SC_BSYD), the Hard X-ray beam line(SC_HXR), the Soft X-ray Line (SC_SXR) and the LINAC
area beyond the diagnostic line. For each of these regions
the system provides permanent, continuous data acquisitionat the following rates:• 1 Hz, for data archive,• 10 Hz, for accelerator control room displays,• 100 Hz, for physics applications.
This gives a total of 15 continuously running BSA buffers.
In addition, there are 29 user-configurable BSA buffers andfour, 1 second deep, fault buffers for the Machine Protection
System (MPS).
The LCLS-II Timing System also provides the Beam Synchronous Scaler Service (BSSS) which provides real-time
pulse-by-pulse scaler data at rates up to 100 Hz for control
room displays.

COMMISSIONING BEAM PATTERNSThe LCLS-II timing system is very flexible and can be
programmed with a wide variety of bunch patterns. However, for the initial commissioning, we required a set of preconfigured patterns that would allow us to commission eachof the subsystems such as the BPMs, beam loss monitors,MPS, LLRF (Low-Level RF), and other diagnostics.

Figure 5: Standard Patterns for Commissioning.
The standard timing patterns are available for the four SC
final destinations: SC_DIAG0, SC_BSYD, SC_HXR andSC_SXR. The beamline map (Fig. 6) below shows in blue
the diagnostic line, used for emittance studies; in green thepath to the BSYD dump; in yellow to the Soft X-ray LineDump, and in Green to the Hard X-ray Line Dump.

Figure 6: Beamline paths and destinations for LCLS-II.
Depending on the machine mode, the operator can request
beam pulses going to different destinations and some of them
are compatible with each other. The compatibility of these
destinations is described by a lookup table of the machinemodes.
The standard patterns can then be combined to generate
Complex Patterns allowing multiple destinations request
beam at the same time. The Complex Patterns are distinguished in:• Pulse Stealing: all pulses with beam land on the Fixed
Rate Markers distributed as part of the timing pattern.
In case of multiple destination requesting beam, destination priority will determine beam pulses assignment.
• Interleaved Patterns: each destination is assigned anoffset from the closest fixed rate marker to the requested
beam rate.
• Bunch Train: user defined interval between beam patterns

Interleaved Patterns
Interleaved Patterns are patterns where the beam is requested for two destinations, but its request is placed on a
fixed offset in respect to a rate marker coming out of thetiming pattern. The offset is unique to that destination toensure no overlap.
TH1I01

09 Data Acquisition and Processing Platforms

455

Content from this work may be used under the terms of the CC-BY-4.0 licence (© 2023). Any distribution of this work must maintain attribution to the author(s), title of the work, publisher, and DOI

12th Int. Beam Instrum. Conf.
ISBN: 9 7 8 - 3 - 9 5 4 5 0 - 2 3 6 - 3

Content from this work may be used under the terms of the CC-BY-4.0 licence (© 2023). Any distribution of this work must maintain attribution to the author(s), title of the work, publisher, and DOI

12th Int. Beam Instrum. Conf.
ISBN: 9 7 8 - 3 - 9 5 4 5 0 - 2 3 6 - 3

IBIC2023, Saskatoon, Canada
JACoW Publishing
ISSN: 2 6 7 3 - 5 3 5 0d o i : 1 0 . 1 8 4 2 9 / J A C o W - I B I C 2 0 2 3 - T H 1 I 0 1

The subharmonic frequency that allows the two LINAC(LCLS-I and LCLS-II) to be phase-locked together is
71.5 kHz. Between two fixed rate pulses at 71.5 kHz there
are 13 empty pulses. The offset of empty pulses from aFixed Rate marker, per destination, is then selected to be
one of the prime numbers <13, such as: 3 empty pulses for
SC_DIAG0, 5 for SC_HXR and 7 for SC_SXR. Following
this logic, we are then able to schedule independent beam
requests for multiple destinations.

will actually result in 90 Hz to DIAG0 and 10 Hz to BSYD
(Fig. 9). This example shows that in pulse stealing mode any

TPG Graphical User Interface
The Timing Pattern Generator Graphical User Interface(TPGGUI) is python-based code that allows expert operatorsand physicists to program new timing patterns to satisfy
additional pattern requirements. The interface provides apreview of possible conflict between destinations requesting
the same beam pulse. The conflicts are highlighted by ayellow triangle (Fig. 7).

Figure 9: Beam pulses collected in the SC_DIAG0 and
SC_BSYD areas corresponding to the pattern shown in
Fig. 8.
beam pulse overlap in beam request is “stolen” by the higher
priority destination, in this case SC_BSYD. It is noted thatthe beam pulses are equally spaced along the pattern.

SC_DIAG0 and SC_BSYD in Interleaved ModeIn this second example shown in Fig. 10, the requested
rate is the same as in the previous example but the beampulses for SC_DIAG0 are now requested with an offset of
3 pulses from the Fixed Rate Marker at 100 Hz. Now thescheduled pattern does produce a 100 Hz beam as requested
to DIAG0 and 10 Hz beam to BSYD with the difference
that the interval between beam pulses is no longer equallyspaced (Fig. 11). However, during commissioning, this did
not create any problems for the average RF load.
Figure 7: TPGGUI showing conflict on beam pulse requested by two destinations on the same bucket.

COMMISSIONING EXPERIENCE
SC_DIAG0 and SC_BSYD in Pulse Stealing ModeIn this commissioning example shown in Fig. 8, a requested pattern of 100 Hz to DIAG0 and 10 Hz to BSYD

Figure 10: TPGGUI showing beam request with offset pulses
between SC_DIAG0 and SC_BSYD.

SC_HXR and SC_SXR in Interleaved Mode

Figure 8: TPGGUI showing conflict on beam pulse requested by SC_DIAG0 and SC_BSYD in pulse stealing
mode.
TH1I01456

In the third example shown in Fig. 12, the requested rateis 1 Hz to SC_HXR and 1 Hz to SC_SXR. This succeeds
because the beam requested to SC_HXR is offset by 5 pulses
from the 1 Hz rate marker (Fig. 13). In the situation where anoffset is not selected for one of the two destinations, then only
the higher priority destination would receive beam pulses.
All of these offsets and rules need to be communicated in
the timing pattern in order to trigger devices at rates lower
09 Data Acquisition and Processing Platforms

IBIC2023, Saskatoon, Canada
JACoW Publishing
ISSN: 2 6 7 3 - 5 3 5 0
d o i : 1 0 . 1 8 4 2 9 / J A C o W - I B I C 2 0 2 3 - T H 1 I 0 1

Figure 11: Beam pulses collected in the SC_DIAG0 and
SC_BSYD areas corresponding to the pattern shown inFig. 10.

Figure 12: TPGGUI showing beam request on offset pulses
between SC_HXR and SC_SXR.
than beam rate as well. For this reason, selecting a fixedoffset per destination increases the stability and reliabilityof the pattern generation.

CONCLUSION
During LCLS-II Project commissioning we were able
to satisfy all operation requirements and provide patternsflexible enough to run beam to multiple destinations at thesame time. The interleaved pattern concept of dedicatedoffset associated with a destination allows a more stableconfiguration for also triggering devices and data acquisitionat rates lower rate than the beam rate.

Figure 13: Beam pulses collected in the SC_HXR and
SC_SXR areas at 1 Hz corresponding to the pattern shownin Fig. 12.

ACKNOWLEDGEMENTS
Acknowledgement to the timing team, physicists, operation and machine protection system team for all the support
and the great teamwork. The LCLS-II timing system isthe product of the great collaboration between those threegroups. Work supported by the U.S. Department of Energy
under contract number DE-AC02-76SF00515.

REFERENCES[1] LCLS-II Timing System Engineering Specifications,
https://docs.slac.stanford.edu/sites/
pub/Publications/LCLS-II_Timing_System_Engineering_Specifications.pdf

TH1I01
09 Data Acquisition and Processing Platforms

457

Content from this work may be used under the terms of the CC-BY-4.0 licence (© 2023). Any distribution of this work must maintain attribution to the author(s), title of the work, publisher, and DOI

12th Int. Beam Instrum. Conf.
ISBN: 9 7 8 - 3 - 9 5 4 5 0 - 2 3 6 - 3