Proceedings of IBIC2012, Tsukuba, Japan

TUPA37FPGA BASED FAST ORBIT FEEDBACK SYSTEM FOR
THE AUSTRALIAN SYNCHROTRON
Y-R. E. Tan, T. Cornall, S. A. Griffiths, S. Murphy, E. Vettoor,
Australian Synchrotron, Clayton, Australia

The ASLS is a 3 GeV light source open to users since2007. Although not implemented at the start, the plan wasto implement the fast global orbit feedback system at alater stage when the need arose. The beam stability
requirement is that the RMS of the beam motion be <10% of one sigma of the transverse beamsize, ı.
The RMS of the beam motion is different depending on
the location around the ring and ranges between 0.6 ȝmup to 3.5 ȝm. To identify the frequencies that contributeto the overall motion, Figure 1 shows the integrated
spectrum of the beam motion collected during regular
operations from the storage ring BPM electronics,
Instrumentation Technologies’ Libera Electron (Libera),
at a sampling rate of 10 kHz. The largest contributor to
the beam motion is at the mains frequency at 50 Hz andits harmonics with mechanical vibrations contributing to
the noise below 100 Hz. The recent improvements to thebeamlines have meant that some are now seeing theeffects of the 50 Hz beam motion.
To improve the beam stability a fast global orbit
feedback system was proposed. Fortunately there aremany existing solutions to learn from; CLS [1], Elettra[2], SLS [3], Soleil [4], SPEAR III [5], TLS [6] andNSLS II [7]. One requirement desired for the design was
to have a single centralised processing station. The
number of BPMs in the feedback (98 in total) wassufficiently small to make this possible. This has thepotential to reduce hardware costs and simplify the designof the system where synchronisation of multiple
processing stations is not needed.

30201000100200300
Frequency (Hz)40050021.510.500Centre of straight
10% of the beam size
100200300
Frequency (Hz)400500Figure 1: Integrated spectrum of the beam motion at ID5(IVU). The vertical beamsize is based on the model withthe current emittance coupling value of 1%.
The proposed layout of feedback system is shown in
Figure 2 where the outer fibre optic ring is used toaggregate the position data from all the BPMs and theinner ring is used to distribute the corrections to thecorrector power supply units (PSUs). A 1000 Base-T GbE
connection links the outer ring with the inner ring throughthe central processor.

Figure 2: Proposed layout of the feedback system forming
two fibre optic rings (2 Gbps). The FPGAs sendcorrections to the corrector PSUs via serial lines. ThePSU, Liberas and FPGAs have network connections tothe control system.

ISBN 978-3-95450-119-9
Feedbacks and Beam Stability

1c 2012 by the respective authors — cc Creative Commons Attribution 3.0 (CC BY 3.0)
Copyright ○INTRODUCTION

402.5
Vertical int. motion (μm)

An initial design for a Fast Global Orbit Feedback
System based on FPGAs has been proposed for theAustralian Synchrotron Light Source (ASLS). The designuses a central processor (Xilinx Virtex 6) for all thecomputations and fast optical connections to distribute thecomputed data to corrector magnet power supplies. The
network topology consists of two fibre optic rings. The
first ring is used by the Libera Electron's to aggregate thebeam position data at 10 kHz using Instrumentation
Technologies' Grouping algorithm. The second ring isused to transmit the computed data. The cycle frequencyof the feedback is 10 kHz with a targeted total latency ofunder 400 us. We shall give an overview of the design
goals and discuss the merits of the current
implementation. We shall also present the measured
bandwidth of the stainless steel vacuum chamber and test
results from initial prototyping work.

Horizontal int. motion (μm)

Abstract

TUPA37

Proceedings of IBIC2012, Tsukuba, JapanThe design of the system is broken into three parts thatshall be discussed in the following sections: position
aggregation, feedback processing and the corrector
magnets.

c 2012 by the respective authors — cc Creative Commons Attribution 3.0 (CC BY 3.0)
Copyright ○POSITION AGGREGATIONWith the Liberas there were two options for theaggregation of position measurements at 98 BPMs at arate of 10 kHz. The first is the communication controller(CC) developed at DLS [8] and Libera grouping [9]. The
choice was made to use the Libera grouping because thecombination of the Gigabit Ethernet and UDP/IP
protocols opens the possibility of choosing many different
platforms for feedback processing. I-Tech wascommissioned to extend the grouping size from 64 to 128
Liberas. Each Libera sends adds 128 bits to the data
packet with a 336 bit header giving a total of 12880 bitsin one UDP packet. By grouping all BPMs into a single
data packet, much like the CC, the synchronisation andaggregation are handled transparently by the Liberas andsimplifies the design of the central processor. The
expected contribution to the total latency of the feedback
system is approximately 67 ȝs for 98 BPMs, including thepacket encoding and transmission time. This system hasbeen commissioned and has been operational for fewmonths.
Two Liberas are currently used to stream the position
data on a dedicated virtual local area network (VLAN) tothe prototype feedback processor and to a PC in thecontrol room respectively. The PC in the control roomruns the fast acquisition archiver [10] extended to accept
the UDP packets. The archiver is used to access the fastacquisition data and is used as a beam stability monitor.

FEEDBACK PROCESSING
An FPGA solution was chosen to do the feedback
processing because it was the most suitable as a robust
real-time platform to perform the calculations. Other realtime systems such as RTEMS and RTAI [11] were also
considered. However the FPGA platform was favoured
because of its processing potential and within the facility
it was a platform that we wanted more experience indeveloping.
The position data grouped by the Liberas are sent to a
prototype feedback processor via Gigiabit Ethernet. The
processor is an FPGA development board by Hitech
Global (V6 PCIE) with a Xilinx Virtex-6. Here a statemachine hardware design in the FPGA parses each packetto extract status and position information from each BPM.
A processor-less design in Verilog HDL (hardwaredescription language) approach was taken because it
offers the advantage of being able to run the calculations(and other functions) in parallel hardware without thelimitations imposed by the inherently sequential operationof a microprocessor.
As each BPM’s unique identifier, i, and X and Y beam
position information is extracted from the Ethernet
packet, it is plugged into a running sum type calculation,for each unique corrector, n, using the following
algorithm:
for(n=0;n<N_CORRECTORS;n=n+1) begincorrector[n] <= corrector[n] +coeff_X[n][i] * libera_x_pos[i] +coeff_Y[n][i] * libera_y_pos[i];endwhere coeff_X and coeff_Y are the coefficients of theinverted BPM-Corrector response matrix. It should be
noted that the Libera data in the packet doesn’t arrive in a
nicely ordered sequence according to its unique ID, i.
The values for coeff_X and coeff_Y are selected out ofblock RAMs on the basis of the i. These RAM blocks
have to be arranged, one per corrector, so that they can beaccessed simultaneously for all of the correctors. This isbecause the for-loop is flattened during logic synthesis
and the calculations for all the correctors run in parallel.
This means that when the last set of X and Y positions forthe last Libera become available, the matrix calculation iscomplete, with very little delay. Care must be taken tothen check that the Ethernet packet was valid and not tosend the calculations on if it wasn’t.
The communications system chosen to distribute thecalculated corrector values to the FPGA devices
controlling the corrector magnets is the bi-directional
Xilinx Aurora 8B10B protocol (the 8B10B corresponds tothe line coding used) which is a high-speed serial
connection that can be simply implemented into a FPGA
design from a supplied HDL module using the Xilinx
synthesis tools. It is simple to integrate, involving nothingmuch more than writing the values to be sent into a fixed
length register and setting a transmit bit at the transmit
end, and at the receive end, waiting for a receive bit to gotrue then reading the data out of a register. The physicalmedia chosen is fibre-optic with a line rate of 2.5Gbps.
The Virtex-6 FPGA we are using has multiple GTXs (fastserial transceivers) available to use and the development
board (also Hitech Global V6 PCIEs) connects two ofthese GTXs to SFP (small format pluggable) adapters that
can take a fibre-optic hardware module to supply thephysical interface to the fibre-optic cable pairs used. We
need two SFPs on each of the corrector magnet
controllers so we can implement a daisy-chain network.
Each corrector controller takes the relevant corrector
values from the data arriving by the first SFP port andpassing the rest of the corrector values down the line on
the second SFP port. An addressing scheme will be
implemented to identify which values correspond towhich corrector.

CORRECTOR MAGNET AND PSU
The corrector magnets in the SR are trim coils on thesextupole magnets. These are powered by Danfysik 7000
90/120 Amp bipolar power supplies with a measured
response of < 100 Hz. Any modification to the current
PSU was deemed too much of a risk to operations. So thepossibility of adding a trim power supply in parallel to theISBN 978-3-95450-119-9
2Feedbacks and Beam Stability

Proceedings of IBIC2012, Tsukuba, JapanGU
SfP r P0

(1)

were į is the skin depth, ȡ the resistivity, f the frequency,
ȝr the relative permeability and ȝ0 the permeability of freespace. At one skin depth the metal’s conductivity is
reduced to 36.8% and at two skin depths its 13.5%. To
calculate the transmission of the AC magnetic field
penetrating the vacuum chamber the following equation isused,Aexp(L / G )(2)

where A is the fraction of the magnetic field transmittedand L the vacuum chamber thickness. For a 3 mm thickstainless steel plate. It is anticipated that the field willattenuate by 71% at ~2 kHz for an infinite plate model.
To measure the response of the vacuum chamber aDanfisik PSUs was replaced with 100W bipolar Kepco
power supply with a maximum current of 5 A. The Kepco
was controlled by an external voltage from a frequency
generator. The Kepco’s output was configured to supply asinusoidal current with a peak of 230 mA withfrequencies ranging from 1 Hz to 1600 Hz. The
perturbation to the beam was measured by analysing 10
kHz position data from a single BPM in the SR. The
amplitude of the frequency peak at the drive frequency isplotted in Figure 3. The results show that the oscillation
amplitude, and indirectly the magnetic field, decreases by
71% at 350 Hz (horizontal) and 1100 Hz (vertical). The
bandwidth is wide enough given the frequencies of theperturbations are < 500 Hz.

Magnet InductanceDuring the measurement of the response of the vacuum
chamber the peak voltage on the Kepco was alsomeasured. With increasing frequency there was a linear
increase in the peak voltage due to the inductance of themagnet. Using V = LIȦ and the measured data, theinductance was calculated to be 1.7±0.1 mH (horizontal)
and 0.8±0.1 mH (vertical). This compares reasonably wellwith theoretical calculations of 1.96 mH (horizontal) and

15
10500500

1000
Frequency (Hz)

1500

500

1000
Frequency (Hz)

1500

15
10500Figure 3: Amplitude of the peak at the drive frequency
when a corrector magnet is driven by a sinusoidal current
with a peak current of 230 mA. The vertical magnetic
fields that drive the horizontal motion attenuates fasterthan the horizontal magnetic field due to the rectangular
geometry of the vacuum chamber. The multiple lines arefrom different data sets. The variation between data sets
may be due to the instability of the Kepco driving aninductive load.

Magnet Strength Requirement
The largest perturbation is measured at 50 Hz and therequired correction is 2.6 ȝrad. For perturbations induced
by insertion device (ID) motion there are feedforward
tables for the local ID correctors. However during largemovements there will be uncorrected perturbations. Theseare expected to be < 1 ȝrad because the largest rate ofchange among all the local ID correctors is 6 ȝrad/s. As
for slow drifts, over a 2 hour period, the maximum
change in the correctors (slow orbit feedback currently
being used) is at most 4 ȝrad (in both decay and top-up
operation). Therefore the fast corrector will be designed
to provide a maximum correction of 10 ȝrad. For a 10
turn secondary corrector coil around the sextupole thiswould require a current of 0.8 A.

Fast Corrector Trim Coils on Sextupoles
There was concern that the slow and fast PSU would
couple given transformer like arrangement on thesextupole. A test with a custom built 1A low power
amplifier indeed showed a significant coupling between
the slow and fast PSUs. The solution is to ensure that themagnetic field created by the slow correctors areperpendicular to the fields created by the fast correctors.
In this setup the coupling between the PSUs was notdetected. In the SR there are a total of three slow
ISBN 978-3-95450-119-9

Feedbacks and Beam Stability

3c 2012 by the respective authors — cc Creative Commons Attribution 3.0 (CC BY 3.0)
Copyright ○At the ASLS the vacuum chambers are made from 3mm thick stainless steel. The effective attenuation of thefields from the stainless steel chamber was calculatedusing the skin depth equation given by

Horizontal Amplitude (um)Vacuum Chamber Bandwidth

0.65 mH (vertical). The low inductance value reduces thevoltage, and power, requirement of the power supply.

Vertical Amplitude (um)main corrector PSU was also considered. This however
would still involve modifications to the main supply todisable the feedback loops to let the trim supply “takeover”. This was also considered unacceptable. Therefore
the decision was made to separate the fast and slow PSUs
and add secondary corrector coils on the sextupoles forthe fast correctors. To ensure that this can be done anumber of tests were carried out.

TUPA37

TUPA37

Proceedings of IBIC2012, Tsukuba, Japan

horizontal and four slow vertical correctors. Using
additional coils there can be 4 fast horizontal and 3 fastvertical correctors.
To decrease the installation time a multi-core cablewith connectors on either end will be designed such thatwhen the two connectors are joined the inner wires of themulti-core cable will form a multi-turn coil.

Fast Corrector PSU
Given that the requirement for the corrector is only 0.8
A and low voltage requirement because of the low
inductances, the power supply would be quite straightforward to design. A PSU was built to test the coils andthe exercise was useful in determining the cost and effort
required to build all 98 supplies including the interfaces.
There are also commercial sources such Bilt system [12]that is used by Soleil and ESRF for their feedback systemthat is perfectly suited for this purpose.

c 2012 by the respective authors — cc Creative Commons Attribution 3.0 (CC BY 3.0)
Copyright ○CONCLUSION
Thus the feedback system will consist of:x Libera grouping providing synchronised position
data at a rate of 10 kHz via GbE.
x Position data is processed in an FPGA (currentlyonly a matrix multiply algorithm) ready to be
distributed to “slave” FPGAs connected to power
supplies.
x The power supplies will need to be able to drive aminimum of 1 A into a magnet with an inductance of~2 mH.
x The magnet will be secondary corrector coils woundaround the sextupole magnets with a minimum of 10turns.

One of the defining parameters for the bandwidth of thesystem is the total latency of the system. This is expected
to be very similar to Soleil’s calculations [4] which isestimated to be 360 ȝs.

REFERENCES[1] M. G. Abbott et al., EPAC, THPC118, pp3257-3259,
2008. M. T. Heron et al., ICALEPCS, THB003,pp650-652, 2009.
[2] M. Lonza et al., PAC, MOPAN024, pp203-205,2007.
[3] V. Schlott et al., Proceedings of BIW, 2004.
[4] N. Hubert et al., DIPAC, POT019, pp187-189, 2005.
N. Hubert et al., DIPAC, TUPC20, pp189-191, 2007.
N. Hubert et al., BIW, WECOTC02, pp337-341,
2008. N. Hubert et. al, DIPAC, MOOC01, pp27-31,2009.
[5] A. Terebilo and T. Straumann, EPAC, THPCH102,pp3035-3037, 2006.
[6] C. H. Kuo et al., DIPAC, WEPC17, pp346-348,
2007.
[7] Y. Tian and L. H. Yu, PAC, WEODN4, pp1-3, 2011.
[8] I. S. Uzun, CTRL-FOFB-CC-0001, 2009.
[9] A. Bardorfer and T. Karcnik, EPAC, pp1041-1043,
2008.
[10] M. G. Abbott et al., ICALEPCS, THCHMUST03,pp1244-1247, 2011.
[11] E. Bianchi and L. Dozio, “Some experience in fast
hard real-time control in user space with RTAILXRT” Real Time Linux Workshop, 2000.
[12] http://www.bilt-system.com/appli_acc.html(25/09/2012)

ISBN 978-3-95450-119-9
4Feedbacks and Beam Stability