Using Reconfigurable Processors as Control System Elements
Timo Korhonen, Adrian Lawrence*, Ian Page*, Junji Urakawa
KEK, High Energy Accelerator Research Organization
Oho 1-1, Tsukuba, Ibaraki 305, Japan*
Oxford University, Computing Laboratory
Parks Road, Oxford, OX1 3QD, United Kingdom
Abstract

mover tables, 18 in each arc section. The tables are positioned in 10 degree angles relative to each other.

A control system applying reconfigurable computers has
been designed and built for the active alignment system ofATF at KEK. The system measures the relative position ofaccelerator components with a high precision allowing
remote correction of the misalignments. A control systemthat acquires the data and permits the position control wasdeveloped. The system was built using reconfigurable
logic as processing elements. To program the devices, ahardware compiler was utilized. The system structure,divided into functional levels and the tasks andprocessings done at each level is reported.

In a circular accelerator it is not practical to use aglobal reference, so a method that depends only on a few
reference points is required. To measure the relative position of the tables in ATF, diode lasers used as lightsources are mounted on the end of one mover table anddetectors to measure the position of the light spot on theend of the neighbour table. Using a combination of twodimensional position detectors separated by a suitabledistance we can measure position in five degrees of freedom. The position information is shared between the tables through the control system.

1 Introduction

3 Control system

The ATF (Accelerator Test Facility) at KEK has been
built with the purpose of studying the techniques andphenomena related to building a linear collider. The
principal components of ATF are a 1.54 GeV injector
linac and a 1.54 GeV damping ring. One of the primarygoals of the accelerator is producing ultra-low emittancebeams in the Damping Ring (DR). To achieve this goal,
the accuracy of the alignment has to be in the range oftens of microns [1]. An active alignment system, with thepurpose of keeping the components of the damping ring
constantly aligned by compensating for the floor motion,
has been built for the ATF.
The control system for the alignment system utilizes
reconfigurable processors; logic devices that can bedynamically programmed for specific tasks. The key
techonology that makes the programmable logic feasible
as processing elements is hardware compilation. We use
the Handel compiler [2], developed at Oxford, to program
the logic arrays. The compiler makes the programming
very much simpler than using contemporary tools, as thehardware details need not to be worried about. The
computing circuit is generated from source code so thecomputing can be performed very fast.

A data acquisition and control system is required forhandling the measurement data and controlling the movers. The system has to be connected to the main control
system of the ATF so that the information can be sharedwith other components of the control system.
CAMAC [5] is an extremely robust and well established standard and still in wide use. However, the standard is showing its age. It is quite hard to include newfeatures like intelligence at the low level and distributedfeatures. The ATF control system is based on CAMAC fora variety of reasons. A major part of the required hardware components and some basic software for the control
system were readily available.
Using the CAMAC system only, all the I/O operations
would have to go through the central computer. It seems
reasonable to have an intermediate layer to ease the burden of the central control system. To build such a system
there are several possible approaches. The most obvious
one would be to use some kind of fieldbus interface to theCAMAC system and microprocessors to drive the localcontrollers. This kind of solution would have the flexibility that is needed. However, it is not so easy to find aninterface that could be readily integrated into our system.

2 The active alignment system
The ATF damping ring has a racetrack shape. The accelerator components in the circular arc sections areplaced on controllable tables (movers). The tables can be
precisely controlled in five degrees of freedom: horizontal
and vertical, roll, pitch and yaw. There are a total of 36

3.1 Hardware/Software codesign
The performance of programmable circuits is increasing
rapidly. The RAM-based field-programmable gate arrays(FPGA) can be dynamically reconfigured, which makes
them ideal for custom computing devices.
The logic design for the programmable devices can bedone either using schematic capture or a behavioural description like VHDL. A more recent approach is to use ahardware compiler to generate the circuit from a programsource [2]. The programming language approach has numerous benefits: it enables the use of an algorithmic approach even in the lowest-level tasks. The borderline between software and hardware is moved all the way down
to the digital interface. No detailed knowledge about thehardware is necessary to use the circuits.
In the ATF system the Handel language [2] is used. The
design of the Handel language is based on a soundmathematical theory of Communicating Sequential Processes [3]. Handel semantics makes it possible to follow andcontrol the temporal behaviour of the hardware. The
programmer can rely on the hardware implementation towork as written in the software and even verify that thebehaviour is correct.
When programs run directly on hardware the algorithms
do not need any interpreting level and can be very fast. It
is possible for example to integrate a signal processing
algorithm directly into the low-level hardware, or embed
sensor data validation into the driving circuit itself [4]. A
general overview of hardware compilation in sensor applications is given in [7].
4 Control system architecture
The control system has to bind together a mix of sensor
and control hardware and integrate them into an existing
control system framework. Reconfigurable computers areused to glue the components together and to bringcomputing power to where it is needed.
To achieve the proper task distribution, a three-layer
structure was adopted (see figure 1). Each layer isprovided with the required kind of computing capability.
The lowest level has to have abundant I/O but thecomputation is fairly straightforward; the middle levelneeds no direct IO but has to have network connections
and a moderate capability to do floating-pointcalculations. The top level is concerned with thealignment and monitoring and does not need to know
about the processing done in the lower levels. The top
level uses the facilities of the global ATF control system;
the implementation of the lower levels is totallytransparent to this layer.
4.1 Hardware interface layer
The lowest level communicates with the sensor andcontrol hardware and processes requests from the upperlayer. The tasks are performed by FPGA-based processorsthat are programmed in Handel. The controller for one
mover table reads in the data from three analog-to-digitalconverters (ADCs), relays communication data to andfrom the mover controller through an RS-232C serial port
and handles a set of digital I/O ports (figure 3). The communication link is used to transfer the data to upper level.
For the processing, the Accelerator Card from Embedded
Solutions Ltd. is used [11]. The card has no traditional
processors but instead two FPGAs (Xilinx 3195A). One
FPGA (master) is connected to a communication link

driver that enables remote configuration and communication using the board. The second FPGA (slave) has 22 of
its pins available for external input/output. The twoFPGAs are connected with a data bus for communication
between them. External circuits are driven through theFPGA and normally do not need any intermediate logic.
This makes the hardware very simple; the number ofcomponents is reduced to a minimum.
Top-level control-database-user interface-interface to other components

Local control-data collection-coordinate conversions-message routing-automatic feedback (optional)

Hardware interface-ADC readout-RS232 port handling-Signal processing (noise filtering etc.)

Fig. 1 Control system architecture.
The software is simplified because there is no need toworry about complexities like handling interrupts, multiple processes and similar things that would be required in
the case of using a regular CPU. The simplicity makes it
easy to guarantee the robustness of the processors, as allof their behavior is contained within the Handel-C code.

FPGA #1(master)
Xilinx
3195A

FPGA #2(slave)
Xilinx
3195A

Input&output
pins

communication link
Fig. 2 The Accelerator Card block diagram.
The software of the low-level processor nodes responds
to requests from the upper level to access the hardware,for example to read the value of one ADC channel. Oneprocessor unit handles several kinds of hardware and therequests and data have to be addressed to the right unit.
This can be achieved by employing a simple protocol. TheADC′s are driven through a serial interface to minimize

the required pin count. After receiving a request to readan ADC channel, the software has to drive the serial interface. For RS-232C communication and digital I/O there
are also corresponding subroutines that are called in reply
to the incoming requests.
RS-232C
interfaceController

DigitalI/O

ADC #1

mon memory and then informs the processor about thecommand by sending it an event signal.
To gather the data into the common memory, the T805
CPU sends data request packets on the local control
network. Each processor node on the network has a
private address. Using the address information on thereceived packets the root CPU can order the data into thememory block.

AD789112-bit, 8-ch

Main control(CAMAC)

ADC #2Detectors
ADC readout

Detectors
ADC readout

Localcontroller

Localcontroller

Mover control
RS-232C

Mover control
RS-232C

AD789112-bit, 8-ch

ADC #3AD789112-bit, 8-ch

Fig. 4 (Local) control of the Active alignment.

Fig. 3 Hardware level processor functions.
There is still room for addition of more elaborate tasks.
For example, employing a constant monitoring of the lightintensity could be used to implement an interlock system
to detect too large movements of the tables that could
cause harm to the vacuum system.

Localcontrol
network

T805

DualportedRAM3195AFPGACAMAC

ATF(global)controlBUS

Bootstrap logic

4.2 Local control network
At the next level the hardware is bound into a local
control network that can be handled from a single
CAMAC connection. All the commands and data arerouted through this network. The data from the lower
level is gathered and the necessary normalizations andcorrections are applied to convert the raw values into
physical units of displacement.
The SMT-219 board [8] is used as the main processorof the local control. It includes a T805 transputer processor, a Xilinx 3195A FPGA and 4 MB of RAM that isshared by the T805 and the FPGA. The T805 has four 1.5
MB/second serial communication links that are used toconnect the subprocessors.
The SMT-219 is inserted in a module and the CAMAC
signals are connected to the FPGA. This module is usedas the interface between the CAMAC bus and the local
control network. The module is very simple, consisting
only of buffer circuits and a simple digital logic to enablebootloading the SMT219 through the CAMAC bus. Thelogic is active after a power failure or a hardware resetand is disabled after the FPGA has been configured.
The FPGA is programmed to run the CAMAC protocol.
It can access the common memory using direct memory
access (DMA). The T805 gathers the data from the network into a memory block which is accessible through theCAMAC bus. For the commands to be sent to the processor network, the FPGA stores the command into the com-Fig. 5 Block diagram of the CAMAC interface
4.3 Top level control
The top level controls one component of the ATF control system rather than a separate subsystem. Its purpose
is to integrate the active alignment with the other components of the ATF.
The ATF main control system [12] is built using theVsystem [9] control software tools. To make sharing ofdata between the subsystems possible, the data is kept inVsystem real-time databases. The data from the alignment
subsystem is also stored in a database to make the data
sharing simple.
The real-time databases hold the current values of data,
whereas the data from the measurement system is inherently time series data. The database has to be periodicallyaccessed and the data saved to create a time series file.
This can be done by either a special purpose program orusing the Vlogger tool. The time series data can then beaccessed with other tools and programs.
5 Conclusions
Reconfigurable processors have been applied to theontrol of the active alignment system of the ATF damping
ring. Use of the reconfigurable processors made it easy tomatch the computing tasks with the functional division of

the system. All the low-level control tasks can be accomplished by the use of a single module. With the proper
software, this module can take over tasks that used toneed the development of additional hardware for eachtask. Handling of ADCs, serial communication, digital IO
and so on can be done in a single circuit. Adding functions
or changing the interface can be done instantly, even
without stopping the operation of the rest of the system.

ActiveAlignment
Controlsubsystem

devicehandler

Real-time
database

Control/
analysisprogramControl/
analysisprogramControl/
analysisprogramFigure 6. Top level control block diagram
The use of reconfigurable computers allows us to use
the concept of ′virtual hardware′. By analogy with virtual
memory on microprocessors, it is possible to download
new hardware configurations at run-time, thus changing
system behaviour. This may be useful if it is found thatdifferent control regimes suit different operating conditions that might arise during operation.
In a large-scale system the cost advantage of reducing
the number of modules and module types to stock could
be large. The risk of some components like a CPU design
becoming obsolete could be avoided. The work done on
software can be protected; even if the detector hardware ischanged, it can be made to look similar to the software. At
the moment, the relatively high cost of FPGAs however
offsets the cost advantage. However as FPGA prices are
dropping rapidly, this is fast becoming a serious alternative.
For low-level control systems that interact with thehardware, combination of reconfigurable computers with afieldbus standard will undoubtedly become a reality andwill be useful for many control tasks.
For our application, the main advantage of this approach lies in its flexibility: new technology can be incorporated while retaining the compatibility with the presentsystem.
Acknowledgments
The authors wish to acknowledge the continuous encouragement of Professors Y. Kimura and K. Takata. andthe whole ATF group.

The movers and the position measurement system areessential components of the active alignment system.
However, the space does not allow them to be presented
here and here we just want to acknowledge the efforts ofthe people that were involved in the development. Theseparts will be reported separately.
Parts of the hardware were developed in the Computing
Laboratory, in collaboration with the Control EngineeringGroup in the Department of Engineering Science, University of Oxford. The Accelerator Card was reengineered and commercialized for our project by Embedded Solutions, Ltd. Their efforts and contributions aregreatly appreciated.
References[1] Hinode, F., Kawabata, S., Matsumoto, H., Oide, K.
Takata, K., Takeda, S. and Urakawa, J. (Ed.): ATF
Design and Study Report. KEK Report 1995-4.
[2] Hoare, C.A.R. (1986) Communicating SequentialProcesses. J.Wiley & Sons, 1986.
[3] Hoare, C.A.R. and Page, I. (1994). Hardware andSoftware: the closing gap. Transputer Communications 2(2) , 69-90.
[4] Henry, M. et. al. (1996) Programmable Hardware
architectures for Sensor Validation. Control Engineering Practice 4(10), 1339-1354.
[5] IEEE (1976) CAMAC Intrumentation and Interface
Standards, IEEE Document No. SH06437. WileyInterscience.
[6] Page, I., Luk, W. (1991) Compiling occam into FieldProgrammable Gate Arrays. I. Page and W. Luk.
FPGAs, pages 217-283. W. Moore and W. Luk (eds.)
Abingdon EE&CS Books, Oxford.
[7] Page, I. (1997) Hardware Compilation, Configurable
Platforms and ASICs for Self-Validating Sensors.
Proceedings of FPL97. Ed. W.Luk, P.Y.K. Cheung,M.Glesner, Springer Verlag 1997.
[8] Sundance Multiprocessor Systems can be reached at
http://www.sundance.com/[9] Vista (1997): V-System is a product of Vista Control
Systems, Inc. Information about the V-system isavailable on: http://www.vista-control.com/[10] Xilinx (1996): Xilinx Programmable Logic Handbook. Xilinx, Inc.
[11] ESL (1997) Embedded Solutions, Ltd. can bereached at http://www.embedded-solutions.co.uk/[12] Terunuma, H. Hayano, T. Korhonen, K. Kubo, T.
Naito (1997). The status of the control system of theKEK accelerator test facility. (in these proceedings).