The Relativistic Heavy Ion Collider Control System∗
T. S. Clifford, D. S. Barton and B. R. Oerter
Brookhaven National Laboratory
Box 5000, Upton, NY 11973-5000
Abstract
The Relativistic Heavy Ion Collider control system has
been used in the commissioning of the AGS to RHIC
transfer line and in the first RHIC sextant test. Much ofthe controls infrastructure for networks and links has been
installed throughout the collider. Almost all of thecontrols hardware modules needed for early RHIC
operations have been designed and tested. Many of these
VME modules are already being used in normal AGS
operations. Over 150 VME based front end computersand device controllers will be installed by the Summer of1998 in order to be ready for Fall of 1998. A few features
are being added to the front end computer core software.
The bulk of the Accelerator Device Objects (ADOs) which
are instantiated in the FECs, have been written and tested
in the early commissioning. A configuration database has
been designed. Generic control and display of ADO
parameters via a spreadsheet like program on the console
level computers was provided early on in the control
system development. User interface tools that weredeveloped for the AGS control system have been used inRHIC applications.
Some of the basic operations
programs, like alarm display and save/restore, that areused in the AGS operations have been or will be expanded
to support RHIC operations. A model for application
programs which involves a console level manager
servicing ADOs have been verified with a few RHIC
applications. More applications need to be written for theFall of 1998 commissioning effort. A sequencer for
automatic control of the fill is being written with theexpectation that it will be useful in early commissioning.
1 RHICThe Relativistic Heavy Ion Collider (RHIC) [1], now in
its final year of construction at Brookhaven National
Laboratory, will provide counter-rotating particle beams in
two 3.8 km super-conducting rings which will be collided
in four crossing regions to provide interactions forexperimenters. Ions are first accelerated through three
existing Brookhaven accelerators, a Tandem Van deGraaff, a Booster, and the Alternate Gradient Synchrotron(AGS) before being transferred to RHIC where they areaccelerated and stored. Beam fill of the two rings willtake a about a minute with the stored beam expected tolast about ten hours.
∗There have been two successful tests of many of thecomponents of RHIC, including many aspects of thecontrol system. The most recent test, the Sextant Test, [2]finished in January of 1997, brought beam from the AGS
into one sextant of RHIC. Final installation of the collider
is going on now with cool down scheduled for late 1998
and first beam in the rings in January of 1999. Colliding
beams for experiments is expected to take place before theend of 1999.
2 RHIC Control System Overview
The RHIC control system [3] is the result of evolution
from the present AGS control system. VME chassis
distributed around the ring in about 35 equipment
locations contain i/o modules which either directly
connect to accelerator equipment or connect to field buses
which in turn connect to equipment. A single board
computer in the VME chassis provides a host for
Accelerator Device Objects (ADOs), software objects
which buffer data, control the I/O modules and passmessages over an Ethernet backbone to console levelcomputers. Synchronization of equipment is achieved
through serial links connected directly equipment.
The console level computers are UNIX-based
workstations which function as host for applications
programs. The operator interface for the applications usean X-Window graphical user interface allowing XTerminals to serve as windows on the control system.
Manager programs run on the console level computers toprovide hosts for higher level ADOs which can interface tothe front end ADOs.
The current AGS main control room will serve as thecontrol room for both AGS and RHIC operations. A sixth
general purpose console will be added to accommodate
more activity in the control room. RHIC and AGSapplications will be expected to run in a commonenvironment and be used by the same group of operators.
3 Hardware Infrastructure
The controls hardware infrastructure consists of datalinks, the Front End Computers (FECs), and hardware
modules in the FEC which interface either directly or
indirectly to accelerator equipment.
There are six different types of data links. The real time
data link (RTDL) [4] transmits up to 256 different tagged
24 bit data values at a 720 Hz rate. The RTDL system
will be used to deliver data, such as beam energy, magnet

Work performed under the auspices of the U.S. Department of Energycurrent, and corrections, to wave form generators and FEC
utility modules. RHIC will have its own event timingsystem[5] to synchronize the operation of distributedequipment. This system, similar to one in use at the AGS,
distributes a master clock and encoded events from amaster to all equipment locations. Many locally designed
VME modules have inputs for the event link system andthere is a utility delay module that can translate an event
into a standard TTL pulse. There will be a beam
synchronized event system, similar to the real time event
system, for each ring, that will deliver beam synchronized
events to instrumentation hardware through a general
purpose delay module. A beam permit module [6] with apermit link and two quench links, one for each superconducting ring, has been built. The beam permit module
accepts inputs from critical equipment enabling the permit
link only when all its inputs are satisfied and its upstream
permit link is present. The quench link for each ring isused signal the quench detection/protection modules todump the current in the super conducting magnets.
The current FEC SBC module is powered by a Motorola
68040 processor. We are evaluating processor modules
which contain the Power PC processor and expect to use
this newer generation processor in most RHIC FECs. The
current Ethernet configuration to the FECs is a 100 Mbit/s
fiber Ethernet from a hub to each of six equipmentbuildings where it is switched to 10 Mbit/s Ethernet for
distribution to about 30 other locations. This topology willallow for a 100 Mbit/s connection directly to FECs whererequired.
The bulk of the I/O channels in RHIC are for the controlof the magnet power supplies of which there are about1000.
The analog control, digital I/O and analog
measurements for the magnets are split among different
modules. A Wave Form Generator (WFG) board, withtwo channel digital serial fiber output, is used to generatereference voltage for power supplies. The reference
voltages are generated in real time using RTDL values,
events and locally stored tables. A multiplexed ADC(MADC) [7]system has been built with enough distributed
capacity to allow three measurement channels per power
supply to be continuously monitored at 60 Hz with
sufficient memory to save about 10 minutes of data incircular buffers. Both the wave form generators and theMADC system are already in use at the AGS. The lowlevel control and status for the power supplies is doneusing commercial PLC systems. These PLC communicate
over vendor's field bus to commercial VME modules in theFEC.
Commercial PLC systems are also in use in vacuum
controls for controlling valves. The vacuum gauges andpumps use an RS 485 bus for local communication. AnAllen-Bradley co-processor module buffers the commandsand readbacks for these devices. The connection of thissystem to the FECs where the vacuum ADOs reside is via
TCP/IP over an Ethernet port in the Allen-Bradley coprocessor.
The cryogenics control system is a commercial systemwith connection to the RHIC control system for logging
and monitoring via a shared file system.
The Beam Loss Monitor (BLM) system and the Beam
Position Monitor (BPM) system make up the bulk of theinstrumentation channels for the control system. MADC
modules will be used to digitize the loss monitors. Thelogic to detect beam losses, fast and slow, are done in acustom designed VME module, in parallel to the digitizingof the signals. These signals from the BLM system play arole in enabling the beam permit link to allow beam. Inthe BPM system digitization takes place in equipment inthe tunnel. Digital data is brought to the FECs using theIEEE 1394 bus. Commercial VME modules in the FEC
store the BPM data in memory where it is used by theBPM ADOs.
4 Software Infrastructure
The main software infrastructure in the FEC consists ofdrivers for VME modules in the FEC, support forAccelerator Device Objects (ADOs), an event system inthe FEC, the ADOs themselves, and a commun-ications
protocol that exports ADO methods across the network.
The Console Level Computer (CLC) infrastructure
includes ADO support, methods to set and get ADO
parameters across the network and an ADO development
environment.
UNIX style drivers (open, close, ioctl, read, write) which
run under vxWorks, have been written for the VME
modules used in the system. Drivers isolate the details of
the operation of the boards from the higher levels of thesystem.
An ADO is an instance of a C++ class which contain dataand methods for a group of parameters.
The most
important class methods for device control are the set andget methods. The set and get methods can use the device
drivers to access the underlying controls hardware. Tasks
in the FEC provide a context for executing ADO methods(set and get) in response to accelerator events and tonetwork messages. ADO parameters are the common andthe lowest level interface to the control system. ADO
classes have been written to support all the current
hardware.
The application program interface to ADOs on the CLCs
is through the ADO Interface class. The basic methods inthis class are the set and get method which are used to set
and get parameters from ADOs distributed over thenetwork. The class also contains methods to getparameters asynchronously.
Support for development is an important role played by
the CLCs, although not by the actual Console Computers.
Because the crafting of ADOs is such an important part of
the RHIC control system a preprocessor was written togenerate C++ code for the ADO classes from a compact
RHIC ADO definition file. The preprocessor takes care ofautomatically generating the house keeping andcommunications methods for the ADO allowing thedesigner to focus on the important set and get methodsand event code.
5 Utilities and Generic Tools
The RHIC control system makes use of tools that weredeveloped for the AGS. Because RHIC and AGS
operations will use the same control room, operator
consoles, and be run by one team of operators, it isimportant that RHIC and AGS control programs have thesame look and feel. To insure a common look and feel themature AGS GUI library, built on Motif, is used for alluser interfaces. A style guide, based on the GUI library, isprovided for programmers writing user interfaces.
Early on in the development of the RHIC control system
there was a need seen for a generic data driven application
which could be used for testing ADOs and in turn, theunderlying hardware. A Page Editing Tool (PET) with aspreadsheet like table interface was written, using the GUI
library, to support software developers, engineers andcommissioners. The PET program is now widely used formany aspects of development and for operation of someequipment.
A messaging system is in place in the FECs to generate
alarms in the AGS alarm system. A variety of usefulprograms and utilities, such as a program launcher, print
utility, crash logs, and monitors, developed over the years
for the AGS, will be used, as is, by the RHIC controlsystem. Work is currently underway to specify the RHIC
needs in the areas of logging, general data display andsave/restore. It is expected that existing AGS programs
can be modified to fulfill most new RHIC requirements.
Services have been written for both ADOs and AGS
Logical Devices so that the cdev[8] object can be used as acommon interface to the AGS and RHIC control systems.
There are a number of Sybase databases on the console
level database server which support the control systemfrom the device level up to the accelerator physics level.
All the FEC configuration information, including ADO
instances, is managed by the database.
6 Applications
About a dozen applications, most with physics content,
have been identified as required for the first year ofrunning. Only a few of these applications were in any way
tested in the Sextant Test. Most of the applications will bewritten by the RHIC Accelerator Physics staff over thenext year. After initial success in early tests with theADOs, it was recognized that console level ADOs would
be useful for some applications. Many of the application
designs incorporate a manager level program which runs
at the Console Level. The manager provides a context for
ADOs which are a higher level abstraction of accelerator
objects than the rather hardware orientated FEC ADOs.
Adding this manager level to applications allows the userinterface code to be separated from the accelerator physics
part of the application and provides a well definedinterface for programs that may need the higher level
functionality of the application contained in the manager.

The sequencer will use managers. Development ofmanager ADOs is supported by the same tools used tosupport front end ADOs.
In order to prevent unforeseen problems with controls
when the entire accelerator is being controlled, the RHIC
control system will be exercised (without beam) forcontrols
performancemeasurements,
scalingmeasurements and applications prototype testing
beginning in April of 1998.
7 Conclusion
Over the next year about 100 FECs with about 1,400
VME modules and tens of thousands of connections toaccelerator equipment must be installed and checked out
from controls programs. While this work is going on there
will still be development work to be done at various levelsof the control system. At the same time controls will haveto support various subsystems coming on line. Each of theindividual technical task are understood and well in handbut accomplishing of all the tasks that need to be done
over the next year will be a challenge.
The demands on a control system are very great at
commissioning time when workers are trying to get
subsystems to work reliably in a production setting. It will
be a considerable challenge to have an operational control
system ready to support the commissioning effort in 1999.
8 Acknowledgments
The authors would like to acknowledge all the staff,
present and past, of the RHIC/AGS controls section who
have contributed to the work on which this report is based.
9 References[1] S. Ozaki, Proc 1991 IEEE Part. Accel Conf., May 6-9,
1991, San Francisco, p 2901
[2] F. Pilat et. al., RHIC Sextant Test - Accelerator
Systems and Performance, Proc 1997 IEEE Part.
Accel Conf., May 12-16, 1997 Vancover, inpress[3] D. Barton, ICALEPICS '93, Berlin, Germany, 1993,
Nuci. Instr. and Meth. A 352 (1994) p7[4] H. Hartmann, RHIC Real Time Data Link, Proc 1997
IEEE Part. Accel Conf., May 12-16, 1997,
Vancover, in press[5] Charles R. Conkling Jr., , Proc 1995 IEEE Part. Accel
Conf., May 1-5, 1995, Dallas, p 2229
[6] Charles R. Conkling Jr., RHIC Beam Permit andQuench Detection Communications System,
Proc 1997 IEEE Part. Accel Conf., May 12-16,
1997, Vancover, in press[7] R. Michnoff, Proc 1995 IEEE Part. Accel Conf., May
1-5, 1995, Dallas, p 2229
[8] W. A. Watson III, A Portable Accelerator ControlToolkit, Proc 1997 IEEE Part. Accel Conf., May
12-16, 1997, Vancover, in press.