INTERFACING TO ACCELERATOR INSTRUMENTATION*T. J. Shea
Brookhaven National Laboratory
Upton, NY 11973
Abstract
As the sensory system for an accelerator, the beam instrumentation provides a tremendous amount of
diagnostic information. Access to this information can vary from periodic spot checks by operators to highbandwidth data acquisition during studies. In this paper, example applications will illustrate the requirements oninterfaces between the control system and the instrumentation hardware. A survey of the major accelerator
facilities will identify the most popular interface standards. The impact of developments such as isochronous
protocols and embedded digital signal processing will also be discussed.

I. INTRODUCTION
Survey of Instrumentation Groups
In September and October of 1995, instrumentation personnel representing 21 facilities and projects were
contacted via email and asked to answer several questions about instrumentation interface technologies. The
results are concisely summarized as follows:• Less than 10% have built a GPIB instrument• About 20% currently include VXI based instruments and nearly 50% plan to use VXI in the future• nearly 60% use LabVIEW for bench measurements• About 40% use portions of the control system during system development
In addition, their other comments were used to select the topics covered in this paper.

Accelerator Measurements
A very simple measurement setup is illustrated in Figure 1 with the accelerator as the device under test. The
stimulus might drive a broadband kicker, the quad bus, or a steering magnet. The response might be measured bya wide band position monitor, a tune meter, or an orbit monitor. In any case, a correlation plot is produced. Sincetime is the implicit free variable, the measurement equipment is almost always synchronized with the accelerator
timing system. If automatic control of the measured parameter is desirable, the measurement system could be
converted to a feedback system.
An accelerator instrumentation system provides the response measurement. Figure 2 illustrates the signal flow
through a generic instrumentation system. For continuous data streams like those produced by storage ring and
CW linac instrumentation, buffer size is determined by the worst-case latency of the interface. If this data is used
in a feedback loop, the performance of the loop is limited by the loop delay as determined by the total worst-caselatency. Therefore, the interface should provide deterministic latency as well as adequate throughput. An exampleof low latency interface is the VME-based reflective memory used for an orbit feedback test at SPEAR [1].
Although the sampling rate for this experiment was 37 Hz, the goal for a similar system at the Argonne APS isto use a 4 kHz sampling rate and provide a - 3 dB closed orbit correction bandwidth of 100 Hz [2]. Given thehigh performance processor and network hardware currently available, it is unfortunate that these systems muststill utilize a data path that is independent of the accelerator control system.
Referring again to Figure 2, a current trend is to reduce analog complexity and digitize closer to the source.
This can lead to fairly high raw data rates. If calculated parameters are desired, a local digital signal processor* Work supported by the U.S. Department of Energy

Figure 1.

Simplest stimulus-response measurementtransducer

analogprocessing

digitizer

digitalprocessing

calculated
resultbufferinterface

rawdatabufferFigure 2.

Generic signal flow diagram(DSP) can write them to the result buffer at a reduced rate. An example of this simple DSP application is the low
intensity position detector proposed for the fixed target areas at Fermilab [3]. Here, a 1.8 Msample/s 12 bit data
stream is digitally down converted and filtered to a much lower rate, higher resolution stream that can be readthrough the control system interface.

II. INSTRUMENTATION SYSTEM LIFECYCLE
Development
Throughout system development, bench measurements are made using commercial test and measurement
equipment. Software may be developed to fully exercise the system and obtain complete calibration data. This
single user, benchtop environment is well supported by the commercial test equipment and software vendors. The
accelerator control system is often immature during this phase and may lack the extensive instrument drivers,
data analysis tools and the rapid turnaround environment of commercial software packages like National
Instrument’s LabVIEW. Therefore, it is not surprising that much instrumentation is developed without control
system support and that system integration activities dominate a lengthy commissioning phase.
Even with the vast array of commercially available hardware, it is rarely possible to meet system performanceand cost goals with exclusively off-the-shelf purchases. This is particularly true with systems that have highchannel multiplicities. However, the in-house development timeline can be compressed by dividing a system into

concurrently developed modules. Obtaining a fine grained modularity through the use of mezzanine cards allows
the following benefits: chances are better that some modules might be reused in other systems, reduced module
complexity improves testability, off-the-shelf modules are less likely to include expensive unwanted features, andlate design changes will have a reduced schedule impact. In the Relativistic Heavy Ion Collider (RHIC) Beam
Instrumentation Section, Industry Packs are used to achieve this modularity [4][5].

Commissioning and machine studies
Due to the desire to constantly improve accelerator performance, commissioning and machine studies areongoing activities that share the following characteristics:• unusual beam parameters and acquisition scenarios• limited faith in system performance and reliability for these operating scenarios• large data sets
• flexible, rapid turnaround application development tools required• experts on hand to solve problems and provide support

Operations
During operations, required data rates are typically lower than those required by studies and some system
features may not be used. It is convenient to use parameters derived locally from the raw data rather than acquireand display large data sets. Built-in self test sequences can instill confidence in system performance without
requiring the additional test equipment that was used during the development phase.

UpgradesAs a result of operational experience, some systems may require improved performance. The upgrade of theLEP narrow-band orbit measurement electronics is just one example [6]. At the same facility, a new operating
mode using bunch trains impactson nearly all instrumentation systems [7]. The same modularity that wasbeneficial during the design phase can provide similar benefits in these situations.

III. STANDARDS FROM THE TEST AND MEASUREMENT INDUSTRY.
As opposed to the industrial instrumentation and controls products, products from test and measurementcompanies are most similar to accelerator instrumentation electronics. However, their target market is
predominantly benchtop test systems and not distributed instrumentation. The standards described in this section
are used extensively during the development phase of instrumentation systems. Commercial instruments based onthese standards also show up in the operational environment when their performance and flexibility justifies thetypically high capital cost.

GPIB
To this day, virtually all rack and stack instruments sport an IEEE 488 port. With it’s 8 data lines, 8 control
signals, and bulky connector, this port looks fairly archaic, and the following timeline shows the reason.
1965 - Hewlett Packard defines its HP-IB interface for instruments
1975 - IEEE 488 standard is ratified. Defines physical interface based on HP-IB
1987 - IEEE 488.1 is ratified [8]. Evolutionary step from 1975 standard. Commonly called GPIB.
1987 - IEEE 488.2 is ratified [9]. Defines higher level command structure. Revised in 1992.
Because the 1975 standard did not define status reporting, data formats and other software protocols,
manufacturers each invented their own implementations. The resulting mess is partially responsible for GPIB’s
poor reputation with system integrators. IEEE 488.2 defined the command structures that all instruments andcontrollers should support. In 1990, the Standard Commands for Programmable Instruments (SCPI) Consortium
defined an extensible set of instrument specific commands.

Although some instruments allow for binary data transfers, all of the standard commands are ASCII that mustbe parsed in real time. Thus, a one shot request for a small data block incurs significant latency and overhead.
The typical raw transfer rate for a GPIB connection is over 1 MByte/s for long blocks. The bus can have up to15 devices, each separated by about 1 meter with a total electrical bus length of 15 meters.
As an example of real world system performance, a recent test at Fermilab was performed to evaluate
oscilloscopes. The test consisted of digitizing a series of 8 bit samples at 2 Gsamples/s and reading them inthrough GPIB to a LabVIEW application running on a Macintosh Quadra. With the fastest scope, a 2000 point
series was digitized and transferred 10 times in 270 milliseconds. With a 100,000 point series, the same scope
digitized and transferred 100 times in 59 seconds.

VXIVME Extensions for Instrumentation (VXI) is now the card/backplane standard for the test and measurementindustry. The VME Handbook [10] provides a very readable overview complete with comparisons to VME. A
brief summary is included here.
There are four VXI card sizes, but by far the most popular is the C-size card (6U high by 340mm deep) so
this will be the assumed format in the following discussion. To provide room for electromagnetic shielding, thecards are spaced on 1.2 inch centers rather than the 0.8 inch centers defined by VME. P1 and P2 carry the bus
signals defined by revision C.1 of the VME specification. The outer two rows of P2 carry the following signals:• 10 MHz clock• Bussed TTL and ECL triggers• 12 pin local bus• Analog summing bus• Module Identification bus• Additional power distribution
The Module ID bus, along with configuration registers defined in A16 space, allow some level of system selfinitialization. Two broad classifications of VXI instruments can be defined by their adherence to VXI software
standards:
1. Message Based: Usually produced by a test and measurement company that ports its GPIB instruments to VXI. Typically they will use a word-serial protocol similar to GPIB SCPI commands.
The company’s customer base is usually downsizing a rack and stack test system and also increasing throughput. For example, converting an oscilloscope-based digitizing system to VXI will result in a 1 to 3 increase in channel density [11].
2. Register Based: Usually produced by a data acquisition company that moves functionality fromCAMAC, VME, etc., to the VXI format. Register-based devices keep the efficient memorymapped access to data and control/status registers. To an accelerator control system, VXI registerbased devices are accessed like any other memory-mapped VME card. Many modern control systems provide VME-based real time systems. In these cases, the VXI address space can be transparently bridged to the VME address space. At Brookhaven, the AGS to RHIC transfer line position monitor electronics reside in a VXI crate that is transparently bridged to the control system
VME crate [4].
Message based commercial equipment is not conveniently integrated with most control systems. The
messaging architecture of this equipment has a long history in the single user, bench-top test environment. Amodern control system provides its own messaging standard to solve the problem of distributed control overrelatively slow LANs for multiple users. With its large library of instrument drivers, LabVIEW has been used as
an integration tool at several laboratories. Two examples illustrate two different approaches. The Sampled Bunch
Display system at Fermilab acts as a local front end for the Tevatron control system [14]. The GUI provided byLabVIEW is not present on the control consoles. A small set of parameters is provided on the network anddisplayed in a laboratory standard parameter page.
A tune meter at the Berkeley ALS provides another integration example. Here, LabVIEW is again
communicating with a message-based single user instrument, but the interface to the control system happens atthe application layer. With this access to the control system on-line database, the technique could be extended toprovide a data acquisition system with the LabVIEW graphical interface and rapid turnaround development
environment [15]. A system with a similar architecture is in the prototype stage at RHIC and might be used for
instrumentation-related machine studies.

IV. ADDITIONAL STANDARDSThe test and measurement standards described above are currently in use at virtually every accelerator
facility. However, two points were mentioned earlier:
1. In the development and upgrade phases of an instrumentation system, fine-grained modularity (below the VXI module level) can increase testability and reduce costs in the event of a change.
Since Industry Packs are used for this purpose at several accelerator labs, the standard will besummarized here.
2. Register-based VXI cards can present an efficient memory-mapped interface to the control system. However, it is sometimes necessary to distribute the modules in a topology that does not allow cost effective use of VXI packaging. This could happen when the analog functions cannot fit
into the VXI format, or when performance could be increased by placing the digitizers close to
distributed signal sources. In either case, it is desirable to keep the efficient memory-mapped access even though the instrumentation modules are not confined to a physical backplane. A standard under consideration at RHIC is the IEEE P1394 Serial Bus [16].

Industry Pack (IP)A summary of features:• slightly larger than a business card
• 2 connectors for each single wide Industry Pack
• 50 pin user connector simply wired to a header• 50 pin logic connector memory-mapped to one port of the DSP• 16 bit synchronous logic interface for single wide packs• option of 32 bit interface for double wide packs• standard 8 MHz interface or preliminary 32 MHz interface• Many available commercially• very simple to develop in-house for special functions• dozens of competitors, but IP seems to have the broadest market penetration• Other market winner will surely be the PCI mezzanine standard, PMC. But it is a larger format
with demanding design requirements.
Real world performance with the RHIC Beam Instrumentation Section DSP board is quite reasonable. At
8Mhz, a 5.3Mbytes/s throughput from IP to DSP is achieved. This can be doubled to 10.6Mbytes/s with a double
wide industry pack. The Industry Pack specifications also specifies (preliminary) a 32 Mhz interface, which has
been implemented giving a throughput of 12.8Mbytes/s or 25.6Mbytes/s for double width Industry Pack.

IEEE P1394 Serial Bus
Some comments on this standard:• IEEE 1212 Command and Status Register (CSR) standard with a 64 bit address space• 100, 200, or 400 Mbits/s bandwidth• memory-mappe- like architecture is well matched to load-store architecture of current processors.
• real world performance should be comparable to VXI backplane performance. Any standard
based on RS-485, Universal Serial Bus (USB), ethernet, etc. will be significantly slower.
• LAN/WAN technologies would introduce complex protocol stacks and driver software. This is ahigh price to pay for simply moving modules off the backplane.
• asynchronous and isochronous (guaranteed bandwidth and latency) data transfers are supported• maximum of 4.5 meters between nodes is standard, but work is in progress to extend to much
longer distances with cable upgrades and/or bridges• cheap ($50/node in 1995)
An Industry Pack that contains a 100 Mb/s Serial Bus interface has been developed at RHIC. Several of thesedevices will be used to evaluate possible applications in the collider ring position monitor and loss monitor
systems. Results of this study will be reported in the near future.

V. REFERENCES[1] Y. Chung, et. al., Closed Orbit Feedback with Digital Signal Processing, Proc. of the Fourth European Particle Accelerator Conference, London (1994).
[2] Y. Chung, et. al., Digital Closed Orbit Feedback System for the Advanced Photon Source Storage Ring, this conference.
[3] H. Ma and C. Drennan, Position Monitoring of Low Intensity Beams Using A Digital Frequency
Down Converter, Beam Instrumentation Workshop AIP Conference Proceedings 333 (1994).
[4] T. J. Shea, et. al., Beam Instrumentation for the RHIC Sextant Test, Proc. of the Fourth European Particle Accelerator Conference, London (1994).
[5] Industry Pack Logic Specifications Rev. 0.7.1, Greenspring Computers, Menlo Park, CA.
[6] G. Vismara, New Front End Narrow Band Electronics for the LEP Beam Orbit Measurement
System, Proc. of the Fourth European Particle Accelerator Conference, London (1994).
[7] C. Bovet, The Use of LEP Beam Instrumentation with Bunch Trains, Beam Instrumentation
Workshop AIP Conference Proceedings 333 (1994).
[8] IEEE 488.1-1987, IEEE Standard Digital Interface for Programmable Instrumentation.
[9] IEEE 488.2-1992, IEEE Codes, Formats, Protocols, and Common Commands, and Standard
Commands for Programmable Instruments.
[10] The VME Bus Handbook, 2nd Rev., VFEA International Trade Association, Scottsdale AZ,1991.
[11] Steve Jennings, Tektronix Corporation, private communication.
[12] E. L. Barsotti, Jr., Fermilab, private communication.
[13] RHIC Design Manual, August 1993.
[14] E. L. Barsotti, Jr., A Longitudinal Bunch Monitoring System Using LabVIEW and High-Speed
Oscilloscopes, Beam Instrumentation Workshop AIP Conference Proceedings 333 (1994).
[15] J. A. Hinkson, et. al., Automated Tune Measurements in the Advanced Light Source Storage
Ring Using a LabVIEW Application, Proc. of the Fourth European Particle Accelerator Conference, London (1994).
[16] IEEE P1394 Draft 7.1 IEEE Standard for a High Performance Serial Bus, 1994.