![](media/image1.wmf)

Fermilab/CD

Beams-doc-1949-v9

April 7, 2006

Version 9

**Main Injector Beam Position Monitor Upgrade**

**Software Specifications for**

**Data Acquisition**

Luciano Piccoli, Stephen Foulkes, Margaret Votava

Fermilab, Computing Division, CEPA

Brian Hendricks

Fermilab, Accelerator Division, Accelerator Controls Department

***Abstract***

This document contains the specifications for the Main Injector BPM
upgrade's front end software. Expected operating modes and interactions
with the BPM hardware are described. Data structures for communication
with the online software via ACNET are also defined.

[1 Overview 3](#overview)

[1.1 Measurement Types 3](#measurement-types)

[1.2 Hardware Configuration 4](#hardware-configuration)

[1.3 Requirements on the DA Software
6](#requirements-on-the-da-software)

[2 Data Acquisition 6](#data-acquisition)

[2.1 Main Injector Cycles 7](#main-injector-cycles)

[2.2 Buffer Lifetime 9](#buffer-lifetime)

[2.3 Timing 9](#timing)

[2.3.1 Timing Margins 10](#timing-margins)

[2.3.2 Delays 10](#delays)

[2.3.3 Narrow Band 10](#narrow-band)

[2.3.4 Wide Band 11](#wide-band)

[2.4 Data Acquisition Modes 11](#data-acquisition-modes)

[2.4.1 Closed Orbit Mode 11](#closed-orbit-mode)

[2.4.2 User Defined Turn by Turn 12](#user-defined-turn-by-turn)

[2.4.3 Flash Turn by Turn 12](#flash-turn-by-turn)

[2.5 Data Processing 13](#data-processing)

[3 Interface to Online Software 14](#interface-to-online-software)

[3.1 SSDN / ACNET Device Mapping 15](#ssdn-acnet-device-mapping)

[3.1.1 SSDN Mappings and ACNET Devices for Command Lists
15](#ssdn-mappings-and-acnet-devices-for-command-lists)

[3.1.2 SSDN Mappings and ACNET Devices for Timing
15](#ssdn-mappings-and-acnet-devices-for-timing)

[3.1.3 SSDN Mappings and ACNET Devices for Diagnostics
15](#ssdn-mappings-and-acnet-devices-for-diagnostics)

[3.1.4 SSDN Mappings and ACNET Devices for Front End Status
16](#ssdn-mappings-and-acnet-devices-for-front-end-status)

[3.1.5 SSDN Mapping for FTP devices 16](#ssdn-mapping-for-ftp-devices)

[3.1.6 SSDN Mappings for Closed Orbit devices
16](#ssdn-mappings-for-closed-orbit-devices)

[3.1.7 SSDN Mappings for Turn by Turn devices
17](#ssdn-mappings-for-turn-by-turn-devices)

[3.2 Configuring the BPMS 17](#configuring-the-bpms)

[3.2.1 Commands 18](#commands)

[3.2.2 Data Structures 18](#data-structures)

[3.3 Retrieving Data 20](#retrieving-data)

[3.3.1 Headers 20](#headers)

[3.3.2 Non Turn-by-Turn Data 21](#non-turn-by-turn-data)

[3.3.3 Turn by Turn Data 22](#turn-by-turn-data)

[4 Interface to BPM Hardware 22](#interface-to-bpm-hardware)

[4.1 Operation of the EchoTek ADC Boards
22](#operation-of-the-echotek-adc-boards)

[5 Calibration 24](#calibration)

[6 Diagnostics, Test Suite, and Simulation
24](#diagnostics-test-suite-and-simulation)

[6.1 Diagnostics 24](#diagnostics)

[6.2 Self-Testing Procedures 25](#self-testing-procedures)

[7 Monitoring 25](#monitoring)

[A. Main Injector States 25](#main-injector-states)

[B. Change Log 27](#change-log)

[C. References 27](#references)

#  {#section .unnumbered}

# Overview

In compliance with the existing Accelerator Controls software
architecture, the software pieces surrounding the BPM hardware are
divided into 3 layers: the front end software running on the individual
front-end computers in the readout creates in the service buildings
(usually referred to as "houses"), user applications running on analysis
nodes (Windows or Linux), and online software running on a central
server (VAX) and a few DAQ engines (Sun) providing the primary (but not
exclusive) bridge between user applications and the front end software.
Figure 1 shows the software architecture. This document will focus on
the front end software. See the MI BPM Requirements document for the
overall project requirements. [^1]

Figure 1. Software Architecture

The remaining parts of this section will briefly outline the required
measurement types, the hardware configuration and the requirements on
the front end software. Section Data Acquisition will describe the
controls, configurations and data buffering within one front-end create.
Section Interface to Online Software will define the interface to the
online software. The remaining sections will elaborate on calibration,
debugging, alarms and monitoring.

##  Measurement Types

The different types of measurements that will be accessible from the
front end are summarized as follows:

-   **User Defined Turn-by-Turn:** A measurement of the orbit on every
    turn (588 53Mhz RF buckets) of every BPM for 2048 turns, performed
    in wide bandwidth mode.

-   **Injection and Extraction Turn**-**by-Turn:** A wide bandwidth
    measurement of the orbit on every turn of every BPM for each batch
    of the beam injected into the machine. It also must provide the
    capability to measure the extraction turn of at least one portion of
    the beam extracted from the machine.

-   **Flash Frame:** A single orbit measurement, performed in wide
    bandwidth mode for each BPM. Flash frames are collected from the
    injection and extraction turn-by-turn buffers. The flash frame
    consists of the first turn that has beam for injections, or the last
    turn that had beam for extractions.

-   **Averaged Orbit:** A single wide bandwidth measurement that
    consists of the orbit averaged over the first 512 turns of the first
    the flash turn buffer.

-   **Closed Orbit:** A narrow bandwidth measurement collected from all
    BPMs at a rate of 500Hz

-   **Display Frame:** A narrow bandwidth measurement triggered by the
    display frame TCLK (\$7B). This measurement can occur at most once
    per cycle and is collected from all BPMs. Data for this measurement
    is pulled from the Closed Orbit Buffer.

-   **Profile Frame:** A narrow bandwidth measurement triggered by the
    profile frame TCLK (\$7A). This measurement can occur at most 128
    times per cycle and is collected from all BPMs. Data for this
    measurement is pulled from the Closed Orbit Buffer.

-   **User Defined Frame:** A narrow bandwidth measurement triggered by
    a user definable TCLK. This measurement can occur at most 128 times
    per cycle and is collected from all BPMs. Data for this measurement
    is pulled from the Closed Orbit Buffer.

> .

It is important to note that for all variations of the Turn by Turn
measurement, all BPMs should measure the same portion and turn of a
particular batch of beam.

##  Hardware Configuration

Based on the experience from the Recycler BPM system and the Tevatron
BPM system, the Main Injector BPM system is designed to be a VME crate
holding a crate controller with a PCMUCD daughter board, a Timing
Generator Fanout (TGF) board and up to 10 ADC boards. See Figure 2 and
hardware doc #2083.

Figure 2. House (VME crate) Configuration

There are seven houses around the Main Injector ring, each having one
VME crate that handles up to 10 ADC cards, each of which can digitize
data for 4 BPMs for a total of 40 BPMS per house. The specific models
and main functions of the boards are briefly described in the following.

-   The Main Injector TGF Board is based on the Tevatron TGF Board. It
    is designed to perform the following:

    -   Phase lock to the Main Injector RF frequency which varies from
        52.8-53.1 MHz and generate a clock signal 10/7 times the RF
        clock for up to 10 ADC boards

    -   Decode the events transmitted through the TCLK and MDAT systems
        to provide advanced arming signals to the crate controller so
        that the hardware can be configured in time for different
        measurement types or present pre-defined requests for data
        transfer from the crate controllers to the online software.

    -   Decode the events transmitted through the Recycler Beam Sync and
        Main Injector Beam Sync as well as triggering off of the Booster
        Extraction Sync to derive accurate timing and triggering signals
        to synchronize up to 10 ADC boards with respect to the different
        beam arrivals.

    -   Configure the analog transition board, to control the signal
        attenuation and frequency.

-   The ADC Boards are model ECDF-GC814-FV-2 from the EchoTek
    Corporation. The digitized output can be raw ADC count or digitally
    down-converted and filtered to extract the strengths of the 53 MHz
    and 2.5 MHz components. The output of each channel is represented by
    two components: a real (I for in-phase) and an imaginary (Q for
    quadrature) part. Each board has 8 input channels.

-   The crate controller is an MVME5500 from Motorola. This model was
    chosen over the MVME2400 used in the Tevatron BPM and Recycler BPM
    due to its larger memory capacity, gigabit Ethernet capability,
    faster processor and longer service lifetime. This card is
    responsible for the communications between the online software, TGF
    and EchoTek ADC boards. All controls and data transfers to/from
    those boards are preformed by the crate controller.

-   The PMCUCD daughter board is from Techobox Inc. It also decodes the
    TCLK signal and triggers the standard ACNET/MOOC software
    applications accordingly. Since this board belongs to the
    Accelerator Controls Infrastructure, it will not be discussed any
    further in this document.

##  Requirements on the DA Software

In order to comply with the existing Accelerator Controls software
architecture and minimize the time needed for development and debugging,
the DA software must meet the following requirements:

-   All communications between the online software and the front-ends
    will be conducted via ACNET devices. This includes reading data from
    the front end as well as setting acquisition and readout parameters.
    Internal diagnostics, however, do not have this constraint.

-   Data acquisition happens asynchronously from data readout, i.e., the
    EchoTeks can be configured to take data continuously on certain
    triggers, but the data will not be moved to the CPU until later. Not
    all data that is collected is read out to ACNET. This implies that
    the front end must have buffers for each measurement type and manage
    readout requests sent down from the online software.

-   "Event assembly" is done by the online software, not by the front
    end. Therefore a given BPM crate does not need to have any knowledge
    of any other BPM crate(s). The data sent by the crate will however
    include information that will allow data synchronization by the
    online software. That information includes time stamps and turn
    counts from the TGF boards.

-   The front-ends should detect state changes via TCLK reset events.

-   The crate controller will run VxWorks.

-   The front-end software must make use of the GBPM library that is
    used in the Tevatron.

# Data Acquisition

The front-end data acquisition system is located between the online
software and the BPM digitizing hardware. Any access to information and
controls on the electronics boards will pass through the front-end
processor. The information includes beam position data, calibration data
and diagnostics data among other configuration parameters.

Figure 3. Communication Path

The communication between the online software and the front-end
processor is ACNET/MOOC based, while the communication between the
front-end processor and other electronics happens over the VME
backplane. See Figure 3

During a measurement, the digitized and filtered data is first stored in
the memory on the EchoTek boards and then transferred to the crate
controller after the end of beam TCLK event. The data is saved in a set
of buffers on the crate controller so that it may be accessed at another
point in time. The depth and number of logical buffers that reside in
the crate controller will vary depending on configuration and total
physical memory[^2]. The actual buffer implementation will be discussed
in the Software Design Document[^3].

## Main Injector Cycles

Each Main Injector cycle is uniquely identified by a machine state. This
state information is transmitted on the MDAT system and is valid at the
TCLK reset for the main ramp (any one of 20 or so various TCLKs). See
Appendix A for a list of main injector states and cycles.[^4]

Because the operation of the main injector can be very complex, each
machine state can have radically different requirements on the type and
frequency of measurements. The behavior of the front ends is described
by a set of acquisition specifications (one per state) that are
re-configurable by the user. An acquisition specification for a given
state will describe a series of commands to be executed during a cycle
that setup and trigger the EchoTek modules to acquire measurements. A
given acquisition specification can switch to and from narrow and wide
band measurements and between 53MHz or 2.5MHz mode. The format of the
acquisition specification is discussed in greater detail in Section
Configuring the BPMS

Since the measurement requirements for a cycle can be quite demanding,
there may not be enough time for the measurement to be taken and
transferred to the processor. As a result of this, the processor may
leave data on the EchoTek board to be read out after the 'end of beam'
TCLK event has been received. All data that is taken during a cycle will
not be available until after the 'end of beam' TCLK event has been
received.

The front end system is guaranteed a minimum of 50msec at the beginning
of a cycle (i.e., from the 'ramp reset' TCLK to the beginning of the
first measurement) to clear buffers and prepare for data taking. It is
also guaranteed a minimum of 500msec between cycles (i.e., from the 'end
of beam' TCLK event to the next ramp reset) to collect and process all
of the data stored in the EchoTek modules for that cycle.

For each ramp reset, the front end software will:

1.  Determine cycle state

2.  Load acquisition specification for that state.

3.  Execute the commands in the acquisition specification in time order
    (see NOTE)

4.  Upon receipt of the 'end of beam' TCLK, collect and process all
    unread data from the EchoTek modules (i.e., wide band data)

5.  Wait for next reset

-   NOTE: The user defined turn by turn request needs to be assembled as
    a complete event across all houses. Since each house is independent
    from the others, the event assembly is done by the online software
    which is reading asynchronously from each house with a variable and
    long (seconds) latency. Data in the front ends boards needs to be
    protected during this time from overwrites and race conditions. To
    facilitate this, the user defined turn by turn buffer is only
    updated when the pretrigger ACNET device is non zero. That way, the
    pretrigger ACNET device can be set to zero after the measurement is
    taken to preserve that data until it is read out.

## Buffer Lifetime

All buffers (profile, display, flash average, flash first turn, and
flash) must be kept valid from the end of the current cycle until the
'end of beam' TCLK is received for the next cycle of the same type. To
achieve this, data is read from the EchoTeks into a set of buffers that
is separate from the main set. Once the 'end of beam' TCLK has been
received the newly collected data is copied into the state buffers,
replacing data from the previous instance of that state. Data is always
readout to ACNET from the state buffers. See figure 4. The only
exception to this is the user defined turn by turn buffer. It must not
be overwritten unless the user defined turn by turn ACNET device is non
zero.

![](media/image2.png){width="5.988888888888889in"
height="4.2555555555555555in"}

Figure 4 -- Buffer Diagram

## Timing

The DA system will be configured to operate in one of two basic running
modes:

wide bandwidth and narrow bandwidth. The former will be used for turn by
turn measurements, while the latter will be used for the various types
of closed orbit.

### Timing Margins

This system will require periods of time when data acquisition is not
possible. There are currently three identified periods:

-   Start of cycle -- There will be at least 50ms between the TCLK reset
    event and the arrival of beam. This is to allow the front end to
    configure the timing boards and EchoTeks

-   Mode Change -- The front end must be able to change modes in less
    than 10ms, i.e. - going from Closed Orbit to Turn By Turn.

-   End of Cycle -- There will be at least 500ms between the end of beam
    signal and the beginning of the next cycle in order for the front
    end to read data off of the EchoTeks.

### Delays

It should be noted that in a given running state of the Main Injector,
the time delay from the Main Injector turn marker (MIBS \$AA) as seen by
the TGF until the actual bunch passing is fixed but different at each
BPM location. In order to ensure that the ADCs are triggered at the same
time relative to the actual bunch passing of the same turn, 3 layers of
delays are implemented:

-   House Delay -- A delay applied to all BPMs in a given house. There
    is a different house delay for measurements triggered off of BES and
    MIBS/RRBS.

-   Board Delay -- A delay applied to all BPMs connected to a particular
    EchoTek board

-   Channel Pair Delay -- A delay applied to a single BPM

There will be different delays for different particles that circulate in
different directions. Delays can be set using the diagnostics console
application, and are effective at the start of the next Main Injector
cycle. Delays for different injection and extraction points are
specified in the acquisition specification.

For measurements triggered by the BES signal, the timing board will
apply the delay from the BES signal. For all other measurements the
delays will be applied starting at the next MIBS \$AA that is received.

### Narrow Band 

For narrow band measurements, the front-ends do not need to be
synchronized to the same portion of a batch of beam; they only need to
cover the same length in time. The front ends are averaging so much data
that it is not a requirement that the timing be synchronized better than
a turn or two. In this mode, the TGF simply counts the Main Injector
revolution/turn markers and uses that counter to generate an interrupt
at 500Hz to inform the crate controller of a pending data acquisition.
When the pre-set number of samples (burst count) has been collected, the
first EchoTek board will issue a collection-done interrupt to notify the
crate controller for data transfer.

### Wide Band

For wide band measurements, the front-ends in all the houses will need
to be synchronized to the same specific turn, as well as the same
portion of a batch of beam. This is achieved by setting the TGF to wait
for a particular "start event". Upon the occurrence of this "start
event", the TGF generates an interrupt to inform the crate controller of
a pending data acquisition and repeats the following steps for the
desired number of turns: wait for the next "turn event", apply
appropriate time delays and generate trigger signals for the EchoTek
boards. When the pre-set number of triggers has been received, the first
EchoTek board will issue a trigger-counter interrupt. The TGFs have
tight and consistent latency specifications and the main differential
latency is caused by the distribution of the timing signals to all the
sub racks, which can be correct by adjusting the house delays.

## Data Acquisition Modes

Corresponding to the various measurement types it needs to handle, the
data acquisition system has a few different operation modes. These modes
are mutually exclusive due to the need for specific configurations of
the triggering and filtering in the

EchoTek boards for a specific mode. These modes are:

-   Closed Orbit

-   Turn by Turn

-   Flash

-   Raw

-   Safe

### Closed Orbit Mode

This is a narrow bandwidth mode of the BPM system and is triggered by
the data acquisition specification. Several buffers of closed orbit data
will be kept by the front end:

-   **Profile Frame Buffer** -- 128 points deep. Data is taken from the
    most recent closed orbit measurement and placed in this buffer every
    time a TCLK \$7A is received.

-   **Display Frame Buffer** -- A single measurement. Data is taken from
    the most recent closed orbit measurement and placed in this buffer
    every time a TCLK \$7B is received.

-   **User Defined TCLK Buffer** -- 128 points deep. Data is taken from
    the most recent closed orbit measurement and placed in this buffer
    every time a certain TCLK is received. This TCLK is defined through
    an ACNET device.

-   **Fast Time Plot Buffer** -- Data is taken from the most recent
    closed orbit measurement every 2 milliseconds. This buffer is
    independent of state. It tracks several pieces of data useful for
    plotting the following:

    -   Position from a particular BPM

    -   Intensity from a particular BPM

    -   A particular input's I value

    -   A particular input's Q value

    -   Sum signal for each channel (magnitude of A + magnitude of B)

The front end software must be able to store a profile frame buffer and
display frame buffer for each state. At the beginning of each Main
Injector state its profile and display frame buffers will be erased. If
the front end is not collecting closed orbit data and a profile frame,
display frame or a fast time plot is called for, an error value will be
placed into the appropriate buffer.

The measurements are triggered at a 500 Hz rate. This rate is sourced by
the TGF board through a decimate-by-*N* counter to down-sample the Main
Injector RF frequency. The value of *N*, and thus the trigger rate, can
be adjusted through the turn modulus/decimation register on the TGF
board. At 500 Hz, the DA system has about 200 us idle time between
triggers.

### User Defined Turn by Turn

User defined turn by turn measurements can occur at any time during a
cycle, but can only be taken once per cycle. A turn by turn measurement
will be taken when it is enabled in the command list and when the ACNET
device that is passed down with the command has a non zero value. The
turn by turn measurement will be the only one preformed during that
cycle. Upon the successful completion of the turn by turn measurement,
the data will be transferred off of the EchoTeks to the crate
controller. The positions and intensities of the beam at each BPM
location will be calculated in the same fashion as closed orbit data.
The position and intensity as well as the raw I and Q values will be
stored for later retrieval through ACNET. There must be a separate turn
by turn buffer for each state.

### Flash Turn by Turn

The front end software will be able to take turn by turn measurements
for particle injection and extraction, triggered by the relevant beam
sync clock event. The system must support taking 512 turns per
measurement, and be able to store up to 20 measurements for each state.

If there are multiple injections in the same cycle, the system must
provide the capability to measure the *first* turn for each portion of
beam injected in the machine. Successive injections from the Booster
happen at a maximum rate of 15 Hz.

If there are multiple single turn extractions in the same cycle, the
system must provide the capability to measure the extraction turn of at
least one portion of beam extracted from the machine.

The system must generate an Averaged Orbit by averaging the injection
turn-by-turn data for the first injection into the Main Injector in each
cycle. The Averaged Orbit will be the average of the first 16 turns with
beam. This data will be used for machine injection closure.

### Raw Mode

A raw mode measurement is one where the front end configures the EchoTek
boards to take 2048 samples without any digital filtering. This
measurement can be enabled at any point during a cycle and will honor
all delays. Once a raw mode measurement is enabled for a given cycle, it
and the filter command proceding it will be the only measurement run.
Raw mode data can only be read out through the diagnostics application,
I43. There is one raw mode buffer per crate. This buffer can only be
cleared by another raw mode measurement, it is not cleared on the reset
event for each cycle.

### Safe Mode

A safe mode measurement is a wide band measurement. The front end will
setup the EchoTek board to take 12,000 samples continously. No other
wide band measurements can be run during a cycle that has a safe mode
measurement, but narrow band measurements can be run. The safe mode
measurement can only be triggered once during a cycle.

At the end of the cycle, the front end will read out the EchoTeks and
try to find the beam. It will choose one sample with the highest
intensity per turn and put that into the first flash buffer. The
remaining flash buffers will be filled with the raw safe mode data.

## Data Processing

After the EchoTek boards have completed data acquisition, the crate
controller will read out the I-Q pairs. The modulus of each channel
(*M*) is then calculated:

$$M = G \times (\sqrt{I^{2} + Q^{2}} - M_{0})$$

Where *G* and *M*0 are the gain and offset of the electronics. Finally
the beam position

\(D\) and intensity (S) can be determined according to:

$$D = g \times \frac{M_{A} - M_{B}}{M_{A} + M_{B}} - D_{M}$$

$$S = M_{A} + M_{B}$$

Where *g* is a polynomial to convert the unit-less quantity to
millimeters (nominally 26

mm), *D M* is the mechanical offset that was surveyed relative to the
BPMs electrical center before the BPM was installed in the ring.

For each BPM, the position ( *D* ) and intensity ( *S* ) are to be
stored along with the raw I and Q pairs.

### Offsets

The front end will read offsets from the online software database and
apply them to all measurements. Offsets are applied differently for
horizontal and vertical bpms. The following equations will be used:

*horizontal position = bpm_offset - survey_offset + orbit_offset -
electrical_offset*

*vertical postion = bpm_offset + survey_offset - electrical_offset*

# Interface to Online Software

This section defines how data and commands are exchanged between the
front-end DAQ software and the online software. ACNET will be the means
of transportation of data and commands between the front-end DA and the
online software.

Requests may be made in parallel by disjoint applications, and some
mechanism for avoiding conflicts must be designed and implemented.

BPM data read by the online software will be organized according to the
data structures defined in section 3.3. Online applications that make
use of the old system must be changed to handle new data formats.

The supported ACNET protocols will be SETDAT, RETDAT and Fast Time Plot
(FTP). The snapshot protocol (not to be confused with a BPM snapshot)
will not be supported by the front-end DAQ. The front-end must be able
to generate FTP data at a rate up to 500 Hz.

## SSDN / ACNET Device Mapping

The following suggested SSDN numbers will be used to map to the
appropriate ACNET devices. There is at least one ACNET device associated
with each SSDN number.

The SSDN number is an 8 byte field split into 4 2-byte pairs. See the
MOOC front-ends document for a detailed field description:

<http://www-bd.fnal.gov/controls/micro_p/mooc_front_ends.html>

For reading BPM data, this project will use the recycler BPM model of
object id:

-   0x0020 for BPM Control

```{=html}
<!-- -->
```
-   0x0021 for BPM RETDAT/SETDAT

-   0x0022 for BPM FTP data

### SSDN Mappings and ACNET Devices for Command Lists

Each house will have an ACNET device for each possible machine state.
The device will take the form I:xxCMyy where xx is the house number (10,
20, 30, 40, 50, 6S, 6N) and yy is the machine state. Note that the
machine state is represented in hex, so machine state 11 on the MI40
front end would be I:40CM0B. These devices will be used to send command
lists down to the houses, the data structures of which are discussed in
section 3.2.1. The SSDNs for these devices take the form
0000/0020/yy00/2094 where yy is the machine state, which is also
represented in hex.

### SSDN Mappings and ACNET Devices for Timing

There are five ACNET devices for every house that are used for setting
delays. I:BxxBRD (SSDN 0000/0020/0000/2087) is for setting the board
delay, I:AxxHSD, I:MxxHSD and I:PxxHSD (SSDN 0000/0020/0000/2086) are
used for setting the house delays and I:BxxCHD (SSDN
0000/0020/0000/2089) is for setting the channel delay where xx is the
house number (10, 20, 30, 40, 50, 6S, 6N). I:BxxBRD and I:BxxCHD are
array devices.

### SSDN Mappings and ACNET Devices for Diagnostics

There are two ACNET devices for diagnostics, I:BxxACQ (SSDN
0000/0020/0000/2082) for sending an acquistion specification to the
front-ends and I:BxxADV (SSDN 0000/0021/0003/2100) for reading back the
diagnostics buffer.

### SSDN Mappings and ACNET Devices for Front End Status

There are four devices that are used for collecting status from the
front ends. I:xxHIST (SSDN 0000/0020/0000/2092) will return the software
start time, the time of the last mode change, the last TCLK received and
the time the last TCLK was received. I:xxSTAT (SSDN 0000/0020/0000/2091)
will return the status of various timing signals as they are seen by the
timing card. I:xxCONF (SSDN 0000/0020/0000/2093) will return the number
of EchoTeks installed into the crate as well as the closed orbit
frequency. I:xxVER (SSDN 0000/0020/0000/2094) will return the versions
of the MIBPM and GBPM libraries, as well as the firmware versions on the
EchoTeks and TGF. The xx in the device name refers to the house number.

### SSDN Mapping for FTP devices

Each channel can have an I and Q value. Data from a pair of channels
(i.e. A and B plates) can be manipulated to generate intensity and
position information. There will be a position and intensity measurement
for the horizontal channel, and a position for the vertical channel for
both proton and anti proton data. One EchoTek card contains eight
channels, allowing 4 BPMs to be read out. A complete house with 10
EchoTek boards can read out 40 BPMs, each one with 2 channels. The total
channels in a house is 80, and each channel has an I and Q value
associated with it.

The following FTP devices have been defined:

-   0000/0022/0000/22**YY** : This device contains the I, Q, and sum
    signal information for each channel, where **YY** corresponds to
    channel 0x00-0x50. The first element in the device is I, the second
    is Q and the third is the sum.

-   0000/0022/0001/22**YY** : This device contains position and
    intensity information for protons or pbars, where **YY** corresponds
    to a BPM number 0x00-0x25. The first element is position while the
    second element is intensity.

The maximum number of FTP devices provided by one house is 120:

1 I/Q/SUM x 80 channels + 1 p/pbar \* 40 position/intensities = 120

### SSDN Mappings for Closed Orbit devices

The front end will supply ACNET devices for the following closed orbit
measurements:

-   Profile Frame (I:**xx**BPPR for scaled, I:**xx**BRPR for raw)

-   Display Frame (I:**xx**BPDF for scaled, I:**xx**BRDF for raw)

-   User Data (I:**xx**BPUF for scaled, I:**xx**BRUF for raw)

The **xx** corresponds to the house number, which can be 10, 20, 30, 40,
50, 6N, 6S.

Data will be available in both scaled and raw form. There will be one
ACNET device for each house, for each closed orbit measurement type. The
device will return the closed orbit measurements for all the BPMs in
that house. The SSDN for these devices will take the for of
0000/0021/000**Y**/000**X** where:

-   **X** corresponds to the measurement type, which can be one of the
    following values:

    -   **0** -- Profile frame

    -   **1** - Display frame

    -   **2** -- User

-   **Y** corresponds to how the data is represented:

    -   **0 -** Raw I/Q values

    -   **1** - Scaled positions and intensities

### SSDN Mappings for Turn by Turn devices

The front end will supply ACNET devices for the following turn by turn
measurements:

-   User defined Turn by Turn (I:TT**nyyy** for scaled, I:TR**nyyy** for
    raw)

-   Injection/Extraction Flash (I:TF**nyyy** for scaled, I:RF**nyyy**
    for raw)

-   First turn Flash (I:**xx**BPFF for scaled, I:**xx**BRFF for raw)

-   Closed orbit flash (I:**xx**BPFT for scaled, I:**xx**BRFT for raw)

The **n** corresponds to the BPM alignment, V for vertical and H for
horizontal. The **yyy** corresponds the the zero padded BPM number. The
**xx** corresponds to the house number, which can be 10, 20, 30, 40, 50,
6S, 6N.

Data will be available in both scaled and raw form. For user turn by
turn and injection/extraction flash turn measurements there will be one
ACNET device per BPM while for the first turn flash and closed orbit
flash there will be one ACNET device per house. The SSDN for these
devices will take the for of 0000/0021/**ZZ**0**Y**/210**X** where:

-   **X** corresponds to the measurement type, which can be one of the
    following values:

    -   **3 --** User defined turn by turn

    -   **4 --** Injection/extraction flash

    -   **5 --** First turn flash

    -   **6 --** Closed orbit flash

-   **Y** corresponds to how the data is represented:

    -   **0 -** Raw I/Q values

    -   **1** - Scaled positions and intensities

-   **ZZ** corresponds to the EchoTek channel number that a particular
    BPM is hooked up to. This value is only used for the user defined
    turn by turn and the injection/extraction flash measurements.

## Configuring the BPMS

Each MI state has an associated acquisition specification. The
acquisition specification for a given cycle must be known by the
front-end software before the MI cycle is started. The specification for
any cycle can be changed at any time by online users. However if the
cycle is currently running the front-end will wait until the end of the
cycle to update.

The front-end will not perform any checking of the individual commands
in the acquisition specification as they are sent down from the online
software. It is the responsibility of the online software to validate
the command list. However, the front-end software will provide the means
to read back the cycle specification, but no history of commands for any
cycle will be kept.

### Commands

There are to be five commands that can be passed down to the front end:
filter, turn by turn, closed orbit, flash and safe. Each command must
have a delay parameter which will determine when they get run during a
cycle. This delay parameter is measured in milliseconds from the cycle
reset TCLK.

The closed orbit command has no other parameters. While it is active,
closed orbit data will be taken at a 500 Hz rate. The front end will
also listen for profile and display TCLKs, and update the profile and
display buffers accordingly. If a profile or display TCLK is received
when the closed orbit command is not active, the profile or display
buffer will be filled with data containing a position of 888.

The turn by turn and flash commands are similar. The turn by turn
command can only take one measurement per cycle while the flash command
can take up to 20. Both commands need a trigger value, as well as a turn
delay and bucket delay.

The filter commands changes settings on the filter board to specify the
type of beam in the machine, as well as its intensity and frequency.
This also tells the front end which set of delays to load (proton or
pbar).

The safe command will cause a safe mode measurement where the front end
will collect data continuously until the buffers on the EchoTek fill up.
A trigger value, as well as turn delay and bucket delay need to be
supplied to this command.

### Data Structures

The data sent to the front-end DAQ from the BPM library and/or
applications is based on the following C data structures. All data is
big endian. The front-end software will support a list of at most 32
commands during a given cycle. It also at this time supports the four
commands discussed in 3.2.1.

#### Constants

/\* Commands \*/

const long FILTER = 0;

const long TURN_BY_TURN = 1;

const long CLOSED_ORBIT = 2;

const long FLASH = 3;

/\* Particle type \*/

const long MIBPM_PROTON_PARTICLE_TYPE = 0;

const long MIBPM_PBAR_PARTICLE_TYPE = 1;

/\* Filter frequency \*/

const long MIBPM_2_5MHZ_FILTER = 0;

const long MIBPM_53MHZ_FILTER = 1;

/\* Filter attenuation \*/

const long DB0 = 0;

const long DB6 = 1;

const long DB12 = 2;

const long DB18 = 3;

const long DB24 = 4;

const long DB30 = 5;

const long DB36 = 6;

const long DB42 = 7;

const long DB48 = 8;

/\* Data Source \*/

const long MIBPM_BEAM_DATA_SOURCE = 0;

const long MIBPM_CALIBRATION_SOURCE = 1;

const long MIBPM_SOFTWARE_DIAG_SOURCE = 2;

const long MIBPM_HARDWARE_DIAG_SOURCE = 3;

/\* Bunch type \*/

const long MIBPM_UNCOALESCED_BUNCH_TYPE = 0;

const long MIBPM_COALESCED_BUNCH_TYPE = 1;

/\* Data type \*/

const long MIBPM_RAW_DATA = 0;

const long MIBPM_SCALED_DATA = 1;

#### Commands

typedef struct {

int type;

int delay; // Specified in miliseconds

union {

int particle; // Proton or Pbar (FILTER)

int enabled; // (All other commands)

} datum1;

union {

int frequency; // 53 MHz or 2.5 MHz (FILTER)

int bsync; // MIBS, RRBS or BES (FLASH)

} datum2;

union {

int attenuation; // (FILTER)

int turnDelay; // (FLASH) / (TURN_BY_TURN)

} datum3;

union {

int bucketDelay; // (FLASH) / (TURN_BY_TURN)

} datum4;

} MIBPM_STATE_COMMAND;

typedef struct {

int size;

MIBPM_STATE_COMMAND commands \[MIBPM_STATE_COMMAND_SIZE\];

} MIBPM_STATE_COMMAND

##  Retrieving Data

### Headers

The current version of the MI_DATA_HEADER is 1. The status variable
inside the MI_DATA_HEADER structure will be set to a non-zero value if
there is some problem at the crate level. Upon receiving a
MI_DATA_HEADER with a non-zero status value the online software will
ignore the data. All data is big endian.

The time data structure in the MI_DATA_HEADER is the time stamp of the
first data frame.

const int MI_BPMS_PER_HOUSE 40;

const int MI_TURN_SCALED_BLOCK_SIZE 512;

const int MI_MAX_FRAMES 512;

typedef struct BPM_TIME

{

ulong timestamp; /\* timestamp in second (GMT) \*/

ulong nanoseconds; /\* nanoseconds after timestamp \*/

};

typedef struct MI_TRIGGER_INFO

{

int type; /\* MIBS / RRBS / BES / Closed Orbit \*/

int value; /\* trigger value \*/

};

typedef struct MI_DATA_HEADER

{

int endidan_type;

int version; /\* data structure version \*/

int status; /\* transaction status, this is

defined later in this document.

\*/

BPM_TIME time; /\* timestamp \*/

unsigned int turn_number; /\* starting turn number \*/

unsigned int num_turns; /\* number of turns in data \*/

double time_in_cycle; /\* starting time in cycle \*/

int data_type; /\* flash/profile/turn by turn... \*/

MI_TRIGGER_INFO trigger_info; /\* trigger information \*/

int data_source; /\* beam or calibration \*/

int particle_type; /\* proton/pbar \*/

int scaled_data; /\* scaled/raw \*/

int calibration_id; /\* calibration data ID number \*/

int beam_frequency;

int attenuation;

};

typedef struct MI_STATE_DATA\
{

int reset_event; /\* TCLK event which began the beam cycle \*/

int machine_state; /\* MDAT / TCLK RESET \*/

int bpm_state; /\* not yet defined \*/

};

### Non Turn-by-Turn Data

The front end will return either an MI_ORBIT_DATA_SCALED or
MI_ORBIT_DATA_RAW data structure depending on the type of data requested
for all measurements except Turn-by-Turn and FTP. All data is big
endian. The bpm_status array inside the MI_FRAME_DATA data structures
will be set to one of the following values:

-   **0** -- OK

-   **1** -- To little beam intensity

-   **2** -- Alarm level -- if we want alarm limits

-   **3** -- Saturated

-   **4** -- Hardware error

-   **5** -- Channel not in use

typedef struct MI_FRAME_DATA_SCALED

{

MI_DATA_HEADER header;

MI_STATE_DATA state_data; /\* machine state information \*/

int num_detectors;

int frame_number; /\* ordinal number in front-end \*/

int bpm_status\[MI_BPMS_PER_HOUSE\]; /\* status information for

bpms, defined later in this

document. \*/

float positions\[MI_BPMS_PER_HOUSE\]; /\* position values \*/

float intensities\[MI_BPMS_PER_HOUSE\]; /\* intensity values \*/

};

typedef struct MI_FRAME_DATA_RAW

{

MI_DATA_HEADER header;

MI_STATE_DATA state_data; /\* machine state information \*/

int num_detectors;

int frame_number; /\* ordinal number in front-end \*/

int bpm_status\[MI_BPMS_PER_HOUSE\]; /\* status information for

bpms, defined later in this

document. \*/

short i\[MI_BPMS_PER_HOUSE\*2\]; /\* Each channel has an I and Q \*/

short q\[MI_BPMS_PER_HOUSE\*2\]; /\* value, and there are two \*/

/\* channels per BPM. \*/

};

### Turn by Turn Data

The front end will return either an MI_TBT_DATA_SCALED or
MI_TBT_DATA_RAW data structure depending on the type of data requested
for all Turn-by-Turn measurements. All data is big endian.

typedef struct MI_TBT_TURN_SCALED

{

unsigned long turn_number; /\* turn number \*/

float position; /\* position in mm \*/

float intensity; /\* beam intensity \*/

};

typedef struct MI_TBT_TURN_RAW

{

unsigned long turn_number; /\* turn number \*/

short i\[2\]; /\* There is an I and Q value for \*/

short q\[2\]; /\* every channel, 2 channels per \*/

/\* BPM \*/

};

typedef struct MI_TBT_DATA_SCALED

{

MI_DATA_HEADER header;

MI_STATE_DATA state_data;

int status;

int num_turns;

MI_TBT_TURN_SCALED turn_data\[MI_TURN_SCALED_BLOCK_SIZE\];

};

typedef struct MI_TBT_DATA_RAW

{

MI_DATA_HEADER header;

MI_STATE_DATA state_data;

int status;

int num_turns;

MI_TBT_TURN_RAW turn_data\[MI_TURN_RAW_BLOCK_SIZE\];

};

# Interface to BPM Hardware

The front-end software interfaces to the BPM hardware (which consists of
a Timing Generator Fanout Board and one or more EchoTek 814gc boards)
over the VME bus. The front-end software is responsible for configuring
the hardware as well as retrieving data from it over the VME bus.

## Operation of the EchoTek ADC Boards

Each EchoTek board provides 8 receiver channels of 14-bit, 80 MHz
(maximum) analog-to-digital conversion and digital processing in a
single 6U VME slot. The clock for the EchoTeks is supplied by the timing
card. They are clocked a 10/7 times the Main Injector RF clock. They
support 3 operating modes:

1.  Gate Mode -- Data is collected as long as the external sync signal
    is active.

2.  Trigger & Free Run -- Data collection begins at the rising edge of
    the external sync signal, or when the Software Sync Bit is written,
    and continues until the Trigger Clear bit is written.

3.  Trigger & Counted Burst -- A preprogrammed number of samples (burst
    count) is acquired and processed with each occurrence of an external
    sync pulse, or writing to the Software Sync Bit.

In the trigger modes, the external sync signal and trigger a delay
specified in the SYNC_DELAY registers. The SYNC_DELAY registers are 12
bit counters that are clocked at the ADC sample rate. Each register
controls a pair of receivers -- 1 & 2, 3 & 4, 5 & 6, 7 & 8.

The 8 channels on each EchoTek board can be independently configured to
output one of 3 types of data:

1.  Count Data -- The FPGA on board the EchoTek generates a continuous
    counting sequence and puts the data directly into memory. This
    bypasses the whole chain of analog-to-digital conversion and digital
    processing. This mode is intended for checking the data handling
    within the board and check VME interactions between the board and
    system controller.

2.  Raw Data -- The ADC counts are stored into the memory directly,
    bypassing the digital processing. The samples are right justified
    with the two least significant bits always set to zero. Samples are
    stored in pairs to form a 32 bit word.

3.  Receiver Data -- The ADC counts are digitally down-converted,
    decimated and filtered to output an interlaced sequence of 24bit I's
    and Q's. The 24bit I's and Q's can either be truncated to 16bit and
    then packed into 32bit words or directly output as 32bit words.

The memory for each channel is a 128K x 32 bit SRAM operating in a FIFO
fashion.

The procedure to initialize the boards through the driver is as follows:

1.  Read in all the setup files (\*.ini and \*.ch), parse the files and
    then create an array of "ghSetup" structures in the crate
    controller's memory by calling ecdr814gcReadSetup() in the startup
    script.

2.  Install a driver for each board by calling ecdr814gcInstall()

3.  Open the driver by calling open()

4.  Allocate data buffers in the create controller's memory for each
    gray chip channel. The buffers must be 8 bit aligned to the address
    of the corresponding gray chip channel.

5.  Map the allocated buffers to those on the EchoTek boards by calling
    ecdr814gcSetBufferChBrd()

6.  Copy the desired setup from the "ghSetup" array to all the boards by
    calling ecdr814gcCopySetupAll()

7.  Set the channel-pair delays by calling
    ecdr814gcIoctlCopySyncDelayAll()

8.  Program all the Gray chips by calling ecdr814gcProgramGrayAll()

9.  Set up the DAQ conditions by calling ecdr814gcRdSetupAll()

10. Set the trigger counters by calling ecdr814gcSetNumTrigsAll()

11. Reset the boards by calling ecdr814gcIoctlClearAll()

Steps 4 through 11 need to be repeated when changing to a different
operation mode.

After initialization, the boards need to be enabled by calling
ecdr814gcEnableSyncAll(). Then the boards cab be disabled and read out
by calling ecdr814gcReadAll(). These two function calls form a complete
DAQ cycle.

# Calibration

The front-end software is able to return raw data as well as calculated
beam position to the online applications. Raw data does not require any
processing on the front-end whereas calculated beam position requires
the use of calibration constants.

Calibration constants are defined by offline processing. Data used for
calibration will be collected from the front-end systems, running on
calibration mode, and will have its data type marked as calibration
data.

At startup time the front-end downloads current calibration constants.
The calibration constants are retrieved by the front-end system via
ACNET variables. The use of ACNET insures that the front-end
automatically receives the latest calibration set. The calibration set
used by the front-end is identified by a database ID. This ID should be
included in the metadata that is returned when calculated data is
requested by an online application.

# Diagnostics, Test Suite, and Simulation

##  Diagnostics

The front-end software will provide diagnostics data via ACNET devices.
It will provide means for operators or programs to detect a bad or
misbehaving BPM.

Some diagnostics operations follow:

-   Generate closed orbit data and return known closed orbit values

-   Generate turn by turn data and known values for a single turn by
    turn measurement

-   Generate single turn data and return known values for a single turn
    measurement

-   Check BPM hardware -- Run test procedures on the BPM hardware, if
    supported

-   Get Buffers -- Return current contents of all (or selected) data
    buffers

## Self-Testing Procedures

The front-end software should be able to perform tests on itself and on
the associated BPM hardware. Results from self-tests should be available
to user applications. Hardware tests will be performed if supported,
i.e., the hardware should have the capability of receiving triggers from
the front-end and generate data for self-tests. Software self-testing
will be used for validating the data path from the time data is read out
from the BPM until it is ready to be read via ACNET devices.

# Monitoring

The front-end software should periodically send status and statistics
messages to a monitor, via ACNET devices. There should be a central
monitoring application that receives data from all BPM front-ends and
points out BPMs that have problems.

Data from the front-end includes:

-   Buffer Usage

-   Up Time

-   Available Memory

-   Status of processes

-   Number of requests

-   Main Injector Status

# Main Injector States {#main-injector-states .Closing}

  ------------------------------ -------------------- --------- --------------
  **Description**                **TCLK Reset Event** **MI      **Extraction
                                                      State**   Energy**

  Pbar stacking                  \$14, \$29, \$8E,    3         120 Gev
                                 \$80                           

  Pbar stacking cycle to MI      \$14, \$29           3         120 Gev
  abort                                                         

  Protons to Tevatron            \$15, \$2B, \$4D     4         150 Gev

  Tevatron proton cycle to MI    \$15, \$2B           4         150 Gev
  abort                                                         

  Slip stacking and NuMi target  \$14, \$19, \$23,    5         120 Gev
                                 \$8D, \$8E, \$A5               

  Pbar stacking and NuMi target  \$14, \$19, \$23,    7         120 Gev
                                 \$8D, \$8E, \$A5               

  NuMi target cycle to MI abort  \$19, \$23           8         120 Gev

  Pbars from Recycler to         \$2E, \$2A, \$40,    11        150 Gev
  Tevatron                       \$E4                           

  Protons from Tevatron          \$2E, \$2A, \$5D     12        150 Gev

  Slip stacking and SY120        \$14, \$13, \$21,    14        120 Gev
                                 \$80, \$30                     

  Proton Beam from Booster to MI \$13, \$2D           15        8 Gev
  Abort                                                         

  Proton Beam from Booster to    \$16, \$2D, \$93     16        8 Gev
  Accumulator                                                   

  Proton Beam from Booster to    \$16, \$2D, \$85     16        8 Gev
  Debuncher                                                     

  Stacking and SY120             \$14, \$13, \$21,    17        120 Gev
                                 \$80, \$30                     

  Pbar Beam from Recycler to     \$2D, \$E4, \$E0     19        8 Gev
  Recycler                                                      

  Pbar Beam from Accumulator to  \$91, \$2D, \$E0     20        8 Gev
  Recycler                                                      

  NuMi target                    \$19, \$23, \$A5     21        120 Gev

  Pbars from Accumulator to      \$91, \$2E, \$2A,    23        150 Gev
  Tevatron                       \$40                           

  SY120                          \$13, \$21, \$30     24        120 Gev

  Proton Beam from Booster To    \$2D, \$E2           25        8 Gev
  Recycler                                                      

  Proton Beam from Recycler to   \$2D, \$E3           25        8 Gev
  MI abort                                                      

  Pbar slip stacking             \$14, \$29, \$8E,    28        120 Gev
                                 \$80                           

  Tevatron pbar cycle to MI      \$2E, \$1C, \$2A     30        150 Gev
  abort using protons                                           

                                                                
  ------------------------------ -------------------- --------- --------------

Table 1. Main Injector States as of 10/1/05

# Change Log {#change-log .Closing}

  ------------- ------------- -----------------------------------------------
  **Version**   **Issue       **Description Of Change**
                Date**        

  1             11/15/05      Initial Revision

  5             11/17/05      Updated data structures and ACNET devices

  6             12/02/05      Miscellaneous fixes

  7             12/07/05      Added Main Injector Cycles Section

  8             3/24/06       Added signature sheet

  9             4/7/06        Minor wording fixes
  ------------- ------------- -----------------------------------------------

#  References {#references .Closing}

[^1]: Alberto Marchionni et. al., 'Requirements for the Main Injector
    BPM Upgrade',

    Beams-doc-1786

[^2]: Luciano Piccoli, 'MIBPM Front-End Software Memory Requirements
    Review',

    Beams-doc1955

[^3]: Luciano Piccoli, 'Main Injector BPM Software Design',
    Beams-doc-2036

[^4]: Dave Capista, 'MI BPM Configurations for Operational States',
    Beams-doc-1996

    # D. Makers and Concurrence {#d.-makers-and-concurrence .Closing .unnumbered}

    The following persons concur with this document.

    Dave Capista (date)

    Alberto Marchionni (date)

    Steve Wolbers, Project Manager (date)

    Bob Webber, Project Manager (date)