**FVTX Specifications Document**

November 13, 2023

[1 Introduction 2](#introduction)

[2 Geometry and Channel Counts 2](#geometry-and-channel-counts)

[2.1 LDRD 2](#ldrd)

[2.2 FVTX 4](#fvtx)

[3 Sensor Properties 8](#sensor-properties)

[4 Occupancies 8](#occupancies)

[4.1 LDRD 8](#ldrd-1)

[4.2 FVTX 8](#fvtx-1)

[4.3 Data Sizes for Central AuAu 8](#data-sizes-for-central-auau)

[5 Block Diagram 10](#block-diagram)

[6 Front-End Chips 11](#front-end-chips)

[6.1 Input 11](#input)

[6.2 Output 11](#output)

[6.2.1 FPIX 11](#fpix)

[6.2.2 PHX 12](#phx)

[7 Readout Card 12](#readout-card)

[7.1 Input 12](#input-1)

[7.2 Output 12](#output-1)

[7.3 Function 12](#function)

[8 FEM 14](#fem)

[8.1 Input 14](#input-2)

[8.2 Output 14](#output-2)

[8.3 Function 15](#function-1)

[9 DCM 16](#dcm)

[9.1 Input 16](#input-3)

[10 Lvl-1 Requirements 16](#lvl-1-requirements)

[10.1 Data Delivery Specs for Lvl-1 17](#data-delivery-specs-for-lvl-1)

[10.2 Data Segmentation for Lvl-1 17](#data-segmentation-for-lvl-1)

[11 Space for Front-End Electronics
18](#space-for-front-end-electronics)

[12 Radiation Environment for Front-End Electronics
19](#radiation-environment-for-front-end-electronics)

[13 Low Voltage 19](#low-voltage)

[13.1 Power consumption 19](#power-consumption)

[13.1.1 LDRD 19](#ldrd-2)

[13.1.2 FVTX 19](#fvtx-2)

[13.1.3 ROC 19](#roc)

[13.1.4 FEM 20](#fem-1)

[13.2 LV Distribution 20](#lv-distribution)

[14 Sensor Bias Voltages 20](#sensor-bias-voltages)

[15 Calibration System 20](#calibration-system)

[16 Timing Diagrams 20](#timing-diagrams)

[17 Timing and Fast Control 20](#timing-and-fast-control)

[18 ARCNet/Ethernet 22](#arcnetethernet)

[19 Rack Requirements 22](#rack-requirements)

[20 Technical issues 22](#technical-issues)

# Introduction

There are two forward silicon projects that are to be installed into
PHENIX: 1) the LDRD project which puts 4 stations of silicon in the
north side of the vertex region, covering a fraction of azimuth (shown
in Figure 1) and 2) the FVTX project which puts 4 stations of silicon in
both the north and south sides of the vertex region and covering 360
degrees in azimuth (shown in Figure 5). This document attempts to
outline general features of each system and what the electronics
requirements are to read the system out into the PHENIX DAQ.

The LDRD project will use the BTeV-designed FPIX readout chip, bonded to
a BTeV-type sensor **(refs)** and we will need to design the electronics
which interface between the FPIX chip and the PHENIX Data Collection
Modules (DCMs). The FPIX chips provide chip-settable thresholds which
allow the chip to produce zero-suppressed data. The data is delivered
via 1,2,4 or 6 data lines per chip using a data-push architecture which
sends out hits whenever a hit is made which is above threshold.

The FVTX project will use a chip designated the PHX chip which is uses
the same basic architecture as an FPIX chip, but is tailored to the
specific geometry of the FVTX sensors.

The readout electronics that reside between the front-end chips and the
DCMs must buffer data for a number of clocks to allow time for Lvl-1
accepts to be computed and delivered to the electronics and, upon a
Lvl-1 accept, must pull out of the data stream the hits that occurred
during the crossing of interest and package them into a format which is
acceptable to the DCMs.

# Geometry and Channel Counts

This section describes the basic geometry of the LDRD and FVTX projects,
and gives some basic information about channel counts in terms of
readout channels and expected board counts.

## LDRD 

![](media/image1.png){width="4.090944881889763in"
height="4.727165354330709in"}

Figure 1 The silicon barrel region, showing a possible design for the
LDRD silicon planes.

The LDRD silicon detector is to provide four stations of silicon readout
covering a fraction of phi, as indicated in Figure 1. Each station will
be made up of two layers of 8-chip modules, which are based on the BTeV
design **(ref.),** with each layer being made up of twelve 8-chip
modules (**NB: the baseline is 12, but is under investigation**).

An 8 chip module is made up of: a pixel silicon sensor with 50 μm x 400
μm size pixels, 8 FPIX2.1 readout chips bump-bonded to the sensor, and a
High Density Interconnect (HDI) kapton cable which takes care of routing
power, download lines and data lines from the chip and sensor to the
outside world. A picture of an 8-chip module is shown in Figure 2, and a
drawing including dimensions is shown in Figure 3.

![](media/image2.png){width="6.25in" height="1.5in"}

Figure 2 A photograph of an 8-chip module, showing sensor on top, 8
chips underneath, and HDI. The pixels are oriented in this picture with
the 50 μm dimension being in the vertical direction and the 400 μm
dimension being in the horizontal direction.

![](media/image3.wmf){width="6.625in" height="2.675in"}

Figure 3 A schematic of one 8-chip module. Dimensions are in microns.

A first version of an LDRD layer, which is made up of twelve 8-chip
modules, including PC board with I/O connections is shown in Figure
4.![](media/image4.wmf){width="5.989583333333333in"
height="3.2291666666666665in"}

Figure 4 A drawing of one silicon layer for the LDRD project. The yellow
blocks indicate twelve 8-chip modules and the white blocks indicate the
various I/O connectors to power the chips and sensors and to readout the
96 chips. Two of these layers would make up one station for the LDRD
project.

## FVTX

The FVTX and barrel silicon detectors are shown in Figure 5, with a
portion of one arm of the FVTX system circled in red. The FVTX system is
comprised of four layers of silicon at forward rapidity, each covering
360° in azimuth. Twenty four "wedges", each containing four sensors, are
used to make up each of the last three stations, with a wedge shown in
Figure 6. The first station is made up of the same number of wedges, but
each wedge contains 2 sensors rather than 4 as the first station is
smaller in radius than the last three stations. Based on this, you get
forty-eight phi silicon sensor segments making up 360°, each sensor
covering 7.5°. Each sensor on a wedge has two columns of mini-strips
which are 75 μm wide in the radial direction and vary from 3.3 to 12.1
mm in the phi direction, and a row of readout chips runs down the middle
of the sensor. We refer to the combined sensor and readout chips as a
"tower". An example of one of these towers is also shown in Figure 6.
The above segmentation gives us 96 mini-strips in phi at a given radial
position.

The FVTX sensors will be read out with a modified FPIX chip, to be
designated the PHX chip. The design specifications for the PHX chip are
in progress so we will make some guesses as to the chip readout below. A
current version of the PHX specifications document can be found at:
<http://www.phenix.bnl.gov/WWW/publish/brooks/silicon/documents/PHXSpecs_2.doc>
.

![](media/image5.jpeg){width="5.495833333333334in"
height="5.103472222222222in"}

Figure 5 The silicon support structure is shown with one half of the
FVTX (left) about to be inserted into the support system.

Figure 6 A silicon wedge is shown on the left, which is comprised of 4
silicon sensors: 2 front and 2 back covering two different phi positions
and two different radial positions. There are 24 wedges making up 360
degrees in phi. On the right is shown a single silicon sensor (or tower)
which is comprised of two columns of silicon strips which are 75 μm wide
in the radial direction and vary from 2.2 mm to 13 mm in the phi
direction and one row of readout chips down the center of the center.
There are 48 towers making up 360 degrees in phi.

Each of the silicon towers will need to be provided with power, chip
download and calibration lines, and have the data sent ultimately to the
DCMs in PHENIX. The rest of this document outlines a proposed block
diagram that will provide these functions, the required I/O for each of
the pieces, the functions that each piece must provide, and the
technical decisions that must be made before production of a system.

Table 1 gives expected channel and board counts for the FVTX and LDRD
systems based on the above geometries and the electronics readout scheme
which is outlined in the rest of this document. Note, this is a draft
and numbers are subject to change.

![](media/image7.wmf)

Table 1 Some electronics counts for the FVTX and LDRD systems.

# Sensor Properties

Put in sensor-types, electron/hole collection, bias voltages required
(or in sensor bias section)...

The LDRD silicon sensor is based on the BTeV design. The sensors are 250
μm thick n+/n/p type with pixels on the n+ side. Electrons are collected
and the bias voltage required for the sensors is negative. It has pixels
which are 50 x 400 μm.

# Occupancies 

This section lists occupancies expected in the LDRD and FVTX detector
systems, especially for central AuAu events which will be the most
difficult events for the readout system to handle.

## LDRD

For the LDRD detector we calculate from GEANT simulations that there
will be 0.14% track occupancy at the inner-most radii in Central AuAu
collisions (**put in centrality definition here from the simulations)**.
The average cluster size at this same inner radii is 1.3. We calculate
the worst-case data sizes based on these numbers and adding a factor of
2 safety margin. Therefore, we would expect the chips at the inner-most
radii to have to deliver up to (0.0014)\*(2)\*(1.3 strips/track)\*(2816
channels/chip) = 10 hits/chip in Central AuAu. This is the worst-case
data rate because the increased number of strips that are hit as you
move to outer radii changes less quickly than the track occupancy,
resulting in a channel occupancy which slightly decreases as you move to
outer radii.

## FVTX

For the FVTX detector we calculate from GEANT simulations that there
will be 2.8% track occupancy in Central AuAu collisions, taking into
account cluster sizes already. The PHX chip is currently in the process
of being specified but if we assume 256 channels per chip then the data
to be delivered would be up to (0.028)\*(2)\*(256 channels/chip) = 14
hits/chip in Central AuAu. This is similar enough to the data rates from
the LDRD project that for now we will specify the details of the LDRD
and assume that the FVTX project will be able to fit into the same
readout scheme since we have the flexibility of designing the PHX chip
in a way that will assure that this is true.

## Data Sizes for Central AuAu

![](media/image8.wmf)

Table 2 Calculations of data sizes and readout rates for the FVTX
detector with various assumptions on data lines coming out of the chip,
number of chips that are ganged together for one FPGA, etc. A detailed
descript of the table is given in the text.

**Table 2** gives a number of calculations for the FVTX detector for
central AuAu events, making an assumption on noise levels (as being
high, to be conservative), number of data lines per chip and number of
chips that are ganged together to go to one FEM. The purpose of the
table was initially to estimate memory sizes needed for an FPGA which
would buffer data and extract events for the DCMs, but includes a few
other useful numbers as well. The table information is as follows:

-   **Layers ganged** indicates the number of layers that might be
    ganged together to go to one FEM. In the rest of the table, we are
    currently assuming 256 channels/PHX chip (but it is as yet
    undefined)

-   **Columns 2,3,4** propagate the current assumption that we have 256
    channels/PHX and the number of layers ganged together into
    chips/board and channels/board

-   **Occupancy** gives the central AuAu per strip occupancy for
    vertical layers (clustering taken into account) and a safety factor
    of \*2.

-   **Strips/Hit** is a placeholder for cluster size, but it has been
    taken into account in Occupancy

-   **Interactions/64 clocks** indicates that in AuAu we only expect one
    interaction within 64 clocks of buffer time

-   **Real Hits/64 Clocks** gives you the total hits expected in a
    central AuAu event for the chips that are ganged together, based on
    the previous columns

-   **Real data size/64** clocks assumes 24 bits/hit

-   **Noise** is a very high-end estimate of what noise might do to you
    if you had 0.1% noisy channels.

-   **Clocks** is the number of clocks for which we need to buffer data
    in the FEM (currently an upper-end estimate of 64 clocks is used)

-   **Noise hits/clock** is cacluated cumulative noise for 0.1% noise
    rate for 64 clocks.

-   **Noise Data Size** assumes 24 bits/hit

-   **Buffer needed for 64 clocks** adds noise hits and real data hits
    for 64 clocks. Even in this very conservative estimate a very modest
    buffer size of only up to 36 kbits is needed.

-   **Number of readout lines** (1,2,4,6) is number of parallel readout
    lines that are assumed per chip. The more the lines, the more data
    can be pumped out in a given amount of time.

-   **Readout Time/Data word** assumes that you will sync the readout
    clock at some level with the beam clock so that you get an integral
    number of words out per clock (not necessary but a starting
    assumption). Based on this, you would pick slightly different clock
    speeds depending on the number of data lines out. The clock speeds
    assumed are given in the last column and are picked to be \~\<= 140
    Mbps since 140 Mbps has been verified to work well.

-   **Readout Latency** calculates how many beam clocks it will take to
    get one chip's worth of data out from a central AuAu event for 1,2,4
    or 6 data lines out per chip. Note that from these conservative
    estimates you would pick 4 or 6 data lines per chip if you wanted to
    get all the data out within close to 4 beam clocks.

-   **Noise hits/chip needed to fall behind** is the number of sustained
    hits you need per chip for the data coming out of the chip to equal
    the rate of data produced. Any more than this would cause the output
    data stream to not keep up with data generation.

![](media/image9.wmf)

![](media/image10.wmf)

Table 3 The same data rate calculations as Table 2, but for the inner
portion of the LDRD detector.

# Block Diagram

A proposed block diagram for the FVTX/LDRD silicon readout is shown in
Figure 7. The diagram is composed of 4 parts: 1) the front-end readout
chip which contains the preamp, ADC and digital output and will be
bonded to the silicon sensors, 2) the readout card which will be located
near the silicon sensors and will reduce the data size and do some
modest packaging, 3) the FEM which is proposed to be located in the
counting house and will package data for the DCMs, and 4) the PHENIX
DCMs and Lvl-1 boards which will receive data from the FEM and will also
reside in the counting house.

Figure 7 An electrical block diagram for the FVTX/LDRD silicon system,
going from the front-end readout chip to the DCMs:

The rest of this document will describe the I/O and functional
requirements for each board and also includes sections on support
systems such as LV, sensor bias, T&FC, slow controls.

# Front-End Chips

## Input

6 pairs of serial lines per module for downloading chip settings. The 6
lines are comprised of: two clocks, control data in, control data
readback, programming enable, and reset.

## Output

### FPIX

The FPIX chip can have 1, 2, 4 or 6 LVDS pairs of serial lines per chip
for transmitting data, each of which nominally run at 140 Mbps. Our
baseline proposal is to use 4 data lines per chip. In addition, one
clock line (pair) per chip will be available. The data stream that comes
out of the FPIX chip consists of 24 bit words for every hit that is
above threshold (which is set per chip) as well as "synch words" which
come out whenever there are no real data hits being produced by the
chip. The synch words have a fixed bit pattern and allow readout systems
to determine where bit 0 is within a data word and... The 24 bits per
hit includes a beam clock number (7 bits), a row (5 bits) and column (8
bits) to indicate which channel was hit within the chip, 3 bits of ADC
information, and a end of word bit.

### PHX

Please see PHX specifications document for current state of PHX design.
(<http://www.phenix.bnl.gov/WWW/publish/brooks/silicon/documents/PHXSpecs_2.doc>
)

### Chip Download and Readback

The input lines to the FPIX chip allow for serial download and readback
of chip settings including: thresholds (one set of eight values value
per chip), kill and inject masks for each of the channels on a chip, BCO
synchronization, hardware resets...

The downloads are accomplished by setting the shift control line high
and sending the serial command in. The serial command includes 5 bits
for the chip address, a 5 bit register, a 3 bit instruction and a data
string for the WRITE command. You can optionally send a wild-card for
the chip address and to send the same command to all chips.

# Readout Card

The readout card is to interface between the front end chips and the
FEM. We describe here the various I/O associated with the readout card,
the functions it must provide, and then a proposed block diagram for the
card.

## Input

-   Data lines from PHX, FPIX chips

-   One clock line (pair) per chip

-   Serial readback from PHX, FPIX chips (?)

-   Calibration control from FEM

-   Serial download stream from FEM, to be passed to chips

-   FPGA JTAG data from FEM, ARCNet/Ethernet data for reprogramming FPGA

-   T&FC information for control of calibration system (to trigger the
    DAC)

-   LV and HV to distribute to chips and sensors?

## Output

-   Serial data stream (fiber) out to FEM. Stream is to contain info
    outlined in 7.3. Include status word to indicate results of error
    checking?

-   Data stream out should also have inserted into it slow-controls
    information as needed: current, voltage, temperature monitoring,
    etc. Or should it be in dedicated slow-controls data stream?

-   Serial download stream, to chips

-   Calibration pulser to chips, coming from on-board DAC

-   LV and HV to chips and sensors?

## Function

The readout card must provide the following functions:

-   Deserialize data coming from the N number of FPIX/PHX chips

-   Strip off synch words that are in the data stream

-   Arrange data words and pass them on to the FEM in such a way that
    the following information is maintained or added:

    -   beam crossing number (at least 7 bits to cover the 120 crossings
        covered by the GTM ??? or at least 6 bits to cover the 64-deep
        buffer we must have ???)

    -   pixel/strip location within chip

    -   chip ID

    -   ADC bits (3 bits)

-   Pass download strings to FPIX/PHX (and do readback)

-   Provide calibration pulses from on-board DAC. Control coming from
    FEM, coming from ARCNet/Ethernet (for setting of pulse heights), and
    trigger coming from T&FC.

-   Should also have on-board memory to download chips with default
    configurations (specify size of memory needed)

Based on the above specifications, we have produced the following block
diagram for the ROC:

![](media/image11.wmf){width="5.997222222222222in"
height="3.9694444444444446in"}

Figure 8 A block diagram for the Read-Out Controller Board.

The ROC consists of:

-   an FPGA to manage the detector and slow controls data streams

-   a serdes (here represented as integral to the FPGA)

-   fiber drivers to send the data stream out to the FEM

-   a fiber transceiver to manage the slow controls data.

-   An on-board DAC to provide calibration pulses to the front-end chips

-   On-board memory for downloading default configurations

-   ...

## Serdes Calculations

**The number of serdes** channels required per ROC can be calculated
based on the expected maximum occupancy, the desire to get all data out
to the Lvl-1 board within 4 beam clocks (4\*106 ns = 424 ns) (see
section on Lvl-1 requirements) an assumption that the output serializer
lines will run at some particular speed, and some assumption of the data
format transmitted out of the ROC. We outline here a couple of possible
data output formats and what the implications would be for numbers of
serdes channels.

**Format 1: retain 24 bits/word**

From Section 4, we expect the occupancies in the FVTX chips (for the
LDRD) to be 10 hits/chip in central AuAu events (taking into account a
safety factor of \*2). If we retain 24 bit words then this produces 240
bits/chip (plus a little for chip ID). If we want to get the data out in
four beam clocks (424 ns), then we need to put the data out at (240
bits/chip)/(424 x 10-9) = 0.566 Gb/sec/chip. If the serdes output lines
are running at 6.375 Gb/sec (based on internal Altera serdes) this means
we can serialize data from 6.375/0.566 = **11 readout chips/serdes**.
(Note a few more bits would be needed to add chip ID to each block of
data) If we have 96 chips/ROC then we need **9 channels of serdes and
each serdes would be handling 44 bits.** It would probably make most
sense to take a little extra delay and send out 48 bits or 2 data words
per serdes. This could be done with 8 channels of serdes.

**Slower serdes speeds:** If you were to run the serdes at **2.5 Gbps**
rather than 6.375 Gbps then the above calculations would give **24
channels of serdes, each serdes handling 16 bits**.

+--------------+-------------+----------------------+-----------------+
| **Serdes     | **Data      | **Number Serdes      | **Bits per      |
| Speed**      | Format**    | Channels**           | Serdes**        |
|              |             |                      |                 |
| **(Gbps)**   |             |                      |                 |
+--------------+-------------+----------------------+-----------------+
| 6.375        | full        | 9 (8)                | 44 (48)         |
+--------------+-------------+----------------------+-----------------+
| 2.5          | full        | 24                   | 16              |
+--------------+-------------+----------------------+-----------------+
| 1.6          | full        | 48                   | 8               |
+--------------+-------------+----------------------+-----------------+
| 6.375        | Move BCO    | 6                    | 64 bits         |
+--------------+-------------+----------------------+-----------------+
| 2.5          | Move BCO    | 16                   | 24              |
+--------------+-------------+----------------------+-----------------+
| 1.6          | Move BCO    | 24                   | 16              |
+--------------+-------------+----------------------+-----------------+

Table 4 Calculations of how man serdes channels are required, and the
number of bits per serdes, depending upon the serdes speed and the
assumption that you want to get all data out within four beam clocks.

**Format 2: move BCO and chip ID into header**

If we move the common BCO and chip ID into header words we can reduce
the data size to ship out. In this case we have in the header(s): BCO =
7 bits, ChipID = 4 bits (depends on number of chips). The data per chip
would be (10 hits)\*(12 bits channel ID + 3 bits ADC) = 150 bits. Total
data size = 11+150 = 161 bits. Data rate out per chip needs to be (161
bits)/(424 x 10-9) = 0.38 Gb/sec/chip. This gives 6.375/0.38 = **16
readout chips/serdes.** If we have 96 chips/ROC then we need **6
channels of serdes**.

**Format 3: smarter encoding of channel ID ?**

**Delay to check for synch words:** If we are to strip off synch words,
then a word from a given chip can not be shipped to the FEM until all of
the 24 bits have been received at the ROC. If we have 4 data lines per
chip, each line running at 140 MBps, then it will take (24 bits)/(4
lines per word)/(140 Mbps) = 43 nsec to receive a given word. At this
point, all bits associated with this particular word cycle could be
shipped to the FEM, but then you would need to wait an additional 43
nsec before shipping the next word. If we use the above number of serdes
then we will not incur anymore than the total needed delay of (10
words/chip)\*(43 ns/word) = 430 nsec.

# FEM

The FEM must provide the interface between the ROC and the DCMs and
Lvl-1 boards. It must buffer data while waiting to see if there is a
Lvl-1 accept for a given beam crossing and upon a Lvl-1 accept extract
the hits from the crossing of interest and package it to send to the
DCMs. PHENIX nominally allows 40 beam clocks for Lvl-1 to make a
decision, but currently delivers the decision within approximately 12
clocks. The FEM must also be capable of processing up to 5 successive
Lvl-1 accepts. This means we must be capable of buffering data for at
least 40 beam clocks plus any additional time required to sort and
deliver 5 successive events. On the other hand, if we are to fit within
the current Lvl-1 latency then we should deliver the data to the FEM
within 12 beam clocks???

In addition we expect the FEM to act as a master controller for various
slow-controls data including the downloads to and readbacks from the
front-end chips, the download of calibration constants, the download to
the FPGAs, any front-end board and chip monitoring information etc.

We describe here the various I/O associated with the FEM, the functions
it must provide, and a proposed block-diagram for the board.

## Input

-   Serial stream of data from ROC (fiber) (As outlined in ROC section,
    input might included slow controls information as well as data)

-   Readback from FPIX/PHX chips (passed through ROC) (?)

-   T+FC (beam clock, Lvl-1 accept)

-   ARCNet/Ethernet to pass:

    -   Slow control info to FPGAs on FEM and ROC

    -   Initialization stream to go to readout chips

    -   Control information to calibration system on ROC

## Output

-   Chip download stream, from ARCNet or Ethernet (to ROC)

-   Packaged data to DCMs

-   Readback from chips (coming from ROC and going to outside world)

-   Split input data stream and send to Lvl-1 board(s)

-   Strip off slow-controls info from ROC and send to slow-controls data
    stream

## Function

-   Buffer data

-   Grab data from event of interest upon a Lvl-1 accept

-   Build packet for DCM and ship to DCM

-   If data being collected is calibration data, tag the packets as such
    before shipping to DCM

-   Pass download stream to readout chips

-   Receive readback stream from chips (for debugging)

-   Pass calibration control to ROC (coming to FEM via ARCNet/Ethernet)

Based on the above specifications we have produced the following block
diagram for the FEM:

![](media/image12.png){width="5.444444444444445in"
height="4.198611111111111in"}

Figure 9 A block diagram for the Front-End Module Card.

The FEM is comprised of:

-   a fiber receiver to receive the data coming from the ROC

-   a transceiver to handle slow controls communication with the ROC

-   an FPGA to manage the detector and slow controls data stream

-   on-board memory for downloading default configurations

-   a fiber driver to send packaged data to the DCMs

-   circuitry to interface to T&FC and Ethernet for slow controls.

-   In addition, the input data stream should be split off to go to
    Lvl-1 boards.

-   ...

# DCM

The DCMs are common throughout PHENIX. We expect that for both the LDRD
and the FVTX projects we will be using the new generation **(ref)** of
DCMs being produced by Nevis. Below we describe the input expected by
the DCM and the constraints that the DCM puts on our data stream.

## Input 

The input DCM package must contain the PHENIX standard header and
footer. Details for rest to be worked out. The data words contained in
the PHENIX standard DCM packets are shown here:

0 start marker

1 detector id

2 event number

3 module address

4 flag word

5 beam clock counter

6 channel number, adc value

... ...

N-1 longitudinal parity word

N summary word

The maximum data size that should be delivered to the DCMs I get from
Chi:

"The new FEM to DCM link will most likely to run at 80MHzX 16 bits
(20bit after encoding) mode. Conservatively you should use the 40MHz 16
bits to calculate the **maximum event size per link**. So the number is
50 microsec/ 25ns = **2000 16 bits words per event per link."**

# Lvl-1 Requirements

We expect both the LDRD and FVTX detector systems to provide data for
Lvl-1 boards that are to be designed by Iowa State University. For the
LDRD detector, the data is expected to primarily be used to test out the
Lvl-1 system and for the FVTX detector we expect the Lvl-1 to provide
critical rejection power in high luminosity running. It is not clear
whether the FVTX system will be able to provide trigger rejection in all
collision systems, but for now we attempt to meet the goals of having
the FVTX system able to provide data for Lvl-1 for all systems,
including Central AuAu collisions. These central AuAu collisions will
provide the most stringent requirements on the electronics, and have
been taken into account in calculations above.

## Data Delivery Specs for Lvl-1

We set as a goal to deliver the data to Lvl-1 within four beam clocks
based on the following information from John Lajoie:

Here are some notes/thoughts on the LL1 requirements for the FVTX/LDRD
detectors.

The LL1 latency that we are currently running is about twelve clock
ticks (beam clocks). This is the time after a crossing of interest
occurs that the LL1 accept shows up at the FEMs. The spec is a maximum
of forty, but we haven\'t needed a latency that large. We could go to a
longer latency, if necessary, but each subsystem would have to shift
their timing and then check it with data. Not impossible, but relatively
disruptive.

Of the twelve clock ticks, it breaks down roughly as follows:

1)  one clock for data transmission of LL1

2)  seven clock ticks for LL1 decision (dominated by BBC LL1, which
    takes seven clock ticks. The MuID LL1, a newer system with orders of
    magnitude more data, takes three.)

3)  three clock ticks for GL1 decision

4)  one clock tick for accept flyback to FEMs.\
    \
    (This is all good to ±1 one clock tick.)

The relevant number of the FVTX/LDRD is (2), the maximum LL1 latency,
which is currently at seven clocks. This needs to include the time delay
to get the data from the FEM to LL1, or the time we need to wait for all
the chips to \"push\" their data up.  **If we plan on 4-5 clock ticks to
get the data from the detector** that leaves us with 2-3 clock ticks to
complete the calculation and get the decision data up to GL1. (Note that
I have already included the one clock tick to get the data from the FEM
to LL1 in (1) above.)

Based on the STTR Phase-1 studies, with enough logic in parallel we
believe the we can complete the LL1 calculation in 1-2 clock ticks, so
let\'s say 2-3 in case we need to buffer an output stage to get data to
GL1. Based on this analysis, everything sort of \"fits\" to within a
clock tick.

## Data Segmentation for Lvl-1

Input from John Lajoie on segmentation of data going to Lvl-1 boards:

LL1 needs the data to come to it on an octant (or fraction of an octant)
basis, since the LL1 algorithm needs to get a segment of each plane in
order to do its job. Let me emphasize that there is way too much data to
be cross-stitching between LL1 trigger processors so we do need to get
the data from the FEMs in some rational manner.\
\
So, for example, what we would like to see would be each fiber coming to
LL1 be data from some phi segment of a single plane. Four fibers (one
from each plane) would give sufficient data to process triggers for that
phi segment.

# Space for Front-End Electronics

Figure 10 shows a cutaway of the barrel and forward silicon layers. The
space(s) where electronics might reside include (1) the space between
the forward four silicon stations, which gives 3 areas which cover
approximately 5.5 cm in z and 360 degrees in phi and (2) at the back-end
of the full silicon system. At the back of the silicon systems (one
shown to the right in this picture) there is 8 cm of space in z and
approximately 360 degrees in phi which can be used to house electronics,
but must be shared among the barrel pixel, barrel strip, and forward
silicon systems. The radial extent of the electronics area that is open
for use is approximately 45 cm. **(?? Check this ??)** It is in one of
these areas that we must consider housing the ROC, along with any
support structure and cooling that might be required for the ROC.

![](media/image13.jpeg){width="6.875in" height="5.898611111111111in"}

Figure 10 A cutaway view of the 4 barrel silicon layers and the one arm
of 4 forward silicon layers. Part of an annular space covering
approximately 360 degrees, where silicon electronics can reside, can be
seen at the top right.

# Radiation Environment for Front-End Electronics

The estimated dose at r=10cm for 10years of running at RHIC II
luminosity is 400-600 kRad, based on measurements taking during Run 6
with chipmunks at r=36 and 66 cm. Please see
<https://www.phenix.bnl.gov/cdsagenda//askArchive.php?base=agenda&categ=a06203&id=a06203s1t2/moreinfo>
for details.

# Low Voltage 

\*\*\*This all needs to be checked (just from memory here)\*\*\*

## Power consumption

### LDRD 

**FPIX**

The FPIX2.1 chip requires 2.5 V analog and digital power. The power
consumption is approximately 90μW/channel. This means that (2162688
channels)\*(90x10-6 W) = 192 Watts of power must be provided to 768
chips for the LDRD system.

### FVTX

**PHX**

### ROC

Voltage(s) required:

Power Consumption:

FPGA+Fiber Drivers+...

![](media/image14.wmf){width="5.998611111111111in"
height="3.4756944444444446in"}

### FEM

Voltage(s) required:

Power Consumption: FPGA + fiber drivers + ...

## LV Distribution

To boards, to front-end chips.

# Sensor Bias Voltages

Requirements and distribution.

# Calibration System

A calibration system will be required to pulse the various channels of
the FPIX and PHX chips. The pulsing will be provided by a DAC on board
the ROC and the control of the calibration system will be passesd via
Ethernet through the FEM to the ROC.

# Timing Diagrams

# Timing and Fast Control

The Timing and Fast Control (T&FC) is provided to the FEMs by the
Granule Timing Module (GTM). The T&FC provides the FEMs with two clocks:
Beam Clock and 4 times Beam Clock, information of whether there was a
Lvl-1 accept for a given crossing, and mode bits (which can tag a
crossing as a particular type of event) for each crossing of the 120
between RHIC fiducials. The mode bits are used, among other things, for
tagging events which are calibration events.

An introduction to the GTM is copied here from the web document (
<http://www.phenix.bnl.gov/phenix/project_info/electronics/timing/tc/current_tc.html>
)

This document describes the Phenix Granule Timing Module. The Granule
Timing Module, which resides on a single width 9U VME-P board, provides
distribution of the timing information to the front end modules (FEM)
for the Phenix Detector. The timing module performs three primary
functions. First, it is a mode bit scheduler, which outputs
predetermined mode commands on a clock by clock basis to the FEM\'s.
Second, it provides readout enable strobes to the FEM\'s, and monitors
the number of level-1-accepts generated by the level1 system. Third, it
provides low jitter distribution and generation of timing signals,
namely the Beam Clock, Beam ClockX4 (generated on board via PLL), and
LVL-1 Accept. The module is interfaced via the VME bus. The block
diagram showing all inputs and outputs from the module is shown in
figure 1.

![](media/image15.png){width="7.34375in" height="5.25in"}\
*figure 1. Granule Timing Module Block Diagram*

We will require at least the following functions to be provide by the
T&FC inputs:

-   The Lvl-1 accept information will be required by the FEM so that it
    can extract from the data stream the hits of interest.

-   Beam clock distribution can originate from the T&FC.

-   The Mode Bits will be required to tag events which are coincident
    with a calibration pulse.

-   ...

# ARCNet/Ethernet 

# Rack Requirements

# Technical issues 

from Dave Winter:

so i can attempt to start this list with the following tasks that we
need to pursue:\
\
- shakedown our understanding of the fpix readout\
- build test setup for serialization/optical translation of fpix
communications\
\-- development of knapp readout controller (this is the board he
brought to fnal, but modified to output fiber signals, among other
changes)\
\-- development of knapp fem (this is the pci board to communicate with
knapp RC that anuj has seen)\
\-- write the code to drive the knapp fem and its testing\
\-- write/maintain the pld logic of both boards\
\-- develop communication protocol knapp btn RC and FEM\
\-- converge on what specific chips would be used in final prototype,
and plan to build boards with them\
- investigate what would be necessary to implement reprogramming of the
RC plds and associated eproms (absolutely necessary if they are to be
located down on the detector)\
- simulations that are realistically coupled to the specs of the readout
that we are developing

My own list:

**FPIX/PHX**

1)  How does chip behave under bad conditions:

    a.  noisy channels, a large amount of data at once, higher sustained
        data rates than typically expected

2)  Continued checking of all functionality (FNAL, Columbia work
    already)

3)  Any real data/telescope data analyses needed? Look at noise,
    resolution, clustering... with real data

4)  Channel-to-channel variations in threshold, gain, offsets...

5)  Cross-talk under various conditions

**ROC**

Technical questions:

1)  Any issues with multiple lines/chip, multiple chips; any issues with
    power consumption or speed in general if you need to sample at 4\*
    data rate...

2)  What segmentation (how many chips to gang together): how many lines
    can be handled, failure analysis, lvl-1 needs/desires...

3)  Copper or fiber out: how to decide?

4)  Format of data out

5)  FPGA JTAG data from where? Presumably from FEM rather than directly
    from ARCNet/Ethernet.

6)  How to pass download data to chips from FEM. Direct pass-through or
    something else?

7)  Spacephysical layout of board(s) and location

8)  Any SEU, radiation environment issues?

9)  Power consumptions placement and cooling

10) Optimizing segmentation: matching well to lvl-1, matching number of
    serdes channels?, failure-rate analysis, keeping I/O lines per board
    manageable, cost/board\*number of boards,

Proposed starting point for above questions:

1)  Clock will be provided by chip. Can decide clockless or not at a
    somewhat later date

2)  For LDRD, try "maximum configuration": one full ½ plane = 96 chips
    ganged together, and investigate issues

3)  Fiber as baseline

4)  ...

5)  ...

6)  ...

7)  Just to basic checks to look for show-stoppers right now. Look at
    "collar" around planes or back of enclosure.

8)  ...

9)  ...

**FEM**

1)  Back-end defined, so just a matter of making block-diagram for FPGA
    code and working on it ???

2)  Download to chip all handled here? How do we pass lines to each
    chip?

3)  Splitting data in to go to Lvl-1

4)  How to handle FPGA reprogramming for this board and passage to ROC?

5)  How to handle download to chips, passing to ROC.

6)  Passage of clocks from T+FC to ROC?

**General:**

-   Calibration handling

-   Power handling

-   Clock handling, what speed to run various clocks

-   Monitoring of chips/boards: temperature, voltage, current,

-   Debugging options to find source of hardware problems?

-   Error handling: check chip data looks reasonable, check FPGA
    configuration, ... Do we want to have any resets included with error
    handling or just pass status words?

Test Boards Required ???? Just place-holders right now, not real
proposals

-   Clockless readout: receive multiple lines/chip and multiple chips,
    FPGA on board. What all do we want to check on data out? Same data
    as with using clock? Just check synch words?\...

-   ROC: Add serdes to board

-   FEM: FPGA + serdes to receive ROC input. Add receiving of T+FC
    (lvl-1 accept, beam clock), fiber output for DCM,

Test DAQ Developments (working toward PHENIX DAQ): ??? again, just
place-holders

-   Chip exercising can use FNAL setup: new test bench, Uffizi...

-   ROC: need to read from serdes

-   FEM: initially need to just read DCM packets. Add trigger at some
    point (right off the bat?). Add real DCM. Add real GTM...