**Pierre Auger Project Electronics**

**Front End (FEv2.x) Board**

**Kwame Bowie**

September 1, 2000

##### Revised: 6/13/2001

1.  **Introduction**

The Front End (FE) board is the digitization and triggering board for
the Surface Detector Electronics system. The Surface Detector
Electronics system consists of a CPU board (CPU), a Slow Controls board
(SC), a Time Tagging board (TG), an Ethernet board (ETH), a GPS receiver
(GPS), and the Front End (FE) board. The FE board is directly linked via
the Local Bus to the all of the subsections. The block diagram Fig. 1
shows the basic layout.

Figure 1: Surface Detector Electronics Interconnections

2.  **Operation**

> The FE version 2 board consists of the FEv2.x board developed at
> Fermilab and the Surface Detector Trigger granddaughter-board,
> developed by Michigan Technological University (MTU). Together these
> two boards function as the digitization board, the data storage board,
> and the trigger generation board for the Surface Detector. The main
> functions of the board are listed below:

-   FE Board must take 6 channels of analog data from photo multiplier
    tubes and convert it to digital data.

-   FE Board must store this digital data in memory deep enough to hold
    25.6 μs of data (6.4 μs pre-trigger and 19.2 μs post-trigger).

-   FE Board must transmit this data to the CPU based upon trigger
    conditions.

The following are the specifics of the board operation based upon the
Surface Detector and Front End board specification.

1)  The analog to digital data conversion (ADC) is to be done at a
    sampling rate of 40MHz and with a precision of 10 bits per channel.

2)  There will be six (6) channels of analog inputs from the photo
    multiplier tubes (PMTs). The digitized data from these channels
    shall be used generate the trigger(s).

3)  The digitized data shall be stored onboard and read out at known
    intervals after trigger generation.

4)  The stored data shall be comprised of ¼ pre-trigger data and ¾
    post-trigger data.

5)  There will be two (2) types of triggers: fast and slow based upon
    amplitude, slew rate, and time over threshold measurements.

6)  The memory will be double buffered so that an event may be accepted
    while data is being read out.

7)  Data readout will comply with the PPC403 CPU specification for DMA
    transfer in Fly-By-Burst Mode.

8)  The FE board must be air-cooled with no fan.

9)  The board must operate at temperatures between --20°C and +60°C.

To ensure that prototypes were available for initial testing, two (2)
versions of the FE board have been produced: Version1 and Version 2. All
Front End Boards receive the following nomenclature: FevM.N where M is
the version number and N is the design iteration. This nomenclature is
used when referring to the different versions of the FE board from this
point on.

### 2.1 Version 1.x

The first version, Version 1.x, of the FE board consists of the FE
motherboard and an analog daughter card. FEv1.x supports the use of a
digital daughter card by providing a connector and signal paths for a
digital daughter card. The digital daughter card was to be developed in
parallel at a different location, but that option was not pursued. The
digital daughter card was to produce a digital trigger signal, which
could be logically OR'ed with two other triggers (Analog and External)
to control data flow into the buffers.

To provide maximum flexibility in available configurations, the FEv1.x
board design was split into three (3) separate boards: the FE
motherboard, the Analog Daughter Card, and the Digital Daughter Card. By
separating the FE Board design into distinct parts, several engineers
could work on various portions of the board in parallel. This has two
desirable effects: first by making distinct parts on separate boards, a
delay or design change on one board will have minimal effect on the
others as the design processes become independent. Second, by separating
the boards, performance analysis may be done with different FE
configurations. It becomes possible to have several interchangeable
analog front ends or digital daughter boards and evaluate the
performance of each in the actual system to get a true performance
metric.

### 2.2 Version 2.x

Version 2.x of the Front End board will have the same physical
dimensions and stacking- connector positioning, but will eliminate the
need for first-in-first-out memory chips (FIFOs) or a PLD. In place of
the FIFOs and PLD will be a PGA socket for an ASIC designed to integrate
all of the digital logic into a single chip. As a result, the digital
daughter card connector along with all digital logic will be gone from
Version 2.x. Also, the ASIC will handle a more advanced triggering
scheme, which will be based solely upon the digitized data so no analog
trigger will be present.

3.  **Input signals**

Input signals to the FE board originate from three (3) PMTs. Each PMT
outputs two (2) analog signals. The three PMTs are set up to have very
different output gains. To simplify nomenclature we simply call the two
PMTs high gain and low gain. The PMT outputs appear at the FE board as
six (6) single-ended, negative-edged signals on 50-Ohm cables from the
PMT bases. The cables are terminated at high frequency SMA sockets. The
high gain channels have an input voltage range of 0 to --0.3V and the
low gain signals have a voltage span from 0 to --1.5V.

-   6 single-ended signals

-   50 Ohm cable impedance from bases

-   High gain range (0 to --0.3V)

-   Low gain range (0 to --1.5V)

The FE board has the ability to handle the 6 input signals in either
single ended form or in a pseudo-differential form via jumper selection.
The pseudo-differential method provides the ideal solution in low-noise
environments as it eliminates possible ground offsets between the FE
board and the PMT bases. To connect with the coax cable, we have
selected standard, right-angle, PCB-mount, SMA connectors AMP part #:
221790-1.

> **4. Shaping/Amplification/Filtering**
>
> **4.1 Overvoltage protection**

Before any filtering, shaping, or amplification is done, the amplifiers
must be protected from over-voltage conditions. This actually serves 2
purposes: first, it prevents large voltage spikes from exceeding the
maximum op-amp voltage ratings. Reliability will certainly be a major
concern in this project as the stations are predicted to be serviceable
only at very long intervals. Replacing op-amps after any large event or
noise spike is not an option. The second reason for including surge
protection on the analog input signals is to lower the power consumption
within the analog portion of the FE board. To achieve the over-voltage
protection goal, we use series-connected dual pin diodes. For the high
gain channels we only use 2 per channel since we can clamp after one
diode drop since the signal is limited to 300mV. However, for the low
gain channels, the signal range is much larger (1.5 V) so we need at
least 3 diode drops in the signal direction before we clamp the signal.
So for the low gain channels, each channel receives 3 dual pin diodes.
The pin diodes that were initially selected for clamping were the
Agilent HSMP3822 series connected dual pin diodes. However, due to
availability and pricing, a replacement had to be selected. The
replacement part that was chosen was part number MA4P274ST from
Richardson Electronics. The only important parameters when selecting the
replacement parts were price, effective series resistance, and effective
capacitance. All of these parameters were deemed acceptable for the
Richardson replacement part. The input circuits are shown below in
Figures 4.1 and 4.2.

![](media/image1.wmf){width="5.051388888888889in" height="2.4375in"}

**Figure 4.1: High gain channel input circuit**

![](media/image2.wmf){width="4.560416666666667in"
height="2.3805555555555555in"}

**Figure 4.2: Low gain channel input circuit**

**4.2 Amplification**

The very next step after surge protecting the inputs is to amplify the
inputs. Amplification is necessary to enable the analog inputs to match
the ADC input range. The ADC device that we have selected has a 0 to 1V
input span, so the inputs must be amplified accordingly. The term
amplification is used as a generalization of the operations performed on
the inputs. The operations are actually polarity inversions,
amplifications, and/or attenuations. All three of these terms may be
generalized and considered amplifications by a gain factor. The sign and
absolute value of this gain factor determines whether the amplification
is a polarity inversion, an amplification, or an attenuation. In our
case, all high gain channels have a gain factor of -10/3 and all low
gain channels have a gain factor of -2/3. Thus the high gain channels
are actually inverted then amplified, whereas the low gain channel is
inverted then attenuated. The Analog Devices AD8011 Current Feedback
Op-Amp was selected because it provides excellent gain vs. bandwidth
characteristics and is a very low power device, dissipating nominally
1mA of current. The 1^st^ op-amp in each channel is responsible for
amplification. Thus, by examining the input circuits in figures 4.1 and
4.2, we can see that the gain is roughly -2.7 for the high gain channels
and -0.46 for the low gain channels.

**4.3 Filtering**

After the amplification stage, the inputs are filtered. This filter is a
simple, 2-pole, anti-aliasing filter. The signals that we intend to
recover span a wide range of frequencies and have spectral components
that also span many frequency ranges. The desired signal range is from
DC to the maximum frequency resolvable by the ADCs. However, the station
resides in a relatively high-noise environment and must reject the
broadband RF noise. An anti-aliasing filter provides good rejection of
unwanted noise signals above the Nyquist frequency. The sampling
frequency of the ADC is 40 MHz so a simple 2-pole active filter with
cutoff frequency at 20 MHz works very well. The circuit in figure 5.3
shows the 2-pole active filter used for both the high gain and the low
gain channels. The filter itself has a gain of 1.44, and this brings the
input-to-output gain of the amplifier and filter sections to 3.86 for
the high gain channels and 0.67 for the low gain channels.

![](media/image3.wmf){width="4.308333333333334in"
height="2.673611111111111in"}

**Figure 4.3: Filtering circuit**

5.  **Analog to Digital Conversion**

The signal lines are single-ended analog waveforms which represent the
filtered, shaped, and amplified PMT outputs. These six signals are the
inputs for the analog-to-digital converters (ADCs). Since the signals
are single ended, they are all referenced to the board's analog ground.
By all signals referencing the same analog ground, it becomes possible
for small return currents to appear at the inputs. This was an area of
intense scrutiny as there is no consistent documentation on the best way
to handle the ground connections for Analog Devices ADCs. The final
circuit that was agreed upon has the ADC's ground connections isolated
from each other, and only returning to their own ground planes in order
to prevent digital ground currents from returning to the analog input.
Figure 5 below shows the ADC circuit that is used in the final FEv2.2
board. Also visible in figure 5 is the pedestal selection circuit. The
pedestal selection circuit is a simple voltage divider between VREF an
AINN. The circuit simply shifts the midpoint of the ADC span downwards.
The desired pedestal was 5% of maximum range, which corresponds to
roughly 50 counts. Theoretically a 9/10 voltage divider should provide
this correct range. However, in reality there is already a small offset
of 20 to 40 counts. This is probably a result of the low pass filters
averaging high frequency noise. If this noise is not symmetric about
analog ground (which it probably isn't) then the averaging leaves a DC
component of the signal. To counteract this initial offset, the voltage
divider had to be set to shift the midpoint less than theoretical
predictions would suggest. As a result, the voltage divider for the
FEv2.2 was selected to be a ratio of roughly 96/100. The actual values
were R23 = 562 Ohms and R51 = 13 KOhms.

![](media/image4.wmf)

> **Figure 5: Analog-to-Digital Converter circuit**

The sampling device that was selected was the Analog Devices AD9203
flash ADC chip. This chip was selected due to its very low power
consumption with a high conversion rate and decent precision. The AD9203
utilizes both edges of the SYSCLK signal. The ADC inputs are clocked
into the flash converter's sample and hold amplifiers on every trailing
edge of the SYSCLK signal. The digital outputs are latched onto the data
bus at each rising SYSCLK edge. To enable such a high sampling rate at
decent precision and low power, the AD9203 incorporates a pipelined
flash architecture. The intrinsic delay of each input sample is 7 rising
SYSCLK edges.

6.  **Digital Interface**

The FEv2.x boards must interface with the PowerPC (PPC) 403GCX local
station CPU in order to transfer the accumulated data. The transfers
occur once a data buffer is filled and ready to transmit the ADC data.
The PPC CPU is signaled that the FE is ready to send data by the FE
asserting the EVTCLK(F/S) signal on the bus. The PPC processor then
determines when it would like to conduct a data transfer. To ease the
load on the processor, the DMA fly-by-burst memory transfer mode was
selected. By deciding to use the DMA transfer mode, the ALU and
registers of the CPU can still execute instructions uninterrupted, but
cannot access the data bus. The fly-by-burst mode is the fastest and
simplest of all of the DMA transfer modes, which is very important as
the CPU data bus is locked out at this time. The transfer uses 1 wait
state for the memory to enable adequate time for the FE boards to place
the necessary data on the bus. With one wait state, the data is put onto
the bus at a 20MHz rate. Each buffer consists of 2 FIFOs therefore
reading both of these 1024 word buffers will take roughly 102.4 us.
However, after many CPU write cycles, the CPU intervenes to allow the
DRAM to refresh and row switch. So a full data transfer ends up taking a
little longer than 125us. Specifics of the DMA fly-by-burst protocol can
be obtained from the PPC403 User's Manual. One very important note about
the digital interface to the CPU is that the PPC devices use IBM's bit
ordering. This caused a slight problem as in most cases D31 is assumed
to be the highest order bit, whereas with the PPC CPU, bit D31 is the
lowest order bit. The bit swapping is done on the circuit board layout
as software bit swapping is computationally expensive and the PGA
socket's pins were already defined. Also, current versions of the
trigger boards differ from the FE board spec. The FE spec. allocates
1024 words of data for each event, but the trigger board implements 768
words due to the memory limitations of the current PLD selected for the
trigger board. However, the pre-trigger/post-trigger relationship
remains the same. This is not a problem but should be noted to ensure
that the CPU knows when valid data ends.

7.  **History**

This section is dedicated to explaining the differences between the FE
version 2 board, both cosmetic and otherwise. It is important to be able
to visually determine which revision of the FEv2.x board one is
examining. Without knowing which version of the FEv2.x board is being
analyzed, it is impossible to find the correct schematics to
troubleshoot it. Although the FEv2.2 board achieves all of the design
specs., the other iterations are more than capable for development
purposes. For that reason, it is perceived that all versions of the FE
board may remain within limited use at different locations.

**7.1 FEv2.0**

The FEv2.0 board although functional and useful for development was
riddled with problems. The first, and possibly the most serious was the
fact that the entire data bus was bit-swapped. This problem made the
FEv2.0 unacceptable for production even if no other problems were found
simply because bit-swapping code had to be developed. Although this code
does not seem to be a considerable problem, as the code is quite easy to
write, it puts a tremendous load on the CPU. The bit-swapping was a true
bit swapping and not a simple endian-ness switch. For that reason, every
bit of every word had to be moved. For 768 data words, with 32 bits
each, this gives a CPU processing overhead of at least 24568 operations
for each DMA transfer. The CPU operates at only twice the DMA transfer
speed so this processing overhead was much too high. As a result, it was
immediately apparent that there would be a revision, FEv2.1. Another
problem that presented itself was the fact that the trigger board could
not access memory fast enough to send data to the data bus without
dropping the first word. As a result, some signals were dropped in favor
of DRAM control signals, so that the trigger board would have enough
time to prepare data for the DMA transfer. These changes were
implemented in kludges for the FEv2.0 board but were added to the FEv2.1
revision design. The final problems with the FEv2.0 board were related
to the analog amplification and shaping section. The first of the
analog-related problems was noise. The FEv2.0 was significantly noisier
than the specification tolerated. True system-level noise estimates
could not be done at Fermilab due to a lack of hardware, but with a
stable power supply and data read from the actual socket, it was
determined that rms errors due to noise were less than 2 LSBs. The
design spec demanded rms noise within the actual system to be limited to
less than 1 LSB. It was found that the major problems with the FEv2.0
with respect to noise were twofold. First, the analog signal lines were
too far from the ADCs that were measuring the analog signals. Secondly,
the grounds were handled poorly. The analog signal traces traveled over
the digital ground plane, which exhibits quite a bit of high frequency
ground bounce. Even worse, a high Q inductor was placed within the
analog ground return path effectively peaking any small ground
perturbations that do occur. The true nature of these grounding problems
was not fully understood until the FEv2.1 board was tested. The last
major problem with the FEv2.0 board was the unacceptable amount of cross
talk. One of the major problems with the original FEv1.0 board was that
it provided only one analog ground reference point and return path for
all channels. The ideal arrangement of the analog daughter board
should've been: analog signal -- analog ground -- analog signal --analog
ground, etc. However, the connector was actually arranged: analog signal
-- digital ground -- analog signal -- digital ground. This did not allow
each analog channel to have its own reference point and ground return
path. As a result, a large signal on a given channel could cause small
analog ground perturbation of a few millivolts. While this perturbation
would not be seen by the driven channel as it is in phase and just
provides a rejected common-mode component, the un-driven channels all
reference the same ground point and as a result, see the ground ripple
as a signal of a few millivolts which is significant at our scale. There
was no easy way to kludge this fix, so solutions to this problem were
untested until the next design iteration.

The FEv2.0 boards are very easy to distinguish from the other version 2
FE boards. These are the only FEv2.x boards with an analog daughter card
connector and surface mount pads for comparators and a PLD.

The silk-screened text says that this board is Rev. C, designed October
6, 2000.

The schematic is listed as FEv2_0.pdf.

**7.2 FEv2.1**

The FEv2.1 was the first redesign of the FE version 2 board and remedied
a few of the more definite errors. The databus bit swapping and DRAM
control signals were correctly fixed in the FEv2.1 board. Also, the
cross talk problem was remedied by using a single analog ground plane
for all signals referencing analog ground. These were problems that had
the sources already identified and solutions already developed. As a
result, these problems were fixed without any problems in the FEv2.1
board. The only problem that did not have its sources clearly identified
and its solutions completely developed was the noise problem. The FEv2.0
board while in the actual system was said to exhibit 5 to 6 LSBs of rms
noise. This was unacceptable and needed to be fixed for the engineering
array (EA). The sources were not completely identified, so the solutions
were only based on hypotheses and the FEv2.1 was designed to be a test
bed for possible fixes to noise problems. This led to the use of many
redundant parts and jumpers to allow for a maximum number of different
board configurations. Prior to finishing assembly of FEv2.1, Fermilab
received a local station CPU board. This was essential because even with
the FEv2.0, the noise from the FE board run without the station and PLD
was almost acceptable for the EA. When the FEv2.1 was tested from a
stable supply without the trigger board, noise was less than 1 LSB so
testing of noise performance would absolutely require the real system
setup to get an adequate noise estimate. Also, it was determined that to
ensure a channel was working, it would be beneficial to include a
pedestal. Changes to the pedestal value should be implemented by
changing component values. It was debated whether it would be better to
set all channel's pedestals at a single point or whether it would be
better to be able to individually trim each channel's pedestal. A
compromise was reached by implementing both solutions and selecting the
best one via jumpers. A comprehensive list of changes between FEv2.0 and
FEv2.1 is as follows:

-   fixed data bus bit ordering

-   removed DMADXFER, TC0, and, TC1 were replaced with CAS3, LA23, and
    DRAMWE

-   referenced all analog related parts to AGND on a single analog
    ground plane (including both ADC grounds)

-   added voltage divider for pedestal setting

-   added reference and amplifier for alternative pedestal setting

-   added ground jumper to select chassis or DGND for AGND return path

-   changed amplifier supply rails from +/- 5V to +/- 3.3V to keep
    outputs within ADC input range even if pin diodes fail. This change
    eliminated DC-DC converters

-   added pull-up network for EXT_TRIGGER as it is active-low (negative
    logic)

-   removed remnants of analog daughter board (24-pin connector, digital
    GND plane on lower half of board).

These are all of the changes that were made in the FEv2.1 board design.
After several days of testing different configurations with the real
system (local station CPU and trigger board), the best configuration for
this board was determined that performed almost well enough for the EA
(but not quite). As a result, another design had to be done. The next
design (FEv2.2) would incorporate all of the design changes that were
needed from FEv2.0 to FEv2.1 and take advantage of all that was learned
while configuring the FEv2.1 board to perform at its best performance
point. While configuring, the FEv2.1 board, it was determined that the
major noise sources were radiated and conducted digital noise from the
station and the trigger board. Without both the trigger board and
station, the FEv2.1 had rms errors of 0.5 to 0.9 LSBs. With the station
powering the FEv2.1 board, the rms error jumped to between 1.0 and 1.2
LSB rms. Finally, when the trigger board was added, the errors jumped to
1.3 to 1.9 LSBs rms error. Taking this knowledge in, it was decided that
it might be a good idea to re-separate the ADC ground connections and
more importantly to add buffers on the digital outputs. These changes
should handle most of the conducted noise, and would appear in the
designs for the FEv2.2. To conquer the radiated noise problems, the main
technique was spacing and shielding within the layout. By moving the
analog outputs closer to the ADCs, less radiation is picked up between
these sections. Also, shielding the data bus and moving the amplifier
and ADC sections further from the databus, help reduce radiated RF noise
and pick-up. Finally, for FEv2.2 it was decided that the amplifier and
reference were more trouble than they were worth as that circuit
performed worse than the voltage divider at the ADCs (probably due to
trace length).

The distinguishing characteristic of FEv2.1 is the large number of
throughhole-jumpers at the ADCs.

Also, the digits 02-01 appears in the upper left hand corner of the
board.

The board is labeled on the silkscreen info section as 5 February 2001
Rev. 2_1

The schematic file for this version is FEv2_1.pdf.

**7.3 FEv2.2**

FEv2.2 is the final version of the FE board, and shall be produced in
large quantities (50 boards) for the EA. The FEv2.2 finally solved the
noise problems that could not be solved with FEv2.1. Most of the
problems were diagnosed after several weeks of analyzing FEv2.1.
However, the solutions could only be proposed and not tested because of
FEv2.1 layout constraints. FEv2.2a is the prototype for FEv2.2 and is a
test board that incorporates several jumper options. The jumperable
options on the FEv2.2a board are as follows:

-   DGND to DGND or AGND at each ADC

-   DGND and AGND tied at each ADC

-   Pseudo-differential input or single-ended input at SMA connector

-   AGND return through DGND or CHASSIS

-   Buffer or transceiver

These are the only jumperable options for the FEv2.2a, and the FEv2.2
board eliminates options 1 and 2, as it is the actual board to enter the
small production run for the EA. These jumpers were eliminated to enable
the FEv2.2 board to be assembled more easily as the SMT jumpers do not
have parts associated with them.

The design changes from the FEv2.1 to the FEv2.2 board were very slight;
mainly noise suppression and correcting the surge protection on the
low-gain channels. The only design changes visible from the schematics
were the removal of the reference-amplifier circuit, the separation of
the grounds at the ADCs, the insertion of buffers at ADC outputs, and
the clock driver using all fanouts. The design changes were deemed
redundant and more power hungry, but were more in line with good design
practice than the previous FEv2.x versions. The main changes to the
FEv2.2 design were layout related changes. The initial purpose of FEv2.2
was to shift the layout so that shielding would be simpler, as it was
known that a good portion of the rms noise was radiated digital noise.
The remainder of the changes were made to ensure that good design
practices were followed. The layout changes focused around three key
goals: keep the ADCs close to the amplifiers, move the amplifiers and
ADCs away from the databus, and shield extensively. To accomplish these
goals, extensive shuffling of the layout had to be done. First, the
layout was migrated to 8 layers to enable room for traces while shifting
and shuffling parts. Second, ½" thick rectangular perimeters surrounded
the ADC and amplifier sections to enable separate shield cans to be
placed over each of these sections. Then both the ADCs and amplifiers
were shifted as far right and close together as possible to ensure that
the databus is as far away from these sections as possible. The ADCs
were then shifted down substantially so that they were very close to the
amplifiers. Then layer stack was setup such that the clock and low order
data bits were not on the same layer as the analog signals to prevent
high frequency radiated switching noise. Finally, the data bus lines
were shielded at the connector by a ground plane on every other layer
starting with the top and bottom. All of these changes brought the rms
noise down to 0.3 to 0.5 LSBs of noise. This meets the design spec. for
the production run next year. As a result, minimal design changes will
be needed for the FEv2.2 board as it becomes part of the unified board.

The distinguishing characteristics of the FEv2.2 boards are the large
ground shields on both the top and the bottom of the board. The FEv2.2a
board differs from the FEv2.2 board by eliminating the 2-pin and 3-pin
SMT jumpers at each ADC and shifts the analog ground shield planes
slightly to the left to give more space for the channel 0 ADC.

Schematic file FEv2_2.pdf is the schematic for FEv2.2a

Schematic file FEv2_2_2.pdf is the schematic for FEv2.2