# Presentation: 317636

## A proposal for a full upgrade of the L1CTT/DFE for Run2b

**Meenakshi Narain**
**Boston University**

**Outline:**
**Why do we need a full upgrade?**
**The new proposed design..**

## Groups

**Simulation and Algorithm development:**
**Kansas: **
**Graham Wilson, Carsten Hensel**
**Manchester: **
**(Liang Han), Terry Wyatt**
**Notre Dame: **
**Mike Hildredth**
**Hardware:**
**Boston University: **
**Meenakshi Narain, Eric Hazen, Ulrich Heintz, Shouxiang Wu**
**FNAL:**
**Marvin Johnson, Stefan Gruenendahl, Jamieson Olsen **

## L1CTT Upgrade Motivation

**In Run IIa and RunIIb, the L1 Track Trigger Provides:**
**CFT tracks for ****L1Muon**** Seeds**
**High p****T**** ****isolated track**** trigger capability **
**CFT tracks +CPS clusters ******** embryonic electrons**
**Found tracks for STT ******** b-tagging**
**We believe the Run IIa L1CTT will fail at the occupancies we expect in Run IIb, resulting in**
**Many more fake high-p****T**** tracks for all systems**
**The loss of rejection from combining subsystems**

## Physics Motivations

**Must trigger on high-p****T**** leptons with high efficiency, low fake rate: **
**HW********bb********e or bb********, HZ********bbll**
**Cal-Track match is important/necessary here**
**Must provide clean track samples for b-tags with the STT**
**HZ********bb********, Z********bb**
**Could (Must?) provide high-p****T**** tracks for hadronic tau triggers**
**H********(gets you 35% equivalent lumi boost)**
**Cal-Track match is important/necessary here**
**(Correlations between Systems is Necessary)**

## Run IIb Luminosity Projections

## Run IIa at high luminosity

**Even at modest occupancies, the high-p****T**** single track trigger would fire on ~2% of events (>100kHZ)**

**Isajet MC events**

**Nominal 5E32**

**(1 high pT track)**** **

**(1 high+1 medium pT)**** **

## L1CTT Granularity

**Tracking trigger rates sensitive to occupancy**
**Upgrade stategy:**
**Narrow tracker roads by using individual fiber hits (singlets) rather than pairing adjacent fibers (doublets)**
**Cal-track matching**

**Use full fiber resolution to restrict roads**

## L1CTT Algorithm Results

**With baseline version of new L1CTT:**

| Scheme/
pT range | Tracking Efficiency (%) | Rate of Fake Tracks (% events) | Resources |
| --- | --- | --- | --- |
| ABCDEFGH (RunIIa)
(pT>10 GeV) | 96.9 | 1.02  0.10 | 11k X 8 |
| abcdefgh
(pT>10 GeV) | 98.03  0.22 | 0.056  0.009 | 9.4k X 16 |
| abcdefgh
(5 GeV<pT<10 GeV) | 99.20  0.14 | 0.89  0.11 | 8.9k X 12 |
| abcdEFGH
(3 GeV<pT<5 GeV) | 98.40  0.20 | 4.5  0.2 | 11.3k X 12 |
| abcdEFGH
(1.5 GeV<pT<3 GeV) | 95.15  0.32 | 25.4  0.2 | 15.5k X 12 |

## L1CTT architecture

**A multistage system**
**Analog Front End (AFE):**
**Signals from the tracker**
**Mixer**
**Sort signals in trigger **
**sector wedges**
**Digital Front End (DFEA):**
**Track Trigger logic**
**Octant (CTOC) **
**Combine track information**
**from several DFE boards**
**CTTT **
**Construct track trigger terms**
**Trigger Manager**
**Construct 32 AND/OR terms **
**used by the L1 Trigger Framework **
**in forming the trigger decision **

## L1CTT architecture

**The approved upgrade:**
**Replace Digital Front End **
**daughter  boards (DFEA):**
**Need to fit 3-5 times **
**more equations**
**Rebuild these cards using **
**larger FPGAs**
**Use XC2V6000 Chip with 6M**
**sytem gates**

## Equation Resource study

**As a function of eff /number of equations:**

- Even the worst case equation scenario
- all singlets in 4 pT bins – fits in two XC2V6000

## DFEA daughter board

**First prototype with XC2V4000 FPGAs in hand and tested**

## Lessons from Run2a commissioning

**Current “approved plan” **
**upgrade of the DFEA daughter boards. **
**Provide a 5x increase in available logic resources.**
**With the experiences over the last year while commissioning the present Run2a L1CTT system a few very important issues have surfaced:**
**Lack of extensive testability of the  input and output information from the DFEA**
**Excessive Firmware download times **
**currently about four  hours  (with 2 parallel jobs) **
**may need restarts (5 automatic, 2-3 manual)**
**Limited by the slow 1553 interface **
**Scales up to about 12 hours/job for RunIIb equation file sizes**
**Clock and SCL signal distribution (dependent on upstream info)**
**A shortage of spare backplanes (there are none)**

## Desired Improvements 

**Reduce FPGA configuration download times:**
**From many hours/ half a day to minutes  **
**Concerns about reliablility of the download of firmware 3-5x the existing size. Permits and improves reliability.**
**realistic times for debugging and updates**
**Improve Testability:**
**Provide diagnostic access to the processor FPGAs on the DFEA DB. **
**Improved ability to inject test data and capture input data for in-situ testing.**
**Enhance post-manufacturing testing (boundary scan or other type)**** **
**Provide a means to inject SCL-provided control signals **
**(****RF Clock, First Crossing, etc) to an entire crate for bench-test and commissioning. This will permit stand-alone operation of the DFEA system for debugging and commissioning**
**Move input cable plant from  front of the crates to the rear.**
**Provide LED status indicators on the front panel.**
**Improve reliability over the existing daughterboard connectors**

## The Proposal

**In order to add testability to the system, one needs:**
**A new motherboard (daughterboard – to a less extent) design.**
**Improved diagnostics: input and output buffers, and L3 capability.**
**To alleviate colossal download times we need:**
**A new DFE crate controller with a                                           faster connection to the D0 online computers.  **
**A new DFE backplane design **
**Utilize this chance to put the  cables to the back.**
**No  transition boards.**
**Detailed specifications of the boards are being developed – follow links through the Run2 trigger web page**

## DFE Backplane

**Standard VME size, 21 slot, 6U monolithic backplane with custom connectors **
**Slot 1 is used for the DFE crate controller, DFE modules are located in slots 2-21. **
**In slots 2-21, the backplane will support up to ten input cables and four output cables passing through the backplane.**
**The backplane carries a simple read/write bus for communication with the DFE modules. **
**Also carries clock and control bits from the SCL receiver **
**The DFE boards can also share data with their neighboring boards**
**A high-voltage (48V) power distribution scheme is under consideration.  **
**Eliminates potentially hazardous high-current low voltage supplies **
**allow more power to be distributed over the backplane with smaller conductors, eliminates heavy cables, remote sensing oscillations**
**However the concern is here noise in the calorimeter – tests will be done before deciding on this scheme.**

## DFE Crate Controller

**The controller must download firmware to the track-finding FPGAs on the logic boards via the backplane read/write bus.  **
**In addition, it provides external access to the bus for configuration, monitoring and debugging.  **
**The existing DFE controller:**
**flash memory for local FPGA configuration storage, and a 1553 control bus interface for external access.**
**The new controller will prove a direct interface to the read/write bus via a high speed (gigabit class) fiber optic link (also simpler).**
**The flash memory and microcontroller will be eliminated.  **
**The firmware for the track-finding FPGAs will be downloaded directly over the fiber optic link at very high speed, allowing fast firmware changes for updates and debugging. **
**In addition, test patterns may be downloaded to the DFEA boards and captured output data read out over the link.**
**It is currently planned to use a commercial gigabit-ethernet fiber interface module to simplify the controller logic such that it may be implemented in a single FPGA.**

## A suggested solution based on CMS design:

** ****VMEbus Controller with Gigabit Ethernet **
**A custom board designed and developed at OSU**
**Based on XILINX Virtex-II Pro**
**Optical transceiver (for Gbit Ethernet) **
**Communicates with stand-alone PC via Ethernet**
**Inexpensive:  ~ $600 each, tests begin soon. **

## DFE Mother Boards 

**Redesign for increased testability **
**Direct read/write access to daughter board FPGAs**
**Status LEDs on front panel**
**Replace or eliminate unreliable spring contact connectors**
**Eliminate rear transition modules**

## DFEA daughter board

**First prototype  in hand and tested**
**Will need to make a few changes due to the new design of the MB etc**
**Improved system level diagnostics**
**By improving ability to send test patterns**
**Capture input/output buffers and enhanced Level3 support**
**Improve reliability over the existing spring-contact PCB surface daughterboard connectors**

## Incremental Equipment Cost

|  | Cost k$ |
| --- | --- |
| Backplane  |    14 |
| Crate Controller |    7.5 |
| Optical download and control link |     5K |
| Test equipment + splitters + crate |     17 |
| Motherboards  |     61 |
| MB Prototypes + test stand	 |     25 |
| Total |     129 |

**+Labor Costs**

## New Schedule

**Aim for: **
**installation in Summer of 2005**
**A slice test during Fall 2004 shutdown.**

## Installation & Commissioning

**The proposed scheme is driven by the desire to enhance testability and reliability of firmware downloads. **
**Eases Installation and commissioning **
**Plan to run on a partial crate of the new system on the platform in parallel with the existing DFEA using LVDS splitters**
**Key to successful commissioning of the new system**
**Advantages:**
**Can assemble the whole crate outside of the collision hall**
**Extensive testing of the entire chain possible before putting in collision hall**
**Enhanced testing capability**
**Use Run2a Data derived test vectors to verify **
**Take out the old crates (2 of them) and replace them with new ones**
**Use the existing Run2a tools used for commissioning**
**Low level changes will be transparent at user level**

## Summary

**Proposal to upgrade the entire L1CTT DFE hardware**
**Based on lessons learnt from Run2a L1CTT commissioning**
**Provide more in-situ testing capability**
**Add more I/O buffers and L3 readouts**
**Reduce the firmware download times significantly**
**Use high speed fiber optic links**
**Installation a bit simpler**
**extensive tests can be performed prior to installation **
**Commissioning times will be reduced**
**Faster downloads**
**Uses existing infrastructure**
** **

## DFE Motherboards

**The Digital Front End (DFE) Motherboard is a general purpose, high bandwidth platform for supporting reconfigurable logic such as FPGAs.**
**It is intended for applications where a DSP or other microprocessor is too slow.**
**The DFE motherboard is a 6U x 320mm Eurocard with fully custom hard metric backplane connectors. **
**Eight point-to-point links bring data onto the DFE motherboard at an aggregate data rate of about 12 Gbps.**
** ****Each physical link consists of five twisted pairs (Low Voltage Differential Signals) **
**These are terminated with hard metric female connectors on the DFE backplane and passed through to the DFE motherboard. **
**After entering the DFE motherboard, the ten links are sent to receivers, which convert the serial data back to a 28 bit wide bus running at 53 MHz. **
**These busses, in turn, are buffered and routed to the two daughtercards. **
**The outputs from the daughterboards are fed back to the motherboard, **
**converted to serial data by channel link transmitters and SLDB (serial link daughter boards) mounted on the motherboard.  **
**The serial data are passed through the backplane to the output cables. **
**A “local bus” bridged to the backplane  bus will allow direct read/write access to the individual FPGAs on the DFEAs for debug and monitoring.  **
**Additionally, a dedicated bus for cross-communication between the two DFEA daughterboards will be provided.**

## Core Trigger Menu Simulations

**Total L1 bandwidth budget= ~3 kHz**

**Total rate:                      ~15kHz ~30kHz ~3.2kHz**

**Additional headroom available from**
** ****topological cuts available in upgraded L1cal**
**Higher muon p****T**** threshold with upgraded CTT**

**1E32    2E32   2E32**