# Presentation: 317612

## Nevis FVTX/LDRD Update

- Dave Winter
- LANL FVTX Meeting
- 28-29 Aug 2006

## Overview

- _Goal: develop a test setup to:_
- Read out one or more FPIX chips
- Explore the technical issues of the FPIX in the PHENIX context
- Develop a systems-level concept of how to read out the pixel detector

- Optical
- transceiver

- FPGA

- User

- FPGA

- Optical
- transceiver

- FPIX

- D. Winter - LANL FVTX Meeting

- 29 Aug 2006

## Test Setup Design

- Interface board with smarts “close” to chip(s)
- Clock recovery (eg. via digital phase follower)
- Handle interaction with FPIX chip(s)
- Provide services to FPIX, eg. calibration pulser signals
- Remote receiver connected via fiber
- Remoteness relaxes requirements for power, temp, size, etc.
- Interfaces to PHENIX environment (timing signals, etc)
- User interface
- Flexibility to do either interactive or batch work with FPIX
- Allow entire system to be stressed

- Optical
- transceiver

- FPGA

- User

- FPGA

- Optical
- transceiver

- FPIX

- D. Winter - LANL FVTX Meeting

- 29 Aug 2006

## Nevis FPIX Test-stand

- _FPIX Interface_
- Altera Cyclone
- 2Mb serial eprom
- CYP15G0101DXB serializer
- Agilent HFBR 53A3DEM
- Interfaces with FNAL FPIX test board

- Optical
- transceiver

- FPGA

- RAM

- PCI Card

- FPGA

- Optical
- transceiver

- FPIX
- Board

- Interface card

- _PCI Card:_
- PCI-based
- Altera ACEX
- CYP15G0101DXB serializer
- 256k x 36bits SRAM
- Provides I/O buffers
- Agilent HFBR 53A3DEM

- _User PC_
- Linux
- Interactive shell to send commands to all three boards
- Can tx/rx basic FPIX commands

- D. Winter - LANL FVTX Meeting

- 29 Aug 2006

## Interface to FPIX test-board

- Interfaces to FPIX test board to send FPIX commandset and receive status/pixel data
- Includes 6-bit DAC for ability to inject pulses to FPIX
- Uses resistor network + caps to implement fast rise time/slow recovery
- Not shown: serdes + bidirectional fiber connection to pci card

- FNAL Connector/Cable, Power

- D. Winter - LANL FVTX Meeting

- 29 Aug 2006

## PCI test card

- Communicates with interface card via bidirectional fiber
- Essentially a passive manager of packet traffic
- Routes commands and pixel data to appropriate places
- Based on 32-bit PCI standard
- Communicates I/O management to host PC via set of registers
- Uses DMA to transfer data to/from host PC

- D. Winter - LANL FVTX Meeting

- 29 Aug 2006

## PCI Card-to-Host Interface

- FPIX Communication with user implemented using three channels
- Treated as ringbuffers
- Allocated from reserved physical mem segments mapped to userspace
- Channel 0, 1, 2 are for control commands out, control command responses, and pixel data in (respectively)
- FPGA as a “passive” router
- Insensitive to internal format of packets
- Only requires that they be 32-bit and complete (16-bit in the future)
- First 2 bits of packet header determine which channel(s) to send FPIX packets
- Control of PCI card done through PCI registers

- Channel 0

- Channel 1

- Channel 2

- D. Winter - LANL FVTX Meeting

- 29 Aug 2006

## User Interface

- A User Application Programming Interface (API) is under continuous development
- Currently based on 32-bit word packet structure
- API has been used to implement a shell-like interface to interactively work with FPIX test board
- Readback status of FPIX
- Manipulate registers in FPIX
- Inject pulses to FPIX (interface still in testing stage)

- Packet I/O

- Packet
- Formatting

- Memory
- Management

- Command-line
- shell

- API

- Example
- Using API

- D. Winter - LANL FVTX Meeting

- 29 Aug 2006

## Status

- PC user environment facilitates remote development (VERY important!)
- Eric implemented a relay-controlled switch for PS power, to enable remote power cycling of FPIX+interface boards
- Demonstrated ability to communicate with FPIX, interface card, and PCI card
- Attempts to perform full readout held up by subtle bug in interface’s fpga code
- Recently fixed
- All required steps are available in API/interface
- Fire Fighter Reset
- Write Alines + Operation Reset (only if not using default of 1)
- Read back status (ie. Synch word)
- Write needed registers (thresholds,kill mask,currents,Vref, etc)
- Write RejectHits=0
- Set SCR
- Set SendData
- Just need to try it out... (now that the latest bug has been fixed)

- D. Winter - LANL FVTX Meeting

- 29 Aug 2006

## Future

- Working on FPGA 16-bit packet structure (interface card)
- Modify next generation of interface card to be able to interface to 8-chip module
- Cable inputs based on FNAL test board specs
- Interface to HDI (4- and 8-chip versions)
- Fast serial readout (current implementation limited to ~80 MB/s)
- Redesign PCI card to include basic inputs required of FEM
- Mode-bits, Lvl1-accept, GTM, DCM busy
- User interface API to be made compatible with 16-packets
- Read out pixel data!
- Interface pixel data with existing (Sergey’s) sensor characterization macros
- Timescale for next generation of test setup: ~ few weeks-months

- D. Winter - LANL FVTX Meeting

- 29 Aug 2006

## Auxiliary Slides

- D. Winter - LANL FVTX Meeting

- 29 Aug 2006

## User-PCI Card communication

- Three communication channels defined
- Channel 0: Control commands to PCI and FVTX
- Channel 1: Response to Control Commands
- Channel 2: Data from FPIX
- Channels implemented as ring buffers (Bruce’s ABC architecture):
- A & B are r/w pointers (which is which depends on direction of data)
- C is word that contains info about how the hardware should wrap ptrs
- Two fiber channels: Transmit & Receive
- Data to/from fiber buffered in on-board SRAM
- Channel data passed between user and card thru reserved physical PC RAM managed by user process
- 3 RAM regions + PCI registers mapped to user space
- PCI card executes DMA transfers to/from RAM
- Packet structure defined to allow PCI and FPIX commands to be passed back and forth

- D. Winter - LANL FVTX Meeting

- 29 Aug 2006

## PCI Card Registers

- Standard 32-bit 33 MHz PCI card
- 32 32-bit registers starting at PCI device base address
- 0: Device and Vendor ID (readonly)
- 1: Status/Command - most bits nonexistant, but some R/W
- b0: Status/Command - most bits nonexistant, but some R/W
- b1: BM Bus Master enable - enables dma activity
- 2: readonly class code and revision id
- 4: Base Address - pci address of first register
- 16: CSR (Control and Status Register)
- 17, 18, 19: ABC regs for PCI to fiber transfers
- 20, 21, 22: ABC regs for fiber to PCI port 0
- 23, 24, 25: ABC regs for fiber to PCI port 1
- CSR (Control and Status Register)
- CSR[2..0] enable dma activity for each of the 3 channels
- CSR[5..3] enable fiber activity for each of the 3 channels
- CSR[6] is a flag to indicate loss of signal on fiber input
- CSR[9..7] are AEB flags for each of 3 channels
- CSR[31] = 1 resets PCI Card

- D. Winter - LANL FVTX Meeting

- 29 Aug 2006

## User space mapping

- Accessing any of the registers or the channels’ ringbuffers is as easy as accessing the appropriate array element

- Max RAM given to kernel

- Top of RAM
- Installed in PC

- Channel 0

- Channel 1

- Channel 2

- PCI Card
- Registers

- 4G Physical Address Space

- Userspace array

- Reserved phys RAM
- Managed via mmap

- Control
- Information

- PCI Reg

- Ch2

- Ch1

- Ch0

- Base address of PCI card (assigned by BIOS)

- D. Winter - LANL FVTX Meeting

- 29 Aug 2006

## Packet Structure

- 4-bit Commands (H11-8)
- 0: normal fpix command with H[23..12]=shift count followed by data words: 13 bits plus data
- 1: generate firefighter reset for fpix
- 2: return recent sync word (10-bit status) in H[21..12]
- 3: test pulse with 6-bit amplitude in H[17..12]
- 4: force hard reset of fpixtest and fpix
- 5: read/write CSR of fpixtest card

- Header

- Data payload
- >= 0 words

- 32-bit words

- Header:

- H0-1: Route packet to ch2,1 (for returning packets)

- H31-24: data word count

- H7-2
- unused

- H11-8: 4-bit cmd

- H23-12: 12-bit data field (nature depends on data)

- D. Winter - LANL FVTX Meeting

- 29 Aug 2006

## Packet Structure: Data Words

- Outgoing: typically will be commands to be delivered directly to FPIX
- For data to be written to FPIX, shift count is contained in data field of header
- Incoming: Echoed FPIX command + result of response to command (where applicable)
- If data is included (for those FPIX commands requiring it), ordering is done according to expectation of FPIX
- Most-significant bit first for FPIX command
- Least significant bit first for data bits
- Could change in the future (ie. let ROC FGPA reorder bits as needed)

- D31-19: 13-bit FPIX command

- (optional) Data bits

- D. Winter - LANL FVTX Meeting

- 29 Aug 2006

## PCI Card-Interface Card Communication

- Currently the only path of communication to interface board is fiber
- Takes data (control commands) queued up in SRAM, serializes, and sends over fiber
- Receives data (control commands and FPIX data) from fiber, deserializes and queues up to SRAM
- Packet format can contain either interface or fpix commands, PCI card doesn’t care
- Only cares if a full packet is present to send

- D. Winter - LANL FVTX Meeting

- 29 Aug 2006