# Presentation: 944660

## DOM  MBHardware

- DOM  MBHardware

- CDR and PRR,
- May 5, 2004
- Gerald Przybylski

- Lawrence Berkeley National Laboratory

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Once upon a time, in 1996...

- Dave Nygren, Buford Price, Francis Halzen & JPL- Use Stewart Kleinfelders ATWR (SCA)
- Two DOMs, deployed Jan ’97            (Author recruited to take ‘em to the pole that year for deployment)
- Built around a PIC,  a One-Time, EPROM Programmable, Microcontroller, with ADC
- Data:  Push Mode,  FSK modulation
- Marginal stability,                      but it produced!

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## SwAmp vs. JPL DOMTypical AMANDA Analog OM,       JPL Digital optical module

- SwAmp vs. JPL DOMTypical AMANDA Analog OM,       JPL Digital optical module

- 7.5us

- 132ns

- Thanks to
- Tim Miller and Doug Lowder

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Lessons Learned

- PMT Waveforms ARE Interesting
- Software Underestimated
- One Time Programmable CPU Worrisome:- No Post Deployment Evolution- Front Loaded Software Development- Not Enough Memory or Speed
- Event Rate Limit by Slow ADC in PIC (dead-time)
- JPL Version Not Designed to be Scalable
- Get to Network ASAP

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Scaling up:for AMANDA

- String 18:40 Optical Modules, January 15, 2000

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Hybrid:Fiber & Digital

- Compromises/problems...
- +EIM and Noise Storm Immunity
- +100 PE linear dynamic range
- -Fiber ~7 PE dynamic range (without tricks)
- -Fiber Attrition
- -PMT HV Power Supply problems

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Design Drivers for IceCube

- Pulse Fidelity/Waveform Quality
- Wide Dynamic Range
- Event Timing Accuracy
- Low Dead-time
- Low Power Consumption
- Adequate CPU and Memory
- Remote Reprogramability,  Software,  and Firmware
- Off-Board Interfaces:    PMT Power,  and Flasher Board
- Cable Sharing:  Bidirectional,  Multi-Drop, Power,  **NO Fiber**
- Long Lifetime/High Reliability/Cost Tradeoff
- Self Identification – No mapping files
- Built-in Calibration and Monitoring Features
- IceTop & In Ice DOMs Identical

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Digital Optical Module Block Diagram

- FPGAandSOPC

- CPU

- CPLD

- Flash

- Flash

- PMT Power

- SDRAM

- SDRAM

- ATWD

- ATWD

- Monitor
- & Control

- LPF

- LC

- x16

- x2

- x0.25

- Flasher
- Board

- Pulser

- DACs & ADCs

- Corning Frequency Ctl (was Toyocom)

- 4Mb    4Mb

- 16Mb
- 16Mb

**+/**-5V, 3.3V,  2.5V, 1.8V

- 64 Bytes

- Delay

- Trigger (2)

- Oscillator

- 20 MHz

- 40 MHz

- MUX

- (n+1)

- (n–1)

- DOR

**OB-LED**

- x 2.6

- x 9

- 10b

- 10b

- 10b

- 10b

- 8b

- 32b

- 16b

- 8b

- 8b, 10b, 12b

- 1 megabaud

- DC-DC

- Configuration
- Device

- 8Mbit

- GTP, LBNL, 26 Mar, 2004

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

- Rev4.2

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Design Methodology

- Schematic Entry (Mentor)
- Layout Tools (Mentor)
- Simulations of Hardware subsystems
- Simulations of Firmware
- Testing in full scale prototype; 4 Major Revisions
- Prototyping in isolation
- Trade Studies
- Interface and Envelope Coordination with UW
- Circuitry Reviews; Reliability Reviews; HALT; HASS
- Framework Testing

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## For more information:

- Hardware: See http://rust.lbl.gov/~gtp/DOMContact Gerald Przybylski
- FirmwareContact Thorsten Stezelberger
- TestingContact Azriel Goldschmidt
- SoftwareContact Chuck McParlandContact Arthur Jones
- All of whom are at LBNL

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Part II:  Subsystems

- Power conversion and conditioning
- Memory: Boot Flash, Flash, and SDRAM
- Configurable Logic: FPGA and CPLD
- Communications: DOM to Surface, LC
- PMT Signal Acquisition
- Local Clock
- Control, Monitoring and Forensics

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Waveform Quality

- Digitize at the PMT Anode to Overcome Dispersion in Transmission Line
- - PMT Waveform risetime of a few ns- Cable risetime from 1us to 2us- Improve noise immunity- High ATWD sampling rate, 128 Samples

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Wide Dynamic Range

- x0.25, x2, x16 gain inputs to ATWD- Achieve around 14 bits from 10 bit part
- 40 count SPE = 5mV into ATWD channel 0
- 800 PE = 1V into ATWD channel 2
- PMT saturates at 5 to 6V into 50 ohms - Divided by 4 at input of ATWD channel 2- System limited by PMT saturation!

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Event Timing Accuracy

- Synchronous Triggering- Requires Delay Line, but- Simplifies Noise Removal- 25ns Course Clock- ~ 3ns ATWD sample- < 1ns resolution from curve fitting
- Clock Phase Tracking- < 2ns jitter in 2us rise-time pulse

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Low Dead Time

- 128 ATWD samples per hit = ~ 420ns- Up to 4 Channels- ~50us to digitize and store data
- Two ATWDs per OM- Acquire into B while Digitizing A- or Deployed spare...
- Acquire fADC data for > 4000ns @ 40 msps
- Digitizing when Local Coincidence “Tag” is present

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Low Power Consumption

- High Performance, Low Power Amplifiers, Comparators,   and Multiplexers
- High Performance, Low Power Data Converters- ATWDs,  fADCs,   and Communications DAC
- Coolrunner II CPLD
- ARM Hard Core CPU/FPGA
- Laptop SDRAM Main Memory
- High Efficiency Switcher Power Supplies

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Adequate CPU and Memory

- SOPC with ARM CPU and 400K Gate FPGAReplaces 256 Pin FPGA + 256 Pin CPU- Wider, Faster Bus... no bottle-necks- Improved speed – power- Bus to FPGA Bridge  - for general I/O- FPGA to Bus Bridge  - for Event DMA- Dual Port Ram  - for Communications

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Remote Reprogrammibility

- FPGA as Configurable Peripheral Controller- Communications Rx Tx- PMT Event Acquisition- Calibration Controller
- CPLD as Glue Logic- Mediates Rebooting- Off-Board Peripheral Interface and Firewall- Serial DAC and ADC interfaces

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Off-Board Interfaces

- HV Control Pin-out Doubled-up - Enhance the reliability of Ribbon Cable - Digital Control and Read-out Interface
- 48 Pin Flasher Interface- Power-Down mode- Just Another Memory Mapped Peripheral- Firewalled through the CPLD- Event initiation on command from FPGA- Flasher Pulse Current Monitor

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Cable Sharing

- AMI Modulation, demonstrated in String 18
- RAPCal, demonstrated in String 18- Round Trip- Drift Tracking
- Two DOMs per DOM Hub Channel + Half the Cables, saving $$$, and Flights+ Half the DOM Hubs, saving Power and Space- Double the Data Rate- Higher Firmware Complexity- Higher Cable Complexity- Higher Deployment Complexity

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Cost and Reliability

- Industrial Grade, or Better-40°C to +85°C ...- Mil-Spec too costly, and narrow selection
- Vetted Component Manufacturers
- Qualified PCB maker,  and Board Loader
- HALT and HASS Testing
- Manufacturers Reliability Information Studied
- STF Testing
- Component Cost ~ 10% to 15% more- e.g...  10 x Cost for Corning Oscillator

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Self Identification

- 64 bit Unique Digital ID in Each Flash Chip
- Configuration Building Software- Executed when DOMs are powered up
- Plug and Play- A or B identified by Jumper during manufacture.- Any DOM may be assigned A or B- Any A DOM and any B DOM may be installed on any pair. - Cable pairs may occupy any DOM-Hub Input
- Ident in PMT HV System,  and in Flasher Board

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Built In Calibration Pulsers

- Front End Pulse Generator- Generates PMT like signals
- On-Board Single LED Pulser- Optimized for narrow pulses- PMT Transit Time Calibration Tool
- Daughter Board: Bright, Long Range LED Pulser- Geometry Calibration Tool- Shower Simulator
- Crude arbitrary waveform pulser circuit

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Built-in Monitoring/Forensics

- 24 slow ADC channels monitoring- Voltages- Currents- Temperature- Pressure
- 8 fast signal monitor inputs- Clock x1, Clock x2, - Communications input, - Upper and lower Local Coincidence, - LED Pulse Monitors
- Post deployment failure investigation

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## IceTop “vs.” IceCube

- One OM per Pair doubles data rate.
- Insulation moderates extreme temperature (-80°C)
- Dual Range operation spans air shower energy range.
- Active, critical involvement in meeting needs.

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## The End

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## XA Block Diagram

- CPU Side
- (Stripe)
- __________
- “PLD” Side

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004

## Glossary

- ADC, Analog to Digital Converter
- AMI, Alternate Mark Inversion
- ATWD, Analog Transient Waveform Digitizer
- ATWR, Analog Transient Waveform Recorder
- ARM, Advanced Risk Machine
- CPLD, Complex Programmable Logic Device
- CPU, Central Processing Unit
- DMA, Direct Memory Access
- DOM, Digital Optical Module
- FPGA, Field Programmable Gate Array
- FSK, Frequency Shift Keying
- HALT, Highly Accelerated Life Testing
- HASS, Highly Accelerated Stress Screening
- JPL, Jet Propulsion Laboratory
- LED ,Light Emitting Diode
- PCB, Printed Circuit Board
- PIC, Programmable Intelligent Computer.
- PMT, Photomultiplier Tube
- PSK, Phase Shift Keying
- RAPCal, Reciprocal Active Pulsing Calibration
- SCA, Switched Capacitor Array
- SOPC, System on a Programmable Chip
- STF, Simple Test Framework
- SwAmp, Swedish Amplifier for Analog Optical Modules over Twisted Pair

- Lawrence Berkeley National Laboratory

- Review, May 5, 2004