# Presentation: 526715

## Clock Distribution for IceCube

- Gerald. Przybylski
- Lawrence Berkeley National Laboratory,
- Design  Review, September 16, 2005

- http://www.phys-astro.sonoma.edu/people/students/baker/SouthPoleFoucault.html

## History

**string 18** implementation: o Rubidium module slaved to GPS Clocko fan-out port for each domcom card.
**string 21** implementation: + OCXO GPS clock, Symmetricom ET-6000+ Passive fan-out to each DOM Hub (DSB Card)+ Sub-nanosecond skew and jitter demonstrated+ Simple and Reliable,  but not scalable

## MCU Requirements

- Straightforward; **conceptually very simple**
**5ns  absolute accuracy** (skew and jitter),  within the IceCube counting house  > Across all DOM Hubs (at DOR cards)  > Fixed and stable offset from Universal Time, Coordinated (UTC)  > Based on Scattering Length in the ice
- Distribute **10 MHz**, **1Hz**   and **T**ime **V**alue **S**tring
- Free from Metastable states/events; no glitches
- Measurable and Verifiable
- Single driver per output-port;  no shared drivers
- Robustness requirements in ERD    x   Mainly dealing with  satellite drop-out and loss of availability    x   Also dealing with tracking multiple GPS clocks
**Phase accuracy**:   **0.4ns** at fan-out,  **0.7ns** at DSB, and **1.0 ns** at DOR

## IceCube + IceTop + AMANDA

- IceTop same as IceCube
- IceCube to AMANDA  *** Zeuthen/Wuppertal;     Install 05/06;    Holger et. al.  ***Fiberoptic transmitter driven by GPS clockFiberoptic receiver drives TWR  (GPS4TWR)Autonomous from the MCU Clock Fan-Out subsystem   10 MHz BNC,  1Hz BNC,  IRIG-B BNC10 ns precision with respect to IceCube time
- AMANDA to IceCube*** Same playersTrigger system signals over Fiberoptic link to IceCube counting houseDepends on DOM Main Board(s) on a “String 81”

- Justification:
-    “We realized for Amanda that any artifial jitter below 15 ns in the MC has no influence on reco accuracy or background rejection.We then said 10 ns just for safety. Then others came and said that for nearly vertical tracks close to a string scattering may be negligible and a 5 nsec request makes sense.”  -- Christian

***NOT COVERED ***
***IN DETAIL ***
***IN THIS TALK***

## Our GPS clock

- ET-6000:
- Time to first fix:  <  two minutes
- Outputs operational: < five minutes
- Timing accuracy better than 2 μS & frequency accuracy better than 1E-8
- Full system accuracy (100nS) within one hour.
- 10 MHz output,  1Hz output, Time burst output
- GPS from  US Naval Observatory clock ~2x10-15 Accuracy

## MCU Brick-Walls

- ET-6000 Specifications:
- 1Hz output is:     positive (rising) edge on time,      within ±100 nanoseconds relative to either UTC or GPS     with **six or more** satellite averaging with 95% confidence.   (± 150ns peak)
- 40ns RMS accuracy   (jitter not specified)
- Cannot “Vote” multiple clocks; Neighboring clocks don’t track!x Tracking Algorithms in GPS clock “PLLs”x Variations in path, and multipathx Constellations and satellite switch-over

## Unlock Behavior

- Power Up:+ Sync to Satellites within an hour> Elapsed Time Format until “Tracking”- Hic-ups while searching; every 2 hours...
- Potential Loss of lock:x Clock Firmware (a-la  TrueTime 2000 problem)x Power Outagex Misadventure (Murphy...)x Wind/Weather damage to Antenna (speculative)x National Security outage (speculative)

## Flywheeling/Freewheeling

**TCXO** Clock “continues” for hours
- Optional **OCXO** “continues” > 1 week     **Aging            **±5 x 10-10 per day, ±5 x 10-8 per year     **Phase Noise **         -115 dbc/Hz @ 10Hz         -94 dBc/Hz @1Hz
- Optional **Rubidium Oscillator clock ** ~100ns per day slew WRT    **Aging           **< ± 5 x 10-8 over 20 years,     **Phase Noise ****          **-90 dBc/Hz @10Hz       ~ -80 dBc/Hz @ 1 Hz Ru: Good short term stability, best hang timeOCXO: best short term stability, good hang time
- Based on Symmetricom ET6000 series product specifications/experiences11

## 2004-2005 Implementation

- One GPS clock,   an  ET6000-OCXO
- Simple Passive fan-out   (resistive splitters)
- All 9 DOM Hubs driven; All clock BNCs used!
- 0.35ns Jitter and Skew measured in situ at NPX
- DOR Firmware Improvements fixed NPX GPS “glitches”
**Not scalable to 90 Hubs**

## 2005-2006 Implementation

- 2U chassis with 24 port fan-out  (2 cards)o Modulates 1Hz signalo RJ-45 distribution cables carry  10 MHz , modulated 1Hz, & TVSo All balanced signaling
- < 0.6ns  skew and jitter measured on the bench
- Passed MOAT:   sps-ichub04, sps-ichub05, domhubjr, domhub51
- Good Noise Immunity
- Stepping stone; scalable to needs of IceCube

## Design Goals/Drivers

- >  90 ports plus Spares
- Meet accuracy, jitter and skew requirements
- Convenience:   Single distribution cable per DOM Hub
- Measurability/Verifiability:   Easy to confirm phase across all ports
- Reliability/Robustness:    Quality components.   No electrolytics.
- Noise Immunity:   Balanced signals to DSB cards
- Minimize hard connections between racks:    Magnetic coupling
- Modularity/Extensibility/Maintainability
- Hot-Swappable Port Cards
- Independent Port Drivers for each signal
- Low power
- Off the Shelf Components
- No heroic solutions
- Simplicity! (no programmable logic in clock distribution)

## Signaling Details

- Balanced 10 MHz 500mV P-P through ‘ethernet magnetics’- High common-mode immunity- Suppress EMI emission & RFI pick-up- Avoid ‘Ground Bounce’ pick-up in the counting house- Commodity components: compact, inexpensive...
- Modulate*  the 1Hz signal (180deg Phase Modulation)* required to pass through ethernet magnetics...
- RS-422/RS-485 differential serial+12/-7V CM range

- Registered 1Hz

- U47-4

- Modulated

- 1Hz

- 1Hz/1PPS

- 10 MHz

## Rev 0 Fan-Out Card

- 12 Port
- 0.6ns port to port skew, worst pair
- <200 ps jitter
- Symmetry matters!
- Passes MOAT
- Revised DOR Firmware now supports Modulated 1Hz
- Inputs directly from GPS clock in Stand-Alone configuration
- Passes Fluorescent Lamp noise immunity test
- Status LEDs   &  Test Header

## 2006 Implementation and Beyond

- VME form factor conditioner card (1)        in 2006
- VME backplane fanout mezzanine card (1)       in 2006
- VME form factor 12 channel fan-out cards (9)       proto now
- VME form factor Monitor card (1)        2006 goal      (Least well defined)

- ...

- DSB

- GPS Clock

- DOM
- Hub

- RJ-45 Cable (90+)

- Coax Cable (2)

- Serial Cable

## Port Card Features, Rev 1

- 2-wideVME form factor;   12 RJ-45 port
- Common 10 MHz from Backplane
- Common modulated 1Hz from Backplane
- Common Serial from Backplane
- LVDS inputs, instead of direct GPS clock signals
- One Point Signal conditioning
- 2 minor schematic corrections
- Skew tweaks
- Additional Monitoring points: P2 and Header (e.g. demodulated 1Hz)

## Backplane Fan-Out

- Active Piggy-back card mated to “Back-of-Crate” style VME backplane over Socket “2”
- Independently drive each EVEN socket
- Match phase at each driven socket
- 1-to-10 LVDS-to-LVDS fan-out chips
- Match skew to each driven socket

*Industry Standard “Back of Crate” *
*style Backplane*

*Piggy-back*
*Card*

## Conditioner card

- Driven by GPS clock 10 MHz, 1Hz, and Serial
- Modulate 1Hz signal
- Control skew by design
- Ensure symmetry of port signals
- Drive LVDS backplane fan-out
- Status indicators/LEDs
- Prototyped on the Fan-Out card

## Monitor Card

- Occupies an **Odd Slot** in VME card cage (e.g. #1)
- Implement GPS clock Monitoring specified by PDR Document
- Could contain FPGA, & SOPC, or SBC in a DIMM form factor
- Could report via ethernet
- Scope TBD

## Watch List/Wish list

- “Lock” status from GPS clocks
- “Tracking” status from GPS clock
- “Power” status from GPS clock
- Parse time strings for error conditions
- Parse GPS Clock console port output   Rich set of Status bits/words   Satellite constellation
- Monitor phase offset between multiple GPS clocks

## Verification

- Reference signals on header10 pin header on DSB10 pin header on Fan-Out card4 pin header on DOR card (inside DOM Hub)

## The  Bottom Line

- Fan-Out card comfortably meets the 5ns Requirement
- Our GPS clock is a good choice, for the money
- No speed bumps this year
- On track for the final assaultSingle unit, flexible,  modular designBuilds on previous successesAvoids heroics and death marches.

- Fin

## Verifications in situ

- IceCube        AMANDA
- Opt 1: Measure Fiber Round-Trip time (2 fibers)
- Opt 2: Use **Portable Atomic Clock** - packaged PRS-10- battery power
-      Steps:   1. Measure/calibrate WRT IceCube clock   2. Transport clock to MAPO   3. Measure fiber distribution signals in MAPO against Atomic Clock   4. Transport clock back to IceCube Clock   5. Check CalibrationRepeat 1 – to – 5 until satisfied
- Measure/Verify within 1ns should be achievable.

- 4”