# Presentation: 470354

## The CERN LHC central timingA vertical slice

- Pablo Alvarez
- Jean-Claude Bau
- Stephane Deghaye
- Ioan Kozsar
- Julian Lewis
- Javier Serrano

**Notes:**
I am going to describe a vertical slice through the LHC timing system.
That is from the level LSA timing service down to the timing receivers cards in the front end computers. IE Hardware and Software 

## Talk layout

- LHC Injector Chain Timing
- Timing Distribution Overview
- Central Timing Overview
- Reflective Memory
- UTC Time and GPS
- The Multitask Timing Generator
- The Controls Timing Receiver Module
- Safe Machine Parameters Hardware Checking
- LHC Software Architecture and Event Tables

**Notes:**
Just read them out ...

I am going to talk about, the LHC injector chain and its timing system
Actually there are two chains one for ions and one for protons, today I will concentrate on the proton chain

## The LHC Proton Injector ChainStrongly time coupled

- LHC

- CNGS

**The LHC Proton Injector Chain****Strongly time coupled**

**Notes:**
Actually there are two chains one for ions and one for protons, today I will concentrate on the proton chain

To fill the LHC with protons many accelerators are involved

This slide shows the accelerators, transfer lines, and dumps implicated in filling the LHC

The beam exits the Linac at 50mev tell the story

## CERN accelerator networksequenced by central timing generator

- PSB

- CPS

- SPS

- LHC

- Experimental
- area

- Experimental
- area

- Experimental
- Area

**Notes:**
The central beam and cycle manager pilots all CERN accelerators except for the LHC
Some of these machines are part of the LHC injector chains
There are two one for Ions, and one for Protons. Today we will take the simpler case Protons as an example.
The LIC accelerators cycle in sequences, each accelerator providing beam for the next as the energy increases.
Notice the time relations between the machine cycles in the injectors.

## CBCM Sequence Manager

**CBCM Sequence Manager**

**Notes:**
This is a screen shot of the program that configures the LIC super cycles.
Notice the strong coupling in time. All cycles are a multiple of a basic period.
A Beam is a series of cycles as shown that results in particles being extracted towards an end user.
The CBCM executes beams not cycles.
The LHC is more like an end user in this sense.
Once the beam sequences are configured they can be downloaded onto the CBCM.

## The LHC Beam 

- PSB1

- PSB1

- PSB2

- PSB2

- PSB3

- PSB3

- PSB4

- PSB4

- CPS Batch 1

- CPS Batch 2

- CPS Batch 3

- CPS Batch 4

- SPS Cycle for the LHC

- SPS injection plateaux

**The LHC Beam **

- LSA Beam request:
- RF bucket
- Ring
- CPS batches

- Extraction
- Forewarning

- LHC Injection plateaux

- Injection

- Injection

- Extraction

- Extraction

- The LHC timing is only
- coupled by extraction

*start-ramp*
*event*

- Extraction
- Forewarning

**Notes:**
Here is the beam needed for manufacturing a nominal LHC proton beam.
12 such beams must be executed to fill one LHC ring.
Notice the strong time coupling in the injectors.
The only time coupling between the LHC and its injector chain is the beam transfer.
Only the extraction forewarning need be exchanged between the two timing systems.
The LHC then parts company and lives its own life, there is no basic period for the LHC, the processes are fluid in time and can run into each other or start stop at any time.
Only extraction, and subsequently the LHC injection timing are coupled.

## Talk layout

- LHC Injector Chain Timing
- Timing Distribution Overview
- Central Timing Overview
- Reflective Memory
- UTC Time and GPS
- The Multitask Timing Generator
- The Controls Timing Receiver Module
- Safe Machine Parameters Hardware Checking
- LHC Software Architecture and Event Tables

**Notes:**
Just read them out ...

I am going to talk about, the LHC injector chain and its timing system
Actually there are two chains one for ions and one for protons, today I will concentrate on the proton chain

## Timing Distribution Overview

- RS485 drop nets distribute timing around the CERN accelerator complex
- Long distance transmission over optical fibers
- One timing network for each accelerator
- Hundreds/Thousands of timing receiver modules distributed around the complex
- One timing generator drives one timing network

## Timing Frames

- Timing frames are Manchester encoded at 500kbit using a 1MHz clock.
- Each frame carries 32 data bits, parity, start and stop bits.
- One frame transmitted each 125μs, 8 per millisecond.
- The frame data is broken into bit fields
- 4 Bits _**A**_ccelerator 	[A]
- 4 Bits frame _**T**_ype 	[T]
- 8 Bits _**C**_ode		[CODE]
- 16 Bits _**P**_ayload 	[PAYLOAD]
- Some frames are recognized by the hardware and cause special treatment
- Two UTC frames carry the time of day in their payload
- Millisecond frames are always sent in phase with the PPS
- Telegram frames are stored in double buffers
- Event frames cause counters to be loaded and triggered and may produce bus interrupts

- Millisecond

- Millisecond

- 8 frames

**Notes:**
The timing distribution network carries events generated by the central timing to (100s or soon 1000s of) timing receivers all around the complex.
The timing networks carry frames at 500KB/S encoded with a 1MHz clock.
Each frame carries a 32bit data payload and start stop and parity information.
They are transmitted at a rate of 8 frames per millisecond.
Each millisecond the millisecond timing frame is always sent leaving room for 7 other frames.
Events are encoded ....

The millisecond events are in phase with the UTC second

## Talk layout

- LHC Injector Chain Timing
- Timing Distribution Overview
- Central Timing Overview
- Reflective Memory
- UTC Time and GPS
- The Multitask Timing Generator
- The Controls Timing Receiver Module
- Safe Machine Parameters Hardware Checking
- LHC Software Architecture and Event Tables

**Notes:**
Just read them out ...

I am going to talk about, the LHC injector chain and its timing system
Actually there are two chains one for ions and one for protons, today I will concentrate on the proton chain

## Central TimingOverview

- LHC
- Central Timing
- Generation

- CBCM
- Controlling
- Injector Chain

- Gateway
- FESA API

- LSA
- Timing
- Service

- SPS destination request
- R1,R2

- CPS Batch Request 1,2,3,4

- LIC Timing

- HIX.FW1K

- LHC Timing

- Inhibits
- Requests
- Interlocks
- SIS TI8/TI2
- & SPS Dump
- Inhibits

- LHC Fill Requests:
- 	Bucket
- 	Ring
- 	Batches

- Master
- ON/OFF

- CTR

- LHC Fill Requests:
- 	Bucket
- 	Ring
- 	Batches

- Reflective
- Memory
- Link

- CBCM
- Sequence
- Manager

- LIC
- Sequence

- TI8/TI2
- Dump

- Request
- TI8/TI2
- Dump

- LHC
- User

- Request
- LHC User
- Normal
- Spare

**LSA changes Allowed**

- LSA Master

- SEX.FW1K
**SPS.COMLN.LSA_ALW**

**Central Timing****Overview**

**Notes:**
This simplified diagram shows the logical components of the central timing we shall talk about.
The sequence manager builds sequences as we have seen and controls which sequences execute
The LSA sequencer takes mastership of the CBCM during the fill.
It makes requests for beam to the FESA API on the LHC gateway system.
The request passes across shared memory to the LHC central timing where the event stream is produced,
A small number of signals are exchanged between the LIC timing and the LHC timing to control SPS destination (LHC Ring) and CPS batch count.

## Talk layout

- LHC Injector Chain Timing
- Timing Distribution Overview
- Central Timing Overview
- Reflective Memory
- UTC Time and GPS
- The Multitask Timing Generator
- The Controls Timing Receiver Module
- Safe Machine Parameters Hardware Checking
- LHC Software Architecture and Event Tables

**Notes:**
Just read them out ...

I am going to talk about, the LHC injector chain and its timing system
Actually there are two chains one for ions and one for protons, today I will concentrate on the proton chain

## Reflective Memory

- LHC Gateway
- Implements FESA API

- 2Gbit/S
- Token ring

- VMIACC-5595
- Single Mode
- Hub

- 64Mbyte
- VMIPMC-5565
- Reflective memory

- LHC Timing generator A

- LHC Timing generator B

**Reflective Memory**

- Reflective memory:
- A and B must always be
- in the same state.
- If no restrictions for
- switch over

- Protects token ring

- Identical
- except
- ID event

**Notes:**
The LHC timing generation crates include a hot standby.
The two systems are identical except for their identity code which is transmitted over the LHC timing.
A manually operated switch selects which crate is actually connected to the control system.
The gateway receives the identity event from the timing so that alarms can be handled. The hot standby may be updated during the run, and then switched to master.
If we are to switch crates, the two timing systems must be in an identical state if the LHC is not to be disturbed. Reflective memory should be identical in all three systems.
The HUB protects the token ring is a system is powered down or rebooted

## Talk layout

- LHC Injector Chain Timing
- Timing Distribution Overview
- Central Timing Overview
- Reflective Memory
- UTC Time and GPS
- The Multitask Timing Generator
- The Controls Timing Receiver Module
- Safe Machine Parameters Hardware Checking
- LHC Software Architecture and Event Tables

**Notes:**
Just read them out ...

I am going to talk about, the LHC injector chain and its timing system
Actually there are two chains one for ions and one for protons, today I will concentrate on the proton chain

## UTC Time and GPS

**GPS**
- One pulse
- per Second

- GPS

- Symmetricom
- XLI

**PLL**
- One pulse
- per Second

- Phase locked
- 10MHz

- Basic Period
- 1200/900/600 ms

**Advanced (100us)**
- One pulse
- per Second

- Synchronized 1KHz
- (slow timing clock)

- Phase locked
- 10MHz

- Phase looked
- 40 MHz Event
- encoding clock

- 40MHz PLL

- Synchronization
- module in each
- timing generator
- crate

- RS485
- Timing

- MTT
- Multitask
- Timing
- Generator
- MTT

- UTC time (NTP or GPS)

- Event
- tables

- External events

**UTC Time and GPS**

**CERN **
**UTC **
**Time**

- Set once on startup
-  & on Leap Seconds

- RS485
- Timing

- Control System

**CERN **
**UTC **
**Time**

- 25ns steps

- Timing receiver

- Symmetricom
- CS4000
- portable
- Atomic Clock

**Notes:**
GPS 
Clock generation
Setting UTC time
Calibration

## Talk layout

- LHC Injector Chain Timing
- Timing Distribution Overview
- Central Timing Overview
- Reflective Memory
- UTC Time and GPS
- The Multitask Timing Generator
- The Controls Timing Receiver Module
- Safe Machine Parameters Hardware Checking
- LHC Software Architecture and Event Tables

**Notes:**
Just read them out ...

I am going to talk about, the LHC injector chain and its timing system
Actually there are two chains one for ions and one for protons, today I will concentrate on the proton chain

## Multitask Timing Generator MTT

**Hardware multitasking for 16 tasks**
- 32 local registers per task
- 218 Global registers
- 6 Memory mapped IO registers
- Timing frame out register
- VME P2 in register ...
**Host processor access to all registers**
- Implements general purpose CPU
- Op-codes are triadic: AND SrcREG,SrcREG,DstREG – AND 0x7,VMEP2,TMP
- Arithmetic and logical
- Move indexed, literal, register
- Wait value, relative
- Conditional branch
- Interrupt host
- Tasks defined from host via Task Control Block
- PC
- PC Offset
- Processor Status Word
- Command and status registers allow host access to running tasks

**Notes:**
Event tables mirror the LHC machine processes that start and stop independently. So we need to do the same thing with event tables.

Local and Global registers

## MTT hardware module

- See: The LHC central timing hardware implementation
- P. Alvarez, J. Lewis, J. Serrano CERN, Geneva, Switzerland
- This conference

**Notes:**
Show what happens, VMEP2 
Host access

## MTT External Events Task

**MTT External Events Task**

**Notes:**

Don’t go into details, just say what this program does.

I.E. Waits for and reads the P2 and sends corresponding events out

## Talk layout

- LHC Injector Chain Timing
- Timing Distribution Overview
- Central Timing Overview
- Reflective Memory
- UTC Time and GPS
- The Multitask Timing Generator
- The Controls Timing Receiver Module
- Safe Machine Parameters Hardware Checking
- LHC Software Architecture and Event Tables

**Notes:**
Just read them out ...

I am going to talk about, the LHC injector chain and its timing system
Actually there are two chains one for ions and one for protons, today I will concentrate on the proton chain

## Controls Timing Receiver CTR

- Hybrid
- PLL

- Delay

- Frame
- Decoder

- TGM

- TGM

- UTC

- Content Addressed
- Memory - Triggering

- Counter
- Configurations
- Loader

- 22Bit 50MHz
- Counters

- Bus
- Interrupt

- Output

- Timing

- 40MHz

- Lookup

- Load

- Trigger

- Start
- Previous
- External
- Event
- Clock
- Previous
- 40MHz harmonic
- External
- Modes
- Once
- Multi-pulse
- Burst
- Trigger
- Event frame
- Wildcard
- Telegram
- Action
- Output
- Bus interrupt
- Output
- TTL/TTL bar
- Pulse width

- CTRV – CTRI – CTRP formats

## Talk layout

- LHC Injector Chain Timing
- Timing Distribution Overview
- Central Timing Overview
- Reflective Memory
- UTC Time and GPS
- The Multitask Timing Generator
- The Controls Timing Receiver Module
- Safe Machine Parameters Hardware Checking
- LHC Software Architecture and Event Tables

**Notes:**
Just read them out ...

I am going to talk about, the LHC injector chain and its timing system
Actually there are two chains one for ions and one for protons, today I will concentrate on the proton chain

## Safe Machine Parameters Distribution

- Reads thresholds

- @1kHz-24Bit 108

**Safe Machine Parameters Controller**
- for LHC

- Energy A

- I_beam1   & 2

**Reads status**

- Energy B

- BEM

- BCT “A”

- BEM

- Management
- Critical
-  Settings

- (CTRV)

- CTRV

- CTRx

- LSA

- BCT “B”

- EXP

- Line
- driver

**LHC**
**Timing**
**Generator**

**SMP @ 10Hz 16Bit 10****10**
**(Flags, E & Int.)**

- CTRV

- CTRV

- CTRV

- CTRV

**BIS**

- CTRV

- CTRV

- CTRV

**Events, UTC, & Telegrams (including SMP)**

**Safe Machine Parameters Distribution**

- CTRx

- CTRx

- CTRx

- EXP

**Flags TTL**
**Hw Output**

- If length > 5m

**Notes:**
As the LHC central timing is involved in machine critical safety parameter distribution, it is monitored by hardware that dumps the beam if it fails.

## Talk layout

- LHC Injector Chain Timing
- Timing Distribution Overview
- Central Timing Overview
- Reflective Memory
- UTC Time and GPS
- The Multitask Timing Generator
- The Controls Timing Receiver Module
- Safe Machine Parameters Hardware Checking
- LHC Software Architecture and Event Tables

**Notes:**
Just read them out ...

I am going to talk about, the LHC injector chain and its timing system
Actually there are two chains one for ions and one for protons, today I will concentrate on the proton chain

## LSA and FESA

- The FESA API is implemented _**on the LHC timing gateway**_
-  Accesses timing generators across reflective memory
- Implements
- Load or Unload event table
- Get running tables list
- Set event table run count and synchronization event
- Stop or Abort event table
- Set telegram parameters
- Send event
- Read status of tasks and MTT module

## Event Table Processing

- Load object
- Initialize TCB
- Run task

- Assemble
- task

- Translate
- and Merge

- HX.Start-Ramp 1.01 0x5
- HX.Start-Freq   0      0x05

- waitr MSFR,1001
- movv 0x14020005 EVOUT
- movv 0x14030005 EVOUT

**Event Table Processing**

- FESA API

- LSA

- Event
- Table

- Table Compiler

- Translation

- Event Table
- template

- Table Task

- Assembler

- Object Code

- Reflective
- Memory

- Task
- Loader

- MTT
- Program
- Memory

- Hot Standby

- Data

- Process

## Conclusion

- Event tables model LHC machine processes.
- Reuse of existing timing boards was facilitated by using FPGAs and writing new VHDL.
- The LHC timing is monitored by hardware.
- Reflective memory has increased reliability.