SuperKEKB Controls< kazuro.furukawa @ kek.jp >SuperKEKB Controls

Kazuro Furukawa
For Control Group
February 8, 2011.

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

1

SuperKEKB Controls

Contents
Based on Presentations at the Review-2010
 Control System
 Simultaneous Injections to KEKB and PF

Achievements in KEKB ControlsRequirements to ControlsSolutions and Developments
Summary

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

2

SuperKEKB Controls

KEKB Construction and Operation
Distributed EPICS-based Control System 2 + 1 Layers
 Operation programs (OPI, Channel Access Clients)
 Equipment controls (IOC, Channel Access Servers)
 Device hardware controllers (over Fieldbuses)

Operation Programs in Scripting Languages
 SADscript for Beam Handling Mathematica-like language to SAD Engine
 EPICS CA, Event-driven, List, Plot, GUI, Optimization, etc

 Python Object-oriented, EPICS, GUI, etc
Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

3

SuperKEKB Controls

Requirements to Controls in SuperKEKB
Hearing and Interviews to each Group Not much different from the present system other than speed requirements

 Mostly the same requirements to KEKBHowever, Additional requirements may come later Should be prepared for them

 Should catch up with the current technology
 for reasonable maintenanceability

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

4

SuperKEKB Controls

SuperKEKB Plan For nano-beam scheme with 40-times higher luminosity
 Many new facilities could be required Start based on the existent environment
 With additional concept of CA everywhere Collaborate with device groups for better global controls
 Replacement of old installations such as CAMAC
 Solutions not only VME but also embedded EPICS if possible, etc

 Faster networks for the groups who can build
controllers by themselves
 Better connection to operational environments
 Keeping SADscript environment, etc
 Monitoring at offices
Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

5

SuperKEKB Controls

SuperKEKB Plan (Examples) Archiving scheme and viewer
 Maybe existing KEKBlog and channel archivers
 New viewer should be developed (in CSS?) Alarm handler Continuation of KEKBalarm or Simulation on CSS or Python
 Under evaluation

 Operational Log
 In house, two versions with different origins
 Postgres + (Python/Zope and Flash/Flex)

 Scripts
 SADscript/Tk, Python/Tk, (decreasing Tcl/Tk)

 Displays Edm/Medm and CSS (Control System Studio from DESY/ORNL)
Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

6

SuperKEKB Controls

Interfaces to Devices
VME as Device-side Computers
 EPICS IOC (I/O controller), CA (Channel Access) server

For Device Controllers
 CAMAC for RF, Vacuum, Beam-Transport It is not impossible to maintain them, however, availabilityof mechanical components as well as chips
 Hope to replace them at least at Vacuum and BT systems

 VXI interfaces (from VME) for ring BPM
 ARCnet (from VME) for Magnet, and other systems
 May be replaced with Ethernet-based controllers

 GPIB over network, etc
Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

7

SuperKEKB Controls

Embedded EPICS IOC
orCA Everywhere

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

8

SuperKEKB Controls

Accelerator Controls at KEKB and Linac
VME + Unix (1990~)
 Standard configuration
 With many third layer field networks

Every controller on IP network (1993~ at linac) Single layer in physical, two layer in logicalEPICS (1995~)
 Globally sharable efforts

Every controller with EPICS IOC (2005~)
 Channel Access everywhere For longer-term maintenanceAccelerator Review

Kazuro Furukawa, KEK, Feb.2011.

9SuperKEKB Controls

Transition of Controls
Mini
Computer

Unix

Unix

Unix

VME

VME/IOCIOC

Field
Networks

TCP/IP

ChannelAccessDeviceController

DeviceController

DeviceIOC

Mini
ComputerDeviceController
Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

10

SuperKEKB Controls

Embedded IOC in Yokogawa’s PLC
More than 150 PLCs were employed at Linac
 All through TCP/IP network since 1993
 Successful to reduce resource consumption

Now Linux CPU is available (2008~)
 533MHz CPU, 128MB RAM, 2xEthernet, USB, etc
 Utilize realtime feature of Kernel 2.6
 EPICS PV response time <150µsec (incl. module delay)

Ladder sequence CPU can coexist
 Variable sharing possible

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

11

SuperKEKB Controls

Simpler PLC Usage under EPICS
Conventional PLC usage
OPI
CA Clientswith asynchronous accessIOC(Logics)

OPICA Clients

LadderCPU(Logics)

FAM3 PLC
I/O Modules

F3RP61IOC

FAM3 PLC
I/O Modules

If necessary, we can combine
OPICA Clients

F3RP61IOCLadderCPU

FAM3 PLC
I/O Modules

Logics are confined in PLC, and management is easier
Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

12

SuperKEKB ControlsMany medium-speedcontrollers implemented
 KEKB, Linac, J-PARC, PF,
cERL, …, Taiwan/TLS,(and other Asian Institutes)
 <http://www-linac.kek.jp/cont/epics/f3rp61/>

Image processing module available as well
Event-receiver (EVR) module being
Developed by SINAP/Shanghai Good example of Global CollaborationAccelerator Review

Kazuro Furukawa, KEK, Feb.2011.

13

SuperKEKB Controls

LLRF in MicroTCA (µTCA)
 MicroTCA was chosen as a platform for LLRF controller
 Synergy between SuperKEKB, cERL, XFEL/DESY, ILC, etc
 ATCA (2003)
 New computing standard for telecommunication and industry
After CompactPCI (1993), for reliability with higher performance Many serial interconnects on backplane

2.5Gbps each (10Gbps in the future)»One card covers all phones in a small city with a certain implementation

 IPMI surveillance/remote-management for reliability

 AMC (Advanced Mezzanine Card for ATCA)
 Serial interconnects, IPMI, good part of ATCA


Like IP-module to VME, PMC to cPCI MicroTCA (2008)
 AMC card itself is powerful Direct slot-in AMC cards in a Box
Commercial I/O cards for industry are available

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

16

SuperKEKB Controls

Standard EPICS
CA as “Software bus”

CA

IOC

Accelerator ReviewIOC

IOC

IOC

Kazuro Furukawa, KEK, Feb.2011.

17

SuperKEKB Controls

Channel Access on MicroTCA Backplane
CA on Hardware “bus”

CA
GbEther
IOCIOCIOCIOC

MicroTCA
Picture by J.Odagiri
Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

18

SuperKEKB Controls

Let’s employ Channel Access on µTCA
Backplane !
Let’s embed EPICS IOC on to Each µTCA/
AMC Card !

Miura and Furukawa
Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

19

SuperKEKB Controls

IOC on MicroTCA
Natural to put IOC on µTCA LLRF Controller
Chose GbEthernet as a main media on thebackplane interconnect
 Somewhat unique Some other institutes chose PCIe as the media

Chose PowerPC core on Virtex5
 ML507 test card from Xilinx as a good referenceLinux on PowerPC No realtime processing is necessary at Linux level In the future, we may use realtime (<100µs) feature of LinuxAccelerator Review

Kazuro Furukawa, KEK, Feb.2011.

20

SuperKEKB Controls

EPICS IOC on MicroTCA LLRF Controller
 Linux 2.6 on Virtex5
 Boot from Flash (or over network)
 EPICS from NFS
 Slight difficulties because …
 Lack or experiences Relatively straight forward

 EPICS 3.14.9 (for now) Driver to FPGA (with mmap)
 Scalar values directly, and waveform/arrays through ringbuffer Channel access on backplane interconnect
 Directly connected to outside at first Possible to install gateway at local CPU if necessary Collaboration between RF group, Mitsubishi Elec. Tokki System
Co. Ltd., and Control group has been going wellAccelerator Review

Kazuro Furukawa, KEK, Feb.2011.

21

SuperKEKB Controls

BPM at Linac and BT
 BPM read-out for 0.1mm resolution
 For KEKB/PF Injection, Not for SuperKEKB

 Using ~30 coefficients per BPM

 Many signals are combined
into one waveform digitizer Recent Embedded IOC Solution
 Again reduction of resources
 (and much helped by Dr. Hu from Shanghai)
100 BPMs

24 x DPO7104

Clients

 Old configuration
100 BPMs
Accelerator Review

19 x TDS680B

19 VMEs~5 Unix

Clients
Kazuro Furukawa, KEK, Feb.2011.

22

SuperKEKB Controls

BPM DAQ Tektronix DPO7104 can acquire data at >50Hz.
 With embedded EPICS, under pulse-to-pulse beam modulation

 Beam modes are recognized by events through CA network.
 Missed less than once in million times

 Clients can monitor data of an interested beam mode.
 24 oscilloscopes are installed for Linac, and 4 for BT.
 200 BPMs are synchronized for Linac and BT.

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

23

SuperKEKB Controls

Several other Embedded IOC
Other oscilloscope-based IOCs
 8-electrode and other monitors (M. Satoh et al)

TDC with Linux/ARM (Armadillo) Timing consistency surveillance (S. Kusano et al)Magnet PS with Linux/ARM(or FPGA) Two prototypes for SuperKEKB (T. Nakamura et al)

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

24

SuperKEKB Controls

Event-based Controls and Timing

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

25

SuperKEKB Controls

Simultaneous Injection
 SuperKEKB HER/LER injections under shorter lifetimes
 PF top-up injection for higher quality experiments
 Simultaneous (non-topup) injection to PF-AR not to disturb SuperKEKB operation

 Achievement in KEKB complex
 Reduced the Beam Switch Time from 10-120 seconds to 20ms
 Beam currents are kept within 1mA (~0.05% for KEKB), 0.05mA (~0.01%, PF) Much more complicated with bucket selections at damping ringand main ring
 Should add PF-AR as well not to disturb SuperKEKB operation
 Through damping ring 4rings! with different beams
 Beam Juggling
Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

26

SuperKEKB Controls

Event System for Simultaneous Injection
 MRF’s series-230 Event Generator / Receivers
114.24MHz event rate, VME64x and VxWorks v5.5.1
50Hz fiducials
 EPICS R3.14.9 with DevSup v2.4.1
More than hundred 17 event receivers up to now
50Hz-Analog/Timing data
CentralEvent Generator
Multi/single-mode fiber
Timing precision is < 10ps.
SH_A1
KL_B5/B6

SB_BInjection

SB_Ae­ GunARC

 < 1ps with external module.

e– BT (PF: 2.5GeV, 0.1nC)

Cont-ABC>100ns

KL_51/52
SB_C

SB_1

SB_2

SB_3

SB_4

SB_5

96nse+ BT (KEKB: 3.5GeV, 2nC)

Event Receivers e Target+Cont-1

Cont-2>100nsCont-3Cont-4Cont-5e– BT (KEKB: 8GeV, 2nC,
PFAR: 3.0GeV, 0.1nC)

96ns

Energies will be changed and DR will be added
Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

27

SuperKEKB Controls

Fast Parameters
Parameters switched via event system
Each parameter has at least 4 flavors (HER,LER,PF,PF-AR and maybe stealth study modes)

LLRF phase/timing : 14x4
Overall energy profile, dual-bunch energy equalization, final energy adjustment

HP RF timing : ~60
Energy profile and backup management

Gun voltages, picosecond delay : 4
Beam charge selection, dual bunch selection, bunchingPulsed magnets/solenoid : 14
Beam transport selection, orbit controls, positron focusingKEKB injection RF phase, timing, bucket selection : 2x3
BPM : ~100x3

Damping ring injection should be added
Integrity monitors soonSome of the modules will be developed with SINAP/Shanghai
Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

28

SuperKEKB Controls

Three Virtual Accelerators
 Controls and instrumentations are essentially mode-dependent, andmutually independent Selecting a real machine out of three virtual machines Managing three parameter sets (four under SuperKEKB environment)e− Gune– BT (PF: 2.5GeV, 0.1nC)

PF Injection�

ARC

e+ Target

Event-basedControl Systeme− GunARC

KEKB-LER Injection�

Primary e– (4GeV, 10nC)

e+ BT (KEKB: 3.5GeV, 0.6nC)

e+ Targete− GunARC

KEKB-HER Injection�
e+ Targete– BT (KEKB: 8GeV, 1.2nC)

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

29

SuperKEKB Controls

Three-fold Independent Closed Loops Feedback loop software act on one of three virtual machines Managing independent parameter setse− Gune– BT (PF: 2.5GeV, 0.1nC)

PF Injection�

ARC

e+ Target

Event-basedControl
Systeme− GunARC

KEKB-LER Injection�

Primary e– (4GeV, 10nC)

e+ BT (KEKB: 3.5GeV, 0.6nC)

e+ Targete− GunARC

KEKB-HER Injection�
e+ Targete– BT (KEKB: 8GeV, 1.2nC)

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

30

SuperKEKB Controls

For SuperKEKB Complex
 Slightly More Complicated Conditions with DRHarmonic number of SuperKEKB-MR (509MHz) is 5120
Common frequency between Linac-MR is 10.38MHz (49 buckets, 96ns)
DR should have common frequency, RF chosen to be 509MHz
2x2 bunches, bunch separation of 49-bucket , kicker rise/fall time of 100ns
Jitter (wait-time) of HP modulator (50Hz) must be <~2ms Harmonic number of DR was chosen to be 230

 In order to Select All the Buckets in SuperKEKB MR
 Active (Pulse-to-pulse) LLRF controls necessary at linac
 Better LLRF monitor is required Dependency between pulses increases

 For PFAR Injection
 Positron have to be used to share the beam-transport
 Independent circumference controls will interfere

PF can use 2.5GeV electron with accidental synchronization (<~300ps)

 More investigation underway
Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

31

SuperKEKB Controls

Other Developments

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

32

SuperKEKB Controls

Linac BPM
Resolution ~10µm required Emittance preservation with beam-based alignmentConditions
 FTE of ~1.0
 Two-bunch measurement apart by 96ns
 Large dynamic range (~x100) for four rings
 Pulse-by-pulse

Present evaluation
 Libera single-pass box, with modifications
 Embedded EPICS IOC possible, maybe with EVR interface

 If not applicable, modifications to other solutions
Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

33

SuperKEKB Controls

Wire-scanners at Linac and BTCAMAC elimination, if possible
 VME read-out evaluation this year for newinstallation

Pulse-by-pulse Read-out
 Controlled by event system Noise elimination with the same detection
condition?

For lower emittance Maybe correction by surrounding BPMs needed

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

34

SuperKEKB ControlsMore Measurement Technique
Switching between Four Rings
 Challenging to improve beams during operation

Event-based Controls may Help More
 No-destructive measurements with four beams
 Stealth (used beam-pulse) measurements
 With beam deflector

 Dithering pulse-by-pulse If very good resolution was achieved

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

35

SuperKEKB Controls

PPS, MPS and others
Personnel Protection System
 Will be managed by Safety group (Mimashi-san) Will have Embedded EPICS to PLC system

Machine Protection System Continue to use KEKB Solution
 ARCnet read-out could be replaced by Ethernet Interface
 TTL (50ohm) and Relay signal interfaces
 ~1µs signal transfer

More Device Level Developments
 For Vacuum, Magnet, and so on
Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

36

SuperKEKB Controls

Conclusion
Steady Improvements/DevelopmentsCollaboration between Groups is Essential
International Collaboration is also Proceeding
Controls Have Interface to Every System Controls can Enjoy Accelerators
 With Phronesis (Greek: Ability to understand the Universal Truth)

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

37

SuperKEKB Controls

Thank you for your patience

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

38

SuperKEKB Controls

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

39

SuperKEKB Controls

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

40

SuperKEKB Controls

RF Group

MicroTCA based LLRF Controller Single-width full-height module Without physics experiment extension (we started earlier) Front-panel connectors (rather busy)

 Digital part and Analog part are separate
 ADC 16bit, 130Msps, x4
 DAC 16bit, 500Msps, x4
 Virtex5 with PPC440
 RAM 640MB, Flash 64MB
 Also monitor card with the same digital part
 ADC 14bit, 400Msps, 1.4GHz, x2

 Fabrication subcontracted
 Mitsubishi Electric Tokki System Windriver Linux (ML507 is supported)
Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

41

SuperKEKB Controls

RF Group
Mitsubishi

Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

42

SuperKEKB ControlsMore development
FPGA controller is ready (Sep.2010)
EPICS IOC and local display application
 With EDM under development

µTCA management capabilities over IPMI
Commissioning in 2011 for STF and cERL
 Slightly later for SuperKEKB

Future Comparison to ATCA
 Move out of Windriver Linux (?)
 Redundant System (?)
…
Accelerator Review

Kazuro Furukawa, KEK, Feb.2011.

43