ACCELERATORRELIABILITYAVAILABILITY
L. HARDY, ESRF

WAO 2003 – JAPANTHE BASICSAVAILABILITY:AVAILABILITY fraction of TIME during which a systemmeets its specification.
à High availability required if continuous service is priority
RELIABILITY: PROBABILITY that a system can perform
its intended function for a specified time interval understated conditions.
à High reliability required when repair of sensitive subcomponents are long (or difficult)A BRIEF HISTORY OF RELIABILITY…

September 1944
First RELIABILITY MODEL:Von BRAUN AND LUSSER:« for a series system:
Rs = R1 * R2 * * Rn
where Ri is the probability ofsurvival for 1 element. «V-2 rockets reliability
problems …

A BRIEF HISTORY OF ACCELERATOR NON-RELIABILITY…

1924: Ising:
the first
Linac model

1928:
Wideröe: amodel basedon RF voltage

March 1936: thefirst external
cyclotron beam: 5.8
MeV deuterons

The race to accelerators physics principles…

A BRIEF HISTORY OF ACCELERATOR NON-RELIABILITY…

1941: 184-inch
cyclotron (>100 MeV)
for U235 / U238
separation (Berkeley)

1947: proton
Linac built under
supervision ofAlvarez

1959: CERN 24 GeVPS is the highest
energy accelerator inthe world

PS inauguration (1960)

The race to higher intensities, higher energies…

A BRIEF HISTORY OF ACCELERATOR RELIABILITY…

( + PSI, TRIUMF)
1972: meson
1981: DEDICATED Xfactories. E.g.: Los
ray sources. FirstAlamos (LAMPF):
one is SRS (UK)
800-MeV beamachieved

MEDICALAPPLICATIONSLAMP aerial view

Rosen: « more particles per unit
time rather than more energy perparticle ! »USERS WANT RELIABILITY
AND AVAILABILITY !!

APPROACH WITH CONCRETE EXAMPLES• SINQ:
SINQ a continuous spallation source• LANSCE:LANSCE a pulsed and versatile proton source
as examples of « extrapolable » accelerators for ADS• ESRF:
ESRF an X-ray source

SINQ:
SINQ a Continuous Spallation Source (Switzerland)
870-keVproton

72 MeV pcyclotron

590 MeV-1.8mA cyclotron:~ 1 MWSpallation targetSINQ:
SINQ a Continuous Spallation Source (Switzerland)
Availability vs beam intensity
100
90
80

1989
1991
1992
1993

60

1994
50

1995
1996

40

1997
1998

30

1999
2000

20

2001
10

1900

1800

1700

1600

1500

1400

1300

1200

1100

1000

900

80
0

70
0

60050040030020010000

AVAILABILITY (%)

70

Beam intensity - µA

Courtesy of P. Schmelzbach/PSISINQ:
SINQ a Continuous Spallation Source (Switzerland)Fighting this is the real
challenge!
Fighting this is aquestion of budget !

Courtesy of P. Schmelzbach/PSISINQ:
SINQ a Continuous Spallation Source (Switzerland)
A few events only are responsible for large downtime:-Cooling: missing redundancy-Magnets: savings on spare parts, time consuming repairs-RF: use components until it fails
According to PSI cyclotron experts: no technologicalobstacles. More a financial problem:-Replace 25-year old power supplies (in progress)-Fully assembled spare parts for magnets (in progress)-Redundancy of cooling water plant (to be decided ?)-Better interchangeability of sub-equipment to decrease MTTR

SINQ:
SINQ a Continuous Spallation Source (Switzerland)
What about short trips / reliability ?
• Short interruptions after beam trips: ≥ 20 s to ramp the intensity.
About 10000 per year = 1% of the beam time
• Electrostatic elements in the Ring Cyclotron: responsible for mostof the beam trips. Seems to be critical as the power increases (1.5 ->1.8 mA). The behaviour of the electrostatic elements is far frombeing understood. R&D is needed (design, sensitivity to RF-leakageand X-Rays, surface physics, beam halo, …)(Extensive R&D work is carriedout to understand RF arcs, e.g:
Werner et al. )Investigation of Voltage Breakdown Caused by Microparticles. Werner et al. PAC 2001 proceedings

SINQ:
SINQ a Continuous Spallation Source (Switzerland)

GOAL: reduce RF trips and MDT ! HOW DID THEY DO IT ?
•Better conditioning of RF cavities,•Improvement of preventive maintenance:•Limited lifetime components (RF tubes are replaced after a
pre-determined operation time)•‘Unlimited’ components periodically inspected, tested.
• DO NOT turn-off beam during self-recovering µ-sparks (< 200µs) in a cavity + automated ramping procedure to recover RFpower within 5 seconds and full beam after 20 s
•A failure = opportunity to learn and improve the design•Shutdown RF driver in a few µs when a spark is detected (thusavoiding evaporating metal)

WAS IT WORTH MAKING SO MUCH EFFORT ?
Inputs from P. Sigg/PSI

SINQ:
SINQ a Continuous Spallation Source (Switzerland)

Courtesy of P. Sigg/PSI

SINQ:
SINQ a Continuous Spallation Source (Switzerland)
When the failure is there: reducing the Mean Down Time
•Improved fault diagnostics and event data logging•Ready-to-operate units available (spare parts)•Design for fast interchangeability of equipment•Modular design at all levels
This policy applied for 15 years at PSI dramatically increased
their reliability / availability.

It mainly required ideas, manpower,willingness to improve and RE-design whennecessary !
Inputs from P. Sigg/PSILANSCE:LANSCE a pulsed Spallation Source (Los Alamos)

1 for H+, 1 for H-750 keV
100 MeV

750 µs pulseà 0.25 µs

800 MeVLANSCE:LANSCE a pulsed Spallation Source (Los Alamos)
Extensive and thorough reliability/failure studies weredone at LANSCE (ref: Marcus Eriksson MSc thesis)
Overall statistics:
H+ beam
1.6 trip / hour (4655 trips/2870 hours)
General availability: 86 %H- beam
0.8 trip / hour (4020 trips / 5144 hours)
General availability: 85 %Main problem = repetitive failures (reliability)« Details » of the weak points:
H+ Injector
90 % of trips < 1 minute !
70 % of all trips (3584 trips in 2870 hours)
30 % ‘only’ of total downtime
H- Injector
40 % of trips < 1 minute !
26 % of all trips (3584 trips in 2870 hours)
11 % only of total downtime
BUT, these are Cockroft-Walton type injectors. This can
NOT be extrapolated for future accelerators! WHY ?

Nowadays, many ion sources have been designed with thepurpose of having a high rate of reliability. Example:SILHI source on IPHI project (CEA)

Courtesy of P-Y Beauvais (CEA)

Parameters

Oct. 99

June 2001

Energy (keV)9595

Intensity (mA)

75

114

Duration (h.)

104

162

Beam off number

1

7

MTBF (h)-23.1

MTTR (mn)

2.5

2.5

Availability (%)

99.96

99.8

LANSCE:LANSCE a pulsed Spallation Source (Los Alamos)
Behind the injectors, the RF system is the most sensitiveequipment:-8 % of the beam trips (mainly coming from High Voltage System)-23 % of the total downtime
PRELIMINARY CONCLUSION
Failure database of existing facilities are of primary
importance to understand and correct weakpoints.
HOWEVER, they must NOT be extrapolated ‘blindly’for future machines ! Technologies are evolving …

The ESRF: a third generation X-ray source

Particles

Electrons

Energy6 GeVIntensity

200 mA

Hours/year 5600

The ESRF: a third generation X-ray source

X-ray beam availability (INCLUDING Refills)
98.5%
98.0%
97.5%

AVAILABILITY

97.0%
96.5%
96.0%
95.5%
95.0%
94.5%
94.0%
93.5%
AVERAGE
1997

AVERAGE1998

AVERAGE1999

AVERAGE2000

AVERAGE2001

AVERAGE
2002

The ESRF: a third generation X-ray source

MTBF and Mean Down Time over the years
9 periods of delivery > 120 hours without a single failure !!
57.8

60

46.1

50

38.0

Hours

4030

31.9

30.6

30.1

20
Mean Timeof 1 failure

101.5

1.1

0.8

0.9

1.1

1.2

2002
AV
ER
AG
E

2001
AV
ER
AG
E

2000
AVER
AG
E

1999
AV
ER
AG
E

1998
AVER
AG
E

AV
ER
AG
E

1997

0

The ESRF: a third generation X-ray source

All failures are recorded in a database, analysed èsolutions and strategies are proposed.
1. Electrical mains drops (mainly due to storms) are detected andcompensated for by 10 Diesel engines (total = 10 MVA)

10 X

Courtesies from JF Bouteille

COSTS

RESULTS

Investment: 6 M€

150 drops/year covered without
starting the Diesel engines

Maintenance: 60 k€ / year

60 severe drops/year compensatedby starting the Diesel engines

BOTH REPETITIVE AND LONG TRIPS NOW AVOIDED !

The ESRF: a third generation X-ray source
2. Complete redundancy of RF system
Cavity#1

Cavity#2

Cavity#3

Cavity#4

5MV - 500kW

5MV - 500kW

Dummy
Load

CavityTest Stand
Dummy
LoadKlystron

HVPS
100kV-22A H.V.P.S

Klystron
HVPS
100kV-22A H.V.P.S

TRANS#2 1.3MW RF

TRANS#1 1.3MW RFBooster+ third klystron installed on a new third pair ofcavities à decrease RF power per cavity window

Courtesy of JM Mercier

The ESRF: a third generation X-ray source

RESULTS
1998 1999 2000 2001 2002
Number ofRF trips

120102785337

Time lost dueto RF (h)7036332714.3

The ESRF: a third generation X-ray source
3. THE POWER SUPPLIES SWITCHING BOARD:
An efficient system to minimize power supplies failure
downtime.

Dummy load1200 V – 800 A
Courtesy of JM KOCH

COST: 360 000 €Switching
Time=20mn

The ESRF: a third generation X-ray source•Fighting all failures, long and repetitive, is a top-priority
at ESRF•Recording, analysing failures and defining strategies tomake them disappear is now almost a FULL-TIME job !
•Long failures ARE AVOIDABLE but this costs (a lot of)money (because of redundancy). Real origins are generally
quickly and well identified.
•Short failures ARE AVOIDABLE but this costs manpower,time, because origins are generally long and difficult toidentify.
•Strategies to avoid human mistakes remain a challenge!
(the fifth cause for time lost at ESRF in 2001 …)

A BRIEF FUTURE OF ACCELERATOR RELIABILITY…

ACCELERATOR DRIVEN SYSTEM à-SPALLATION SOURCES-NUCLEAR WASTE TRANSMUTATION-SUB-CRITICAL NUCLEAR REACTOR !
10th March 2003

2015

2050

Typical HPPA
parameters:

Feasibility
demonstration

Industrial applications

800 MeV – 10mA proton< 10 TRIPS/YEAR… and even 2-3 / year
for an industrial
accelerator !

IS IT REALISTIC ? WHAT IS THE PRICE TO PAY ?

Dream Machines: the price to pay, the efforts to make1. AVOID LONG TRIPS: generally a question of money (redundancy)•Diesel engines : 6 M€ for 10 MW + 60 k€/year•Power supply switching board: 360 k€•RF redundancy , water pumps redundancy, Control system
redundancy, etc, etc•Preventive maintenance: Frequent shutdowns must be foreseen

Excellent cost study made by R. Ferdinand et al. on the ESS Linac:
A conservative and reliable Linac for ESS costs 50 % more than a‘nominal’ design: 157 M€ + 77 M€ reliability = 234 M€
The yearly operation cost is increased by about 1.3M€

Dream Machines: the price to pay, the efforts to make

Cryogeny: reliability aspects
Turbines in the Cold Box are the most sensitive elements: at low
working temperature, any impurity becomes a solid pellet which candamage the turbine wheel. So, the important point for the turbine
reliability is the impurities level control in the helium flow

Cryogeny experts agree that 5000 hours are necessary
to improve the system reliability (‘childhood diseases’),train Operators, etc. Afterwards, reliability isexcellent : > 99 % !
KEK: 137000 hours experience: reliability = 99.2 % !
Fermilab: 76000 hours experience: reliability = 99.5 % !
CERN: 120000 hours experience: 99.3 % !
BUT … Sub-component redundancy is THE key point !!
Thank you to C. Commeaux for his inputs

Dream Machines: the price to pay, the efforts to make
2. Avoid REPETITIVE trips:
Lot of REALISTIC R&D in progress and now showing promising
results:- Ion source reliability (SILHI, CDPADS in China, others)> 100 hours run @ high intensity without single failure !
- Realistic schemes for ACTIVE REDUNDANCY sections fromADS Linac systems. An active spare section can take over astopped section within 50 msec !(Huge work of Kozodaev et al.)Kozodaev et al.

- RFQ design optimization
Now a standard for low-energy ion injectors-Vane shape optimization (<->minimization of electric field)-High pumping capacity à minimisation of bursts and hence
sparksà Beam interruption probability is minimized

A CHAIN IS AS STRONG AS ITS WEAKEST LINK !
FAILURE-Design optimization (margin !)-FAULT DIAGNOSTICS !!

-Preventive Maintenance-Rigorous Spare part Policy-Experience from otherInstitutes-Experts on standby 24hours/day ready to intervene-Realistic Operation Schedule-Operator’s training-Redundance-Fast interchangeability ofcomponents (<- design …)-Avoid human mistake withautomation when necessary-Repair procedures

CONCLUSION•Reliability / availability is now a priority forparticle accelerator designers: great progress !
• Many existing devices ARE already reliable !
• R&D for future ADS is difficult BUT REALISTIC.
Promising results have ALREADY been seen
• Parallel efforts must NOT be neglected (spareparts, training, procedure, etc)

Achieving « Dream Machines » is no longer a
dream but will require the best expertise forall links in the design chain