# OPeration before LHC operation: The SM18 Magnet tests facilty

V. Chohan, CERN, Geneva, Switzerland

**ABSTRACT**

The Large Hadron Collider under construction at CERN has 1706 cryo
magnets that need testing under cryogenic conditions prior to installing
them in the tunnel. These tests are not feasible at the manufacturers
and hence, are carried out at CERN at a purpose built facility on the
site (SM18). To fulfil the tests requirements & needs of various groups
such as the magnet production follow-up engineers and accelerator
physicists, it was important to set up and run operations at SM18 in a
simple & transparent manner. This presentation will give an
understanding of the various issues that had to be taken into account to
set up Operation. A flavour of the operation tools that were necessary
or developed will be given. These include web-based tests information
systems, development of precisely defined 'to do list' of tests
sequences, associated methods, procedures with strict check-lists,
electronic logbooks and so forth. With the setting up of an Operation
Team comprising of a few CERN Accelerator operation staff and large
number of external collaborators, particular care had to be taken of
several issues to permit (the first ever) unique aspect of setting up
operation of the tests facility in SM18.

## Introduction 

LHC (Large Hadron Collider) will use the most advanced superconducting
magnet & accelerator technologies ever employed. It will consist of two
colliding beam storage rings, installed in the 27 km tunnel used in the
past for the LEP machine. The two rings are constituted by large
double-bore, two-in-one magnets. The two superconducting magnet channels
will accelerate the protons to 7-on-7 TeV before they are made to
collide. To bend the 7 TeV protons around the ring, the LHC
superconducting dipoles must be able to produce fields of 8.33 Tesla
which require a current of 11850 A, possible only under the cryogenic
regime of super-fluid helium at 1.9 K. 1232 Dipole magnets (with
correctors) are used for bending the beams around the circular tunnel.
Besides, 360 Short Straight Sections with arc quadrupoles are needed for
the standard focusing, defocusing lattice functions. To direct the beam
towards the interaction zones, 114 Matching & Dispersion Suppressor
region magnets integrated in Special SSS (IR-SSS) need to be installed
in the tunnel.

The testing of the above magnets at cryogenic temperature is
prerequisite to their installation in the LHC tunnel. These tests are
not feasible at the manufacturers; hence a facility was built at CERN
(SM18). The facility consists of 12 test benches arranged in 6 clusters.
Each bench is fed independently with a cryogenic feed box (CFB). The
benches in a cluster share common electronics and power source. This
means that at any time, only one bench of a cluster can undergo cold
tests within the limits of cryogenic availability. The dwell time of a
magnet on a test bench accounts for the electrical and mechanical
connection of the magnet to the test bench, cooling down time, cold
tests, warming up, disconnection and the final removal of the magnet
from the test bench. The total activity takes typically 120 hours. To
complete the magnet tests in time for the LHC start-up and for better
utilisation of assets per 24 hour period, a round the clock operation
had to be implemented at SM18. CERN being a laboratory with years of 24
hr. accelerator operation experience for physics, it was natural that
concepts and methods of operation envisaged for SM18 were in line with
known & established ways of carrying out 24 hr. operation.

## ![](media/image1.jpeg){width="3.2423611111111112in" height="1.6027777777777779in"}

Figure 1: View of SM18 Tests Facility

## OPERational ISSUES

For an operation group to take over the running of the facility, it was
important to evaluate the "who's" and "how's" of operation. So the first
question which arose was who would operate this facility. One should
recall that in 1999-2000, CERN had a budgetary crisis and outsourcing
the tests operation was no more an option. Hence it was imperative to
make use of existing CERN staff for the tests purposes. However, due to
the need of CERN accelerators for physics experiments and LHC detector
tests, it was not possible to shut down the accelerators and liberate
operation staff. The management did provide 7 staff having experience in
running some of the CERN accelerators like the PS, PS Booster and the
SPS, but the bulk of the operation staff required for 24 x 7 operation
had to be found elsewhere. It was at this point that a CERN-Dept. of
Atomic Energy (DAE), India collaborative effort came into being. It was
agreed to utilise human resources from India for the purpose of testing.
Since then, up to 18 qualified personnel from India participate in the
magnet tests on a strict one year rotational basis. For India, the
objective is to expose as many technical engineers to modern high
profile work in a large international project, hence the one year strict
rotation. The Indian technical engineers also come from 4 different
national establishments with specific backgrounds and work cultures not
directly related to operation or CERN type of work.

The difficult question was however the "how's". To understand the
problems, it is useful to know the conditions that existed in mid 2002
(the time when operation had to be set up for the running of the tests
facility). The work environment which existed then is described below:

-   Considering that the cryo magnets are in a way unique, there was a
    lot of unknowns and scientific curiosity among the various groups
    such as the magnet designers, magnet production teams and
    accelerator physicists. Tests were conducted with a research &
    development mindset. There was no definition of a tests sequence for
    routine testing.

-   The Tests were conducted by magnet and equipment specialists.
    Partially automated test systems which existed then were considered
    good enough. Tests were conducted by connection and disconnection of
    cables from the magnet, test bench and flicking switches in the
    electronic racks on the shop floor. The experts found it difficult
    to understand the complexities involved to run a facility in a round
    the clock fashion.

-   The equipment for the some tests was of laboratory type, which was
    unsuited for 24 hrs operation. Many measurement systems depend on
    mobile racks (even today), imposing a lot of cable connections /
    disconnections leading to possible errors.

In addition, the role and responsibility of the operation team was not
clearly defined. The operation team was initially instructed to test
only dipoles. When the testing of arc-SSS was moving too slowly, the
management had to transfer this activity as well to the operation team.
The responsibility of the operation staff initially was only to conduct
tests. As other operational factors came into being, their
responsibility increased to analysing & interpreting results, data
logging and evaluating the first-level goodness of the magnets. Thus a
patchwork approach was followed in defining the role and responsibility
of the operation team even though enough experience existed at CERN to
have done it otherwise (for example, experience from a new accelerator
start-up, running-in, commissioning and routine operation).

To start and run operation in this environment has been a challenging
task. It was imperative to formulate an operational methodology taking
into account all these factors. Techniques had to be fashioned to be
flexible yet at the same time giving a semblance of order.

## OPERATIONAL METHODOLoGY

Some guiding principles in setting and running operation successfully
are to create an ambience of transparency with a sound knowledge base.

1.  **Transparency:** Transparency is critical in operation as it
    instils confidence in the operator with regards to what has to be
    done and when. In addition it must be stated that the operation team
    from CERN and India had absolutely no previous experience in testing
    of superconducting magnets. To set up operation, the following
    initiatives, some of them simple, obvious but necessary had to be
    implemented:

-   A "To Do List" was created, which described the minimum tests to be
    performed on a magnet. The tests were sequentially numbered &
    prefixed with the nature of tests i.e., "Preparatory test" (PREP) or
    "Power test" (PT) as shown in Fig 2. Demands on conducting
    additional tests are still made, but as a rule, it was agreed that
    these would be carried out only if requested in advance.

![](media/image2.png){width="3.047222222222222in"
height="4.209027777777778in"}

Figure 2: Tests list for Dipole magnets

-   In this day and age of computers, we still had to use a paper log
    (Magnet Tests Report) of tests conducted on a magnet as the readings
    for various tests had to be taken on the shop floor. The Magnet
    Tests Reports were designed in an easy-to-use manner with
    operational notes /checklists appended wherever necessary. The flow
    of tests in the templates obeys the "To Do List". This ensured that
    the tests were carried out systematically, efficiently and in a fail
    safe sequential manner.

```{=html}
<!-- -->
```
-   Transfer of information between shifts is essential for smooth
    operation. In a dynamic environment of SM18, where additional tests
    have to be carried out often, it assumes even further importance. To
    facilitate smooth transfer of shifts authority & information, a
    white board was added in the control room. The white board
    summarises the tests done or pending on a magnet. Additional
    information, instructions as well as tests priorities are also
    written on this board. Hence information transfer between shifts has
    become seamless and transparent.

2.  **Knowledge base:** It is essential to provide and increase the
    knowledge of the operation team for superior performance and
    thoughtful execution of tests.

-   Routine error-free operation is only possible by means of organised
    methods and procedures. A documentation strategy was developed which
    facilitates easy integration of rotating staff to the tests
    operation. The documentation consists of several levels of
    complexities from a single sheet "Tests made easy" to a brief but
    complete description of each test, further example trouble shooting
    aids as well as full checklist of the work to be carried out. These
    documents are written by the operation staff for the operation team,
    in consultation with experts. Based on the above strategy, a
    specific website was designed and is routinely used by the operation
    team.

-   Training is part of human resource development. The shut-down period
    of SM18 facility in December/January is utilised for this purpose.
    Talks by equipment and magnet experts are arranged. Factory visits
    to a manufacturer's site has help the operation team to visualize
    the magnet as being not a coloured tube that arrives on a test bench
    but a complex engineering product. "Look & feel" awareness for cryo
    magnets is generated by these visits.

-   To ingrain safe operating practices among the operation team, it is
    mandatory for the operation team to attend safety courses in
    electricity and cryogenics. Procedures & Instructions to follow in
    matters of safety are documented and available at all times in the
    control room. A call-out list of equipment and services experts is
    maintained.

## Additional tools 

To improve operational efficiency and smooth the work flow, operational
tools were designed and implemented. It is true that the best
suggestions come from the personnel on the shop floor. At SM18 too, many
innovative ideas and proposals emerged from discussions held with the
operation team. Some of these schemes were developed thanks to the
ingenuity of the operational personnel. A few of these are described
below.

\(1\) The development of SM18 Tests Management System (SMTMS) was based
on fulfilling a need. The operation team were required to fill out 11
WORD reports per cryo-dipole after the tests were completed. They
detested this time consuming 'bureaucratic' activity. The solution was
to generate it automatically, hence SMTMS was developed. It uses the
CERN backbone in Web based services and features of the ACCESS database.
Tracking of tests progress and tests statistics were spin-offs of this
utility. These extras have enhanced the utility of this application in
the control room. SMTMS and its features are covered in detail in
another paper at this workshop. Tests information available in SMTMS is
shown in Fig. 3.

![](media/image3.wmf){width="2.582638888888889in"
height="1.929861111111111in"}

Figure 3: Real time information on Tests

\(2\) With 12 benches in operation on a round the clock basis, the tests
scenario in SM18 is very dynamic (with several magnets leaving, several
arriving and other magnets in different stages of cooling / testing /
warming). Hence it is necessary to coordinate efficiently the above
activities which involve different actors like the Tests Operation, the
Cryogenics and the magnet movement /connection / disconnections crew.

![](media/image4.jpeg){width="3.1791666666666667in"
height="1.7479166666666666in"}

Figure 4: Main Page of E-Traveller

![](media/image5.pct){width="2.5055555555555555in"
height="2.9583333333333335in"}

Figure 5: Work flow page of E-Traveller

> A computer based tool in the form of a workflow manifest (called
> e-Traveller) was created. The interface of this software tool with
> mobile phones helped in alerting & informing the relevant teams of the
> requirement of their services on a particular magnet. This tool also
> reduced the verbal communication between teams; it helped the Indian
> collaborators, whose knowledge in French is limited, to interact with
> predominantly French speaking other teams. Fig. 4 & 5 show different
> pages of this tool.
>
> ![](media/image6.png){width="2.759027777777778in" height="2.20625in"}

Figure 6: Integrated Resource Usage Display

\(3\) Resources such as water, power, and cryo capacity are limited in
SM18. The electronics too is common between benches of a cluster. For an
operation team to work in such an environment requires them to juggle
with these resources for the tests. It is necessary that the operators
know the real time status of the resources to effect a change. For these
reasons it was important to integrate the resources information on a
single window. A graphical display to highlight the status of the
various resources was generated by providing interfaces with software
programs controlling the resources and running under different operating
systems. This enabled the operation group to optimise resource usage and
prevent resource conflicts from arising. The resource display is shown
in Fig. 6.

## COLLaboration issues

For an external collaboration effort from another continent to succeed,
it is necessary to provide a friendly and cordial atmosphere which
belies the fact that the personnel are in a totally alien work & social
environment. Providing help to the external collaborators in matters of
accommodation, transportation and other infrastructural needs due to
alien culture and language enabled them to focus on work. As a matter of
policy, the CERN & Indian personnel work together round the clock in
mixed teams. Shift rotation plans are drawn accordingly, which helps in
building a healthy work attitude, team spirit & camaraderie. Members of
the operation team are encouraged to partake in novel culinary
experiences, celebration of festivals and sports activities, all of
which help in team building and creating a conducive working atmosphere.

Considering that the control room has personnel from various CERN
accelerators principally from two different control room experiences
(ex-MCR, ex-PCR), Indian personnel from four different establishments
and experiences and non-CERN contract personnel handling the cryogenic
systems, it is a unique arrangement. This is the first time at CERN, if
not elsewhere, that such a large and diverse collaboration has been
successfully assembled for 24 hr. operation. It may also serve as an
example for collaborative operation of a world accelerator.

**CONCLUDING REMARKS**

> There is no universal "setting up & running" methodology to guarantee
> operational success. Each facility has its own characteristic problems
> that need to be addressed & solved. Methods implemented for testing
> the LHC Cryo magnets at SM18 have been described. Operational
> simplicity has been achieved as well as the synergy gained among the
> various actors, thanks to some of the techniques implemented. It is
> hoped that this paper gives an insight to the complexities of the LHC
> project and the spirit of "Operation before LHC Operation".
>
> **ACKNOWLEDGEMENTS**

I would like to thank all the past & present members of the SM18
Operation Team from DAE-India & AB-Operation, CERN for their effort in
streamlining the operational methodology and bringing the tests
programme to the present level. I would acknowledge the effort of George
Hemelsoet & Eric Veyrunes AB-OP, D.Bakker AB-CO and J.Axensalva AT-ACR
for providing the operation team with the various tools & solutions. The
author is grateful to Martin Mascarenhas, BARC-DAE, India for assisting
in preparing this paper.