# DETECTOR Control SYstem of BESIII

Xihui Chen^a,b^, Xiaonan Li^a^, Xiaoxi Xie^a^\
Cuishan Gao^a^, Yinhong Zhang^a^, Jian Min^a^, Zhendong Nie^a^, Yigang
Xie^a^

^a^Institute of High Energy Physics, Chinese Academy of Sciences,
Beijing 100049, China

^b^Graduate School of the Chinese Academy of Sciences, Beijing 100049,
China

Abstract

In the upgrade project of Beijing Electron Positron Collider (BEPC) II,
a novel Detector Control System (DCS) for Beijing Spectrometer (BES) III
is under developing. The principal task of the DCS is to supervise and
control the status of the BESIII detector and to guarantee a safe
operation to the detector. The DCS must provide a uniform and coherent
interface to the detector operators even though there are a large number
of distributed I/O channels from large variety of equipments. For this
reason, the DCS is hierarchically organized and divided into three
layers: Front End Layer (FEL), Local Control Layer (LCL) and Global
Control Layer (GCL). In the FEL, ranges from simple sensors up to
complex computer-based devices like embedded systems and Programmable
Logical Controllers (PLC) are utilized. A LabVIEW based software
framework has been developed for the LCL. Network communication and web
server technologies have been used for the GCL. This paper will give a
detailed introduction to the design and implementation of the DCS. Its
performance and reliability will also be discussed in the last.

## INtroDUCTION

The BESIII experiment \[1\] at the upgrading Beijing Electron Positron
Collider (BEPCII) with an energy range of 2--5GeV, a peak luminosity of
10^33^cm^-2^s^-1^ at 3.78GeV \[2\] is designed for high-precision
measurements and new physics searches in the tau-charm energy region.
The BESIII detector has a cylindrical symmetry with a total length of
5.6m and a radius of 2.6m and consists of four subdetectors for
different tasks like particle identification, track reconstruction,
energy measurement and muon detection.

The BESIII experiment brought several special challenges to the design
and implementation of the control system. First of all, there are about
9,000 data points covering dozens of physics parameters need to be
monitored or control. To provide a uniform and coherent interface to the
operators, the DCS must be organized in hierarchy and the different
drivers in the bottom layer have to be wrapped into a uniform interface
to communicate with the top layer. Secondly, the DCS must be able to
work continuously and stably for more than ten years in the harsh
environment due to radiation and magnetic field. Finally, the budget of
the DCS is only 0.4% of the total cost of BESIII.

## SCOPE Of the Dcs

The principal task of the DCS is to supervise and control the status of
the BESIII experiment and to guarantee a safe operation to the detector.
The status of the BESIII experiment includes not only the status of the
subdetectors but also the readout electronics, the PC farms, the
experimental hall environment and other parameters from external
systems. The term external systems indicates that the control of these
systems is outside the scope of the DCS although the communication with
them is required, such as the DAQ, the BEPCII accelerator, the
Superconducting Magnet and the Beam Pipe. The DCS is divided into six
subsystems by the target to monitor or control: Temperature and Humidity
(TH), Low Voltage (LV) power supply, High Voltage (HV) power supply
\[3\], VME crates, GAS control and Safety Interlocking (SI).

## Dcs Architecture

Even though there are thousands of distributed data points in the DCS
need monitoring or control, only one or two operators will keep watch
the whole BESIII experiment during concurrent physics datataking. So the
DCS must provide a uniform, coherent, simple and friendly interface to
the operators who may be physicist, engineer or student. On the other
hand, the DCS must also provide the functionality required to operate
the different BESIII detectors in stand-alone mode during building and
commissioning.

For these reasons, the DCS has to be organized hierarchically \[4\] as
shown in figure 1. The DCS is divided into three layers: Front End Layer
(FEL), Local Control Layer (LCL) and Global Control Layer (GCL). Simple
sensors up to complex computer-based devices like embedded systems and
Programmable Logical Controllers (PLC) are utilized in the FEL to
acquire the different parameters or execute control commands of the
BESIII experiment. The LCL acquires data from the FEL and offers
supervisory and control functions such as displaying, archiving and
alert handling. Each Local Control Station (LCS) can work either in
stand-alone mode or integrated mode to satisfy the different stages of
the BESIII experiment. In the GCL, the Global Control Station (GCS) will
gather the summery information from the LCS and provide a uniform and
simple interface to the operators. All the data acquired by the LCS will
be stored in the Database Server and published to the Internet by the
Web Server. The GCL will only work in integrated mode during concurrent
physics datataking

![](media/image1.png){width="3.1819444444444445in"
height="2.373611111111111in"}

### 

### Front End Layer

The FEL includes all the hardware components of the DCS, which are used
to convert the physics signal such as temperature and pressure into
digital data or execute control commands sent from the LCS, such as
remote ON/OFF of the VME crates or the HV power supply.

For the Temperature and Humidity (TH) monitoring, about 1,000
temperature and humidity sensors are distributed over the subdetectors,
electronics, PC farms, cavern and control room. Both the temperature and
humidity sensor are digitized sensor, in which the temperature sensor is
DS18B20 from DALLAS and the humidity sensor is SHT75 from SENSIRION.
Both of the sensors have been qualified for the long lifetime operation
in the harsh environment due to radiation and magnetic field according
to a strict radiation test.

To collect the data from the sensors and transmit it to the LCS, three
customized boards have been developed by the DCS team at a low cost. Two
of them are DAQ boards which are used to gather the data from DS18B20
and SHT75 separately \[5\]. The third board is an USB/RS485 adapter
\[6\] to transduce the data from RS485 to USB. Fieldbus RS485 is
selected to transmit data from the DAQ boards to the LCS because of its
high reliability in transmitting data over a long distance up to 1,200m.
A quasi-Modbus protocol has been implemented to define the transmission
format of the data. Because of the unaccessible of the boards during
experiment, a bootloader program is resident in the boards for firmware
reloading or upgrading online.

For the LV monitoring, the same technologies as in the TH monitoring
system have been adopted except the TH sensors are replaced by
amplifiers.

In the HV monitoring and control system \[3\], the CAEN SY1527LC HV
power supply mainframe \[7\] has been adopted and it supplies an
approach of remote monitoring and control via TCP/IP. So no hardware and
software need to be implemented in the FEL and the communication program
on the LCS side can be developed based on the CAEN HV Wrapper \[7\].

Similarly, in the VME monitoring and control system, the W-Ie-Ne-R VME
crate \[8\] also provides a way to access the crate remotely via CAN
bus. To make the communication with computer easier, an OPC server for
the W-Ie-Ne-R VME crate has been developed by the IT-CO-FE group at CERN
\[9\].

Both of the Gas and SI system selected SIEMENS PLC S7-200 for control
operation and data acquisition due to its high reliability. The OPC
server for this PLC has also been provided by SIEMENS.

### Local Control Layer

The LCL is composed of several LCS to provide the full monitoring and
control of the subdetectors and other support systems. There will be a
unique LCS per subdetector during detector building and commissioning
and each LCS can work independently, which provides the maximum
flexibility for the operation of the subdetectors. During concurrent
physics datataking, the LCS will be rearranged by subsystems as figure 1
showed.

To acquire data from the FEL and offer supervisory and control
functions, a software framework for LCL has been developed based on
LabVIEW \[10\]. LabVIEW is a graphical development environment, which
not only simplifies and accelerates the development of the framework
dramatically but also has a low cost compared with other commercial
SCADA software such as PVSS-II, iFIX and WinCC. The architecture of the
LCL software framework is shown in figure 2.

![](media/image2.jpeg){width="3.2583333333333333in"
height="1.8854166666666667in"}

The framework provided all the basic functions for the subsystems on LCS
such as configuration, data acquisition, error and alert handling,
logging, data storage and retrieve, system management and GUI. Each
function is represented by a package of SubVIs, which makes the
framework highly flexible and scalable. It needs only little
modifications on the configuration and driver package when apply the
framework to a new specified subsystem.

### Global Control Layer

The GCL includes a GCS, a database server and a web server. The overall
supervisory and control of the BESIII detector will be performed from
the GCS. In normal situation, the GCS will only display the summary
information from the LCS and some key parameters such as run number,
luminosity, current and lifetime. However, when abnormal status occurs,
the corresponding alarm or error messages will be popped up. The network
communication between GCS and LCS is implemented by Shared Variable
which is a new feature of LabVIEW 8.20, for the reason that both of the
programs on LCS and GCS are implemented from LabVIEW. On the other hand,
the network communication between GCS and external systems such as
BEPCII, DAQ and luminosity monitoring system are implemented based on
the Distributed Information Management System (DIM) \[11\] because of
its availability both in Linux and Windows. DIM is a communication
system for distributed / mixed environments, it provides a network
transparent inter-process communication layer.

The database server in GCL will store all the data of the DCS, which not
only includes the original data but also the summery and configuration
information. MySQL \[12\] has been selected as the database management
system (DBMS) because of its fast performance, high reliability and free
of cost. A dedicated web server is used to publish the DCS data to the
Internet and PHP based program has been written for the generation of
the dynamic web pages, so that the remote users can access the real time
or history data in graph or text.

## Performance and reliability

Because of the parallelism of the readout chain, we just need to study
the performance of the heaviest loaded readout chain of each subsystem.
From the real-size prototype tests, we got satisfactory results. For the
TH and LV system, the maximum update cycle is 10s corresponding to a
load of 600 temperature sensors. For the VME and GAS system, the maximum
update cycle is 30s due to the heavy load on OPC server. The update
cycle for the HV system can reach less than 1s because of the fast
transmission speed on TCP/IP. The response time of SI PLC system is less
than 1ms for electric connection.

The risks that weaken the system reliability are mainly from three
aspects as follows, and corresponding efforts have been made to minimize
these risks: 1) Ageing and brokenness of the sensors and circuit boards
in the harsh environment due to radiation and magnetic field: the
sensors and DAQ boards which will be assembled in the radiation district
have passed a strict radiation test; To assure the long lifetime of the
purpose-built boards, all of them have passed a 24-hour thermal ageing
test with a temperature of 60 centigrade. 2) Database crash or
brokenness of hard disks: to recover the database in the shortest
possible time, all the database files will be backed up automatically at
regular intervals and RAID 5 has been adopted to enhance the reliability
of the hard disks. 3) Virus and hackers' attack: in the network
architecture of the DCS as shown in figure 1, only the web server will
be connected to the Internet. The operating system of the web server and
database server is Linux, which means less possibility of virus
infection. In addition, firewall will be installed on every machine and
the IP filtering mechanism will be employed also.

By far, a full function LCS for the Main Drift Chamber (MDC) cosmic rays
test has been established and kept running stably for more than four
months. The result from the test shows that the LCS is highly stable,
flexible and scalable and the GUI is very friendly to the operators.

## Conclusion

The special requirements and limits of the BESIII experiment brought
several challenges to the design and implementation of the DCS. To
provide a uniform and coherent interface to the operators, the DCS is
hierarchically organized and divided into three layers for different
functions. Large variety of equipments ranges from sensors up to
embedded systems and Programmable Logical Controllers (PLC) have been
utilized in the FEL. A LCL software framework has been developed based
on LabVIEW to simplify the implementation of new subsystems. The GCS
will provide a uniform and simple user interface to the operators for
the overall supervisory and control of the BESIII experiment. The
performance of the readout chain and the reliability of the DCS have
been tested and the results satisfy the requirements of BESIII
experiment.

## References

\[1\] BESIII Collaboration, "BESIII Preliminary Design Report", 2004.

\[2\] BEPCII Preliminary Design Report, 2002.

\[3\] Xihui Chen, Xiaoxi Xie, et al. "Design and Implementation of the
Labview based HV System of BESIII Slow Control". Proceedings of the 13th
National Conference on Nuclear Electronics & Nuclear Detection
Technology. October, 2006. P.108-113.

\[4\] Xiaoxi Xie, Xihui Chen, et al. "The computer architecture and data
acquisition solution of BESIII slow control system". Nuclear Electronics
& Detection Technology, 2006 Vol.26 No.3 P.291-295.

\[5\] XIE Xiao-xi, XIE Song, et al. "The application of 1-wire bus used
in the BES Ⅲ slow control system". Nuclear Electronics & Detection
Technology, 2005 Vol.25 No.5 P.475-480.

\[6\] Xihui Chen, Song Xie, et al. "Design of USB/RS485 converter and
its application in slow control data acquisition system of high energy
physics". Nuclear Electronics & Detection Technology, 2005 Vol.25 No.1
P.88-92

\[7\] http://www.caen.it

\[8\] http://www.wiener-d.de/

\[9\] http://itcofe.web.cern.ch/itcofe/

\[10\] http://www.ni.com/labview/

\[11\] http://dim.web.cern.ch/dim/

\[12\] http://www.mysql.com/