DATA LOGGING FOR TECHNICAL SERVICES AT CERN
R. Martini, H. Laeger, P. Ninin, E. Pfirsch and P. Sollander
CERN, Geneva, Switzerland
Abstract
A data-logging system for the technical infrastructure of CERN was developed and has been used for over two years by
the Technical Control Room (TCR) and by equipment groups. Hardware and some software are common to the SPS/LEP
accelerator control system, though some particular features had to be implemented to meet specific requirements of theTCR. One of these requirements is the need for data-logging throughout the year without any interruption. Currently,
around one thousand measurements are logged at rates ranging from one to fifteen minutes; data are kept online for periods
varying between one month and one year, after which they are automatically archived. This paper gives a description of
the data-logging system, including the database structure, the interface tools for data management and data display, thedata-acquisition mechanism, and the integration of the system in the existing control environment. An account is givenof the experience gained and the problems encountered as well as the evolution of the system.

I. INTRODUCTION
A complex infrastructure of basic technical services, comparable to the industrial services of a medium-sized town,
is required to run the accelerators as well as all other activities at CERN. These services include electricity supply, watersupply, heating and ventilation, and personal safety systems such as fire and gas detection. To ensure the smooth runningof these services the Technical Control Room (TCR) monitors the state of this infrastructure 24 hours a day and 365 daysa year, and takes corrective actions where needed.
For many years a number of measurements, mainly electrical and fluid data, had been recorded on paper. The
recording devices were expensive to maintain, and could be non-operational due to trivial causes such as lack of paper
or ink. Logged data was becoming increasingly necessary to analyse the behaviour of equipment prior to malfunction,
to help in optimizing performance, or to reduce cost, e.g. energy consumption. In order to respond to a growing numberof measurements that required logging, a computer-based system was envisaged which could be linked to the existing
control system.
The start of this project coincided with a project in the SL control group to log a large number of measurements
relevant to the performance of the LEP accelerator. A collaboration was undertaken to devise a solution that would shareas many resources as possible [1].

II. REQUIREMENTS, CONSTRAINTS, DESIGN CRITEREAThe TCR required the following basic functionality from the data-logging system:– different measurements should be logged at different frequencies (1 to 30 minutes)– data for different measurements should be kept online for different periods (days to months)– data should be archived– logging parameters should be changeable at any time by non-specialists– adding new measurements should be possible by non-specialists– logged data should be available for display in graphical or in raw form
– summary data (averages, maximum and minimum values within a period of time) should be kept if required– data should be logged without interruption– problems should be reported to TCR operators for remedial action.
A. Design Constraints
The design of the product was heavily influenced by the LEP logging system [1], which is an ORACLE-based solution,
implemented for storing key measurements for LEP operations. The table structure defined for this project was used in
the TCR logging system.
The measurements to be logged are all on the control network [2], which consists of Equipment Control Assemblies(ECAs) connecting the equipment, linked to Process Control Assemblies (PCAs). Data is accessed differently dependingon the type of equipment being treated. The standard communication modules, which allow users at a workstation toobtain data from the equipment, were used in the data logging application.
The TCR has a policy to hold all elements of the surveyed equipment under its responsibility in an ORACLE reference
database (known as STRefDB), so that control system applications can be data-driven. The data logging application
conforms to this policy.

B. Product Components
The product was broken down into the following components:– data storage– data management– data access– archiving– data display.
C. Concept
Data logging is based on the concept of logging groups which cover a number of similar measurements to be logged ina similar way, such as humidity from different regions of the LEP tunnel. All measurements within a logging group have,by definition, the same parameters (logging period, retention time, start and end dates). Logging groups are identified bya unique number. This organization of data is transparent to the user selecting measurements for display, where they arepresented according to the equipment systems and subsystems to which they belong.
D. Environment
The database is currently running on a SUN SPARC 2000 station. The logging and archive programs, written in Cand Pro*C, run on an HP-UX workstation. The user interfaces may run on any X-terminal that can connect to the control
network.

III. PRODUCT DESCRIPTION
A. Data Storage
An ORACLE database was chosen as the repository for all logged data, following the analysis done by the SL control
group. The structure of the tables, as well as the storage method used, was that devised by the above group, i.e. theuse of a set number of pre-filled time slots for each group of measurements read, whose corresponding rows are updated
sequentially with time stamps and measurement values [1]. This strategy was adopted to facilitate future attempts at data
correlation.
A set of tables is used to store data for a logging group; this consists of one table for logged data, one for the timestamp, and one for the time slot management. There are as many sets of tables as there are logging groups. A strict naming
structure of the tables is used: Tx− LOG for logged data, Tx− TIME− SLOTS for time slot data, and Tx− TIME− MAN fortime slot management data, where x is the logging group identifier.
B. Data Management
This enables users of the data-logging system to specify the measurements to be logged and to set the logging parameters according to their needs. A logging group can be created by choosing measurements from the reference databaseon a form-based user-interface. In a similar way modifications to a logging group’s parameters can be made. A designated table on the logging database is then updated with the logging group number in order to inform the creation andmodification handler that a change has been requested.
For measurements that are managed by services other than the TCR and which are therefore not described in theSTRefDB, the previous facility cannot be applied. For these, a set of scripts has been developed in order to allow for thecreation of a new logging group and for the modification of logging parameters. These scripts are run by the data-logging
specialist, as no tool has been developed to handle data validation, program malfunction, or communication with thelogging programs.
One module of this component then builds or alters the logging tables required for a logging group according to
the chosen parameters. The TCR reference database is accessed in order to obtain the logging system parameters. Therequired table sizes are calculated using the number of measurements to be logged, the logging frequency, and the retention
period. Control tables in the logging database are updated with the logging parameters of the system so that the data
access programs can determine what to access and where to store the data. There are two control tables, one which holds
the parameters relating to the logging groups, the other which contains the parameters (physical address, conversion
factor) relating to the measurements in each logging group. Any errors in the table manipulations are signalled and canbe recovered.
C. Data AccessThis component is responsible for obtaining the measurement values from their source, and for storing them in thedatabase. There exists only one program, the logging black box, which is used to read any type of equipment whose
access method has previously been determined. One instance of this program is run for each logging group. Theseprograms are launched by a logging server which then monitors their behaviour (Figs. 1 and 2). This server must always
be running.

Figure. 1. Checks on the logging server program

Figure 2. Server checks on logging processes

In order to determine how many logging black boxes to run, the server program reads the logging control tables.
Once a black box program has started, it must first determine what kind of equipment it will be accessing, since data fromdifferent sources are read and interpreted differently. It will then determine which data elements to access and where
the data will be stored; this is done once again by consulting the control tables. Should access to the database become
impossible, the data is written to the file system and will be written to the database once the latter is available (Fig. 3).
D. Archiving
Since data is kept online for rapid access for typically one to two months, it is archived regularly to the file system.
This is done for each logging group by the archive program, which determines the number of logging systems runningand their data retention periods by reading the control tables. Archiving consists of exporting the table data to the filesystem and compressing the data. The name of the compressed file contains the logging group code and the archive date,thus permitting the data to be automatically restored should this be required.
CERN has a legal obligation to keep a 10 year record of the pH and temperature of the water which it discards into
the local rivers. For other data, the archive retention policy is in the process of being determined.
In addition, the archive program also updates summary tables for groups that require them.
E. Data Display
Once the data is stored on the ORACLE tables, it is publicly accessible by any tool that interfaces with the database,e.g. EXCEL can be used to retrieve data and display it graphically. However, since such tools do not present a sufficiently
well-tailored interface to the user for selecting the measurements for display, and also lack functionality such as zooming,a specific module was developed to satisfy these goals.
The data display module is divided into two parts, one for managing the selection of measurements, the other forhandling the graphical display. The data selection part consists of a menu-driven user-interface which guides the user
by presenting the available measurements using names with which he is familiar rather than the table and column namesthemselves. The information on which the menus are built is obtained once again from the logging control tables. Oncethe required measurements have been selected, the graphical display program is called with the appropriate display
parameters. The display program uses in-house graph templates built on top of the Xplore package, allowing the use of
XRT Motif widgets without the need for X-Windows programming. The graph displayed can be updated with new data
if so requested. Should the data required for display not be available online, the archive is searched for a file referring to
the data requested. If the file is found it is restored to the database and the request is processed accordingly.
This program uses the X-Window protocol for data display and thus must be run on an X-Window emulator if usedon PCs and Macintoshes.

IV. PERFORMANCE, PROBLEMS, PERSPECTIVES
The logging programs are generally reliable, though certain problems have been experienced during the evolution ofthe product over the past two years.
Accessing a remote database requires that the unavailability of the machine running the database, or the databaseserver itself, must be correctly handled. The product uses a number of standard tools, either provided commerciallyor made in-house. Each new release of a tool may lead to incompatibilities or may imply a modification of some partof the logging application. These can sometimes be overlooked and cause problems that are not immediately detected.
Access to the measurements, which are distributed over a wide area across a number of networks, may not always bepossible. Errors in obtaining data are now logged themselves in a separate file for each logging group; these can thenbe analysed to determine whether equipment is functioning correctly. The expansion of the application to cover morethan the originally planned number of measurements to be logged led to changes in the configuration of the machinerunning the logging programs and to an optimization of the files used for error reporting and archiving. Original design
weaknesses are gradually being corrected. Our experience leads us to conclude that the use of one single facility for SLand TCR does not permit optimization for both parties; it is envisaged to create two independent set-ups.
The product has been of significant use to many different services at CERN and as a consequence is constantly
evolving. For example, new requests for logging different data sources mean adaptation of the data access modules.
Though diagnostics on the logging processes have steadily been improved, better treatment of data access errors isoverdue. Moreover, with the advent of new control software [3] in the technical services domain, a move towards more
event-driven logging will be made, and this will of course have a significant impact on the product.

References[1] R. Billen et al., “LEP Accelerator Logging System Using On-line Database”, Nucl. Instrum. Methods Phys. Res.
A352 (1994) pp. 296–299.
[2] P. Lienard, “Evolution of the SPS and LEP Communication Control Network for the SPS and LEP Accelerators” inProc. ICALEPCS’93, Nucl. Instrum. Methods Phys. Res. A352 (1994) pp. 170–172.
[3] P. Ninin et al., “The Technical Data Server for the Control of 100 000 points of the Technical Infrastructure atCERN”, these Proceedings

Figure. 3. Data access and storage module