Network Communication Libraries for the Next Control System of the KEK e·/e+ Linac
Norihiko Kamikubota,Kazuro Furukawa, Kazuo Nakahara and Isamu Abe
National Laboratory for High Energy Physics (KEK)
Oho, Tsukuba, lbaraki 305, Japan
II. NElWORK COMMUNICATIONLIBRARY

Abstract
The network communication libraries for the nextcontrol system of the KEK Linac have been developed. They
are based on TCP/IP sockets, and show high availability
among the different operating systems: UNIX, VAXNMS,and MS-DOS. They also show high source portability ofapplication programs among the different computer systemsprovided by various vendors. The performance and problems
arepresented in detail.

A. Principlesfor the Ubrary
The library "SCLIB" has been developed for theTCP/IP protocol. It was designed as a tool for real-time datatransfer between processes, which is different from a filetransfer tool (like FTP), a file-sharing tool (like NFS), and anworkstation tr

I. INTRODUCTION
The KEK 2.5-Ge V electron/positron linac has been
controlled with a distributed processor network since its first
operation in 1982 (1,2]. Since, however, the system resources
have become inadequate for increasing demand, we haveintroduced several subsystems in order to extend the systemcapability [3]. Furthermore, we have studied the possibility ofsystem rejuvenation by a complete replacement of theminicomputers and their associated fiber-optic network withnew ones. The proposed next control system comprises Unixbased workstations as a man-machine interface, an Ethernet asa high-speed communication network, and VME stations as
front-end systems [4].
It is apparent that the tools for communication
between workstations and VME stations are needed in theproposed system. In addition, some of the subsystems (theoperator's console subsystem which comprises DOS-based
personal computers [5], a diagnostic expert system for thelinac injector developed in a Unix workstation [6], a beamcurrent monitor developed in a VAXstation [7], and so on) are
expected to be used with the next control system. Thus,
communication availability between different operating
systems is important in our case.
We have developed a network communication librarycalled "SCLffi" for the media of Ethernet with the TCP/IP
protocol. Details are described in section II. Another libraryused to control the magnet power-supplies in the KEK linac,called ."MGLIB", has been developed as an improved versionof the SCLIB. The features are demonstrated in the section m.
A discussion related to these libraries is presented in section
IV. We hope that our present experience will provide good
guidance for those who intend to introduce a similar network
communication system.

318""
clientprocess

I~

SCLIB
(sendm)\..

~Ethernet- T(TCP/IP protocol}

workstation 2r
SCLIB(recvm)'-,I

serverprocess~~rmain() client process */{int sc_open(),sendm(), sc_errend(),ac_c:lose();r sclib •1
extern Int sc_errno; error c:odeholder for sclib functions*I

rsd • sc_open(•servioe_name@nodename•): r open connection •tif( sc_erroo< O) sc_errend();1• error message & exit*/
rtn • sendm( ad, •message•);If( sc_errno < 0) sc_errendQ;r send a charcter siring •1

sc_c!ose(sd );r close connection•t}

Figure 1
Example of transferring a character string from a client
process (workstation 1) to a server process (workstation2). The basic flow of the SCLIB function calls is alsoshown.

Table 1
Computer systems used in the present performance tests. An
operating system and a rough estimate of the CPU power foreach computer system is also given.

Comouter

Ooeratin2 SystemMIPSDEC DECstation5000

UL1RIX

24

Sun SPARCstation!

SunOS

12

NeXTNeXT

Mach

10

Mitsubishi Mx:300011

OS60/UMX(System-V based)

3

SXA(System-V based)

3

Fujitsu A-60

DEC VAXstationll/GPX VMS
WIN/fCP installed
NEC PC9801ES(16MHz,80386)

have found several "small" differences in the socket functions
provided by different vendors, and have tried to include vendordependent parts within the library as many as possible. The
use of the C language is also preferable from this viewpoint.

B. CommunicationThroughput
In order to study the overall communication
throughput, including library overhead, test programs with theSCLIB functions have been prepared. We have carried out
measurements of the data-transfer times between two computer(a)100o VAXatallonll/GPX
x VAX1tallonll/GPX+ VAXllallonll/GPX
x VAXatallonll/GPX• VAXalallonll/GPX------NEC PC9B01ESVAXolallonll/GPX
FACOMA-60
SPARCalallonl
DECalallon5000

50

1MS-DOSPCTCP installed..

«l10....
'v

interactive communication tool (like telnet). The libraryprovides C-language functions. A typical example is shown infigure 1, together with the basic flow of the SCLIB function
calls.
The principles that we considered when we starting todevelop the library are the following: The first principle iscommunication availability between different operating
systems. We have adopted an inter-process communication
technique called "stream socket", which is based on theTCP/IP protocol.
It provides the basic network
communication functions with error-handling schemes for C
language. The socket .is one of the standard inter-process
communication methods in Unix-based workstations, and isalso available on other computer systems.
Since the socket functions require various kinds ofnetwork parameters, it is almost impossible to use by those
who are not familiar with this field. Thus, the second principle
is to. prepare easy-to-use functions for application
programmers. As shown in figure 1, the SCLIB functions
used in a client process require only two parameters: thedestination of data ("service_name@nodename") and the data,
itself ("message"). In addition, the existence of an errorhandling routine (sc_errend) makes the client program verysimple.
The third principle is to ensure high source
portability of application programs among computer systems
provided by various vendors. This leads to an easy replacementof the hardware of our present control system. Actually, we319

6 Ll.LuL~--'--'-'-.LUJ10LJ,.1-.L..'-'-U-L1u.02=-...1.....L..1..u..~,------'--~

packetsize (byte)• DECalallon6000 -- FACOMA-BO
X SPARC1tallonl -Nexl
o SPARC1tallonl
X DECalallon5000
Ill

Jj 101...
...
Ill"'~b.,....
Id10°""

10 1

10 2

packet size (byte)

Figure 2
Measured data-transfer times between the computer
systems listed in table 1.
(a) Data-transfer times between a VAXstationll/GPX
and other computer systems.
(b) Data-transfer times for other combinations ofcomputer systems. Two of them (SPARCstation! andDECstation5000) are the result of the case that two
processes are in the same workstation.

· systems (processes) for the seven computer systems listed intable 1. The results for typical cases are shown in figure 2.
The following points canbe pointed out.
(a) The data-transfer time takes 0.2-30 ms for 1-1000 bytes ofdata, depending on the CPU power of the computer systems
used.
(b) The data-transfer time remains almost constant as the size
increases from 1 to 100 bytes. It typically becomes twice as
the size increases from 100 to 1000 bytes.
(c) The best record among combinations of computers in table
1 was found in the case that the two processes are at the sameworkstation, DECstationSOOO.The data-transfer time was 0.21 ms for the 1-1000 bytes data.
(d) Assuming a data size of 1 kB, the overall throughput isevaluated to be 400-1000 kB/s for data transfer between typicalworkstations (10 or more MIPS).
In addition, the dips observed at around 1000 bytes are
considered to be caused by the buffering scheme of the TCP
protocol.

daemon process in the gateway. The daemon process controls
the power-supplies according to the received message. It thenreturns a reply message which includes the result of thecontrolled power-supply. The SCLIB functions areused at themessage interchange between two processes.
An example of MGLIB function calls is also shownin figure 3. Here, an initializing function (sc_open for theSCLIB) is not necessary, since initialization is carried out
automatically at the first call of the MGLIB functions.

B. Sourceportability
Some problems concerning source portability havewod<statlon1r

C. Discussion"-The round-trip response time between the operator anda local control device was studied in the present control system[l]. It was evaluated to be of the order of 100 ms for our dataof 128 bytes. According to the results of our present
measurements, one or two orders better throughput can beexpected in the next control system. If workstations withgreater CPU power (for example 40-100 MIPS) become
available in the future, further improvement in thecommunication throughput can be expected.
The measured communication throughput seems to be
sufficiently high for most of the applications required tocontrol the accelerator. However, it should be noted that thetime required to open network connection, which usually takes
0.1-5 seconds and is necessary each time to start inter-process
communication, is not included in the values given here.

application "'process

I ISCLIB
MGLIB
II

1_-1
workstation2r

I I"''"~a.Ethernet(TCP/IP p,otoco/)

~
11:

MGLIB

I "'

daemon process
I SENDQ I\.

_jPresentControlSystem(MELCOMl0/30)

ID.MAGNETCONTROL
LIBRARY

300 magnets)
in total(A. Introduction

malnO r appllcaHonprocess•1

The development of applications used to control themagnet power-supplies of the KEK linac was possible onlywith the minicomputers used in the present control system(Mitsubishi MELCOM 70/30) and its backup workstation(Mitsubishi MX300011). In order to realize a better
environment for software development, we first tuned theworkstation serving as a gateway between an Ethernet and thepresent control system [3]. We then developed a library called"MGLIB" in order to make it possible to control the powersupplies at any of workstations connected with the Ethernet.
The mechanism of how a power-supply is controlled
is shown in figure 3. When an application process calls aMGLIB function, an information message is sent to the320IInt mgO,mg_emmd(l; r mgllb functions•1axtem Int m1Larrno: error
holder of mgllb •;rcode

r1n• mg( "CUR', •magnet_name•,&curr ): r get magnetcurrant •,If( mg_errno< O) mg_arrandQ: r error m861111ga& exit •1

Figure 3
Example of an application process which controls a magnet
power-supplyof the KEK linac.
Inter-process
communications
betweenan application
process(workstation 1) and a daemon process (workstation 2) are
carried out with the SCLIB functions (see text). Workstation
2 serves as a gateway between the Ethernet and the present
control system.

arisen during the development of the MGLIB. The mostserious one is the differenceof byte order in the expressionof
numericalvalues. We have describedbyte-conversionroutines
for each computersystem with C preprocessorstatements.
Since the conversion routines are described in theMGLIB library,applicationsourcesare expectedto show highportability. Actually, sample programs for basic operation of
the magnet power-supplies show perfect portability among
the computer systems in table 1. In addition, the basic
availabilityof the SCLIB/MGLIBfunctionsis also checkedin
a Hewlett Packard workstation with HP-UX, and a Force
68030-basedVME system with OS9.

C. DiscussionThe typical throughput of a single control action isobtained as 100400 ms. This value is understood to be theround-tripresponse time in the present control system.
After introducingthe next control system, we should
re-develop a new daemon (server) process with a VME
environment. It would not be easy since the development ofsuch a server process requires deep understanding of thenetwork parameters and workstations, even with the present
SCLIB functions. However, the sources of applications
developedso far with the MGLIB functions will be available
withoutany modification,even in the next control system.

IV. DISCUSSION
One of the problems concerning an Ethernet is
response delays due to network packet collisions, which
inevitablyoccur in an Ethernet when the network traffic rateis considerably high. An easy answer to this is to replace an
Ethernetwith a FDDI (Fiber DistributedData Interface) fiberoptic network system. It is a token-ring type network and ismore suitable for a real-time purpose. In addition, we would
expect one order better communicationthroughputsince it iscapable of a communicationrate of 100 Mbit/s. Moreover, it
is worth noting that a replacement is possible without any
softwaremodification.
The present libraries provide a possibility to use an
Ethernet as a high-speed data-transfernetwork with low cost.
There exists a plan to use these libraries for the control
system of the TENKO-100, a 100-meter long laserinterferometer aimed for detecting gravitational waves now
under construction in ISAS (The Institute of Space andAstronauticalScience)[8,9).

VI.REFERENCES[1] K. Nakahara, I. Abe, R.P. Bissonnette, A. Enomoto, Y.
Otake, T. Urano and J. Tanaka, Nucl. Instr. Meth.
A251(1986)327(2) K. Nakahara, I. Abe, A. Enomoto, Y. Otake and T. Urano,Nucl. Instr. Meth. A247(1986)153[3] K. Furukawa,N. Kamikubota,K. Nakaharaand I. Abe,
proceedings(Int. Conf. on Acceleratorand Large Experimental
ControlSystems,VancouverCanada, October 1989),Nucl.
Instr. Meth. A293(1990)16[4] K. Furukawa,N. Kamikubota,K. Nakaharaand I. Abe,
"UpgradePlan for the Control System of the KEK e-/e+
Linac", this conference[5] K. Nakahara,I. Abe, K. Furukawa and N. Kamikubota,
proceedings(Int. Conf. on Acceleratorand Large Experimental
ControlSystems,VancouverCanada, October 1989),Nucl.
Instr. Meth. A293(1990)446[6] I. Abe, M. Kitamuraand N. Nakahara, "A Diagnostic
Expert System in PF Linac Control", this conference(7) K. Furukawa,N. Kamikubota,T. Shidara, K. Nakahara
and G. Horikoshi,proceedings (14th. Int. Conf. on HighEnergy Accelerators,TsukubaJapan, August 1989),ParticleAccelerators29(1990)245[8] N. Kamikubota,K. Furukawa, K. Nakahara,Y. Ogawa
and A. Asami, "A study of the Control System for LongBaselineLaser Interferometer",the proceedingsof the Sixth
Marcel GrossmannMeetingon General Relativity (MG6) inpress, Kyoto Japan, June 1991; KEK preprint 91-124[9] N. Kawashima,T. Yokota, Y. Ogawa and N. Kamikubota,"Developmentof 100 m Laser InterferometerGravitational
Antenna(TENKO-100)",the proceedingsof the Sixth Marcel
GrossmannMeeting on General Relativity (MG6) in press,Kyoto Japan, June 1991

V.ACKNOWLEDGMENTS
We would like to thank Professor A. Asami, head of
the Injector-Linacdivision in KEK, for his support and kindencouragement.We also thank the staff and operators of thelinac for their valuablediscussionsand assistance.

321