IOC performance on F3RP61
Hiroshi Kaji (KEK)2F3RP61••Embedded CPU module– for PLC FA-M3– Linux OS– compact size
We started to use as IOC since 2008.

F3RP61

FA-M3- Stepping motor controller formovable beam mask at KEKBManufacturer

Yokogawa

Base

PLC FA-M3- Power supply for electro-static
septum and septum magnets at theslow extraction line of J-PARC MR.

TypeCPU moduleCPU

32bit, 533MHz

Memory128 Mbyte- Current monitor at the neutrino
beam line of J-PARC MR.

Storage

64 MB flash ROM

OSELDK Linux 4.2

Network port

10BASE-T/100BASE-TX (2ch)

We studied detailed performances before largely adopting at SuperKEKB.

3Read/Write performance with IO modules
Process time for following access is studied.

Analog write

DA08-1X

Binary read

XD32-3F

Process time is studied at two level
Module access driver level (driver level):- This driver is released by manufacturer.
- Intrinsic process time for F3RP61 and FA-M3 is measured.

Record level:- Followings are called from ai, ao, and bi records,
record support ⇒ device support⇒ driver support ⇒ module access driver- Practical process time for EPICS user is measured.
For each study, average of 1000000 processes is taken to avoid
influence of fluctuation by environmental condition.

OPICA

read
write
read

XD32-3F

AD08-1R

DA08-1X

Analog readAD08-1R

Accessed Module

F3RP61

Access type

4

Driver level measurement
Expended time to call ‘module access driver’ 1000000 times ismeasured.
- Measurement is performed with both real time and processor
time. (Note, Linux Kernel version: 2.6.26.8-rt16).
- There is no difference.
- Only one of those results is shown here.
(These numbers have already been divided by 1000000.)

Access type

Accessed Module Process time

Analog read
Analog write
Binary readAD08-1R
DA08-1X
XD32-3F

15.73 ms
12.34 ms
13.02 ms

5Record level measurement
1001 records linked with each IO module areproduced and chained via FLNK.
Dtime and process time are calculated after
process first record.
Dtime: = time stamp(last) – time stamp(first)
Process time: = Dtime/1000

All results from now are
studied with R3.14.12.1.

ai record

Process time (ms)ao record

Process time (ms)

Iterate this procedure 1000 times
with 1 second interval.
Mean values are treated as the result.

bi record

Process time (ms)Record level

Driver levelai

26.46 ± 0.01 ms

15.73 msao

24.32 ± 0.01 ms

12.34 msbi

23.17 ± 0.01 ms

13.02 μs

Typically process time is:= Driver level + 10 ms

6

subArray and compactSA
The subArray (SA) and compactSA (CSA) are studied as examples of soft records.
Process time and memory usage on F3RP61 are measured.
waveform (source record)

Both records take out part ofarray from source record.
Same information is stored inVAL field.

The compactSA is developed at
KEK to suppress memory usage
and is not included in public EPICS
version.
compactSA

subArray
VAL field

Copy whole array in its buffer.

VAL field

Copy only selected elements of array.
All results from now are studied with double type array.

7

How to measure process time
Waveform: 600000 elements
SA, CSA: 10 elements

Process that 10 elements are taken out
from waveform is studied.
- compared between SA and CSA- compared with changing number of elements ofwaveform

CSA
SA

Average of 10000 processes:- One waveform and 100 SA/CSA records areproduced and chained via FLNK.
- Process time is calculated with time stamps ofwaveform and last SA/CSA.
- Measurement is performed 100 times.
(The right plot is an example with 600000 elements of waveform.)

Process time (ms)Process time:SA
4.03 ms/record
CSA
3.77 ms/record

8Process time on F3RP61 is ~4ms.

Difference of process time seems tobe increased with number ofelements for waveform.
However there is no clear correlation.

Process time (ms)Process time of subArray/compactSA

They are at most
500 ns at >1000000 elements.

There is no significant difference of process time between SA and CSA.

9How to measure memory usage
1. Total memory usage on linux is measured,simply ‘free’ command, with changing
number of SA/CSA records.
2. Memory usage for one record isdetermined from slope of the measurement.
Measurement is performed with changing thenumber of elements of waveform and SA/CSA.

Memory usage (kbyte)

Waveform: 200000 elements
SA, CSA: 100 elements

compactSA
As a example, result of following condition is shownon the right plot.
- waveformwith 200000 elements- SA/CSA
with 100 elements

Number of records
Memory usage:subArraycompactSA

1570 kbyte/record
2.83 kbyte/record

10Comparison with number of elements
Waveform: n element (x-axis)
SA:
100 elements

subArray○ compactSA○Waveform: n element (x-axis)
CSA:
100 elements

Memory usage (kbyte/record)

Memory usage (kbyte/record)

Memory usage is significantly suppressed in case of CSA.
Its dependence on number of elements of SA/CSA and waveform is reasonable.
Waveform: 200000 elements
SA:n element (x-axis)

Waveform: 200000 elements
CSA:n element (x-axis)

Number of elements
Memory usage of SA depends on numberof elements of “parent” waveform.

Number of elements

Memory usage of CSA depends on its
number of elements.

11Summary• F3RP61 is CPU module which:– we use as IOC.
– we largely adopt at SuperKEKB.

• The process time of ai, ao, and bi records are studied.
– Intrinsic process time of device is ~15ms.
– Read/write process needs ~25ms on EPICS.

• The performance of soft record is studied with subArray andcompactSA.
• Process time is ~4ms and there is no significant differencebetween
subArray and compactSA.
• The memory usage is significantly suppress by using compactSA.

– For example, waveform(200000) case,they are 1570 kbyte for SA(100) and 2.83 kbyte for CSA(100).
– Its dependence on number of elements (array length) is reasonable.

12

Backup slides

13More information• F3RP61 software:
http://www-linac.kek.jp/cont/epics/f3rp61/• Application example:
http://www-linac.kek.jp/cont/epics/f3rp61/#pre
http://www-linac.kek.jp/cont/epics/f3rp61/#art

14

Process time dependence on EPICS version
Same study as page 5 is performed for both
EPICS versions R3.14.11 and R3.14.12.1.

aiR3.14.11
R3.14.12.1

Process time (ms)aoProcess time (ms)bi

R3.14.11

R3.14.12.1

ai27.21 ± 0.02 ms

26.46 ± 0.01 ms

ao

24.18 ± 0.02 ms

24.32 ± 0.01 msbi

22.71 ± 0.02 ms

23.17 ± 0.01 ms

Process time (ms)There are small differences.
But it is <1ms.

15

Memory usage for others
Memory usage of followings is determined by same method as page 9.
Results are compared with those of MVME5500(VxWorks).

OS

sequencer

F3RP61ELDK Linux20.7 kbyte

MVME5500

VxWorks 5.5.1

25.6 kbyte

aiao

0.90 kbyte

MVME5500 1.05 kbyte

F3RP61bibombbi

mbbo

0.95 kbyte 0.75 kbyte

0.83 kbyte

1.20 kbyte

1.24 kbyte

1.12 kbyte 0.91 kbyte

1.00 kbyte

1.37 kbyte

1.40 kbyte

16

Process time dependence on CPU
Process time of subArray is compared among different servers.

Condition:
waveform(5000elements,double) ⇒ subArray(20elements)
CPU

533MHz3658 MHz

3481MHzOS

ELDK LinuxRed Hat Linux

Windows XP

Process timems

1.94 μs2.26 μs