Evaluating EtherNet/IP and CIP Safety
Communication over 5G
Shinji Kobayashi, Hiroaki Yamada
Technology Development Division HQ.
Industrial Automation Company
OMRON Corporation
Presented at the ODVA
2023 Industry Conference & 22nd Annual Meeting
October 18, 2023
El Vendrell, Spain
Abstract
5G wirelessization is being considered for use in industrial networks in factories. While the communication
performance requirements for various use cases in factories have been identified, there is limited
information available regarding the specific evaluation of communication performance achieved through5G wireless transmission in factory settings.
In this study, we evaluate the communication performance of EtherNet/IP controller-to-controller
communication using Sub6 of 5G SA in the 4.8GHz band. We measure packet loss and latency during
5G transmission of EtherNet/IP communication. The communication evaluation is conducted using
Implicit communication between Controller 1 node and N nodes (N: 1 to 7). Implicit communication
employs both multicast and unicast communication methods, with RPI intervals ranging from 1 to 200
milliseconds.
Furthermore, we conduct measurements of safety reaction time and stability evaluation over a period of
24 hours by transmitting CIP-Safety over 5G for robot cells, serving as a specific use case.
Through these evaluations, we clarify the expected communication performance in 5G factory use cases.
Keywords
5G Sub6, EtherNet/IP, CIP Safety
Definition of terms
BBU:
BS:
DL:
EPI:
iperf

Baseband Unit
Base station; a device that provides wireless communication services in Private 5G
Downlink; transmission of data from a the 5G base station to a user device
Expected Packet IntervalCommand-line tool used to measure network performance by establishing a client-serverconnection and testing throughput, latency, and packet loss.
NDAC:
NOKIA Digital Automation Cloud
RPI:
Requested Packet Interval
RRH:
Remote Radio HeadSA:
Stand-Alone; a network architecture in Private 5G
Safety Reaction Time: The time required for the system to enter a safe state in a worst-case scenario
after the occurrence of a safety-related input (press of an emergency stop pushbutton
switch, interruption of a light curtain, opening of a safety door, etc.) or device failure.
2023 ODVA Industry Conference

1©2023 ODVA, Inc.

Sub6:
TDD:
UL:
VPN:

Sub-6 GHz; a frequency band
Time Division Duplex; a type of communication method used in wireless communication
Uplink; transmission of data from a user device to the 5G base station
Virtual Private Network

Introduction
In factories, there is a shift from the previous dominant wireless technology, Wi-Fi, to Private 5G.
Consequently, there is a consideration on the wirelessization of industrial networks that have traditionallyrelied on wired connections. 5G offers features such as high-speed and large capacity, low latency, andmassive connectivity. However, these features are not all maximized at the same time. Depending on theactual operating environment and use case, some features may be prioritized over others. Additionally, it
should be noted that not all wired networks in factories will be replaced by wireless networks through 5G
adoption, as the feasibility depends on some specific applications.
The acceptable latency for communication required in various applications is illustrated by NICT (National
Institute of Information and Communications Technology) in Figure 1.Except mission-critical applications
for some control and safety use, many applications require an acceptable latency ranging from 10
milliseconds to 10 seconds.
At Omron, based on the communication capabilities of 5G Sub6, we have determined and set a target
range of 10 milliseconds or more for the wirelessization of industrial networks, transitioning from wired
cables to 5G.
Delay
Tolerance

Control

1msec
Machine,Robot

10msecAGV with rails,
Rotary equipment

100msec

1sec

10sec

100secAGV w/o rails

Inline inspection

Quality

Machine operation/production recording
LoggingPreventive maintenance for tools,
Machine monitoring
Positioning,
Motion analysis,
Inventory control

Management

Preventive maintenance for machines

Facility environment control
Work instruction

Display
SafetyAndon board
Emergency warning,
Safety reaction

Dangerous behavior detection
Vital sign monitoring

Figure 1: Permissible Latencies of Representative Wireless Use Cases [1]Although the requirements for wirelessization in various use cases in factories have been identified, there
is limited information available regarding the actual communication performance achieved whentransmitting industrial networks over 5G.
Additionally, according to the Voice of the Customer (VOC), there is an issue where the wired network
between production equipment hinders the flexible placement of production equipment during theconfiguration of production lines or layout changes.
Therefore, with the aim of validating the 5G wirelessization of Ethernet/IP for each controller thatmanages the production equipment, we conducted a performance evaluation of EtherNet/IP over 5G
transmission as a use case for controller-to-controller communication.
2023 ODVA Industry Conference2©2023 ODVA, Inc.

In 5G, we utilized BS in the 4.8GHz band for SA mode. To evaluate communication performance forcontroller-to-controller communication, we used EtherNet/IP's Implicit communication, varied the RPI(cyclic period), and measured communication latency and packet loss of the communication packets. We
evaluated both Multicast and Unicast communication methods, which are commonly used in EtherNet/IP
communication. These evaluations have revealed the expected level of communication performance
when transmitting EtherNet/IP over 5G in Factory Automation (FA) applications.
In addition, we performed a specific use case evaluation of CIP Safety over 5G transmission for FA
applications. In this case, we assumed a robot cell line that integrates three robot cells with a single
safety line controller. Firstly, we measured the basic characteristics of CIP Safety packet transmission
over 5G, including its latency and packet loss.
Next, to verify whether the system's safety functions adequately perform their roles when transmitting CIP
Safety over 5G, we measured the Safety Reaction Time. The Safety Reaction Time was measured bytransmitting safety process communication of CIP Safety over 5G, pressing the emergency stop button in
the robot cell, and measuring the time until the power was cut off.
The latency, the packet loss, and the Safety Reaction Time measurements were conducted for both 5G
transmission and wired LAN, and the results were compared.
Finally, we kept the three robot cell safety controllers and the safety line controller powered on for 24hours to verify the presence of safety communication errors caused by communication interruptions orcommunication delays. This allowed us to validate the stability of the line under CIP Safety 's 5G
transmission.
Based on the above, the current performance of the FA application that transmits CIP Safety over 5G has
been clarified.
System Configuration for Evaluation of EtherNet/IP over 5G Transmission
The system configuration which EtherNet/IP packets with Multicast are transmitted over 5G between
controllers, and the packet loss and communication delay of the transmitted packets are measured isshown in Figure 2.

5G

5G Base Station

5G Wireless Area
5G Antenna（4x4 MIMO）

Core Network
NDAC
Edge Server
Medium+5G RRH

5G BBU(ASIBx1,ABILx1)

5G
Terminal

NDAC
SecureGateway

NDAC
Backbone
Switch1000BASE-T

VPN Router(Server)

VPN
RouterGPS
Antenna

PacketCapturepcapngpcapngpcapngpcapngpcapngController
NX701

NX102

Figure 2: System Configuration for EtherNet/IP over 5G Transmission between Controllers

2023 ODVA Industry Conference

3©2023 ODVA, Inc.

The system consists of the following components:
 Core Network: Consisting of the NDAC Edge server, NDAC Switch, and NDAC Secure Gateway(Nokia), to control the 5G system.
 5G Base Station: Comprising a BBU and RRH, the 5G Base Station connects to omnidirectional
antennas (4x4 MIMO) in the 5G Wireless Area. It converts digital communication frames from the
network system into 5G wireless signals. The 5G signal operates in the 4.8GHz band (n79) with asignal bandwidth of 100 MHz. The TDD frame structure follows the synchronization method ofJapan's communication carrier band, with a transmission ratio of 3.25:1 for DL and UL.
 5G Wireless Area: This area is established by the arrangement of omnidirectional antennas (4x4MIMO) and 5G devices, forming a wireless space. In the evaluation environment, SHARP's local
5G devices were used, which achieved a DL throughput of 784Mbps (4x4 MIMO) and an UL
throughput of 51Mbps (SISO).
 EtherNet/IP Network: For the measurement purposes, we used one controller NX701 (Omron)and N (1 to 7) controllers NX102 (Omron). The controllers were connected to the 5G devices via
Packet Capture (Profitap: Profishark1G+) and a VPN router (YAMAHA: RX830). The Packet
Capture device is equipped with a GPS antenna, enabling time synchronization between packet
captures using GPS clock synchronization. By comparing the timestamps of the captured packets
from each capture device, it is possible to accurately measure the communication latency between
the packet captures. The VPN tunneling communication is used to ensure that multicast
communication frames reach all 5G devices by virtually constructing an L2 network. Since the
network system, including the 5G base station, operates primarily on L3 communication,
EtherNet/IP's multicast communication, which is L2 communication, cannot be directly transmitted
to other layers. Therefore, we encapsulated EtherNet/IP packets using L2TPv3 tunneling with the
VPN router, and performed tunneling communication between the VPN client and VPN server. The
VPN server (YAMAHA: RX1220) was connected to the switch in the Core Network.
In this configuration, Multicast packets from the NX701 connected to the 5G devices are transmitted
through the Edge Server in the Core Network to the VPN server. There, the packets are replicated androuted back through the Edge Server to the VPN clients under the N 5G devices in the 5G Wireless Area.
Finally, the packets are forwarded to the NX102 controllers.
5G Wireless Configuration
The wireless configuration used for this evaluation utilizes the 4.8GHz band with a signal bandwidth of
100MHz and follows the carrier band synchronization method used in Japan, as shown in Figure 3(a).
The DL to UL ratio is 3.25:1, indicating a configuration which DL has a higher priority than UL. At the timeof conducting this evaluation, this synchronization method was the only one available in Japan, andtherefore, it was adopted for the evaluation. The introduction of the semi-synchronous method, as
depicted in Figure 3(b), which provides a DL to UL communication performance ratio of 1.25:1, began
after December 2022 and has been spreading in Japan.

Figure 3(a): Synchronization pattern in Japan employed in the experiment2023 ODVA Industry Conference

4©2023 ODVA, Inc.

Frame 10ms
6 symbols or less

4 symbols or less

6 symbols or less

DDDDDDGGGGUU UU

Slot
0.5ms

4 symbols or less

DDDDDDGGGGUU UUSlot Number

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

Frame Structure

D

D

D

S

U

U

D

S

U

U

D

D

D

S

U

U

D

S

U

U

DDDDDDGGGGUU UUDDDDDDGGGGUU UU

10 symbols or less ( No downlink)

10 symbols or less (No downlink)

D: Downlink slot
U: Uplink slot
S: Special slot

Figure 3(b): Semi-synchronization pattern in Japan
Figure 3: 5G NR TDD configuration [2]The throughput evaluation results of the 5G device (SHARP: Local5G Router01) used in this evaluation,
conducted through iperf in the 5G experimental environment, are shown in Figure 4. A DL throughput of
784Mbps (4x4 MIMO) and an UL throughput of 51Mbps (SISO) were achieved.

1200

Throughput [Mbps]

1000

784

800
600
400
200

51
0

DL

UL

Figure 4: Results of throughput evaluation test using iperf on 5G terminal

2023 ODVA Industry Conference

5©2023 ODVA, Inc.

Measurement Configurations and Conditions for EtherNet/IP over 5G Transmission
The variations in measurement configurations (Type I to VIII) are shown in Figure 5.
N (1,2,4,7)

Ⅰ

Send

Receive

ReceiveN (1,2,4,7)Receive

Multicast

VPN

Ⅴ

Send

Receive

Receive
Receive

Receive

Multicast

Send
VPN

Unicast

Send
UnicastReceiveN (7)Receive

Send
Unicast

N (7)

Ⅱ

Send

Receive
Unicast

Send

Receive
Unicast

Send

Send

Receive
UnicastⅥ

N (1,2,4,7)Receive
Unicast

Send
Send
Receive

Receive

Receive

Send

Receive

VPN

Receive

ReceiveN (1,2,4,7)

Send
UnicastReceive

Send
UnicastMulticast

Unicast

Send
UnicastReceive
Send

Receive
Unicast

Send
UnicastReceive

Ⅲ

Receive
Unicast

Send
UnicastⅦ

N (7)Receive

Send
UnicastReceive
Receive

Send
UnicastVPN

Send
Unicast

Send

Ⅳ

Send

Receive
UnicastReceive
Receive

Receive
Unicast

Send

Unicast

Send
UnicastReceiveN (7)Receive

Send
Unicast

Send
UnicastMeasurement Area

Ⅷ

Receive

Send

Receive

UnicastReceive

Send
Unicast

Send
UnicastCommunication Load

Figure 5: Variation of Measurement Configuration for EtherNet/IP over
5G Transmission Evaluation between Controllers

2023 ODVA Industry Conference

6©2023 ODVA, Inc.

The performance measurement conditions are as follows: Measurement Parameters: Packet loss and latency were measured.
 Communication Modes: Measurements were conducted with "w/VPN" and "w/o VPN"
configurations. In particular, "w/VPN" was mandatory for multicast while "w/VPN" and "w/o VPN"
were set for unicast.
 Number of NX102 Nodes (N): Measurements were conducted with 1, 2, 4, and 7 NX102 nodes.
 RPI (Controller-to-Controller Implicit Communication Period): RPI values were set at 1, 5, 10,
20, 50, 100, and 200 ms.
 Number of Connections: Each NX102 node had 32 connections, with a maximum of 224
connections for NX701 (N nodes × 32). In conditions where the RPI period is short, the maximum
achievable number of connections for each RPI condition was set. Based on the specification
constraints of NX102, the maximum resource of 32 connections was varied across two test
configurations. In Fig.5, when there is no communication load, all 32 connections were used formeasuring the communication. When there is a communication load, 16 connections were used formeasuring the communication, and the remaining 16 connections were used for the opposite
direction communication load. The reason for using the maximum resources is to maximize theload.
 Data Size: The Implicit communication from the controller had a uniform data size of 600 bytes.
 Number of Measurements: Each condition combination was measured three times, with aduration of 3 minutes per measurement.

2023 ODVA Industry Conference

7©2023 ODVA, Inc.

Table 1: Required Communication Bandwidth for Uplink (UL)
under Packet Loss and Latency Measurement Conditions
Type

RPI [ms]

UL required communication bandwidth [Mbps]

Measurement Additional Load
Conditions
Conditions# NodesⅠ

1->N Multi.

-, VPNⅡ

1->N Uni.

-,-

Ⅲ

1->N Multi.

N->1 Uni., VPN

Ⅳ

1->N Uni.

N->1 Uni., -Ⅴ

N->1 Uni.

1->N Multi, VPN

Ⅵ

N->1 Uni.

N->1 Uni., -Ⅶ

N->1 Uni.

-, VPN

Ⅷ

N->1 Uni.

-, -

1
57.5
57.7
46.4
23.4
149.5
57.5
86.3
115.2
92.2
149.5
57.5
86.3
115.2
92.2
149.5
57.5
114.9
183.9
160.9
149.5

1
2
4
7
7
1
2
4
7
7
1
2
4
7
7
1
2
4
7
7

5
37.3
37.8
39.0
28.1
165.7
37.3
56.2
94.0
103.7
165.7
37.3
56.2
94.0
103.7
165.7
37.3
74.5
149.0
179.3
165.7

10
18.9
19.2
19.8
20.7
121.7
18.9
28.5
47.8
76.6
121.7
18.9
28.5
47.8
76.6
121.7
18.9
37.8
75.7
132.5
121.7

20
9.8
10.1
10.6
11.5
62.0
9.8
14.8
24.8
39.9
62.0
9.8
14.8
24.8
39.9
62.0
9.8
19.5
39.0
68.3
62.0

50
4.3
4.6
5.2
6.0
26.2
4.3
6.5
11.1
17.9
26.2
4.3
6.5
11.1
17.9
26.2
4.3
8.5
17.0
29.8
26.2

100
2.4
2.7
3.3
4.2
14.3
2.4
3.8
6.5
10.6
14.3
2.4
3.8
6.5
10.6
14.3
2.4
4.9
9.7
17.0
14.3

200
1.2
1.4
1.7
2.1
7.2
1.2
1.9
3.3
5.3
7.2
1.2
1.9
3.3
5.3
7.2
1.2
2.4
4.9
8.5
7.2

Table 2: Required Communication Bandwidth for Uplink (DL)under Packet Loss and Latency Measurement Conditions
Type

RPI [ms]

Measurement Additional Load
Conditions
Conditions# NodesⅠ

1->N Multi.

-, VPNⅡ

1->N Uni.

-,-

Ⅲ

1->N Multi.

N->1 Uni., VPN

Ⅳ

1->N Uni.

N->1 Uni., -Ⅴ

N->1 Uni.

1->N Multi, VPN

Ⅵ

N->1 Uni.

N->1 Uni., -Ⅶ

N->1 Uni.

-, VPN

Ⅷ

N->1 Uni.

-, -

1
2
4
7
7
1
2
4
7
7
1
2
4
7
7
1
2
4
7
7

DL required communication bandwidth [Mbps]

1
57.5
114.7
183.4
160.5
149.5
57.5
114.8
183.7
160.7
149.5
57.5
114.8
183.7
160.7
149.5
57.5
114.9
183.9
160.9
149.5

5
37.3
73.9
147.2
176.8
165.7
37.3
74.2
148.1
178.1
165.7
37.3
74.2
148.1
178.1
165.7
37.3
74.5
149.0
179.3
165.7

10
18.9
37.3
73.9
128.9
121.7
18.9
37.6
74.8
130.7
121.7
18.9
37.6
74.8
130.7
121.7
18.9
37.8
75.7
132.5
121.7

20
9.8
18.9
37.3
64.7
62.0
9.8
19.2
38.1
66.5
62.0
9.8
19.2
38.1
66.5
62.0
9.8
19.5
39.0
68.3
62.0

50
4.3
7.9
15.3
26.3
26.2
4.3
8.2
16.1
28.0
26.2
4.3
8.2
16.1
28.0
26.2
4.3
8.5
17.0
29.8
26.2

100
2.4
4.3
79
13.4
14.3
2.4
4.6
8.8
15.2
14.3
2.4
4.6
8.8
15.2
14.3
2.4
4.9
9.7
17.0
14.3

200
1.2
2.1
4.0
6.7
7.2
1.2
2.3
4.4
7.6
7.2
1.2
2.3
4.4
7.6
7.2
1.2
2.4
4.9
8.5
7.2

In Table 1, when comparing the required communication bandwidth for multicast communication in Type I
and unicast communication in Type VII, it becomes apparent that the required communication bandwidth
for multicast communication is reduced, indicating that multicast communication allows for communication
traffic reduction.
This is because in multicast communication, after UL transmission, packets are replicated at the VPN
server on the core network side, reducing the required bandwidth consumed by UL. Therefore, the2023 ODVA Industry Conference

8©2023 ODVA, Inc.

utilization of multicast communication can be an effective means to reduce communication overhead in
5G networks with limited bandwidth.
Measurement Results for EtherNet/IP over 5G Transmission Evaluation between Controllers

Table 3 presents the packet loss results obtained from measurements under each condition.
Table 3: Measurement Results of Packet Loss over 5G for EtherNet/IP between Controllers
Type

RPI [ms]

Measurement Additional Load
Conditions
Conditions# NodesⅠ

1->N Multi.

-, VPNⅡ

1->N Uni.

-,-

Ⅲ

1->N Multi.

N->1 Uni., VPN

Ⅳ

1->N Uni.

N->1 Uni., -Ⅴ

N->1 Uni.

1->N Multi, VPN

Ⅵ

N->1 Uni.

N->1 Uni., -Ⅶ

N->1 Uni.

-, VPN

Ⅷ

N->1 Uni.

-, -

1
2
4
7
7
1
2
4
7
7
1
2
4
7
7
1
2
4
7
7

Packet Loss [%]

1
0.659

5
0.001
10.714

10
0.000
0.000
5.066

20
0.000
0.000
1.494

0.000
4.161

0.000
0.000
13.551

0.000
0.000
0.000

0.000
0.062

0.000
0.000
0.109

0.000
0.000
0.000

0.000
2.865

0.000
0.000
3.390

0.000
0.000
0.000

50
0.000
0.000
0.000
1.595
0.004
0.000
0.000
0.000
0.001
0.005
0.000
0.000
0.000
0.000
0.002
0.000
0.000
0.000
0.000
0.002

100
0.000
0.000
0.000
0.000
0.004
0.000
0.000
0.000
0.000
0.005
0.000
0.000
0.000
0.000
0.002
0.000
0.000
0.000
0.000
0.002

200
0.000
0.000
0.000
0.000
0.004
0.000
0.000
0.000
0.000
0.003
0.000
0.000
0.000
0.000
0.003
0.000
0.000
0.000
0.000
0.002

Table 3 presents the packet loss results obtained from measurements under each condition. The green
cells indicate no occurrence of packet loss, while the color transitions to yellow and red as the packet loss
increases. The gray cells represent conditions where the connection could not be established, resulting in
the inability to conduct measurements.
In Table 3, the conditions that achieved successful communication without packet loss are highlighted
within the blue border. Additionally, to examine the impact of packet loss on the UL communication
performance limits, the purple borders from Table 1 are transcribed.
Here are the packet loss measurement results for each condition:
 Type I [1->N Multicast, VPN]: No packet loss was observed up to RPI 10ms for 1 and 2 nodes,
RPI 50ms for 4 nodes, and RPI 100ms for 7 nodes. Communication was successful without any
issues.
 Type III [1->N Multicast & N->Unicast, VPN]: No packet loss was observed up to RPI 5ms for 1
node, RPI 10ms for 2 nodes, RPI 20ms for 4 nodes, and RPI 100ms for 7 nodes. Comparing withType I results, it was observed that at 4 nodes, communication was possible without packet loss
even with a shorter RPI of 20ms.
 Type V [N->1 Unicast & 1->Multicast, VPN]: No packet loss was observed up to RPI 5ms for 1
node, RPI 10ms for 2 nodes, RPI 20ms for 4 nodes, and RPI 50ms for 7 nodes. Communication
was successful without any issues.
 Type VII [N->1 Unicast, VPN]: Similar results were obtained as in Type V results. Comparing withType I results, it was observed that at 7 nodes, communication was possible without packet loss
even with a shorter RPI of 100ms.
 Type II, IV, VI, and VIII: Packet loss was observed in all conditions.

2023 ODVA Industry Conference

9©2023 ODVA, Inc.

Comparing the results within the blue border and purple border, in Type I: 1->N Multicast VPN
configuration, it was observed that under conditions with fewer connected nodes and longer RPI, there
was no packet loss, and satisfactory communication was achieved as expected. However, underconditions with a larger number of connected nodes and shorter RPI, packet loss occurred andconnections could not be established, resulting in the outcomes deviating from the predictions based on
the throughput performance of 5G devices.
In Type V and Type VII: N->1 Unicast VPN configuration, there were several conditions where the blue
border and purple border overlapped, indicating favorable communication outcomes. However, in cases
like Type V, where there was multicast communication in the opposite direction as the communicationload, packet loss occurred as a result of the influence of the multicast communication.
The measurement results for latency (average values) are shown in Table 4.

Table 4: Measurement Results of Latency over 5G for EtherNet/IP between Controllers
Type

RPI [ms]

Measurement Additional Load
Conditions
Conditions # Nodes

Ⅰ

1->N Multi.

-, VPNⅡ

1->N Uni.

-,-

Ⅲ

1->N Multi.

N->1 Uni., VPN

Ⅳ

1->N Uni.

N->1 Uni., -Ⅴ

N->1 Uni.

1->N Multi, VPN

Ⅵ

N->1 Uni.

N->1 Uni., -Ⅶ

N->1 Uni.

-, VPN

Ⅷ

N->1 Uni.

-, -

1
2
4
7
7
1
2
4
7
7
1
2
4
7
7
1
2
4
7
7

Latency [ms]

1
419.4

5
24.2
344.2

10
22.1
25.1
136.1

20
22.0
22.9
74.3

22.2
601.8

20.2
23.0
751.5

19.8
22.3
31.2

19.1
21.0

18.0
19.3
20.4

18.8
18.0
20.5

20.7
731.9

19.6
18.9
1474.5

18.6
17.6
20.0

50
19.9
20.9
25.3
86.3
25.4
19.6
21.1
25.1
31.6
26.9
16.8
19.1
20.0
22.2
21.5
17.7
18.3
19.1
21.8
21.9

100
21.4
21.4
23.2
27.5
20.6
21.2
22.0
25.2
26.0
22.5
19.4
17.6
19.6
21.4
21.4
18.6
18.5
19.6
19.9
21.2

200
20.9
20.9
21.7
24.6
18.7
20.4
21.1
22.0
23.4
18.4
17.9
17.9
18.3
20.1
19.9
18.6
16.9
18.6
19.5
19.2

The latency measurement results indicate the conditions where packet loss did not occur, representing
successful communication in this experiment (transcribing the blue border from the Packet Loss in Table
4 and the purple border from the UL communication performance limit in Table 1). The latency
measurement results show that the green cells have low latency, while the latency increases as the cells
become yellow or red.
Here are the latency measurement results for each condition:
 Type I, III [1->N, Multicast, VPN]: In the range of successful communication indicated by the blue
border, latency values between 19.6ms and 31.2ms were obtained. For example, with Type I at
10ms, while maintaining an RPI of 10ms, an additional latency of 22.1ms will be added before thedata arrives. One condition (Type I: 1->N Multicast, VPN, 4 nodes, RPI 20ms) did not yield the
expected results, and packet analysis was conducted. The analysis revealed that there were
duplicate packets not being output from the VPN server for the input packets, indicating insufficientthroughput performance of the VPN multicast. This indicates that even if the UL communication
bandwidth is reduced through multicast communication, if the VPN multicast throughput
performance is not sufficient, the VPN server becomes a bottleneck in communication.
 Type IV, VI [N->1, Unicast, VPN]: In the range of successful communication indicated by the blue
border, latency values between 16.8ms and 22.2ms were obtained.
2023 ODVA Industry Conference

10©2023 ODVA, Inc.

 Type II, IV, VI, VIII [Unicast w/o VPN]: Despite the communication volume being lower than thethroughput limits of the 5G devices for both UL and DL required bandwidth, packet loss occurredand communication was not successful for any of the conditions. Unfortunately, the specific causesof packet loss could not be identified even after conducting the packet analysis. The systemcomplexity and the time consumption to pinpoint the cause of the problem are some of the issuesof 5G.
From the results above, multicast communication was achieved using L2TPv3 tunneling with a VPN
router for EtherNet/IP, which is a Layer 2 communication protocol. In the 1->N multicast communication,
latency ranging from 19.6ms to 31.2ms was observed. Additionally, in the N->1 unicast communication
with VPN, latency ranged from 16.8ms to 22.2ms. Furthermore, whether the delay being longer than the
RPI is a problem or not requires judgment based on the purpose of the application being used. It was nottreated as a problem in this context.
In this evaluation, it is possible that achieving a small latency at the level of 10ms was difficult due to theuse of synchronization with the communication carrier band in Japan. The TDD frame structure, which
maintains a communication performance ratio of 3.25:1 between DL and UL, may have resulted in lower
UL performance, making it challenging to achieve a latency of 10ms or lower.
Multicast communication can help reduce UL communication load, but the VPN multicast functionality of
the VPN server became a bottleneck in this setup. Therefore, in the case of multicast communication
between controllers using 5G transmission, the VPN server's VPN multicast performance becomes a
critical factor in determining the system's performance.

2023 ODVA Industry Conference

11©2023 ODVA, Inc.

System Configurations for Evaluation of CIP Safety over 5G Transmission
As a use case for an FA application utilizing 5G, we assumed a system that integrates three robot cells
into a single safety line controller. Based on this use case, we transmit multicast CIP Safety packets over
5G between controllers and measure packet loss, communication delay, and the Safety Reaction time of
CIP Safety packets. The system configuration for this measurement is shown in Figure 6.

5G

5G Base Station

5G Wireless Area
5G Antenna（4x4 MIMO）

Core Network
NDAC
Edge Server
Medium+5G RRH

5G BBU(ASIBx1,ABILx1)

5G
Terminal

NDAC
Backbone
SwitchNDAC
SecureGateway

1000BASE -T

VPN Router(Server)

VPN
RouterSafety Line
Controller

SafetyController

Robot Cells

Figure 6: System Configuration for CIP Safety over 5G Transmission Evaluation
between Manufacturing Equipment
The system components of this configuration are as follows: Safety Controller: One single safety line controller using NX102+SL5500 (Omron) oversees thesafety within the control system and integrates the safety controllers of the three robot cells.
 CIP Safety Network: Similar to the configuration for measuring packet loss and latency in an
EtherNet/IP network, each controller is connected to a 5G device via packet capture and a VPN
router. Additionally, a VPN server is connected to the switch in the Core Network."

2023 ODVA Industry Conference

12©2023 ODVA, Inc.

Measurement Configurations and Conditions for CIP Safety over 5G Transmission
The variations in the measurement configurations are shown in Figure 7 and Figure 8.
Safety Line
Controller

SafetyController

SafetyController

SafetyController

Send

Receive

Receive

Receive

Receive

Send

Multicast

UnicastReceive

Send
UnicastReceive

Send
UnicastMeasurement Area

Receive
Receive
Receive
Receive
Unicast

Send
Send
Send
Send
Send
Send
Send
Send

Receive
Receive
Receive
Receive
Unicast

Send
Send
Send
Send
Receive
Receive
Receive
Receive

Send
Send
Send
Send
UnicastReceive
Receive
Receive
Receive

Send
Send
Send
Send
UnicastReceive
Receive
Receive
Receive

VPN

Send
Send
Send
Send

Send
Send
Send
Send
UnicastReceive
Receive
Receive
Receive
Unicast

Send
Send
Send
Send
Receive
Receive
Receive
Receive
UnicastReceive
Receive
Receive
Receive
UnicastReceive
Receive
Receive
Receive
Unicast

Send
Send
Send
Send

VPN

Receive
Receive
Receive
Receive
Unicast

Send
Send
Send
Send
UnicastReceive
Receive
Receive
Receive

Receive
Receive
Receive
Receive
Unicast

Send
Send
Send
Send

Send
Send
Send
Send

VPN

Communication Load

Figure 7: Measurement Configurations for CIP Safety over 5G Transmission
between Safety Controllers with Multicast and VPN

2023 ODVA Industry Conference

13©2023 ODVA, Inc.

Safety Line
Controller

SafetyController

Send
Send

SafetyController

Receive
UnicastReceive
Unicast

Send
Receive
Receive

SafetyController

Unicast

Send
Unicast

Send
UnicastReceive

Receive

Send
Unicast

Send
Send
Send
Send
Send
Send
Send
Send

Receive
Receive
Receive
Receive
Unicast

Send
Send
Send
Send
Receive
Receive
Receive
Receive
Receive
Receive
Receive
Receive

Send
Send
Send
Send
UnicastReceive
Receive
Receive
Receive
Send
Send
Send
Send
Send
Send
Send
Send

Receive
Receive
Receive
Receive
Unicast

Send
Send
Send
Send
UnicastReceive
Receive
Receive
Receive
UnicastReceive
Receive
Receive
Receive
Unicast

Send
Send
Send
Send

Receive
Receive
Receive
Receive
UnicastMeasurement Area

Receive
Receive
Receive
Receive
Unicast

Send
Send
Send
Send
UnicastReceive
Receive
Receive
Receive
Unicast

Send
Send
Send
Send
UnicastReceive
Receive
Receive
Receive

Receive
Receive
Receive
Receive
Unicast

Send
Send
Send
Send

Send
Send
Send
Send

Communication Load

Figure 8: Measurement Configurations for CIP Safety over 5G Transmission
between Safety Controllers with Unicast Only
2023 ODVA Industry Conference

14©2023 ODVA, Inc.

The conditions for performance measurement are as follows: Number of CIP Safety Connections: CIP Safety connections were established between thesafety line controller as the target and the three safety controllers as originators, as well as
between each of the three safety controllers as targets and the safety line controller as theoriginator.
 EPI (CIP Safety Communication Period): The EPI for safety process communication was set to
60ms (fixed).
 Data Size: CIP Safety packets required for this use case were set to 46 bytes.
 Communication Load: To simulate real-world usage, standard process communication was used
to add communication load to CIP Safety communication. A maximum of 24 standard process
communication connections per safety controller (12 as originators and 12 as targets) were
established between each safety controller, including the safety line controller. The RPI (Request
Packet Interval) was varied at 1ms, 5ms, 20ms, 50ms, 100ms, and 200ms to introduce different
communication loads, and CIP Safety packet loss and communication delay were measured.
 Wired LAN Measurement: The safety line controller and the three safety controllers wereconnected to an L2 switch (CISCO: CBS350-16T-E-2G-JP) via Ethernet cables for measurement
using packet capture.
 Safety Reaction Time: Safety reaction time was measured between the emergency stop circuit
connected to the safety controller and the safety relay circuit connected to another safety controller(★->★). An oscilloscope was used for measurement, and the time was measured from the voltage
drop in the emergency stop circuit after pressing the emergency stop button until the safety relay
opened and the voltage of the connected power supply dropped. The system configuration andcommunication flow for safety reaction time measurement are shown in Figure 9.
 Number of Measurements: Referring to ISO13855:2010 [3], 10 measurements were conductedwithout using standard deviation determination, considering the worst-case scenario.

5G

5G Base Station

5G Wireless Area

Core Network
NDAC
Edge Server
Medium+5G RRH
5G Antenna（4x4 MIMO）
5G BBU(ASIBx1,ABILx1)

5G
Terminal

NDAC
Backbone
SwitchNDAC
SecureGateway

1000BASE -T

VPN Router(Server)

VPN
RouterSafety Line
Controller

SafetyController

Robot Cells

Figure 9: System Configurations and Communication Flowfor Measurement of Safety Reaction TimeIn addition to performance measurement, we conducted a 24-hour continuous communication test
between the safety line controller and the safety controllers of the robot cells to verify the stability of thesafety system with CIP Safety over 5G transmission and to check for any timeout errors. The operating
conditions for the continuous communication are as follows
 Timeout Error Trigger Condition: The timeout error condition was set to 120ms, which is EPI(CIP Safety Communication Period) multiplied by 2, as the default setting for Omron safetycontrollers.
 Number of CIP Safety Connections: Same as the conditions for performance measurement.
2023 ODVA Industry Conference

15©2023 ODVA, Inc.

 EPI (CIP Safety Communication Period): Same as the conditions for performance measurement.
 Data Size: Same as the conditions for performance measurement.
 Communication Load: Same as the conditions for performance measurement.
Measurement Result for Evaluation of CIP Safety over 5G between Manufacturing Equipment
Table 5 presents the measurement results of packet loss and latency (average measured value) for CIP
Safety packets in both 5G and wired LAN environments. It shows the results for multicast and unicast
communication of CIP Safety packets in 5G and wired LAN.

Table 5: Measurement Results of Packet Loss and Latency over 5G
and Wired for CIP Safety between Manufacturing Equipment
Measurement Conditions
RPI for Comm. Load [ms](EPI: 60ms Fixed)
5G Multicast w/ VPN

Packet Loss [%]
1

5

10

20

50

Latency [ms]
100

200

1

5

10

20

50

100 200

0.000 0.000 0.000 0.000 0.000 0.000 0.001 20.7 20.0 17.0 16.3 13.3 12.2 11.6

5G Unicast Only

0.002 0.009 0.009 0.004 0.006 0.002

19.4 17.6 16.5 13.0 12.8 12.3

Wired Multicast w/o VPN 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.02 0.02 0.02 0.01 0.01 0.01 0.01
Wired Unicast Only

0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.02 0.01 0.02 0.01 0.01 0.01 0.01

In 5G multicast, packet loss occurred only when the standard process communication load RPI was set to200ms. In 5G unicast, packet loss occurred under all conditions. However, there were no observed
packet losses in the wired LAN environment.

2023 ODVA Industry Conference

16©2023 ODVA, Inc.

Focusing on the condition of the standard process communication load RPI of 20ms, Figure 10 presents
a histogram of the latency distribution of CIP Safety packets.
5G Murticast

5G Unicast
6000

6000

4906
4609
4513
4562
4341
4306
3924
3778

10ms

3284
2759
2702

2620

2165
2000

0

1018
684
433
112
3

319
6 56

3645
2222

3000

2726

2484

1440
1362
939

1000

1

1

0

Latency[ms]

810
751

566
68

11

Latency[ms]

40000

40000

35000

10µs

30000

25000

25000

Number of packets

17628

15000
10000

20000
16382
15000
10000

5000
53

50

45

57

67

56

49

45

51

35

15

14

12

17

11

13

12

10

11

10

13

14

18

16

14

21

13

18

18

26

19

13

22

14 2…
15

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
50<

0

10µs

30297

5000
0

5505
22

16

22

31

33

17

22

32

22

21

32

15

16

23

15

28

26

27

13

30

27

30

30

24

36

27

40

46

40

38

32

41

27

33 661
32

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
50<

Number of packets

35010

30000

20000

0

Wired Unicast

Wired Multicast

35000

10ms

3408

2000

1574
1100

1000

3976

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
50<

3000

4203
4458

4577

4000
Number of packets

4000

5094

5000

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
50<

Number of packets

5000

5598
5451

Latency[μs]Latency[μs]Figure 10: Histogram of Latency over 5G and Wired for CIP Safety between
Manufacturing Equipment（20ms RPI for Communication Load）
Comparing 5G multicast and unicast, 5G multicast shows a slightly broader shape in terms of CIP Safety
packet latency performance. However, there was no significant difference between multicast and unicast.
Both 5G multicast and unicast achieved latency of 30ms or below for almost all CIP Safety packets.
However, there were outliers with higher latency in the multicast case. Although the cause of it has notbeen analyzed, it is assumed to be due to the performance of the VPN multicast (packet retention within
the VPN system) of the VPN server.
Furthermore, in the wired LAN environment, regardless of multicast or unicast, the latency performance of
CIP Safety packets was consistently below 1ms.

2023 ODVA Industry Conference

17©2023 ODVA, Inc.

Figure 11 shows the measurement results of safety reaction time. Safety reaction time was measured
based on the worst-case scenario, without using standard deviation determination, and taking 10
measurements.
1000.0

Safety Reaction Time [ms]900.0

Limitation criteria for inter-robot cell safety reaction time.

800.0
700.0
600.0
500.0
400.0
300.0

163.2

200.6

5G Mulcticast

5G Unicast

200.0

146.9

125.2

Wired Mulcticast

Wired Unicast

100.0
0.0

Figure 11: Measurement Results of Safety Reaction Time over 5G and Wired
for CIP Safety between Manufacturing Equipment(EPI=60ms, Load RPI=20ms、n=10)
The results showed that in 5G, multicast had shorter safety reaction time, while in wired LAN, unicast had
shorter safety reaction time. When comparing 5G to wired LAN, the higher latency in 5G resulted in larger
safety reaction time.
Based on the system configuration depicted in Fig.9, the safety reaction time calculated from Fig.12 was
677ms. Considering those the acceptable limit requirement for this use case is 800ms, it was confirmedthat 5G transmission is sufficiently usable.
SafetySensor/Switch(a) Safety
sensor/switc
h response
time

Safety Input
Unit/Slave(b) Safety
Input
Unit/slave
responsetime(c) Network
reaction time

Safety CPU Unit(d) Safety
CPU Unit
response
time

Safety CPU Unit

Safety CPU Unit(c) Network
reaction time(d) Safety
CPU Unit
responsetime(c) Network
reaction time(d) Safety
CPU Unit
response
time

Safety Output
Unit/Slave(c) Network
reaction time(e) Safety
OutputUnit/slave
response
time

Actuator(f) Actuator
response
time

Figure 12: Safety Reaction Time in Network Configuration between Controllers [4]Finally, in the evaluation of running the system continuously for 24 hours using the system configuration
shown in Figure 6, no timeout errors occurred in either multicast or unicast communication, and thesystem operated stably for the entire 24 hours. This confirms that safe communication between theequipment can be achieved using CIP Safety over 5G transmission.
Based on these results, CIP Safety multicast communication demonstrated practicality in terms ofcommunication stability, obtained safety reaction time, and the evaluation of 24-hour system operation
using 5G transmission. However, in CIP Safety multicast communication, there were instances of packet
loss and latency outliers. If the operation time is extended, timeout errors may occur, which may lead to a2023 ODVA Industry Conference

18©2023 ODVA, Inc.

safety stop on the equipment line. To prevent safety stoppages on the line due to wireless communication
issues, it is desirable to further reduce packet loss and latency outliers.
Conclusion
The conclusions obtained from applying 5G communication for controller-to-controller communication,
evaluating EtherNet/IP performance, measuring safety reaction time of CIP Safety in the robot cell line
use case, and conducting a 24-hour continuous power evaluation are presented below.
 When applying 5G communication for Implicit communication between controllers, multicast
communication from Controller 1 to N nodes (N = 1, 2, 4, 7) achieved an average latency of 19.631.2ms, while unicast communication from N nodes to Controller 1 achieved an average latency of
16.9-22.2ms. This indicates that it can provide the expected performance for AGV control as
defined by NICT in Figure 1. However, it was not possible to achieve the required latency of 10ms
or below for "Machine, Robot control." The results presented in Table 4 indicate that the VPN multicast capability of the VPN server canpotentially become a bottleneck in system performance for EtherNet/IP multicast communication
over 5G transmission.
 When applying 5G communication to CIP Safety in the robot cell line use case (where the operator
operates a safety button), it was confirmed that the achieved safety reaction time met theperformance requirement of 800ms (163.2ms). However, it was not possible to achieve therequired latency of 10ms or below for "Emergency warning" as defined by NICT, which requires
high-speed reaction times where the operator does not make safety judgments (e.g., safety lightcurtains).
 In the system where CIP Safety was transmitted over 5G, it was confirmed that there were no
timeout errors and that stable operation was possible for 24 hours.
Looking ahead, improvements are expected in the UL performance through changes in the performance
ratio of DL and UL in TDD for 5G Sub6, the introduction of URLLC (Ultra-Reliable and Low LatencyCommunications) including 5G devices, advancements in private 5G using 5G millimeter-wave
technology, and improvements in VPN multicast throughput of VPN servers. These advancements are
anticipated to enable the realization of a latency of 10ms using 5G wireless technology. Furthermore, withimproved operational reliability of private 5G, it is expected that stability of 5G communication
applications, including CIP Safety, can be enhanced by reducing packet loss and latency within thesystem.
References[1] "Wireless use cases and communication requirements in factories (abridged edition)",Version 1.0,
February 2018, NICT. https://www2.nict.go.jp/wireless/en/ffpj-wp.html[2] “Promotion of Local 5G and Other Initiatives by the Ministry of Internal Affairs and Communications(MIC)” https://www.soumu.go.jp/main_content/000739007.pdf[3] ISO 13855:2010 “Safety of machinery - Positioning of safeguards with respect to the approach speedsof parts of the human body”[4] “Safety Network Controller, NX-series, Safety Control Unit/ Communication Control Unit User's
Manual” https://www.fa.omron.co.jp/data_pdf/mnu/z395-e1-14_nx-sl5_si_so_csg.pdf?id=3813******************************************************************************************************************************************************The ideas, opinions, and recommendations expressed herein are intended to describe concepts of the author(s) for the possible useof ODVA technologies and do not reflect the ideas, opinions, and recommendation of ODVA per se. Because ODVA technologies
may be applied in many diverse situations and in conjunction with products and systems from multiple vendors, the reader andthose responsible for specifying ODVA networks must determine for themselves the suitability and the suitability of ideas, opinions,and recommendations expressed herein for intended use. Copyright ©2023 ODVA, Inc. All rights reserved. For permission to
reproduce excerpts of this material, with appropriate attribution to the author(s), please contact ODVA on: TEL +1 734-975-8840
FAX +1 734-922-0027 EMAIL odva@odva.org WEB www.odva.org. CIP, Common Industrial Protocol, CIP Energy, CIP Motion, CIP
Safety, CIP Sync, CIP Security, CompoNet, ControlNet, DeviceNet, and EtherNet/IP are trademarks of ODVA, Inc. All othertrademarks are property of their respective owners.

2023 ODVA Industry Conference

19©2023 ODVA, Inc.