User Perspective:

Single hop inter-VLAN routing– a capability that EtherNet/IP I/O can take advantage of– a feature that suppliers of advanced, managed, industrial Ethernet
switches should consider implementing.

Gary Workman

Principal Engineer – General Motors
Gary.C.Workman@GM.com

Technical Trackwww.odva.org

Outline
GM’s Current EtherNet/IP Network Architecture EtherNet/IP Network Size Discussion
Switching vs. Routing EtherNet/IP I/O Traffic
Single Hop Inter-VLAN Routing Proposal
 Description
 Advantages / Limitations
 Alternatives
I/O Device Communication Needs
GM’s EtherNet/IP I/O Network Architecture
Summary / Acknowledgements
Technical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

page 2www.odva.org

GM’s Current EtherNet/IP Network Architecture

Plant Backbone Network
Layer 3 Switch / Router
10.0.0.1
System Switch
10.0.0.200

Processor Switch
10.0.0.202

Processor Switch
10.0.0.201

10.0.0.10

Technical Track© 2014 ODVA, Inc.

10.0.0.11

10.0.0.12

10.0.0.21

10.0.0.22

10.0.0.20

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

page 3www.odva.org

How Large Can a Logical EtherNet/IP Network Be ?
One limit to the logical size of an Ethernet network is the abilityof devices on that network to tolerate extraneous broadcast
Ethernet traffic.
Operational problems occurred on early installations ofEtherNet/IP networks with 300+ nodes due to ARP
broadcast traffic. Two situations caused device failures:
1) A broadcast EtherNet/IP “Who’s there?” message from a
newly activated diagnostic computer.
2) An IT asset inventory application running on a highperformance server searching for in use IP addresses.
GM now limits EtherNet/IP networks to a maximum of 254
devices. Every device in a GM EtherNet/IP network now has
a subnet mask of 255.255.255.0
Technical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

page 4www.odva.org

Communicating EtherNet/IP Implicit Message Traffic
Between PLCs on Different EtherNet/IP Networks
Option 1: Make one PLC be a member of two EtherNet/IP networks

Technical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

page 5www.odva.org

Communicating EtherNet/IP Implicit Message Traffic
Between PLCs on Different EtherNet/IP Networks
Option 2: Route unicast EtherNet/IP traffic across the IT plant network.

Technical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

page 6www.odva.org

Communicating EtherNet/IP Implicit Message Traffic
Between PLCs on Different EtherNet/IP Networks
Option 2: Route unicast EtherNet/IP traffic across the IT plant network.

Technical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

page 7www.odva.org

Switching vs. Routing EtherNet/IP I/O Traffic
Note the device IP addresses
Industrial Ethernet
SwitchCommercial Grade Router /Layer 3 Switch

Programmable
LogicController

10.0.0.10

10.0.0.1
1

Switched
Communication

Ethernet
CapableI/O Block192.168.1.11
Ethernet
CapableI/O Block

Routed Communication

Technical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

page 8www.odva.org

Switched EtherNet/IP I/O Communication
Communicating devices belong to the same logical Ethernet network

Switched
Communication
10.0.0.11

10.0.0.10

Encapsulated IP Packet
I/O BLOCK MAC ADDR

PLC MAC ADDR…

10.0.0.10

10.0.0.11…

User Data

Ethernet Frame
Technical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

page 9www.odva.org

Routed EtherNet/IP I/O Communication
Communicating devices belong to different logical Ethernet networks

10.0.0.1 / 192.168.1.1

Routed Communication
10.0.0.10

Ethernet frame sent:
Router MAC ADDR

PLC MAC ADDR

192.168.1.11…

10.0.0.10

192.168.1.11…

User Data

Router MAC ADDR …

10.0.0.10

192.168.1.11…

User Data

Ethernet frame received:
I/O Block MAC ADDRTechnical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

page 10www.odva.org

Routed EtherNet/IP I/O Communication
Switch with single hop inter-VLAN routing feature

10.0.0.201 / 192.168.1.1

Inter-VLAN
Routed Communication
10.0.0.10

Ethernet frame sent:
Switch MAC ADDR

PLC MAC ADDR

192.168.1.11…

10.0.0.10

192.168.1.11…

User Data…

10.0.0.10

192.168.1.11…

User Data

Ethernet frame received:
I/O Block MAC ADDR Switch MAC ADDRTechnical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

page 11www.odva.org

Single hop inter-VLAN routing capable switches
End devices take advantage of the single hop inter-VLAN routing
feature by having their default gateway addresses point to a
virtual router interface configured on the switch.

10.0.0.1

10.0.0.201

10.0.0.202

10.0.0.200

192.168.1.1

10.0.0.10
255.255.255.0
10.0.0.201

Technical Track© 2014 ODVA, Inc.

192.168.1.1

10.0.0.11

192.168.1.11
255.255.255.0
192.168.1.1

10.0.0.20
255.255.255.0
10.0.0.202

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

10.0.0.21

192.168.1.11
255.255.255.0
192.168.1.1

page 12www.odva.org

Combining single hop inter-VLAN Routing withtrunking VLAN traffic between switches
Note the virtual router interface IP addresses and the end
device default gateway addresses

10.0.0.1

192.168.2.202

192.168.2.201

10.0.0.202

10.0.0.201
10.0.0.200

10.0.0.10
255.255.255.0
10.0.0.201
Technical Track© 2014 ODVA, Inc.

10.0.0.11 192.168.2.11
255.255.255.0
192.168.2.201

10.0.0.20
255.255.255.0
10.0.0.202

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

10.0.0.21

192.168.2.21
255.255.255.0
192.168.2.202

page 13www.odva.org

Combining single hop inter-VLAN Routing withtrunking VLAN traffic between switches
Note the virtual router interface IP addresses and the end
device default gateway addresses.
192.168.2.201
10.0.0.201

192.168.2.202

10.0.0.1

10.0.0.202

192.168.1.1

192.168.1.1

10.0.0.200

10.0.0.10
255.255.255.0
10.0.0.201
Technical Track© 2014 ODVA, Inc.

192.168.2.11 192.168.1.11
255.255.255.0 255.255.255.0
192.168.2.201 192.168.1.1

10.0.0.20
255.255.255.0
10.0.0.202

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

192.168.2.21
255.255.255.0192.168.2.202

192.168.1.11
255.255.255.0
192.168.1.1

page 14www.odva.org

Advantages / Limitations ofSingle hop inter-VLAN routing
Advantages:
1) Limited, local routing capability;2) Multiple restricted access, privately addressed
networks can be built;3) SNMP traffic statistics supporting private networks
are available for remote monitoring and management.
Limitations:1) Only unicast traffic can be routed;
2) Traffic is only routed for end devices that are awarethat a switch supports this service.
Technical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual MeetingAll rights reserved.

page 15www.odva.org

Alternatives toSingle hop inter-VLAN routing
1) Implement a true layer 3 switch (more expensive andmore complex solution, significant IT concerns, every
device requires a unique IP address)
2) Deploy truly isolated Ethernet I/O networks(Controllers must support multiple Ethernet
interfaces, Isolated I/O networks are difficult totroubleshoot)3) Expand the size of allowable networks (every device
on an existing network needs to be reconfigured,
ensure every device on a large network can tolerate
worst case background broadcast traffic conditions)4) Install Network Address Translation (NAT) Gateways(cascaded NAT gateways can become difficult tomanage)
5) Procure a controller Ethernet Interface card thatsimultaneously supports multiple IP addresses.
Technical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

page 16www.odva.org

I/O Device Communication Needs
What is the value in assigning a globally unique IP address toan I/O device?
There are extremely few devices that need to or would benefit from
communicating with a random Ethernet capable I/O device.
There are numerous reasons for isolating and/or securing an I/O
device so that it is only able to communicate with a few otherdevices.

What devices need to communicate with a typical I/O device?
Conversely, which devices does a typical I/O device have alegitimate and productive reason to communicate with?
A controller.
A configuration device, a network monitoring device, atroubleshooting tool.
A time server, a name server, an address server.

If there is a need to communicate with a distantly remotedevice, it will likely take place through a security server.
Technical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

page 17www.odva.org

GM’s Current EtherNet/IP Network Architecturewith multiple DeviceNet I/O networks
Plant Backbone Network
Layer 3 Switch / Router

System Switch

Processor Switch

Technical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

Processor Switch

page 18
www.odva.org

GM’s Proposed EtherNet/IP Network Architecturewith multiple (linear DLR) EtherNet/IP I/O networks
Plant Backbone Network
Layer 3 Switch / Router

System Switch

Single hop Inter-VLAN
routing capable
Processor Switch

Technical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

Single hop Inter-VLAN
routing capable
Processor Switchpage 19
www.odva.org

Summary
Ethernet I/O increases themanufacturing need for IP addresses
by an order of magnitude or more.
EtherNet/IP (unicast) real-time traffic
can be easily and reliably routed.
Single hop inter-VLAN routing is acapability that EtherNet/IP I/O cantake advantage of. It is a feature that
suppliers of advanced, managed,
industrial Ethernet switches should
consider implementing.
Technical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

page 20www.odva.org

Acknowledgements
I would like to gratefully acknowledge
the help and assistance of numerous
individuals from Cisco, Hirschmann,and Rockwell Automation in
conducting the experiments andparticipating in the discussions that
resulted in this presentation and theaccompanying paper.
I would also like to thank ODVA for
allowing me to share my findings,
observations, and opinions with you.
Technical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

page 21www.odva.org

Questions?

Technical Track© 2014 ODVA, Inc.

2014 Industry Conference & 16th Annual Meeting
All rights reserved.

page 22www.odva.org