Reference Architectures for
Industrial Automation and Control systems
Paul Didier, Cisco Systems
Technical Trackwww.odva.org

Control Network types

Isolated Single Controller• Single Controller• 10s of devices• Potentially multiple
switches• Limited non-CIP traffic• Sharing data via sneaker
net or transferable device

Technical Track© 2012 ODVA, Inc.

Isolated Multiple Controller• Multiple Controllers• Up to 100s of devices• 10s of switches, maybe arouter• A few networks• Potentially multiple switches• Controllers sharing data
• Some non-CIP traffic (e.g.
HTTP, file sharing, etc.)

Enterprise Connected• Many Controllers• Up to 1000s of devices• Lots of switches and routersand other network
infrastructure• Many “networks”• Sharing data, applications
and services between
Enterprise and Plant
networks• Could have lots of non-CIP
traffic (e.g.
Voice, Video,
2012 ODVA Industry Conference & 15 Annual Meeting
page 2All rights reserved.
www.odva.org
etc.)
th

AgendaBenefits of Reference Architecture
Overview
Topology and Resilience
Multicast Management and IGMP
Prioritization and QoS
Routing & Layer 3
Security
Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 3www.odva.org

Manufacturing 2.0:
Plant Operations Transformation
Integrated
Business andPlant Data
…Multi-Site

Real-Time
Asset
Tracking,
Monitoring,andNotification

Collaborative
Tools and
Processes

Enhanced
Security andPersonnel
Tracking

IT andControl
SystemFlexibility

Traditional
Plant

Mobile andRemoteWorkers

SensorEnabledManufacturing

Plantof the Future

Real
Time

Restricted andIsolated
Static Environment

Dynamic, Integrated
Mobile, Real-Time
Connected
Environment“Solid State”“Liquid State”
Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

Source: AMR, Industry Week, Cisco Analysis

page 4www.odva.org

Manufacturing and IT Convergence
Creating Challenges and Opportunities

Technology
Convergence

Wide Ethernet
Deployment
Network
Convergence

Organizational
Convergence

Business Model
Innovation

 Business Agility Competitive Advantage

Increasing Business
Pressures
CulturalConvergence
Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 5www.odva.org

Challenges
with Manufacturing Convergence
Organizational
IssuesIndustrial
Applications

 Misaligned
objectives
 Support
requirements
 Different models
and language

 Industrial
protocols andtraffic patterns
 Hardened
products
 Determinism,
latency, etc.
 Motion control

Technical Track© 2012 ODVA, Inc.

Security

 Increased risk
with COTS
technology
 Patching issues
 Implications ofissues Impact on
performance &ease of use

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

MFG and IT
Skill Alignment

 Ease of use Multiple
management
tools Understanding ofindustrial
applications

page 6www.odva.org

Logical Architecture
Built on Industry Standards
Enterprise Zone

Enterprise Network

Level 5Site Business Planning and
Logistics Network

Level 4DMZ

Demilitarized Zone—
Shared Access

Manufacturing Zone

Site Manufacturing Operations
and Control

Level 3

Area Control

Level 2

Basic Control

Level 1

Process

Level 0Cell/Area
Zone

Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 7www.odva.org

Converged Plantwide
Ethernet ArchitectureInternet

Enterprise Network
Levels 4–5
Web

Patch Management
Terminal Services
Application Mirror
AV Server

Apps DNS

FTPEnterprise/IT Integration
Collaboration
Wireless
Application Optimization

Application and Data share
Gbps Link for
FailoverDetection

Demilitarized Zone
(DMZ) Firewalls

Firewall(Standby)

Access Control
Threat Protection

Firewall(Active)

Site Operations and Control
Core
Switches

Manufacturing Zone
Level 3Distribution and Core

AccessSwitchFactory
Application
Servers

Network Services

Multi-Service Networks
Network and Security
Management
Routing

Aggregation
SwitchEtherNet/IP (Industrial Protocols)
Layer 2
Access Switch

Real–Time Control

HMI

Controller
HMI

Drive
Controller
HMI Distributed
I/O
Technical Track
Cell/Area© 2012#1
ODVA, Inc.
(Redundant Star Topology)

Drive
Controller
Cell/Area #2(Ring Topology)

Cell/Area Zone
Levels 0–2
Layer 2 Access

Drive

Distributed I/O2012 ODVA Industry Conference & 15th Annual Meeting
Cell/Area #3
All rights reserved.
(Linear Topology)

Fast Convergence
Traffic Segmentation andManagement
Easeof Usepage 8www.odva.org

Cell/Area Zone Overview
Layer 3Distribution
SwitchLayer 2 AccessSwitchLevels 0–2

Aggregation
Switches

Media andConnectors
Legend:IE Switch

Level 1
Controller

HMI

Level 2 HMI
VFDController

Layer 2
Interswitch UplinkVLAN Trunk,
Layer 2 Resiliency

Controller
IE Switch

Level 0 Device(Drive)Layer 2 Access
Link-Single
VLAN Assigned
to PortHMI

DIO
Cell/Area
Zone

Controller
Controller

Cell/Area
Zone

The Cell/Area Zone Is a Layer 2 Network for a Functional Area of a Production Facility.
Key Network Considerations Include: Environmental constraints
 Range of device intelligence Time-sensitive applications
Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 9www.odva.org

Networking Best Practices – Cell/Area Zone
Best Practices For Reducing Latency and Jitter, and to Increase Data
Availability, Integrity and Security

IP Multicast Control
 IGMP Management
Segmentation
 Virtual LANs (VLANs)
Prioritization
 Quality of Service (QoS)
Apply Resiliency Protocols andmulti-path topologies
 Use Fiber-media uplinks forfast convergence
Defense-in-Depth Security

Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 10www.odva.org

Cell/Area Traffic Flows Cell/area traffic is predominately (>80%)
local, cyclical I/O (a.k.a. Implicit) traffic
Producers generated UDP multi-cast messages
Consumer generated UDP uni-cast messages
Packets are small: 100-200 Bytes, but
communicated very frequently (every 0.5 to 10’sof ms).
Typically un-routable (TTL=1 by application) The rest is informational control andadministration (or Explicit) traffic flows intraand inter-cell/area
CIP-based, non-critical administrative or data traffic
Diagnostic information via HTTP
Status and fault warnings via SNMP or SMTP
Packets are larger, ~500 bytes but infrequent (100s of
ms)Technical Track© 2012 ODVA, Inc.

DMZ
Manufacturing Zone
Engineering Laptop
Network
Management
Mail Gateway

Aggregation
SwitchHMIHMIController

AccessSwiitch
Drive

Access Swiitch

Cell/Area Zone

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

Cell/Area Zone

page 11www.odva.org

Convergence Requirements
Target Cycle
Time

Target RPI

Information/Process (e.g.
HMI)

<1s

100 - 250 ms < 1 sec

Time critical processes (e.g.
I/O)

30 - 50 ms

20 ms< 40 ms

Safety10 - 30 ms

10 ms< 15 ms

Motion500 µs - 5ms

50 µs - 1 ms < 1ms

Requirement Class

Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

Target Network
Convergence

page 12www.odva.org

Resiliency for Industrial Applications
Supporting Multiple Topologies
Ring ConvergenceResilient Ethernet Protocol (REP)Si

Achieves ~50 ms convergencein large, complex networks

Redundant Star ConvergenceMultiple protocol options
Convergence times of <100ms for

Flexlinks and Etherchannel

Tested with Rockwell applications
and multicast traffic
Fast convergence avoids application
reset and improves uptime
Critical for industrial applications

Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

CZ-3750

page 13www.odva.org

Reliability, Availability and Network
Segmentation Cell/Area Zone Topology Options
Redundant Star

Star/Bus Linear

Ring

EtherChannel

MSTP
Aggregation Switches

Aggregation
Switches

Aggregation
Switches

HMI

HMI
Controller

Controllers

HMI

Controllers

HMI

Controllers,
Drives, and Distributed I/O

Cell/Area Zone
Cell/Area Zone

Controllers, Drives, and Distributed I/OControllers, Drives, and Distributed I/O

Cell/Area Zone

Redundant Star

Cell/Area Zone

Ring

LinearTechnical Track2012 ODVA Industry Conference & 15th Annual Meeting
Overall in Network TCO and PerformanceBest
OK© 2012 ODVA, Inc.
All rights reserved.

page 14
Worst
www.odva.org

Cabling Requirements
East of Configuration
Implementation Costs
Bandwidth
Redundancy and Convergence
Disruption During Network Upgrade
Readiness for Network ConvergenceSpanning Tree Protocol (STP)
Most common standard protocol
for network resiliency—IEEE
802.1D
Supports Redundant Star and Ring
Topology
Provides alternate path in case of
failures, avoiding loops
Unmanaged switches don’t
support STP
Versions: STP, RSTP, MSTP and
RPVST+ - there are differences
Coordinate with IT before
implementing

Distribution
Switches
Aggregation
Switch

X

X
F

B

F

B
AccessSwitches

F- Forwarding
B- Blocking

Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 15www.odva.org

Layer 2 Hardening
Spanning Tree Should Behave the Way You Expect
 Place the root where you want it—Distribution Switch

LoopGuard Root primary/secondary macro The root bridge should stay where you
put itRootGuard

STP RootSi

Si

RootGuard

LoopGuard

LoopGuard

UplinkFast
UDLD Only end-station traffic should be seen
on an edge port

UplinkFast

BPDU Guard

BPDU Guard or
RootGuard
PortFast
Port Security

RootGuard
PortFast
Port-security

Standard setup applies the above
Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 16www.odva.org

EtherChannel
 Link Aggregation Control Protocol(LACP) port aggregation—IEEE
802.3adDistribution Switches
Aggregation
Switches Redundant Star Topology
 A way of combining several physicallinks between switches into one logicalconnection to aggregate bandwidth (2 to8 ports)
 Provides resiliency between connectedswitches if a connection is broken

AccessSwitches

Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 17www.odva.org

Reliability, Availability and Network
Segmentation
Cell/Area Zone Topology Options
Redundant Star

Star/Bus Linear

Ring

EtherChannel

MSTP
Aggregation Switches

Aggregation
Switches

Aggregation
Switches

HMI

HMI
Controller

Controllers

HMI

Controllers

HMI

Controllers,
Drives, and Distributed I/O

Cell/Area Zone
Cell/Area Zone

•
•
•
•

Controllers, Drives, and Distributed I/OControllers, Drives, and Distributed I/O

Cell/Area Zone

Cell/Area Zone

Use Fiber over Copper for uplinks
Spanning Tree (MSTP) recovery in Ring topology for CIP Explicit
Messaging such as HMILACP in Redundant Star for CIP Implicit I/O applications
Device Level Ring for device connectivity

Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 18
www.odva.org

Multicast Protocols
 IGMP—Internet Group Management Protocol
 IGMP snooping is used to prevent multicast
from flooding all ports on a VLAN. It monitors
the IGMP packets from end devices

IGMP
QUERIER

IGMP snooping becomes operational as soon as a Querier isdetected
A Layer 2 access switch can act as an IGMP querier
Recommendation to select the distribution switch as acting
querier by giving it the lowest IP on the VLAN

IGMP
SNOOPING Make sure IT is aware of multicast
requirements
 IE switch enables IGMP in standard setup
IGMP v2, Querier and Snooping

Standard setup applies the above

Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 19
www.odva.org

IP Multicast Group Concept
 The device must join a groupin order to receiveits data

Sender andReceiverGroup Member 3

 All members of a group
receive the same data A device can send to a group
without being a member ofthat group

SenderA

B

D“Non”- Group
MemberC

E

Group
Member 1

ReceiverTechnical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

Group
Member 2

Receiverpage 20www.odva.org

IGMP Snooping Summary

Producer-Consumer

 IGMP Snooping provides scalability for
Consumer-Producer Data Models bylimiting the amount of multicast traffic

MulticastTraffic w/o IGMP
Snooping Performance benefits of the ConsumerProducer model are maintained (allconsumers have equal access to data)

Mbps

 In a Consumer-Producer Model traffic
grows exponentially with the number ofhosts unless multicasts are constrained

Multicastwith IGMP
Snooping

Unicast Traffic

Number of Control Devices
Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 21www.odva.org

VLANs in an Industrial Ethernet System
Assign VLANs to devices
when traffic patterns are
known
Limit the flow of produce of
required devices (e.g.:
one VLAN per cell orzone)
Use L3 switch such as IE
3000 to exchange data
between VLANs (i.e. PLC
interlock layer)

Backbone
Network

VLAN
101
Si

Zone
VLAN 102VLAN 103

Cell

Learn
Your Traffic Patterns: Safemap.Sourceforge.Net
Technical Track2012 ODVA Industry Conference & 15 Annual Meetingth© 2012 ODVA, Inc.

All rights reserved.

Si

VLAN 104

Cell

page 22www.odva.org

VLAN
105

VLAN Considerations for Cell/Area zone
Design small Cell/Area zones, segment traffic types into
VLANs and IP Subnets to better manage the traffic
Requires Layer-3 switch or router to communicate between
VLANs
Use Layer 2 VLAN trunking between switches
When trunking, use 802.1Q, VTP in transparent modeSet native VLAN to something other than 1
Use switchport mode host command to assign VLAN to end
deviceDo not use VLAN 1 for EtherNet/IP Control & Information
Traffic
Enable IP directed Broadcast on Cell/Area VLANs with
EtherNet/IP traffic for easy configuration and maintenancefrom IACS applications
Prune unused VLANs for security
Use VLAN 1 for data is viewed as a security risk
Create a Network Management VLAN, don’t use VLAN 1

Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 23www.odva.org

Not All Traffic Is Created Equal
Prioritization Is Required
Control(e.g., CIP)
Bandwidth

Data(Best Effort)

Video

Voice

Low to Moderate Moderate to High Moderate to High Low to Moderate

Random Drop
Sensitivity

High

Low

High

Low

Latency
Sensitivity

High

High

Low

High

Jitter Sensitivity

High

High

Low

High

Control Networks Must Prioritize Control Traffic over Other Traffic Types toTechnical Track
2012
ODVA Flows
Industry Conference
15 Annual
Meeting
page 24
Ensure DeterministicAll
Datawith &Low
Latency
and Low
Jitter© 2012 ODVA, Inc.
rights reserved.
www.odva.orgthQuality of Service Operations

Classificationand Marking

Technical Track© 2012 ODVA, Inc.

Queuing and(Selective) Dropping

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

Post-Queuing
Operations

page 25www.odva.org

Cell/Area Zone QoS Priorities
Output Queue traffic prioritization
Cell/Area Zone QoS
Typical Enterprise QoS
Voice
Priority
Queue 1Video

PTP-Event

Priority
Queue 1CIP Motion
PTP Management,
Safety I/O and I/OCall Signaling
Network Control

OutputQueue 2

OutputQueue 3
OutputQueue 4

Network Control

Voice

Critical DataCIP Explicit
Messaging

Best Effort
Bulk Data
Scavenger

OutputQueue 3

Call Signaling

OutputQueue 4Video

OutputQueue 2

Critical Data
Bulk Data
Best Effort

Note:
the Conference
queue & 15th Annual MeetingTechnicalDue
Trackto queue characteristics of the2012 IE3000,
ODVA Industry
page 26
©
2012
ODVA,
Inc.
All
rightsreserved.
www.odva.org
order of priority is different than general enterprise.
Scavenger

QoS Design Considerations
 Priority for latency and jitter sensitive CIP
I/O traffic
Guaranteed delivery for CIP sync, CIP motion
Minimize impacts by DDoS attacks

 QoS deployed throughout industrial network

Device w/out QoS
marking support

 QoS trust boundary moves from switch
access ports to QoS-capable CIP devices
 For existing CIP devices, marking at theaccess port is based on port number

DIO

Drive

Servo
Drive

HMI

CIP I/O UDP 2222
CIP Explicit TCP 44818

Controllers

CIP Motion
or QoS readydeviceGigabit
Ethernet

Fast
Ethernet

No Trust + Policing + CoS/DSCP
Marking + Queuing
Trusted DSCP + CoS
Marking + QueuingTechnical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

Trusted DSCP +CoS Marking +Queuing

page 27www.odva.org

IP Addressing
Management
OptionDescription

Advantages

Disadvantages

 Simple to commission
and replace In large environments, can beburdensome to maintain Limited ranged of IP addresses andsubnet Not all devices support
 Requires technician to configure IP
address/Mac address when a device
is replaced Adds complexity and point of failure

StaticAll devices hard codedwith an IP Address

Static via BOOTP
Configuration

Server assigns devices
IP addresses
Precursor to DHCP

DHCP

Server assigns IP
addresses from a pool
(NOT
RECOMMENDED for
Cell/Area devices)

DHCP Option 82

 Efficient use of IP
 More complex to implement and adds
Server assigns
a point of failure
consistent IP addresses Address range
Can
reduceMixedenvironments may not workfrom a pool (NOT
administration work load
RECOMMENDED)

Technical Track© 2012 ODVA, Inc.

 Supported by every
device Efficient use of IP
 More complex to implement and adds
address range
a point of failure Can reduce Devices get different IP addresses
administration work loadwhen they reboot

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 28
www.odva.org

Manufacturing Zone Overview
Site Manufacturing
Operations and Control

Manufacturing Zone
Level 3

FactoryTalk Application Servers
 View
 Historian AssetCentre Transaction Manager
FactoryTalk Services Platform Directory Security/Audit
Data Servers

Core
Switches

Layer 3
RouterLayer 3
SwitchStack

Network Services
Remote Access, DNS, DHCP, syslog server
Network and security management

 Highly available Layer 3, routing
services for the Plant network
Provides inter Cell/Area zone
connectivity
 Interconnectivity to the DMZ
 Network and Security
management
Technical Track© 2012 ODVA, Inc.

Manufacturing
Zone - Core

Aggregation
SwitchManufacturing Zone Distribution

 Level 3 Plantwide applications
including the Factory Talk suite Key network-based services such as
Access and Authentication (e.g.
Active Directory), DHCP, DNS, etc.
 Remote Access Server

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 29
www.odva.org

Distribution Layer
Policy, Convergence, QoS, and High Availability
 Availability, load balancing,
QoS and provisioning are the importantconsiderations at this layer Aggregates Cell/Area zones(access layer) anduplinks to core

Si

Si

Core

 Protects core from highdensity peering andproblems in access layer

Distribution

 Route summarization, fast convergence,
redundant
path load sharing

Si

Si

Access HSRP or GLBP to provide
first hop redundancy

Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 30
www.odva.org

Core Layer
Scalability, High Availability, and Fast Convergence Backbone for the
network—connects
network building blocks
 Performance andstability vs. complexity—
less is more in the core

Si

Si

Core

 Aggregation point for distribution layer Separate core layer
helps in scalability
during future growthDistribution
Si

Access Keep the design
technology-independent

Technical Track© 2012 ODVA, Inc.

Si

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 31www.odva.org

Manufacturing Zone Scalability
DMZ
Manufacturing Zone

Small

 Collapsed Core/Distribution
 30-50 Access switches

Core/Aggregation Switches

Cell/Area Zone

DMZ

Medium Collapsed Core/Distribution
 <200 Access Switches

Manufacturing Zone
Core/Aggregation
Switches

Cell/Area Zone

DMZ

Large

 Distinct Core/Distribution
 >200 Access Switches

Layer 3
RouterManufacturing Zone
Core Switches

Aggregation
Switches

Cell/Area Zone
Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 32
www.odva.org

Routing DesignCore and Distribution Routing Design
 Good routing design forms
the foundation of the HA campus design Needed to quickly re-route around failed
node/links while providing load balancing
over redundant paths

Si

Si

Si

Si

 Build triangles not squares for deterministic
convergence Only peer on links that you intend to use astransit Insure redundant L3 paths to avoid blackholes Map the protocol design to the physicaldesignTechnical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 33
www.odva.org

EIGRP in the Plant
Leverage the Tools Provided
 The greatest advantages of EIGRP are gained
when the network has a structured
addressing plan that allows
for use of summarization andstub routers10.10.0.0/16

 EIGRP provides the ability
to implement multiple tiersof summarization and route filtering Relatively painless to migrate
to a L3 access with EIGRP
if network addressing
scheme permits Able to maintain a deterministic
convergence time in very large
L3 topology

Technical Track© 2012 ODVA, Inc.

Si

Si

10.10.0.0/17

Si2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

10.10.128.0/17

SiSi

page 34
www.odva.org

Si

EIGRP Design Rules for HA Plant
Layer 2 Access—Same Rules Apply The basic EIGRP design rules apply
irregardless if the access is L2 or L3
 Minimize the number and time for query
response to speed up convergence

Si

Si

Si

Si

Si

Si

 Summarize distribution block routes
upstream to the core
 Summarize at every major network tierinterface TenGigabitEthernet 4/1
ip summary-address eigrp 100 10.120.0.0
255.255.0.0 5
router eigrp 100
network 10.0.0.0
passive interface <access-layerinterface>

Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 35
www.odva.org

Design Principles—Plant Routing
Manage Your Routing Protocol
 Manage your routing protocol don’t let it
manage you It is easy to just turn it on and forget because
it works
 Optimize your configuration
to ease management and improve
convergence

Distribution

Si

Si

Routing
UpdatesAccess Manage:
Router peering
Route summarization
Route propagation
Failure notifications(LSA and queries)
Protocol timers

Technical Track© 2012 ODVA, Inc.

E.G. Prevent Peering Through the Access:Router(config)#router eigrp 1
Router(config-router)#passiveinterface Vlan 99
Router(config)#router eigrp 1
Router(config-router)#passiveinterface default
Router(config-router)#no passiveinterface Vlan 99

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 36www.odva.org

Logical Framework

Technical Track© 2012 ODVA, Inc.

Enterprise Network

Level 5
Level 4Enterprise
Zone
E-Mail, Intranet, etc.

Terminal
Services

Site Business Planning and Logistics Network

Patch
Management

Application
MirrorFirewallAV
ServerWeb Services
Operations

Web
E-MailCIP

Application
ServerDMZ

FirewallLevel 3

Level 2

Application
Server

Factory
Directory

Engineering
Workstation

DomainController
Site Manufacturing
Operations and Control
Area
Supervisory
Control
Operator
Interface

Factory
ClientFactory
ClientOperator
Interface

Manufacturing
Zone

Engineering
Workstation

Basic Control

Level 1
Level 0

Batch
Control

Discrete
Control

Sensors

Drive
Control

DrivesContinuous
Process
Control

Actuators

ISA-99Purdue Reference Model, ISA-95

Strong Segmentation

Cell/Area
Zone

SafetyControl

Robots

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

Process

page 37
www.odva.org
37

DMZ Deployment
Components and Traffic Flow

Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 38
www.odva.org

DMZ and Secure Remote Access
Guiding Principals



VPN for secure remote accessEnterprise Access and Authentication servers(e.g Active Directory, Radius, etc.)• Remote Access Server• Application level security

Enterprise
Data Center

S S L VP N

IACS Protocols Stay home
Control the Application

Internet

I P S E C V PN

Use IT-Approved Access andAuthentication

Enterprise Zone
Levels 4 and 5

Enterprise
WAN

No direct traffic
No common protocols
Only one path in and out of
manufacturing zone—the firewalls

Demilitarized Zone (DMZ)

Manufacturing Zone
Site Manufacturing
Operations and Control
Level 3Cell/Area Zones
Levels 0–2

Technical Track© 2012 ODVA, Inc.

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

page 39
www.odva.org

Secure Remote Access Remote engineer or partner
establishes VPN to corporate
network; access is restricted to
IP address of plant DMZ
firewallIntrusion protection system (IPS)on plant firewall detects andprotects against attacks fromremote host

 Firewall proxies a client session
to remoteaccess server Access to applications on
remote access server is restrictedto specified plant floor IACS
resources through IACS
application security
Technical Track© 2012 ODVA, Inc.

Internet

EnterpriseConnected
EngineerEnterprise
WAN

Patch Management
Terminal Services
Application Mirror
AV ServerEnterprise Zone
Levels 4 and 5

Enterprise Edge
FirewallSSLVPNEnterprise
Data Center Portal on plant firewall enables
access to IACS data, files andapplications

Cisco
VPN
ClientIPSEC VPN

Remote Engineer
or Partner

Gbps Link
FailoverDetection

Enterprise Zone
Levels 4 and 5

HTTPS

Demilitarized Zone (DMZ)

Firewall(Standby)

Plant-Wide Servers
 Control Center View
 Historian Asset Mang.
 MESServices Platform• Directory• Security/Audit
Data Servers

2012 ODVA Industry Conference & 15th Annual Meeting
All rights reserved.

Firewall(Active)Core
SwitchRemote Desktop
Protocol (RDP)

Remote Access Server
 Engineering Application
Aggregation
SwitchEtherNet/IP

Manufacturing Zone
Site Manufacturing
Operations and Control
Level 3Cell/Area Zones

page 40
www.odva.org