TSN - Report on the IIC Testbed
Paul Didier, Cisco Systems, Inc.
George Ditzel, Schneider Electric, Inc.
Ludwig Leurs, Bosch Rexroth
Jordon Woods, Analog Devices, Inc.
Presented at the ODVA2017 Industry Conference & 18th Annual Meeting
February 21-23, 2017
Palm Harbor, Florida, USA
Abstract
This paper will give a report from the Industrial Internet Consortium’s (IIC) TSN Manufacturing testbed.
The goal of this test bed is to display the value of new Ethernet (IEEE 802) standards, referred to as
Time-Sensitive Networks, in a Manufacturing ecosystem of applications. Four ODVA members are
participating in the testbed: Bosch Rexroth, Cisco, Innovasic (recently acquired by Analog Devices, Inc.)and Schneider Electric.
The testbed will display the following: Combine different critical (e.g. OPC Pub/Sub and ODVA CIP) and best-effort traffic flows on asingle network based on IEEE 802.1 Time Sensitive Networking (TSN) Demonstrate the real-time capability and vendor interoperability using standard, converged
Ethernet
 Evaluate security value of TSN and provide feedback on the secure-ability of initial TSN functions
 Show ability for IIoT to incorporate high performance and latency sensitive applications
 Provide integration points for smart edge-cloud control systems into IIoT infrastructure &application
By the time of the report, the testbed will have conducted at least two plugfests where a variety ofvendors have demonstrated varying levels of integration with TSN-based networking technology. This
paper will report on the applications used, the level of integration and readiness of TSN based standard
technology and report some of the value of TSN to the ODVA community.
Keywords
Industrial Ethernet, Industrial Control, Industrial Internet Consortium, IIC, Motion Control, EtherNet/IP,Time-Sensitive Networking, TSN, software-defined networking, SDN, deterministic Ethernet.

Definition of terms (optional)
Bridge: A system that includes Media Access Control (MAC) Bridge or Virtual Local Area Network (VLAN)
Bridge component functionality and that supports a claim of conformance to Clause 5 of IEEE Std
802.1Q-2014 for system behavior. Note: “Bridge” is often used interchangeably with “switch” in TSN
discussions.

2017 ODVA Industry Conference1©2017 ODVA, Inc.

Centralized Network Configuration (CNC): A centralized component that configures network resources onbehalf of TSN applications (users).
Centralized User Configuration (CUC): A centralized component that discovers and configures application(user) resources in end stations. The CUC exchanges information with the CNC to configure TSN
features on behalf of its end stations.
End station: A device attached to a local area network (LAN) or metropolitan area network (MAN), whichacts as a source of, and/or destination for, traffic carried on the LAN or MAN.
Grandmaster: The time-aware system that contains the best clock, as determined by the best masterclock algorithm (BMCA), in the generalized precision time protocol (gPTP) domain.
Listener: The end station that is the destination, receiver, or consumer of a stream.
Stream: A unidirectional flow of data from a Talker to one or more Listeners.
Synchronized time: The synchronized time of an event is the time of that event relative to thegrandmaster.
Synchronized time-aware systems: Two time-aware systems are synchronized to a specified precision if
they have the same epoch and their measurements of the time of a single event at an arbitrary time differ
by no more than a specified amount of time.
Talker: The end station that is the source or producer of a stream.
Time Sensitive Networking – Flexible Manufacturing Testbed
Time Sensitive Networking is an initiative to enhance Ethernet communication, a foundational piece of theInternet of Things (IoT). The IIC’s TSN Testbed endeavors to apply new TSN technology in amanufacturing system with a wide range of automation and control vendors in order to display these newcapabilities and the value of TSN. Testbed participants consist of manufacturers of equipment intended
for the automation and control systems that make up a large part of the 50 billion things that comprise theIoT. These applications generally require that the pieces, parts and the overall system be very
deterministic in nature.
This deterministic performance is mandatory for many control applications in factory automation as wellas manufacturing for oil and gas, utilities and transportation. TSN is an enhancement of Ethernet(standards IEEE 802.1 and 802.3 comprise what is generally considered Ethernet) to bring moredeterministic capabilities to the network. It is enabling more and more devices, applications and systems
to use the standard, open, inter-connected network that is the basic concept and driver of the IoT. Without
these enhancements manufacturers of industrial control systems have to use specialized non-IEEE
conformant network technology in their products and therefore, they cannot participate in future
developments of the IoT. Thus, TSN is viewed as a very important enhancement and upgrade to standard
networks in order to enable the overall Industrial IoT.
The goals of the TSN Testbed are to:1) Show TSN’s readiness to accelerate the marketplace and2) Show the business value of accelerating the adoption of this technology.

2017 ODVA Industry Conference2©2017 ODVA, Inc.

Time Sensitive Networking – A Brief Overview of TSN
TSN brings a number of enhancements to Ethernet. Testbed participants have chosen to focus initialefforts on three key TSN enhancements. These particular enhancements are considered key to industrial
control systems and are also either published or are nearing publication. They include:
1) Time Synchronization – A distributed, precise sense of time is a base requirement for distributed
automation and control. It is also a requirement for the synchronized acquisition of sensor signals
and activation of output signals needed between the sensors, actuators, controllers that comprisethe manufacturing systems as well as the network infrastructure.
2) Sending scheduled Traffic flows – Based on the precise sense of time, the network infrastructure
must be capable of forwarding critical automation and control traffic on a timely basis. This isconsidered an enhancement to the Quality of Service capabilities in most network infrastructure.
3) Central, automated system configuration – Recently, a key trend in networking has been
Software-Defined Networking which enables automated, easier configuration. Software-DefinedNetworking is particularly important for TSN in Manufacturing because it consolidates application
requirements, develops paths and schedules for the traffic flows and distributes that to therelevant network infrastructure.

Other TSN enhancements, such as Frame Pre-emption, Ingress Policing and more, will be prioritized andincorporated into the testbed when complete.
Testbed Objectives
This testbed is designed to grow and achieve objectives in a progression. The three key capabilities listed
above represent the initial technical milestones of the testbed. In initial plugfests, the focus was onestablishing time synchronization. Once this was accomplished, devices began to send scheduled traffic
over the network. Upcoming plugfests will focus on the centralized, automated configuration aspects ofTSN and on application layer standards for interoperable real time control. At that point will comprise acomplete TSN ecosystem, albeit with a subset of the TSN standards, and will be able to demonstrate thevalue proposition of TSN. Participation is open to all IIC members and offers an unparalleled opportunity
to explore, understand and apply the TSN standards to real-world applications.
In addition to technical objective, the TSN Testbed is interfacing with a range of tools and applications
within the industrial and networking spaces. These tools include programming and configuration tools, aswell as the actual operational input and output devices typically found in these environments. The testbed
is attempting to emulate the entire process of designing a plant as well as operations of that plant based
upon these new capabilities.
The TSN Testbed also provides a unique opportunity to verify TSN functionality and interoperability
thanks to the participation of its twelve member companies representing an effective cross-section of2017 ODVA Industry Conference

3©2017 ODVA, Inc.

functionality. Most of the TSN Testbed participants are from the automation and control markets, butthere are also a number of networking technology companies. These organizations have joined together,
via the testbed, to address the cultural and technical divide between Information Technology andOperational Technology. IIC members participating in the TSN Testbed at the time of this paper are
Belden/Hirschmann, Bosch Rexroth, B&R Industrial Automation, Cisco, Analog Devices, Inc. (ADI), Intel,KUKA, National Instruments, Schneider Electric, SICK AG, TTTech and Xilinx.

Figure 1 - Current TSN Testbed Topology

Testbed Status
The TSN Testbed partners are currently focused on the technical aspects of TSN communications. They
have brought in vendors with infrastructure, devices, switches and test equipment, to conduct rigoroustesting. They have also developed successful demonstrations. Significantly, much of the effort has beenspent ensuring that various parts of the TSN standards interoperate. Applications-layer considerations will
be included in the testbed as soon as the technical interoperability is ensured.
Testbed Outcomes
The testbed team decided from the beginning that the testbed was to more than a simple “proof ofconcept”. The goal is to develop a TSN-based ecosystem in which new applications and TSN features
can be validated. A fixed location was established at National Instruments – the NI Industrial IoT Lab –which provided the team with a permanent home for their collaboration. The Lab is available for anyone totest their product or technology and thus determine how these products work within a TSN ecosystem.
Companies may also validate their progress in larger groups via plugfests.

2017 ODVA Industry Conference

4©2017 ODVA, Inc.

As a result of these plugfests, all participants gain a valuable sense of how their particular
implementations work with these new technologies; what challenges they may encounter, where theyneed to focus development and where they may have misunderstood or require further clarification fromthe standards and technology elements. All participants – the network infrastructure vendors, the enddevice vendors, even the testing tool vendors – have learned what it means to implement TSN, wherethey can enhance, improve or update their product to ensure that they and eventually their customers
gain value. These experiences are not just valuable for the participants, but are being fed back to thevarious standards groups working on TSN.
Two plugfests have occurred to date, offering participants opportunities to test, for example:
 Synchronizing devices to a common, precise sense of time delivered over the network
 Establishing TSN flows between various vendors
 Defining TSN flows in Central Network Controllers (CNCs) and distribute schedules to network
infrastructure Communicating input/output traffic via TSN flows
 Demonstrating TSN’s ability to protect critical flows from high-bandwidth traffic Connecting via gateway non-TSN traffic into TSN flows
 TSN flow requirements
 CNC to network infrastructure (schedule distribution)
 Data consistency over OPC UA Pub-Sub over TSN
The first TSN Testbed plugfest occurred on June 20 – 23, 2016 in Austin, Texas at the NI Industrial IoTLab. Participants include: NI, Cisco, TTTech, GE, Schneider Electric, Kuka, Intel, ADI and Ixia. Depending
upon participant readiness, objectives for the plugfest included:
1. Establish end-device synchronization via 802.1AS network based time services
2. Define TSN flows in Central Network Controller and distribute schedule to network infrastructure
3. Communicate I/O traffic via TSN flows
4. Measure and verify TSN performance with available testing tools
All participants with end-devices achieved synchronization and a subset of participants achieved
all 4 objectives.
On Oct. 3-5 2016, a second TSN plugfest was held in Austin TX at NI Headquarters.
Participants include: NI, Cisco, TTTech, Belden Hirschmann, BoschRexroth, B&R, Schneider Electric,Intel, ADI and Ixia. Depending upon participant readiness, objectives for the plugfest included:
1. Establish end-device synchronization via 802.1AS network based time services
2. Communicate I/O traffic via TSN flows
3. TSN Gateway traffic from one vendor
4. Measure and verify TSN performance with Ixia testing tools
5. Built a demonstration displayed at IOT SWC (Oct. Barcelona) and SPS Drives (Nov. Nuremberg)
All of these objectives were achieved and successful demonstrations of the testbed were held at
the IOT SWC and SPS drives.

2017 ODVA Industry Conference

5©2017 ODVA, Inc.

Figure 2 - From Plugfest to Demonstration

Influencing Standards
It is one thing to build a standard and it is another thing to actually implement and use it. The TSN
Testbed is accelerating this process by early phase usage of the TSN enhancements to the Ethernet
standards. Specifically, the testbed has been influencing two Ethernet standards: IEEE 802.1 and IEEE
802.3. All TSN Testbed work is filtered to Avnu, a standards organization developing the interoperabilityand certification based on IEEE’s TSN. Avnu is helping to define measurement conditions and set-up forfuture conformance testing. Through Avnu, it is channeled back into IEEE [Institute of Electrical andElectronics Engineers].
Many IIC members participate in both organizations. Additionally, both have formal liaison relationships
with the IIC, easing information flow. The IIC maintains active liaison relationships with standards
organizations, open-source organizations, other consortia and alliances (either technology-focused orindustry-focused), certification and testing bodies and government entities or agencies. The purpose of
these relationships is to generate requirements for new standards from the activities taking place within
the IIC. These relationships help eliminate duplication of effort and ensure that new standards andtechnologies necessary to build and enable the Industrial Internet are brought to market more rapidly.

2017 ODVA Industry Conference

6©2017 ODVA, Inc.

SDN-inspired Network Configuration
While the participants in the TSN testbed have managed significant progress in a very short time, theirwork is far from complete. As previously noted, the TSN testbed will evolve to incorporate new use cases,
new applications and interoperation of protocols. In the near future, centralized configuration of the timesensitive features will be achieved, making it possible to coordinate motion control, I/O, controller-tocontroller, and machine-to-machine communications seamlessly across the same network backbone.
Further, data from any or all of these applications can be made seamlessly available to the user via thenetwork or through the cloud.
To manage traffic schedules and paths (including redundant paths) for data and time-sync, a centralized
configuration approach is beneficial. IEEE P802.1Qcc (Stream Reservation Protocol (SRP)
Enhancements & Performance Improvements) enhances the existing Stream Reservation Protocol withthe addition of a User Network Interface (UNI) which allows for a centralized network configuration (CNC)
entity.

Figure 3 - SDN-inspired Network Configuration

A Centralized User Configuration (CUC) entity communicates to the CNC via a standard API. The CUC
may be used to discover end stations, retrieve end station capabilities and user requirements, andconfigure TSN features in end stations.

Figure 4 - CNC - Northbound2017 ODVA Industry Conference7©2017 ODVA, Inc.

This CNC can then provide a centralized means for performing network calculus, scheduling and otherconfiguration via a remote management protocol such as NETCONF or RESTCONF.

Figure 5 - CNC - SouthboundConclusions
While progress to date is far from this lofty vision, these are the first vital steps to making that vision areality. We invite all those interested in shaping that vision to join our efforts to achieve a truly convergednetwork.

Figure 6 - TSN Testbed of the Future2017 ODVA Industry Conference8©2017 ODVA, Inc.

References

StandardDescription

IEEE 1588-2008

IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement
and Control Systems
https://standards.ieee.org/findstds/standard/802.1AS-2011.html

IEEE 802.1AS-2011

Timing and Synchronization for Time-Sensitive Applications in Bridged Local Area
Networks
http://standards.ieee.org/getieee802/download/802.1AS-2011.pdf

IEEE 802.1AS-Rev

Timing and Synchronization for Time-Sensitive Applications in Bridged Local Area
Networks (draft)
http://www.ieee802.org/1/pages/802.1AS-rev.html

IEEE 802.1Q-2014,
Clause 34

Forwarding and Queuing Enhancements for Time-Sensitive Streams (FQTSS)
https://standards.ieee.org/findstds/standard/802.1Q-2014.html

IEEE 802.1Qbv2015

IEEE Standard for Local and Metropolitan Area Networks -- Bridges and Bridged Networks- Amendment 25: Enhancements for Scheduled Traffic
https://standards.ieee.org/findstds/standard/802.1Qbv-2015.html

IEEE 802.1Q-2014,
clause 35

Stream Reservation Protocol (SRP)
https://standards.ieee.org/findstds/standard/802.1Q-2014.html

IEEE 802.1Qcc

Stream Reservation Protocol (SRP) Enhancements and Performance Improvements (draft)
http://www.ieee802.org/1/pages/802.1cc.html

IEEE 802.1Qci

Per-Stream Filtering and Policing
http://www.ieee802.org/1/pages/802.1ci.html

IEEE 802.1Qcp

Bridges and Bridged Networks Amendment: YANG Data Model (draft)
http://www.ieee802.org/1/pages/802.1cp.html

IETF RFC 6020

YANG – A Data Modeling Language for the Network Configuration Protocol (NETCONF)
https://tools.ietf.org/html/rfc6020

IEEE 802.1AB

IEEE Standard for Local and Metropolitan Area Networks - Station and Media AccessControl Connectivity Discovery
https://standards.ieee.org/findstds/standard/802.1AB-2016.html

IETF DetNet

Deterministic Networking (DetNet) (draft)
https://datatracker.ietf.org/wg/detnet/documents/******************************************************************************************************************************************************The ideas, opinions, and recommendations expressed herein are intended to describe concepts of the author(s) for the possible useof ODVA technologies and do not reflect the ideas, opinions, and recommendation of ODVA per se. Because ODVA technologies
may be applied in many diverse situations and in conjunction with products and systems from multiple vendors, the reader andthose responsible for specifying ODVA networks must determine for themselves the suitability and the suitability of ideas, opinions,and recommendations expressed herein for intended use. Copyright ©2017 ODVA, Inc. All rights reserved. For permission toreproduce excerpts of this material, with appropriate attribution to the author(s), please contact ODVA on: TEL +1 734-975-8840
FAX +1 734-922-0027 EMAIL odva@odva.org WEB www.odva.org. CIP, Common Industrial Protocol, CIP Energy, CIP Motion, CIP
Safety, CIP Sync, CIP Security, CompoNet, ControlNet, DeviceNet, and EtherNet/IP are trademarks of ODVA, Inc. All othertrademarks are property of their respective owners.

2017 ODVA Industry Conference9©2017 ODVA, Inc.