Developments inConformance Testing
ODVA Conformance Authority

Developments in Conformance Testing
Presenters– Lance Smith, ODVA

lsmith@odva.org– Hamza Choudhry, ODVA

hchoudhry@odva.org

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

2

Developments in Conformance Testing
Conformance Testing Process – Background– Conformance Testing Purposes• Provide vendor-independent Quality Assurance to industry – Product
Conformance to CIP Specification• Improve Customer Satisfaction with CIP technologies – Ensure
Interoperability between products• Drive Growth of ODVA – Increase the value proposition for ODVA
membershipTechnical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

3

Developments in Conformance Testing
Conformance Testing Process – Background– ODVA Terms of Use Agreement (TOU) Requirements• Vendors must obtain and maintain Declaration of Conformity (DOC) for theirdevices• The Conformance Test process outcome is a DOC granted to a product

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

4

Developments in Conformance Testing
Conformance Testing Process – Vendor development cycle– Plan ahead for your product conformance test
• Follow the current CIP Specification to guide your development• Order/maintain the protocol test software subscription and use it on aregular basis while developing your product• Check the planned contents for the next CT to see what’s up andcoming.

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

5

Developments in Conformance Testing
Conformance Testing Process – Vendor development cycle– Plan ahead for your product conformance test
• Integrate the conformance testing process into your productdevelopment process• Ann Arbor test lab is available on an hourly fee basis for pre-testing
devices if neededTechnical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

6

Developments in Conformance Testing
Conformance Testing Process – The process

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

7

Developments in Conformance Testing
Conformance Testware – (Non-Safety), Current Releases:– EtherNet/IP Conformance Composite Test CT12– DeviceNet Conformance Composite Test CT26

– EZ-EDS 3.11.1.20151008
Find current CCT info at:www.odva.org

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

8

Developments in Conformance Testing
EtherNet/IP CT13 Testware (November 2015)– New features• Originator Connection List Object Test
• Target Connection List Object Test
• Connection Configuration Object Test
• Position Sensor Object Test

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

9Developments in Conformance Testing
EtherNet/IP CT13 Testware (November 2015)– New features• Ethernet Link Object revision 4 and 1 Gbps support• Energy Objects revision 2 and ODOMETER data type support• CIP I/O connection timeout reconnect test and I/O injection test
(Sequence Number Gap Test)• Port Object Revision 2 Routing Capability Test

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

10

Developments in Conformance Testing
EtherNet/IP CT13 Testware (November 2015)– Improvements and Bug Fixes• TCP/IP Interface Object revision 4 support required• Encapsulation Inactivity Timeout update• ForwardClose IP Check required• Connection ID Reuse Check required

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

11

Developments in Conformance Testing

EtherNet/IP CT13 Testware (November 2015)– Improvements and Bug Fixes• Large_Forward_Open support for Class 1/ Class 3 Connection• Type 2 Reset Test for EtherNet/IP devices• RSTP Object available in any Device Profile for EtherNet/IP devices

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

12

Developments in Conformance TestingEtherNet/IP CT13 Testware (November 2015)– Bug Fixes for Existing Objects:• Assembly• File• PortTechnical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

13

Developments in Conformance Testing

EtherNet/IP CT13 Manual Test Changes (November 2015)– DLR Object Test update:a. Verify DUT does not learn MAC address of the active ring supervisorb. DLR Link Speed 10 Mb and Half Duplex are now optionalc. Neighbor_Status Checkd. Unicast and Multicast I/O passing check

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

14

Developments in Conformance Testing

EtherNet/IP CT13 Manual Test Changes (November 2015)– ACD Direct ARP Test – in Automated Tool (ACDTEST.exe)– IGMP support only required if implementing O->T multicast (NMAP)– MS and NS LED indicator optional for Industrial Performance Level

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

15

Developments in Conformance Testing

DeviceNet CT27 Testware (November 2015)– New features• Position Sensor Object Test• Energy Objects: revision 2 and ODOMETER data type support• Port Object Revision 2 Routing Capability Test
• (Note: Vol 3 has Not changed since 2013)

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

16

Developments in Conformance Testing
DeviceNet CT27 Testware (November 2015)– Bug Fixes in Objects:• Assembly• File• PortTechnical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

17

Developments in Conformance Testing
CIP Safety Conformance Testing– Implements Safety Test Plan (Volume 5 Appendix F)– Includes all relevant sections of Standard Conformance Test

• A standard test order IS needed for non-safety product variants• A standard test order is NOT needed for safety productsTechnical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

18

Developments in Conformance Testing
CIP Safety Conformance Testing– Safety Protocol Test
• CIP object extensions for safety (Vol 5 chapters 5 & 6)
– Safety-specific profiles and objects– Impact to existing objects SAFETY IMPACTS SOME OBJECTS• “Black Box” tests are automated (Vol 5 F-3)
• “White Box” tests must be performed by Vendor (Vol 5 F-4)

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

19

Developments in Conformance Testing
CIP Safety Conformance Testing– Refer to the Conformance Test User’s Guide – Appendix E• CIP Safety Adaptation for Conformance Testing

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

20

Developments in Conformance Testing
CIP Safety Conformance Software– Current Composite Test Levels• CIP Safety on EtherNet/IP CT8 ES (CT12 EN)• CIP Safety on DeviceNet: CT7 DS (CT26 DN)• CT1 CIP Safety on Sercos III

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

21

Developments in Conformance Testing
CIP Safety Conformance Software– Planned for 2015-PC2• CIP Safety on EtherNet/IP CT9 ES (CT13 EN)• CIP Safety on DeviceNet: CT8 DS (CT27 DN)• CT2 CIP Safety on Sercos III

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

22

Developments in Conformance TestingCIP Safety Conformance Software– Planned features• Improved test result auditing• Improved test coverage for non-SNCT DUTs
• Improved support and guidance for Originator DUTs

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

23

Developments in Conformance Testing
Conformance Testing Preparation – Common Errors

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

24

Developments in Conformance Testing
Conformance Testing Preparation – Common Errors– TCP/IP Interface Object Test
• DHCP: Following the correct DISCOVER/OFFER/REQUEST/ACK/ARPsequencing

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

25

Developments in Conformance Testing
Conformance Testing Preparation – Common Errors– TCP/IP Interface Object Test
• Report correct status or value during modeswitch for following attributes:
a. Attribute 2 (Configuration Capability)reflects correct DUT capability
b. Attribute 3 (Configuration Control) shows
current mode of DUT
c. Attribute 5 (Interface Configuration)
maintains proper values

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

26

Developments in Conformance Testing
Conformance Testing Preparation – Common Errors– ACD Test (ACD support is optional, but if implemented
ensure it conforms to the CIP specification)• Report correct conflict information in attribute 11(LastConflictDetected), especially AcdActivity andArpPdu• Report correct AcdStatus (bit 6) in attribute 1(Status) if DUT recovers from reconnecting thelink cable• MS/NS LED behave correctly to the conflict ifimplemented• Pay close attention to the Wireshark trace whilerunning Automatic ACD Test, make sure no extra
frames are sent.
• We run Automatic ACD Test under both fixed IP
configuration mode and dynamic IP configuration
mode.
Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

27

Developments in Conformance Testing

Conformance Testing Preparation – Common Errors- Ethernet Link Object Test
• Attribute 1 (Interface Speed) shallreport current speed in use. 0 is onlyused to indicate the speed isindeterminate.
• Attribute 2 (Interface Flags) shallreport correct status and configuration
about the physical interface.

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

28

Developments in Conformance Testing
Conformance Testing Preparation – Common Errors– Ethernet Link Object Test
• Attribute 6 (Interface Control) is a preferred CIP interface, if the DUT
supports auto-negotiation and forced speed/duplex.
• Attribute 9 (Admin State) shall not allow you to disable the last port.
• Attribute11 (Interface Capability) shall indicate the exact capability for theinterface.

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

29

Developments in Conformance Testing
Conformance Testing Preparation – Common Errors– DLR Object Test
• Auto MDIX support is required for both autonegotiate and force speed/duplex modes.
• 100 Mbs and Full Duplex required; 10/1000
Mbs, Half-Duplex optional.
• Attribute 6 (Interface Control) is a preferred
CIP interface since auto-negotiate and forced
speed/duplex are required for DLR device.
Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

30

Developments in Conformance Testing
Conformance Testing Preparation – Common Errors– DLR Object Test• No constant I/O packet loss
during normal ring operation• When DUT passes on a
Sign_On frame, the Source IP
Address of the frame should be
replaced with the DUT’s IPaddress.
Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

31

Developments in Conformance Testing
Conformance Testing Preparation – Common Errors– NV (Non-Volatile) Attributes Behavior• NV attribute values shall be persistent after a power cycle or a Type0 Reset.

• NV attribute values shall be restored to factory default values after a
Type 1 Reset.

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

32

Developments in Conformance Testing
Conformance Testing Preparation – Common Errors– NV (Non-Volatile) Attributes Behavior• Various NV attribute behavior after a Set_Attribute_Single request:
a. Some NV attributes take effect immediately after a Set, such as
PTPEnable in Time Sync Object, Admin State in Ethernet Link Object.
b. Some NV attribute changes have to take effect after a power cycle ora Type 0 Reset, such as SelectAcd attribute in TCP/IP Objectc. Some NV attributes can take effect immediately or after a reset,
there’s a pending bit to reflect such behavior, such as Interface
Configuration, TTL/Mcast attributes of TCP/IP Object; Interface
Control of Ethernet Link Object. If a reset is not needed, the DUT shallbehave under the new value immediately.

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

33

Developments in Conformance Testing
Conformance Testing Preparation – Common Errors– DeviceNet physical layer propagation delay exceeds 312 ns• Many new CAN transceivers and isolation devices are available since theoriginal introduction of DeviceNet.
• Newer doesn’t always mean better! Excessive propagation delay results ina sharp increase in the network CAN error rate.
• Before choosing a transceiver and isolator, verify that the combined
propagation delay does not exceed 120 ns for the CAN transmitter + isolator(isolator data in to CAN state out) and does not exceed 130 ns for the CANreceiver + isolator (CAN state in to isolator data out). Ref: Vol 3, Ch8, Tables
8-2.2 and 8-2.3
Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

34

Developments in Conformance Testing
Conformance Testing Preparation – Common Errors– DeviceNet physical layer propagation delay exceeds 312 ns• Confirm device performance using the bit sample time measurement described inthe test report.

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

35

Developments in Conformance Testing
Conformance Testing Preparation – Tips
– Access documents in the CT installation folder, see readme.htm

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

36

Developments in Conformance Testing
Conformance Testing Preparation – Tips
– Check the latest Sample Test Report Template to get familiar
with the expectation and test contents

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

37

Developments in Conformance Testing
Conformance Testing Preparation – Tips
– Self-testing with the latest CT release– Understand manual test procedures and test tools from theSample Test Report.
– If not sure about the errors according to the CIP specification,contact ODVA staff at conformance@odva.org

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

38

Developments in Conformance Testing

Questions?

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

39

THANK YOU