Fundamentals of Precision Time Protocol
Rudy Klecka
Cisco Systems
October 14, 2015

Abstract

This session will provide a general background on IEEE 1588 Precision Time Protocol (PTP),
how it works, some basic terminology, and its main uses in the market. There will be adiscussion on PTP implementations (with a primary emphasis on Industrial products). The
session will also touch on other related timing protocols and future enhancements to PTP.

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

2ObjectivesConverse
About PTP
Basics…

Understand thedifferent Time
Protocols

Understandthe various
PTP Profiles
and Standards

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

Understandhow PTP relates
to Controls
Engineeringwww.odva.org

3

“Timing is Everything”
Motion Control

Scan-based to time-based control operation

Load sharing of robots

“Timing is Everything”
Line PLC

HMIRing

Slow Loops

DLR Ring

Fast Loops
DLR Ring

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

5Various Time Distribution Protocols
One Way Time Transfer
GPSTwo Way Time Transfer

GLONASSGNSSGalileo
BeiDouNTPIRIG

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

PTP/1588

www.odva.org

6

GNSS – Systems Around the World

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

7

Why (and Why Not) GNSS…
Examples: GPS, GLONASS, COMPASS/BeiDou-2, Galileo, …

 Nearly globally
available✗ Reliability(weaksatellite signal)

 Traceable to UTC✗ Not available indoor(or urban canyons) Government
Requirements forBackup✗ Cost of Equipment

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

8One Way Time Transfer Basics

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

9One-Way Time Transfer (OWTT) Basics

Since my phone only has a cheap oscillator and I just powered it on –initially I have no concept of true time…
Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

10

Steps to GPS Time

1. Frequency Lock to Satellite 1
S1

RTo Frequency Lock – only t1 & t2
timestamps are needed

t1
Delayt2t1’

FFO = ((t2’- t2)-(t1’- t1))/(t1’- t1)
Continually adjusting your frequency (yellow) until it matches GPS (blue)
This gets your cheap Cell Phone oscillator locked to a stable GPS frequency…

t2’

This will allow you to more accurately measure time differences (up next)…
Also note: the Delay here is constant – speed of light through atmosphereTechnical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

11

Steps to GPS Time
S1RS2t1

S3S4

2. Find Time Differences of Arrival (TDOAs)

t1
t1

t2

t2
t1

TDOA S1 – S2 = t2 – t2
TDOA S1 – S3 = t2 – t2
TDOA S1 – S4 = t2 – t2t2These differences are Hyperboloids…
t2

Note: angles on the arrows are the same as the delays
through the atmosphere are the same

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

12

Multilateration – Overlapping Hyperboloids

It takes the intersection of 3of the Hyperboloids to narrow
it to a single point
Source: http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=5507349 & http://masters.donntu.org/2014/frt/gontsa/diss/indexe.htm

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

13

Two Way Time Transfer Basics

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

14

PTP v2 messages and transmission
A set of event messages providing
significant instances and consisting of:

A set of general messages consisting of:- Follow_Up- Sync- Delay_Resp- Delay_Req- Pdelay_Resp_Follow_Up- Pdelay_Req- Announce- Pdelay_Resp- Signaling- Management••••

Mappings: L2 Ethernet, IPv4, IPv6 (others possible)
Transmission modes: either unicast or multicast (can be mixed)
Variable rate and timeout values
Various TLVs and flexible TLV extensions

Basic Message Flow – PTP (TWTT)
Event Messages:
Sync
Del_Req
Pdel_Req
Pdel_Resp
General Messages:
Follow_Up
Announce
Del_Resp
Pdel_Resp_FU
ManagementSignaling

MasterSlavet1Sync
Follow-U p

De(t1)

el _R

t3qt4

Del _Re sp(t4t2)

How do we get there?

First – you Synchronize (or “Syntonize”)

1. Frequency Lock to GMC

MasterTo Frequency Lock – only t1 & t2
timestamps are needed

t1Slave

Sync
Follow-U p(t1)

FFO = ((t2’- t2)-(t1’- t1))/(t1’- t1)

t2

Think of a counter on each system.

From t1 to t1’ the Master uses its clock toCount the time…
From t2 to t2’ the Slave uses it clock toCount the time…

The difference in the number of counts
is how different the two clocks are.

t1'Sync
Follow-Up(t1')

t2'(Assuming the Delays are the same)

Fractional Frequency Offset (FFO) = diff in frequency from Master to SlaveNext – you Time (Phase) Align

2. Time AlignMasterSlave

t1

Sy
nc
Follow-Up(t1)

To Time Align you need t3 and t4 timestamps

Delay
1t2

Once you are Frequency Locked, you canCalculate the Delay and Offset from the Master

De

Delay
2l_q
Ret4

Del_ Re sp(t4)

t3

Calculating Delay
Delay = ((t2 – t1)+(t4 – t3))/2
Delay = (Delay1+Delay2)/2

MasterSlave

t1SyncFollow-Up(t1)Delay is really an Average
An assumption is made:Delay 1 = Delay 2
Assumption is usually wrong(to some degree)

Delay1t2

t1, t2

De

Delay2

l_q
Re

t3

t4

Protocol Mechanisms can correct
for known Asymmetry
Asymmetry cannot be detected

Del_ Re sp(t4)

t1, t2, t3, t4

Calculating Time Offset
Delay = ((t2 – t1)+(t4 – t3))/2
Delay = (Delay1+Delay2)/2
Offset = ((t2 – t1)-(t4 – t3))/2
Offset = (Delay1–Delay2)/2
orOffset = Master Time – Slave Time – Delay

In other words:
If you are Frequency Locked and you assume
Delay1 = Delay2, then any difference is due toerror in Time.

MasterSlave

t1

Sy
nc
Follow-Up(t1)

Delay
1
t2

t1, t2

De

Delay2el _R

qt3

t4

Del_ Re sp(t4)t1, t2, t3, t4

Visualizing Time Error
Time Error

Fast

Slow

Averages to ZeroTimeGMCOCVisualizing Time Error
Time Error

Fast

OffsetSlow

Averages HighTimeGMCOCReal World Problems

Assumptions we made:

MasterSlave

t1Sy
ncFollow-Up(t1)1. Symmetrical Master/Slave & Slave/Master
2. Same delay each time

In the Real World – both are not true
to varying degrees…

The Two Biggest Problems:

1. Asymmetry
2. Packet Delay Variation (PDV)

Delay
1t2

DeDelay
2

el _R

qt4

Del_ Re sp(t4)

t3

Asymmetry••

Forward and backward delays are not identical.
Every Node and Link in the network can introduce asymmetry.

MasterTechnical Track© 2015 ODVA, Inc.

Slave

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

25

Asymmetry: Within a NodeUplink
PHY

If the internal components of the System aresymmetrical, then the delays will be symmetrical.

SwitchASIC

However…

Switch PHY

Downlink

Asymmetry: Within a Node
TimeStamp
Reference
If different PHYs are used on different ports,
Even that slight difference can cause asymmetry

UplinkPHY
SwitchASIC

When trying to get to nanoseconds of Precision,
Every little bit counts…

And where you time stamp matters.
TimeStamp
Reference
Switch PHY

Downlink

Asymmetry: Transmission
MasterTypically, transmission delays are fairly symmetrical:
Bidirectional over same pairs, small differences in pair lengths
However, some things can make it more likely:
EtherChannel, Fiber (especially rings; East v. West)

Nanoseconds matter => Meters matterSlavePacket Delay Variation (PDV)

PDV is primarily due to Varying Queue Delays…
Packet NetworkMaster

Hop 1

Hop 2

Hop 3

Hop 4

Hop n

Slave

PTP Session

Even High Priority packets get behind a 1518 from time to time.
The variance comes from the fact that sometimes you do and sometimes you don’t.

It’s all statistics…

SlaveServoTimeMasterBoundary Clocks andTransparent Clocks

ServoMonitor

Freq
Only

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

30

Boundary Clock v. Transparent Clock – Error Sources
Boundary ClockTransparent Clock
SwitchTimeStamp
Reference

SwitchTimeStamp
Reference

PHY

PHY

Time & Message
Transfer

Time & Message
Transfer

Slave

Servo

Oscillator

TimeMonitor

Servo

Freq
Only

MasterOscillator

TimeStamp
Reference

TimeStamp
Reference

PHY

PHY

Pros and Cons of Boundary Clocks
Pros:
•••••

Breaks up the PTP message domain
Breaks up the PTP timing domain
Spans across VLANs
Shields Slaves from Transients due to hierarchy changes (BMCA)
Filters PDV

Message Transfer

Time Transfer
Cons:•

Adds low frequency (wander) time error (hard to filter)

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

32

Pros and Cons of Transparent Clocks
Pros:
••

Maintains tight timing throughout a domain
Peer-to-Peer TCs can converge faster after network topology changes

Message Transfer

Cons:•Time Transfer

End-to-End TCs can have scalability issues

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

33

Delay and pdelay Mechanisms
Two Types of TCs: End-to-End and Peer-to-Peer

GMS

E2E TC

BC

BC
M

M

S
P

Non-PTP
SYNC

SM

M

Delay_Resp
Delay_Req

Delay
E2E TC

GM

BC

BC

S

S

M
M

PM

Brownfield
P2P TC

BC
S

SYNC
M

S

M

pdelay_Resppdelay_Req

P2P TCTechnical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

34

Peer to Peer Transparent ClockMASTER
Master time = TMt1P2PTransparent
Clock

SLAVE
Slave time = TS= TM + offset

SYNC

Offset = ((t2 – t1)–(t4 – t3))/2
Residency Time (rt)pDelay_Request

Offset = (t2 – t1)– mPD2– correctionField

SYNC

pt1

t2
pt2
pt3

pDelay_Request
pDelay_Response

correctionField = mPD1 + rt

pt2
pt4 pt3pDelay_Response

mPD1= ((pt2 – pt1)+(pt4 – pt3))/2
Technical Track© 2015 ODVA, Inc.

pt1

mPD2= ((pt2 – pt1)+(pt4 – pt3))/2
2015 Industry Conference & 17th Annual Meetingwww.odva.org
All rights reserved.

pt4

mPD : meanPathDelay

35Environmental Aspects

Thermal Changes AffectBase Oscillators

PTP Profiles(And the Proliferation of said…)

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

37

Main Current Industrial Profiles
IEEE 1588-2008
Default (CIP Sync)

IEEE 802.1AS

IEEE C37.238

ITU-T G.8275.1

Segments

Industrial Solutions

AVB(residential)
TSN

Power Industry(SmartGrid substation)

Telecom Mobile
BackhaulSubstation Backup

Transport

IP, L2 Ethernet,
industrial specifics

L2 Ethernet

L2 Ethernet

L2 Ethernet

Transmission

Multicast (default)

Multicastnon-forwardable

Multicastnon-forwardable

Multicast(both address types)

Delay mechanism

Delay (Annex J.3)
Pdelay (Annex J.4)

pdelaypdelaydelay

Clock mode

One- & two-stepTwo-stepTwo-step

Any

BMCA

Default

Alternate

Default

Alternate

TLV Extensions

Optional

Yes

Yes

No

ClocksOC, BC, TC

time-aware bridge andend station

OC, TC(BC in future revision)

T-GM, T-BC, T-TSC

Deployment modelNot defined

Full support

Full support

Full Support+ PHY layer freq.

Performance Specifications
IEEE 1588-2008
Default (CIP Sync)

IEEE 802.1AS-2011

IEEE C37.238-2011

Network limitsNo

7-hop network:• time accuracy• jitter and wander

16-hop (TC) network

ClocksNoTC timeInaccuracy limit+-50 nsecs

Grandmaster

Frequency accuracy

LocalClock:• frequency accuracy• time granularity• noise generation
TimeAware systems:• residence time
• pdelay turnaround time
• Error in rate ratio (orfrequency offset)
measurement

GrandmastertimeInaccuracy limit

White Rabbit

Sub-nanosecond synchronization!
Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

40Real World ImpactPTP Errors Translated to Machine Error

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

41

Using Multiple Unmanaged Switches in a Large System…

42

Simulated Network Load – ITU Telecom Profiles
ITU-T G.8261 – Timing & Sync Aspects in PSNs
•Appendix VI.5 – Test for Two Way Protocols•
Baseline Test (no Network  Master/Slave back to back)•Performance Tests (Network & Load)
Test Case
12Description
Static Packet Load

Network Load
80%
20%

1h80%
1350%

Sudden large and persistent Load Changes

20%

10%14Slow Load Change over extremely long Time
20%

15

Temporary Network Outage

16

Temporary Congestion

17

Routing Changes caused by failures

Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

3h80%
55%

6h12h24h43

www.odva.org

43

Simulated Network Load – ITU Telecom Profiles
Background Traffic

Background Traffic

ITU G.8265.3 TC 13 TM 1

GMC

PDV

Recovered Clock6000.00

Mean = 64 usecs
Minimum = 57 usecs
Maximum = 105 usecs
Std. Dev. σ = 6 usecs
Sample Count = 20000

5000.00

Count

4000.00
3000.00
2000.00
1000.00
0.00
0 To 0.058

0.06 To 0.062

0.064 To 0.066

0.068 To 0.07

0.072 To 0.074 0.076 To 0.078

0.08 To 0.082

0.084 To 0.086

0.088 To 0.09

20 msec/bin
Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

44

Simulated Network Load – ITU Telecom Profiles
Background Traffic

Background Traffic

ITU G.8265.3 TC 13 TM 1

GMC

Recovered ClockQoS Only120.00Mean = -1.6 μsecs
Minimum = -35 usecs
Maximum = 11 usecs
Std. Dev. σ = 7.4 μsecsSample Count = 900

100.00Count80.00
60.00
40.00

20.00
0.00
-35 To -28

Technical Track© 2015 ODVA, Inc.

-24 To -22-18 To -16-12 To -10

1 μsec/bin

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

-6 To -4

0 To 2www.odva.org

6 To 845

Simulated Network Load – ITU Telecom Profiles
Background Traffic

Background Traffic

ITU G.8265.3 TC 13 TM 1

GMC

QoS + PTP

Recovered Clock120.00Mean = 43 nsecs
Minimum = -54 nsecs
Maximum =154 nsecs
Std. Dev. σ = 35 nsecs
Sample Count = 900

Count100.00
80.0060.00
40.00
20.00
0.00
-53 To -40-30 To -20-10 To 0

10 To 2030 To 40

50 To 60

70 To 8090 To 100

110 To 120

130 To 140

150 and over

10 nsec/bin
Technical Track© 2015 ODVA, Inc.

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

www.odva.org

46

Simulated Network Load – ITU Telecom Profiles
Background Traffic

Background Traffic

ITU G.8265.3 TC 13 TM 1

GMC

With and Without PTP – Same Bin Scale

Recovered Clock800.00Mean = -1.6 μsecs
Std. Dev. σ = 7.4 μsecs
Sample Count = 900

700.00Count600.00
500.00200X Difference w/ PTP!

400.00Mean = 43 nsecs
Std. Dev. σ = 35 nsecs
Sample Count = 900

300.00200.00100.000.00-28 To -26

Technical Track© 2015 ODVA, Inc.

-24 To -22-20 To -18-16 To -14-12 To -10-8 To -6

-4 To -2

1 μsec/bin

2015 Industry Conference & 17th Annual Meeting
All rights reserved.

0 To 2www.odva.org

4 To 68 To 1047

So what do these numbers really mean?
16 Axis Star, Linear K6500, Stratix 8000 Switch

16 Axis Star, K350, Stratix 2000 Switch

Multiply your application
speed by this value todetermine position error
due to network jitter

Average System Clock Jitter (Max) ~ 35 nanoseconds

Average System Clock Jitter (Max) ~ 7.4 microseconds

0.000000035s x 6000 RPM/ 60s/min = 0.0000035 Revs

0.0000018s x 6000 RPM/ 60s/min = 0.00074Revs

Note: Sample from Axis 2

Note: Sample from Axis 1, off switch 148

THANK YOU