Patent Description:
Timing synchronization between devices may be important for performing financial transactions, transmitting and receiving telecommunication signals, coordinating transmission arrays, providing video over internet protocol (IP) services, and/or the like. To provide timing synchronization, one or more client devices may use precision timing protocol (PTP) to provide timing information for transport across a network.

<NPL>, provides distributed boundary clocks and distributed transparent clocks. <CIT> provides a method and device for forwarding a clock synchronization message.

According to some implementations, a method may include receiving, by a network device of a network, a timing control packet from a first client device; determining, by the network device, that the network device is in a synchronized state relative to a network grandmaster clock; modifying, by the network device, a first field of a header of the timing control packet to indicate that the network device is in a synchronized state; modifying, by the network device, a second field of the header of the timing control packet to indicate a time at which the network device received the timing control packet from the first client device; and forwarding, by the network device and via the network, the timing control packet toward a second client device.

According to some implementations, a method may include receiving, by an egress network device of a network, a timing control packet from a second network device via the network; identifying, by the egress network device and within a first field of a header of the timing control packet, a first indication that indicates that the ingress network device was in a synchronized state when the ingress network device received the timing control packet from a first client device; identifying, by the egress network device and within a second field of the header of the timing control packet, a second indication that indicates a time at which the ingress network device received the timing control packet from the first client device; determining, by the egress network device, that the egress network device is in a synchronized state; determining, by the egress network device, a time at which the egress network device will transmit the timing control packet; determining, by the egress network device, a resident time of the timing control packet on the network based on the second indication and the time at which the egress network device will transmit the timing control packet; modifying, by the egress network device, a third field of the header of the timing control packet to indicate the resident time of the timing control packet on the network; and forwarding, by the egress network device, the timing control packet toward a second client device.

According to some implementations, a system may include an ingress network device of a network to: receive a timing control packet from a first client device; determine that the ingress network device is in a synchronized state; modify a header of the timing control packet to include: a first indication that the ingress network device is in a synchronized state; and a second indication that indicates a time at which the ingress network device received the timing control packet from the first client device; and forward the timing control packet toward an egress network device via the network; and the egress network device of the network to: receive the timing control packet via the network; identify, within the header of the timing control packet, the first indication and the second indication; determine that the egress network device is in a synchronized state; determine a resident time of the timing control packet on the network based on the second indication and a time at which the egress network device will transmit the timing control packet; modify the header of the timing control packet to indicate the resident time of the timing control packet on the network; and forward the timing control packet toward a second client device.

Precise timing synchronization among devices may be important in many types of communications. For example, high-frequency trading operations (e.g., for trading on one or more stock markets), services provided by telecommunication networks (e.g., <NUM>-based or <NUM>-based telecommunication networks), services provided by professional broadcast television providers, and/or the like may rely on accurate timing synchronization among devices.

To provide precise timing synchronization among devices, PTP may be used. In PTP, one or more devices including grandmaster clock devices act as master timing sources to which other devices synchronize. A device, including a grandmaster clock device, receives timing information (e.g., via a global positioning system receiver, an atomic clock, and/or the like) and distributes the timing information to one or more connected devices via PTP control packets. The connected network devices may further distribute the timing information to additional devices (e.g., acting as a boundary clock or a transparent clock). For example, a transparent clock may perform corrections to the timing information that are based on amounts of time spent traversing network equipment.

Client devices that provide and/or consume timing-sensitive services may use network overlays (e.g., a cloud, multi-cloud overlay, a virtual private network, a multiprotocol label switching network, and/or the like) to communicate among the client devices (e.g., at different geo-locations). A network overlay is a virtual and/or logical network that is overlaid on a network underlay of network devices. The network underlay provides infrastructure upon which the network overlay is instantiated. The network underlay may support multiple network overlays in which client devices of the multiple network overlays are isolated (e.g., in network reachability, protocol, and/or the like) from each other and from the underlay network.

If a client device attempts to synchronize one or more other client devices across the overlay network (e.g., at different sites connected via the overlay network) by providing a timing control packet across the overlay network, the timing information may not include corrections based on amounts of time spent traversing network equipment (e.g., communication media, network devices, and/or the like) of the underlay network. This is because when a timing control packet is tunneled through one or more segments of the overlay network, the network devices in the underlay network may not parse the timing control packet to identify the timing control packet as including timing data that may need a correction. Even if networking devices are capable of parsing the timing control packet, computing and networking resources would be unnecessarily consumed at each hop of the network, and the transport service may incur unwanted delays to allow for the parsing.

The network may include one or more tunneling services, such as a multi-protocol label switching (MPLS) network, a virtual extensible local area network (VXLAN), generic network virtualization encapsulation, tunneling over IP version <NUM>, and/or the like. If the network provides a tunneling service, data and/or control packets to be tunneled through one or more segments of the overlay network may be encapsulated with an identifier to direct the data and/or control packets through the network. Intermediate network devices may not parse the data and/or control packets beyond a tunnel header, so the intermediate network devices do not identify the encapsulated data and/or control packets by type or content. Based on failing to identify a PTP control packet as including timing information that needs a correction, the intermediate network devices may not provide the correction to the PTP control packet, which may result in the timing information being incorrect based on an error from an amount of time spent traversing the network.

Other features of networks may cause further errors in timing information sent over the network. For example, a path from a first client device to a second client device may be different than a path from the second client device to the first client device. This means that an amount of time spent traversing one path cannot be accurately determined as half of a round trip path. Additionally, even if a path from the first client device to the second client device is the same as the path from the second client device to the first client device, lengths of transmit and receive media (e.g., fiber optic cables) may be different. Every meter of difference of fiber optic cables may cause a difference in propagation time of about <NUM> nanoseconds. Transmission and reception media between the same two network devices may differ by <NUM> to thousands of meters, which may cause enough delay asymmetry to cause problems with synchronization across the network. Further, even if the traversing paths are the same and/or the lengths of transmit and receive media are the same, the transmit and receive latency within optics modules of intermediate network devices may be different and may cause latency asymmetry between paths.

According to some implementations described herein, a first network device (e.g., an ingress network device, a provider edge device, and/or the like) receives a timing control packet from a first client device. The timing control packet may be a PTP control packet intended for a second client device accessible to the first client device via the network. The first network device may parse the timing control packet to determine that the timing control packet includes client timing information. The first network device may determine whether the first network device is in a synchronized state (e.g., relative to a network grandmaster clock device). If the first network device is in a synchronized state, the first network device may modify a header of the timing control packet (e.g., within a first field) to indicate that the first network device is in a synchronized state (a first indication). The network device may also modify the header of the timing control packet to indicate a time at which the first network device received the timing control packet from the first client device (a second indication). The first network device may also modify the header of the timing control packet (e.g., within the first field or a second field) to indicate one or more attributes (e.g., a location, a quantity of bits used, and/or the like) of the second indication (a third indication). The first network device may forward the timing control packet toward the second client device via the network. The second client device may be reachable through second network device.

A second network device (e.g., an egress network device, a provider edge device, and/or the like) may receive the timing control packet from the first client device via the network. The second network device may parse the timing control packet to determine that the timing control packet includes client timing information. The second network device may identify the first indication that indicates whether the first network device was in a synchronized state and the second indication that indicates a time at which the first network device received the timing control packet from the first client device (an "ingress time"). The second network device may determine whether the second network device is in a synchronized state (e.g., relative to the network grandmaster clock). If the first indication indicates that the first network device was in a synchronized state and the second network device determines that the second network device is in a synchronized state, the second network device may determine a resident time of the timing control packet on the network. The second network device may determine the resident time based on the second indication and the time at which the second network device will transmit the timing control packet toward the second client device. The second network device may modify the header of the timing control packet (e.g., in a correction field) to indicate the resident time before forwarding the timing control packet toward the second client device.

In this way, the first client device may provide timing information across the network to the second client device without, or with a reduction of, errors caused by time spent traversing network equipment of the network. This may allow client devices to perform operations that rely on accurate timing synchronization, such as telecommunication services, IP video broadcasting, and/or the like. This may also reduce errors in communications between client devices and may reduce computing and network resources that may be used to identify and/or correct the errors in the communications.

<FIG> are diagrams of one or more example implementations <NUM> described herein. As shown in <FIG>, the example implementation(s) <NUM> may include a network grandmaster clock device, an ingress network device, an egress network device, a client grandmaster device, a first client device, a second client device, and/or the like. The network grandmaster clock device, the ingress network device, the egress network device, the client grandmaster clock device, the first client device, the second client device, and/or other included devices may comprise hardware, firmware, or a combination of hardware and software and may include, for example, switches, routers, servers, security devices, and/or the like.

The ingress network device and/or the egress network device may be part of a network that includes a plurality of network devices. The network may include an underlay network that provides infrastructure to support one or more overlay networks. For example, the underlay network may provide a data transport service (e.g., an MPLS network, VXLAN, and/or the like) to transport data packets and/or control packets between the client device and other client devices (e.g., at other physical locations). The first client device and/or the second client device may have visibility to one or more prospective connections provided by the overlay network (e.g., with other client devices accessible via the overlay network). The underlay network and/or the one or more overlay networks may be transparent (e.g., not visible) to client devices. The overlay network may provide one or more prospective connections visible to the first client device and/or the second client device via a service type such as ethernet bridging, IP routing, integrating routing and bridging, and/or the like.

The first client device and/or the second client device may include one or more devices, which may be part of a local network of client devices at a physical location (e.g., an office building). In some implementations, the first client device and/or the second client device may be physical endpoints (e.g., a device assigned to a network address) or virtual endpoints (e.g., an agent of a device that is assigned to a network address). In some implementations, the underlay network is transparent to the first client device and/or the second client device. In some implementations, the overlay network may provide one or more prospective connections visible to the first client device and/or the second client device (e.g., prospective connections with other client devices accessible via the overlay network).

As shown in <FIG>, and by reference number <NUM>, the ingress network device may receive network timing information from the network grandmaster clock device. In some implementations, the ingress network device may receive the network timing information according to a precision time protocol (PTP) control packet from the network grandmaster clock device. The network may comprise one or more network grandmaster clock devices, which may be synchronized with each other. One or more additional network devices may relay the network timing information from the network grandmaster clock device to the ingress network device. The one or more additional network devices may operate as one or more transparent clocks that may perform corrections to the timing information that are based on amounts of time spent traversing network equipment between the network grandmaster clock device and the ingress network device.

As shown by reference number <NUM>, the ingress network device may update a clock of the ingress network device based on the network timing information recovered from the network grandmaster clock device. For example, the ingress network device may use the network timing information to synchronize the ingress network device with the network grandmaster clock. In some implementations, the timing information received from the network grandmaster clock device of the network is independent from timing information received from the first client device.

As shown by reference number <NUM>, the egress network device may also receive network timing information from the network grandmaster clock device. In some implementations, the egress network device and the ingress network device may receive the network timing information from the same network grandmaster clock device. In some implementations, the egress network device and the ingress network device may receive the network timing information from different network grandmaster clock devices of the network that are time synchronized.

As shown by reference number <NUM>, the egress network device may update a clock of the egress network device based on the network timing information. For example, the egress network device may use the network timing information recovered from the network grand master clock to synchronize the egress network device.

In some implementation, the ingress network device and/or the egress network device may receive additional network timing information. For example, the ingress network device and/or the egress network device may receive the additional network timing information periodically (e.g., with a regular update schedule), based on requests by the ingress network device and/or the egress network device (e.g., based on a failure to update a clock based on a previously received and/or scheduled update), based on a detected synchronization error in the network, and/or the like.

As shown in <FIG>, and by reference number <NUM>, the client grandmaster clock device may provide a timing control packet to the first client device. Timing information within the timing control packet may be independent from (e.g., in a separate domain from) the network timing information. In other words, the client grandmaster clock device may not be synchronized with the network grandmaster clock, may receiving timing information from the same or different sources, and may use the same or different timing profiles to provide the timing information to connected devices.

As shown by reference number <NUM>, the ingress network device may receive a timing control packet from the first client device. In some implementations, the timing control packet may comprise a PTP control packet. The timing control packet may indicate that the timing control packet is intended for the second client device by identifying a network address (e.g., and internet protocol (IP) address) as a destination address of the timing control packet.

In some implementations, the ingress network device may provide an interface between the first client device and the network. In other words, for the first client device to use the network as a service to transport (e.g., tunnel) the timing control packet to the second client device, the first client device provides the timing control packet to the ingress network device. The ingress network device may be configured to receive the timing control packet via one or more client-facing interfaces, prepare the timing control packet for transport over the network (e.g., by encapsulating the timing control packet, attaching a label, and/or the like), and forward the timing control packet via one or more network-facing interfaces.

As shown by reference number <NUM>, the ingress network device may parse the timing control packet to determine that the timing control packet is a timing control packet (e.g., including the client timing information for synchronizing the second client device with the first client device). For example, the ingress network device may inspect one or more elements of a header of the timing control packet to determine that the timing control packet is a timing control packet and/or that the timing control packet includes client timing information.

As shown by reference number <NUM>, the ingress network device may determine whether the ingress network device is in a synchronized state. In some implementations, the ingress network device may determine whether the ingress network device is in a synchronized state based on an amount of time that has passed since updating a clock based on network timing information received from the network grandmaster clock device. For example, the ingress network device may be in a synchronized state if an amount of time since a most recent update to the clock satisfies a threshold (e.g., less than a threshold amount of time). In some implementations, the ingress network device may determine that the ingress network device is in a synchronized state by testing the clock of the ingress network device against another clock in the network (e.g., the network grandmaster clock device). In some implementations, the ingress network device may determine that the ingress network device is in a synchronized state or is not in a synchronized state based on a communication from a network entity (e.g., a network controller, the network grandmaster clock device, and/or the like).

As shown by reference number <NUM>, the ingress network device may modify the header of the timing control packet to indicate whether the ingress network device is in a synchronized state. For example, the ingress network device may modify a header of the timing control packet received from the client device, to include an indication that the ingress network device is in a synchronized state. In some implementations, the ingress network device may modify a field of a header of the timing control packet (e.g., a reserved field of the header for a PTP-based timing control packet) to indicate that the network device is in a synchronized state.

As shown in <FIG>, and by reference number <NUM>, the ingress network device may modify the header to indicate a time at which the ingress network device received the timing control packet from the client device. For example, the ingress network device may modify another field of the header of the timing control packet to indicate the time (referred to in some instances as the "ingress time") at which the ingress network device received the timing control packet from the first client device or another device (e.g., a border clock device, a network device of another network along the path from the first client device to the second client device, and/or the like). In some implementations, the ingress network device may modify the header to indicate the ingress time in a field other than a correction field of the timing control packet (e.g., for a PTP-based timing control packet). This may prevent corruption of the existing correction field in cases where a resident time is not determined and/or indicated by the egress network device (e.g., if the egress network device is not in a synchronized state, if the egress network device fails to identify the timing control packet as a timing control packet, and/or the like). A corrupt correction field may cause an error by the second client device when attempting to use the timing control packet along with information stored in the correction field, to synchronize the second client device with the first client device.

As shown by reference number <NUM>, the ingress network device may modify the header to indicate attributes associated with the indication of the time at which the ingress network device received the timing control packet. In some implementations, the attributes include indications of a location and/or a quantity of bits used to indicate the ingress time. In some implementations, the indication of the attributes may be included in a same field as the indication that the ingress network device is in a synchronized state. For example, the indication of attributes may use a first bit and a second bit of a field and the indication of synchronization state may use a third bit and a fourth bit of the same field (e.g., a <NUM>-bit reserve field of the header (e.g., a PTP-based header)).

The indication of the attributes may indicate a field of the header that contains the indication of the ingress time (e.g., a reserve field, a correction field, and/or the like). In some implementations, the indication of the attributes may indicate a field of the header that includes <NUM> bits or <NUM> bits.

As shown by reference number <NUM>, the ingress network device may modify the header to identify the ingress network device as the ingress network device for the timing control packet. For example, the ingress network device may modify a field of the header to include an identifier of the ingress network device. The identifier may be unique to all network devices. The identifier may be used by another entity of the network to manage timing errors reported by ingress and/or egress network devices, to generate a heat map of the network (e.g., to manage loads and/or traffic), and/or the like.

As shown by reference number <NUM>, the ingress network device may forward the timing control packet toward the second client device and/or the egress network device via the network. Before forwarding the timing control packet, and after modifying the header, the ingress network device may prepare the timing control packet for transportation over the network (e.g., by encapsulating the timing control packet, attaching the label, and/or the like). The egress network device may receive the timing control packet via the network and prepare the timing control packet for forwarding toward the second client device (e.g., by popping a label, removing an encapsulation, and/or the like).

In some implementations, the egress network device receives the timing control packet from the ingress network device via the network as the network transports the timing control packet toward the second client device. The egress network device may provide an interface between the network and the second client device, similar to the interface provided by the ingress network device between the network and the first client device. The egress network device may receive the timing control packet and/or other packetized data from the network via one or more network-facing interfaces. When the egress network device receives the timing control packets and/or other packetized data, the egress network device may prepare the timing control packet for forwarding to the second client device (e.g., pop a label, remove an encapsulation, and/or the like).

As shown in <FIG>, and by reference number <NUM>, the egress network device may identify the one or more attributes associated with the indication of the time at which the ingress network device received the timing control packet. For example, the egress network device may inspect the header to identify the location and/or the quantity of bits associated with the indication of the ingress time. The egress network device may inspect the header after preparing the timing control packet for forwarding to the second client device by popping a label, removing an encapsulation, and/or the like.

As shown by reference number <NUM>, the egress network device may identify the indication that indicates the ingress network device was in a synchronized state when the ingress network device provided the indication of the time at which the ingress network device received the timing control packet. For example, the egress network device may identify the indication within a field of the header. In some implementations, as described above, the indication of the synchronized state of the ingress network device and the indication of the one or more attributes associated with the indication of the ingress time may be within the same field of the header.

As shown by reference number <NUM>, the egress network device may identify the indication indicating the time at which the ingress network device received the timing control packet from the first client device. For example, the egress network device may inspect a location (e.g., a field) of the header of the timing control packet to identify the indication indicating the ingress time. The egress network device may inspect the location based on a configuration of the egress network device to inspect the location, from the indication of the attributes associated with the identification of the ingress time, and/or the like.

As shown in <FIG>, and by reference number <NUM>, the egress network device may determine that the egress network device is in a synchronized state. In some implementations, the egress network device may determine whether the egress network device is in a synchronized state based on an amount of time that has passed since updating a clock based on network timing information received from the network grandmaster clock device. For example, the egress network device may be in a synchronized state if an amount of time since a most recent update to the clock satisfies a threshold (e.g., less than a threshold amount of time). In some implementations, the egress network device may determine that the egress network device is in a synchronized state by testing the clock of the egress network device against another clock in the network (e.g., the network grandmaster clock device). In some implementations, the egress network device may determine that the egress network device is in a synchronized state or is not in a synchronized state based on a communication from a network entity (e.g., a network controller, the network grandmaster clock device, and/or the like).

If the egress network device determines that the ingress network device was in a synchronized state when the ingress network device provided the indication of the ingress time and if the egress network device determines that the egress network device is in a synchronized state, the egress network device may use the ingress time to determine a resident time of the timing control packet.

If the egress network device determines that the ingress network device was not in a synchronized state when the ingress network device provided the indication of the ingress time, if the egress network device does not identify the indication of the ingress time, and/or if the egress network device determines that the egress network device is not in a synchronized state, the egress network device may not determine a resident time. Additionally, the egress network device may set one or more of the fields (e.g., the reserve fields and/or the correction field) to a null state (e.g., all zeros, all ones, and/or the like).

As shown by reference number <NUM>, the egress network device may determine a time at which the egress network device will transmit the timing control packet. For example, the egress network device may determine the time at which the egress network device transmits the timing control packet based on a time at which the timing control packet reaches a port for transmission toward the second client device, a time at which the timing control packet passes through a time stamping component (e.g., an application-specific integrated circuit (ASIC)) of the egress network device, a time at which the timing control packet reaches a client attachment interface, a time at which the egress network device expects the timing control packet to exit the egress network device, a time at which the egress network device expects the timing control packet to reach a port for transmission toward the second client device, a time at which the egress network device expects the timing control packet to reach a particular component of the egress network device, and/or the like.

As shown by reference number <NUM>, the egress network device may determine a resident time of the timing control packet within the network (e.g., terminate resident time). The egress network device may determine the resident time based on (e.g., based on a difference between) the ingress time and the time at which the egress network device will transmit the timing control packet.

As shown in <FIG>, and by reference number <NUM>, the egress network device may modify the header to indicate the resident time of the timing control packet on the network. In some implementations, the egress network device may modify a field of the header (e.g. a CF field of a PTP packet header).

In this way, the timing control packet includes information to account for time spent traversing network equipment of the network. Additionally, by using the ingress time and a time at which the egress network device will transmit the timing control packet to determine the resident time, the entire network may operate as a single transparent clock. This may avoid errors in attempts to synchronize across the network that might otherwise be caused by missing corrections by network devices in the underlay network (e.g., network devices not acting as an ingress network device or an egress network device for a particular timing control packet), differing paths between client devices, differing lengths of media connecting network devices along paths between client device, differing transmitter and receiver components of network devices, and/or the like.

As shown by reference number <NUM>, the egress network device may modify the header to remove one or more of the indications provided by the ingress network device. For example, the egress network device may, before forwarding the timing control packet toward the second client device, set fields of the header that were modified by the ingress network device to a null state (e.g., all zeros, all ones, and/or the like). In this way, a device outside of the network may use the same fields for other purposes and may avoid errors by misinterpreting the header.

As shown by reference number <NUM>, the egress network device may forward the timing control packet toward the second client device. In some implementations, one or more intermediate devices and/or networks connect the egress network device to the second client device. The egress network device may provide the timing control packet and/or the other packetized data to the second client device via one or more client-facing interfaces after preparing the timing control packet for forwarding (e.g., popping a label, removing encapsulation, and/or the like) and modifying the header.

In this way, the network, including the ingress network device and the egress network device, may facilitate providing timing information across the network. The network may help to reduce and/or avoid errors that might otherwise be caused by missing corrections by network devices in the underlay network (e.g., network devices not acting as an ingress network device or an egress network device for a particular timing control packet (e.g., not acting as boundary clock or transparent clocks), differing paths between client devices, differing lengths of media connecting network devices along paths between client device, differing transmitter and receiver components of network devices, and/or the like.

<FIG> shows an example PTP message header format <NUM> that may be used in one or more implementations described herein. The ingress network device may use one or more of the reserved fields in the PTP message header shown. For example, the <NUM>-bit reserve field between the messageType field and the versionPTP field may be used to indicate a time stamp type (e.g., attributes of the indication of the ingress time) and/or time stamp validity (e.g., whether the ingress network device was in a synchronized state when the ingress network device provided the indication of the ingress time). For example, the first two bits of the field may be used to indicate the time stamp type (e.g., <NUM> means that the indication of the ingress time is located in <NUM> bits of the reserve field after the correctionField, <NUM> means that the indication of the ingress time is located in <NUM> bits of the correctionField, and/or the like). The second two bits of the field may indicate the time stamp validity (e.g., <NUM> means that the ingress network device was not in a synchronized state, <NUM> means that the ingress network device was in a synchronized state, and/or the like). In some implementations, the <NUM>-bit reserve field between the messageType field and the versionPTP field may be considered the first field of a header, as described herein, that is used to indicate whether the ingress network device is in a synchronized state when the ingress network device receives the timing control packet.

In some implementations, the <NUM>-byte reserve field between correctionField field and sourcePortIdentity field may be used to store an ingress time stamp (e.g., the indication of the ingress time). In some implementations, the PTP message header may use a format to indicate a count of nanoseconds in the reserve field (e.g., using <NUM> or <NUM> bits). In some implementations, the <NUM>-byte reserve field between the correctionField field and the sourcePortIdentity field may be considered the second field of a header, as described herein, that is used to indicate a time at which the ingress network device receives the timing control packet from the first client device.

In some implementations, the <NUM>-byte reserve field between domainNumber and Flags fields may be used to store an identifier of the ingress network device. The identifier may be assigned by a network entity and may be unique for each ingress and egress network device. In some implementations, the identifier may be used for only the purpose of identifying the ingress network device for timing control packets transported over the network.

As indicated above, <FIG> are diagrams of one or more example implementations <NUM> described herein.

<FIG> is a diagram of an example environment <NUM> in which systems and/or methods described herein may be implemented. As shown in <FIG>, environment <NUM> may include a client grandmaster clock device <NUM>, a client grandmaster clock device <NUM>, a first client device <NUM>, a second client device <NUM>, an ingress network device <NUM>, an egress network device <NUM>, a network grandmaster clock device <NUM>, and a network <NUM>. Devices of environment <NUM> may interconnect via wired connections, wireless connections, or a combination of wired and wireless connections.

First client device <NUM> includes one or more devices capable of receiving, generating, storing, processing, and/or providing information associated with communicating (e.g., timing control packets) over network <NUM> via ingress network device <NUM>. For example, first client device <NUM> may include a communication and/or computing device, such as a customer edge device (CE device), a switch, a router, a mobile phone (e.g., a smart phone, a radiotelephone, etc.), a laptop computer, a tablet computer, a handheld computer, a desktop computer, a server, a set-top box, a gaming device, a wearable communication device (e.g., a smart wristwatch, a pair of smart eyeglasses, etc.), or a similar type of device.

Second client device <NUM> includes one or more devices capable of receiving, generating, storing, processing, and/or providing information associated with communicating over network <NUM> via egress network device <NUM>. For example, second client device <NUM> may include a communication and/or computing device, such as a customer edge device (CE device), a switch, a router, a mobile phone (e.g., a smart phone, a radiotelephone, etc.), a laptop computer, a tablet computer, a handheld computer, a desktop computer, a server, a set-top box, a gaming device, a wearable communication device (e.g., a smart wristwatch, a pair of smart eyeglasses, etc.), or a similar type of device.

Ingress network device <NUM> includes one or more devices capable of receiving, storing, generating, processing, forwarding, and/or transferring information. For example, ingress network device <NUM> may include a router, a switch (e.g., a top-of-rack (TOR) switch), a gateway, a firewall device, a modem, a hub, a bridge, a network interface controller (NIC), a reverse proxy, a server (e.g., a proxy server), a multiplexer, a security device, an intrusion detection device, a load balancer, or a similar device. In some implementations, ingress network device <NUM> may be a physical device implemented within a housing, such as a chassis. In some implementations, ingress network device <NUM> may be a virtual device implemented by one or more computer devices of a cloud computing environment or a data center. In some implementations, ingress network device <NUM> may provide one or more interfaces between network <NUM> and a client device (e.g., first client device <NUM>). In some implementations, ingress network device <NUM> may forward timing control packets from first client device <NUM> to second client device <NUM> via one or more hops of network <NUM>. In some implementations, ingress network device <NUM> may perform as an egress network device (e.g., for packetized data provided to first client device <NUM> via network <NUM>).

Egress network device <NUM> includes one or more devices capable of receiving, storing, generating, processing, forwarding, and/or transferring information. For example, egress network device <NUM> may include a router, a switch (e.g., a top-of-rack (TOR) switch), a gateway, a firewall device, a modem, a hub, a bridge, a network interface controller (NIC), a reverse proxy, a server (e.g., a proxy server), a multiplexer, a security device, an intrusion detection device, a load balancer, or a similar device. In some implementations, egress network device <NUM> may be a physical device implemented within a housing, such as a chassis. In some implementations, egress network device <NUM> may be a virtual device implemented by one or more computer devices of a cloud computing environment or a data center. In some implementations, egress network device <NUM> may provide one or more interfaces between network <NUM> and a client device (e.g., second client device <NUM>). In some implementations, egress network device <NUM> may receive timing control packets from first client device <NUM> intended for second client device <NUM> via one or more hops of network <NUM>. In some implementations, egress network device <NUM> may perform as an ingress network device (e.g., for packetized data received from second client device <NUM> via network <NUM>).

Network grandmaster clock device <NUM> may include one or more devices capable of receiving, generating, storing, processing, and/or providing information associated with timing information. In some implementations, network grandmaster clock device <NUM> may receive timing information for the network from one or more of a global positioning system receiver, an atomic clock, another device that receives timing information from a timing source, and/or the like.

Network <NUM> includes one or more wired and/or wireless networks. For example, network <NUM> may include a network of devices to provide one or more tunneling services, such as a multi-protocol label switching (MPLS) network, a virtual extensible local area network (VXLAN), generic network virtualization encapsulation, tunneling over IP version <NUM>, and/or the like. Network <NUM> may include a cellular network (e.g., a long-term evolution (LTE) network, a code division multiple access (CDMA) network, a <NUM> network, a <NUM> network, a <NUM> network, another type of next generation network, and/or the like), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., the Public Switched Telephone Network (PSTN)), a private network, an ad hoc network, an intranet, the Internet, a fiber optic-based network, a cloud computing network, and/or the like, and/or a combination of these or other types of networks.

The number and arrangement of devices and networks shown in <FIG> are provided as one or more examples.

<FIG> is a diagram of example components of a device <NUM>. Device <NUM> may correspond to client grandmaster clock device <NUM>, first client device <NUM>, second client device <NUM>, ingress network device <NUM>, egress network device <NUM>, and/or network grandmaster clock device <NUM>. In some implementations, client grandmaster clock device <NUM>, first client device <NUM>, second client device <NUM>, ingress network device <NUM>, egress network device <NUM>, and/or network grandmaster clock device <NUM> may include one or more devices <NUM> and/or one or more components of device <NUM>. As shown in <FIG>, device <NUM> may include a bus <NUM>, a processor <NUM>, a memory <NUM>, a storage component <NUM>, an input component <NUM>, an output component <NUM>, and a communication interface <NUM>.

Bus <NUM> includes a component that permits communication among the components of device <NUM>. Processor <NUM> is implemented in hardware, firmware, or a combination of hardware and software. Processor <NUM> takes the form of a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field-programmable gate array (FPGA), an ASIC, or another type of processing component. In some implementations, processor <NUM> includes one or more processors capable of being programmed to perform a function. Memory <NUM> includes a random access memory (RAM), a read only memory (ROM), and/or another type of dynamic or static storage device (e.g., a flash memory, a magnetic memory, and/or an optical memory) that stores information and/or instructions for use by processor <NUM>.

Communication interface <NUM> includes a transceiver-like component (e.g., a transceiver and/or a separate receiver and transmitter) that enables device <NUM> to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. Communication interface <NUM> may permit device <NUM> to receive information from another device and/or provide information to another device. For example, communication interface <NUM> may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, or the like.

Device <NUM> may perform one or more processes described herein. Device <NUM> may perform these processes based on processor <NUM> executing software instructions stored by a computer readable medium. A computer readable medium may comprise a non-transitory computer-readable medium, such as memory <NUM> and/or storage component <NUM>, or a transient computer-readable medium such as a transmission signal or carrier wave. A computer-readable storage medium is defined herein as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space spread across multiple physical storage devices.

The quantity and arrangement of components shown in <FIG> are provided as an example.

<FIG> is a diagram of example components of a device <NUM>. Device <NUM> may correspond to client grandmaster clock device <NUM>, first client device <NUM>, second client device <NUM>, ingress network device <NUM>, egress network device <NUM>, and/or network grandmaster clock device <NUM>. In some implementations, one or more of client grandmaster clock device <NUM>, first client device <NUM>, second client device <NUM>, ingress network device <NUM>, egress network device <NUM>, and/or network grandmaster clock device <NUM> may include one or more devices <NUM> and/or one or more components of device <NUM>. As shown in <FIG>, device <NUM> may include one or more input components <NUM>-<NUM> through <NUM>-B (B ≥ <NUM>) (hereinafter referred to collectively as input components <NUM>, and individually as input component <NUM>), a switching component <NUM>, one or more output components <NUM>-<NUM> through <NUM>-C (C ≥ <NUM>) (hereinafter referred to collectively as output components <NUM>, and individually as output component <NUM>), and a controller <NUM>.

Input components <NUM> may be points of attachment for physical links and may be points of entry for incoming traffic, such as packets. Input components <NUM> may process incoming traffic, such as by performing data link layer encapsulation or decapsulation. In some implementations, input components <NUM> may send and/or receive packets. In some implementations, input components <NUM> may include an input line card that includes one or more packet processing components (e.g., in the form of integrated circuits), such as one or more interface cards (IFCs), packet forwarding components, line card controller components, input ports, processors, memories, and/or input queues.

In some implementations, switching component <NUM> may enable input components <NUM>, output components <NUM>, and/or controller <NUM> to communicate.

In some implementations, output component <NUM> may send packets and/or receive packets.

Controller <NUM> may create routing tables based on the network topology information, create forwarding tables based on the routing tables, and forward the forwarding tables to input components <NUM> and/or output components <NUM>.

Controller <NUM> may perform one or more processes described herein. Controller <NUM> may perform these processes in response to executing software instructions stored by a computer readable medium. A computer readable medium may comprise a non-transitory computer-readable medium, such as memory <NUM> and/or storage component <NUM>, or a transient computer-readable medium such as a transmission signal or carrier wave. A computer-readable storage medium is defined herein as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space spread across multiple physical storage devices.

<FIG> is a flow chart of an example process <NUM> for transporting client timing information across a network. In some implementations, one or more process blocks of <FIG> may be performed by an ingress network device (e.g., ingress network device <NUM>). In some implementations, one or more process blocks of <FIG> may be performed by another device or a group of devices separate from or including the ingress network device, such as a client grandmaster clock device (e.g., client grandmaster clock device <NUM>), a first client device (e.g., first client device <NUM>), a second client device (e.g., second client device <NUM>), another ingress network device, an egress network device (e.g., egress network device <NUM>), a network grandmaster clock device (e.g., network grandmaster clock device <NUM>), and/or the like.

As shown in <FIG>, process <NUM> may include receiving a timing control packet from a first client device (block <NUM>). For example, the network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may receive a timing control packet from a first client device, as described above.

As further shown in <FIG>, process <NUM> may include determining that the network device is in a synchronized state relative to a network grandmaster clock (block <NUM>). For example, the network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may determine that the network device is in a synchronized state relative to a network grandmaster clock, as described above.

As further shown in <FIG>, process <NUM> may include modifying a first field of a header of the timing control packet to indicate that the network device is in a synchronized state (block <NUM>). For example, the network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may modify a first field of a header of the timing control packet to indicate that the network device is in a synchronized state, as described above.

As further shown in <FIG>, process <NUM> may include modifying a second field of the header of the timing control packet to indicate a time at which the network device received the timing control packet from the first client device (block <NUM>). For example, the network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may modify a second field of the header of the timing control packet to indicate a time at which the network device received the timing control packet from the first client device, as described above.

As further shown in <FIG>, process <NUM> may include forwarding, via the network, the timing control packet toward a second client device (block <NUM>). For example, the network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may forward, via the network, the timing control packet toward a second client device, as described above.

In a first implementation, the first field comprises a first portion and a second portion, wherein the first portion includes a first indication that the network device is in the synchronized state, and wherein the second portion includes a second indication of one or more attributes of the second field.

In a second implementation, alone or in combination with the first implementation, the one or more attributes include a location of the second field within the header of the timing control packet or include a quantity of bits associated with the second field.

In a third implementation, alone or in combination with one or more of the first and second implementations, the timing control packet comprises a precision time protocol control packet.

In a fourth implementation, alone or in combination with one or more of the first through third implementations, process <NUM> includes parsing the timing control packet to determine that the timing control packet includes client timing information for synchronizing the second client device with the first client device, wherein the network device is determining whether the network device is in the synchronized state based on determining that the timing control packet includes the client timing information.

In a fifth implementation, alone or in combination with one or more of the first through fourth implementations, process <NUM> includes receiving, from the network grandmaster clock, network timing information to synchronize the network device with the network grandmaster clock; and updating a clock of the network device based on the network timing information, wherein the network device is determining that the network device is in the synchronized state based on updating the clock of the network device.

In a sixth implementation, alone or in combination with one or more of the first through fifth implementations, process <NUM> includes modifying a third field of the header of the timing control packet to identify the network device as an ingress network device for the timing control packet.

<FIG> is a flow chart of an example process <NUM> for transporting client timing information across a network. In some implementations, one or more process blocks of <FIG> may be performed by an egress network device (e.g., egress network device <NUM>). In some implementations, one or more process blocks of <FIG> may be performed by another device or a group of devices separate from or including the network device, such as a client grandmaster clock device (e.g., client grandmaster clock device <NUM>), a first client device (e.g., first client device <NUM>), a second client device (e.g., second client device <NUM>), another egress network device, an ingress network device (e.g., ingress network device <NUM>), a network grandmaster clock device (e.g., network grandmaster clock device <NUM>), and/or the like.

As shown in <FIG>, process <NUM> may include receiving a timing control packet from an ingress network device via the network (block <NUM>). For example, the egress network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may receive a timing control packet from an ingress network device via the network, as described above. Needs modification on figures and related description based on that.

As further shown in <FIG>, process <NUM> may include identifying, within a first field of a header of the timing control packet, a first indication that indicates that the ingress network device was in a synchronized state when the ingress network device received the timing control packet from a first client device (block <NUM>). For example, the egress network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may identify, within a first field of a header of the timing control packet, a first indication that indicates that the ingress network device was in a synchronized state when the ingress network device received the timing control packet from a first client device, as described above.

As further shown in <FIG>, process <NUM> may include identifying, within a second field of the header of the timing control packet, a second indication that indicates a time at which the ingress network device received the timing control packet from the first client device (block <NUM>). For example, the egress network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may identify, within a second field of the header of the timing control packet, a second indication that indicates a time at which the ingress network device received the timing control packet from the first client device, as described above.

As further shown in <FIG>, process <NUM> may include determining that the egress network device is in a synchronized state (block <NUM>). For example, the egress network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may determine that the egress network device is in a synchronized state, as described above.

As further shown in <FIG>, process <NUM> may include determining a time at which the egress network device received the timing control packet (block <NUM>). For example, the egress network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may determine a time at which the egress network device received the timing control packet, as described above.

As further shown in <FIG>, process <NUM> may include determining a resident time of the timing control packet on the network based on the second indication and the time at which the egress network device will transmit the timing control packet (block <NUM>). For example, the egress network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may determine a resident time of the timing control packet on the network based on the second indication and the time at which the egress network device will transmit the timing control packet, as described above.

As further shown in <FIG>, process <NUM> may include modifying a third field of the header of the timing control packet to indicate the resident time of the timing control packet on the network (block <NUM>). For example, the egress network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may modify a third field of the header of the timing control packet to indicate the resident time of the timing control packet on the network, as described above.

As further shown in <FIG>, process <NUM> may include forwarding the timing control packet toward a second client device (block <NUM>). For example, the egress network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may forward the timing control packet toward a second client device, as described above.

In a first implementation, the first field comprises a first portion and a second portion, wherein the first portion indicates that the ingress network device was in the synchronized state, and wherein the second portion indicates a location of the second field within the header of the timing control packet or indicates a quantity of bits associated with the second field.

In a second implementation, alone or in combination with the first implementation, process <NUM> includes receiving, from a network grandmaster clock, network timing information to synchronize the egress network device with the network grandmaster clock; and updating a clock of the ingress network device based on the network timing information, wherein the egress network device is determining that the egress network device is in the synchronized state based on updating the clock of the egress network device.

In a third implementation, alone or in combination with one or more of the first and second implementations, the network comprises a multi-protocol label switching network.

In a fourth implementation, alone or in combination with one or more of the first through third implementations, the egress network device provides an interface between the client device and the network, and wherein the ingress network device provides an interface between the network and another client device.

In a fifth implementation, alone or in combination with one or more of the first through fourth implementations, process <NUM> includes, before forwarding the timing control packet toward the client device, setting the first field and the second field to a null state.

<FIG> is a flow chart of an example process <NUM> for transporting client timing information across a network. In some implementations, one or more process blocks of <FIG> may be performed by a system comprising an ingress network device (e.g., ingress network device <NUM>) and an egress network device (e.g., egress network device <NUM>). In some implementations, one or more process blocks of <FIG> may be performed by another device or a group of devices separate from or including the network device, such as a client grandmaster clock device (e.g., client grandmaster clock device <NUM>), a first client device (e.g., first client device <NUM>), a second client device (e.g., second client device <NUM>), another ingress network device, an egress network device (e.g., egress network device <NUM>), a network grandmaster clock device (e.g., network grandmaster clock device <NUM>), and/or the like.

As shown in <FIG>, process <NUM> may include receiving a timing control packet from a first client device (block <NUM>). For example, the ingress network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may receive a timing control packet from a first client device, as described above.

As further shown in <FIG>, process <NUM> may include determining that the ingress network device is in a synchronized state (block <NUM>). For example, the ingress network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may determine that the ingress network device is in a synchronized state, as described above.

As further shown in <FIG>, process <NUM> may include modifying a header of the timing control packet to include a first indication that the ingress network device is in a synchronized state, and a second indication that indicates a time at which the ingress network device received the timing control packet from the first client device (block <NUM>). For example, the ingress network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may modify a header of the timing control packet to include, as described above.

As further shown in <FIG>, process <NUM> may include forwarding the timing control packet toward an egress network device via the network (block <NUM>). For example, the ingress network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may forward the timing control packet toward an egress network device via the network, as described above.

As further shown in <FIG>, process <NUM> may include receiving the timing control packet via the network (block <NUM>). For example, the egress network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may receive the timing control packet via the network, as described above.

As further shown in <FIG>, process <NUM> may include identifying, within the header of the timing control packet, the first indication and the second indication (block <NUM>). For example, the egress network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may identify, within the header of the timing control packet, the first indication and the second indication, as described above.

As further shown in <FIG>, process <NUM> may include determining a resident time of the timing control packet on the network based on the second indication and a time at which the egress network device received the timing control packet (block <NUM>). For example, the egress network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may determine a resident time of the timing control packet on the network based on the second indication and a time at which the egress network device received the timing control packet, as described above.

As further shown in <FIG>, process <NUM> may include modifying the header of the timing control packet to indicate the resident time of the timing control packet on the network (block <NUM>). For example, the egress network device (e.g., using processor <NUM>, memory <NUM>, storage component <NUM>, input component <NUM>, output component <NUM>, communication interface <NUM>, switching component <NUM>, controller <NUM>, and/or the like) may modify the header of the timing control packet to indicate the resident time of the timing control packet on the network, as described above.

In a first implementation, the ingress network device is further to modify the header of the timing control packet to indicate one or more attributes of the header.

In a second implementation, alone or in combination with the first implementation, the one or more attributes include a location of the second indication within the header of the timing control packet or include a quantity of bits associated with the second indication.

In a fourth implementation, alone or in combination with one or more of the first through third implementations, the ingress network device is further to parse the timing control packet to determine that the timing control packet includes client timing information for synchronizing the second client device with the first client device, wherein the ingress network device initiates a determination of whether the ingress network device is in the synchronized state based on determining that the timing control packet includes the client timing information.

In a fifth implementation, alone or in combination with one or more of the first through fourth implementations, the ingress network device is further to modify the header of the timing control packet to identify the ingress network device as the ingress network device for the timing control packet.

In a sixth implementation, alone or in combination with one or more of the first through fifth implementations, the ingress network device provides an interface between the first client device and the network, and wherein the egress network device provides an interface between the second client device and the network.

As used herein, the term traffic or content may include a set of packets. A packet may refer to a communication structure for communicating information, such as a protocol data unit (PDU), a network packet, a datagram, a segment, a message, a block, a cell, a frame, a subframe, a slot, a symbol, a portion of any of the above, and/or another type of formatted or unformatted unit of data capable of being transmitted via a network.

Therefore, from one perspective, there has been described a network device that may receive a timing control packet from a first client device. The network device may determine that the network device is in a synchronized state relative to a network grandmaster clock. The network device may modify a first field of a header of the timing control packet to indicate that the network device is in a synchronized state. The network device may modify a second field of the header of the timing control packet to indicate a time at which the network device received the timing control packet from the first client device. The network device may forward, via the network, the timing control packet toward a second client device.

As used herein, satisfying a threshold may, depending on the context, refer to a value being greater than the threshold, more than the threshold, higher than the threshold, greater than or equal to the threshold, less than the threshold, fewer than the threshold, lower than the threshold, less than or equal to the threshold, equal to the threshold, or the like.

Claim 1:
A method, comprising:
identifying, by an egress network device and based on a timing control packet received from an ingress network device, information that indicates a first time at which the ingress network device received the timing control packet from a first client device;
determining (<NUM>), by the egress network device, a second time at which the egress network device will forward the timing control packet;
determining (<NUM>), by the egress network device based on the information that indicates the first time and information that indicates the second time, a resident time of the timing control packet, wherein the resident time of the timing control packet is determined based on:
determining (<NUM>) that the ingress network device was in a synchronized state when the ingress network device provided the information that indicates the first time; and
determining (<NUM>) that the egress network device is in a synchronized state;
modifying (<NUM>), by the egress network device, the timing control packet to indicate the resident time and to form a modified timing control packet; and
forwarding (<NUM>), by the egress network device, the modified timing control packet to a second client device.