Abstract:
A network component comprising a processor configured to implement a method comprising promoting transmission of a first frame comprising a first timestamp associated with a transmission time of the first frame, recognizing a reception of a second frame having a reception time, wherein the second frame comprises a second timestamp comprising a downstream node delay associated with a downstream node, measuring a total delay between the transmission time of the first frame and the reception time of the second frame, and calculating a transport delay using the total delay and the downstream node delay. Also disclosed is a clock synchronization method comprising receiving a first frame comprising a first timestamp associated with an upstream clock at a reception time, sending a second frame at a transmission time, and measuring a downstream node delay between the reception time and the transmission time, wherein the second frame comprises the downstream node delay.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application claims the benefit of U.S. Provisional Application Ser. No. 60/826,764 filed Sep. 25, 2006 and entitled “System for TDM Data Transport Over Ethernet Interfaces,” U.S. Provisional Application Ser. No. 60/857,741 filed Nov. 8, 2006 and entitled “TDM Data Transport Over Ethernet,” and U.S. Provisional Application Ser. No. 60/886,833 filed Jan. 26, 2007 and entitled “Closed Loop Clock Synchronization,” all of which are by Serge F. Fourcand and are incorporated herein by reference as if reproduced in their entirety. 
         [0002]    This application is related to U.S. patent application Ser. No. (Attorney Docket No. 4194-03200) entitled “Inter-Packet Gap Network Clock Synchronization,” U.S. patent application Ser. No. (Attorney Docket No. 4194-04400) entitled “Network Clock Synchronization Timestamp,” and U.S. patent application Ser. No. (Attorney Docket No. 4194-04600) entitled “Network Clock Synchronization Floating Window and Window Delineation,” all of which are by Serge F. Fourcand, are filed concurrently herewith, and are incorporated herein by reference as if reproduced in their entirety. 
     
     STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT 
       [0003]    Not applicable. 
       REFERENCE TO A MICROFICHE APPENDIX 
       [0004]    Not applicable. 
       BACKGROUND 
       [0005]    Ethernet is the preferred protocol for many types of networks because it is flexible, decentralized, and scalable. Ethernet is flexible in that it allows variable-sized data packets to be transported across different types of mediums using various nodes each having different transmission speeds. Ethernet is decentralized in that it allows the end devices to transmit and receive data without oversight or intervention from a centralized server or party. Furthermore, Ethernet is scalable in that it can be implemented in both small-scale and large-scale networks. These advantages make Ethernet a preferred choice for data distribution in many computer networks. 
         [0006]    Unfortunately, Ethernet does have some drawbacks. When Ethernet packets are transported through the network, the Ethernet packets contend with other traffic being transported over the same links or through the same nodes. The contentious traffic not only includes packets bound for the same destination, but also packets bound for other destinations that are transported over the same link or through the same node as the Ethernet packet. This contention produces burstiness and jitter at the nodes within the network. Some of these problems can be addressed by using resource arbitration and buffers at the nodes, and by prioritizing the packets into high priority data and low priority data. However, these solutions increase network complexity, increase delay, and detract from the inherent advantages of Ethernet. 
         [0007]    The aforementioned drawbacks are part of the reason Ethernet has not been widely implemented in networks carrying time division multiplexed (TDM) data. Specifically, Ethernet does not provide a sufficient Quality of Service (QoS) to meet the stringent jitter and data loss requirements for voice traffic in the public switched telephone network (PSTN) and other TDM networks. Instead, TDM traffic is carried by highly synchronized networks, such as synchronous optical networks (SONET) and synchronous digital hierarch (SDH) networks. Various Ethernet enhancements, such as circuit emulation, provider backbone transport, and pseudowires, have been proposed to address the jitter and data loss issues, but these enhancements fail to couple the flexibility of Ethernet with the high QoS requirements of TDM networks. Thus, a need exists for an improved Ethernet protocol that is flexible, easy to implement, supports the QoS requirements of TDM networks, and is compatible with existing technology. 
       SUMMARY 
       [0008]    In one aspect, the disclosure includes a network component comprising a processor configured to implement a method comprising promoting transmission of a first frame comprising a first timestamp associated with a transmission time of the first frame, recognizing a reception of a second frame having a reception time, wherein the second frame comprises a second timestamp comprising a downstream node delay associated with a downstream node, measuring a total delay between the transmission time of the first frame and the reception time of the second frame, and calculating a transport delay using the total delay and the downstream node delay. 
         [0009]    In another aspect, the disclosure includes a clock synchronization method comprising receiving a first frame comprising a first timestamp associated with an upstream clock at a reception time, sending a second frame at a transmission time, and measuring a downstream node delay between the reception time and the transmission time, wherein the second frame comprises the downstream node delay. 
         [0010]    In a third aspect, the disclosure includes a system comprising a first Ethernet node comprising a first clock, and a second Ethernet node comprising a second clock, wherein the first Ethernet node sends a first Ethernet packet comprising a first frame count and a first timestamp associated with the first clock to the second node, wherein the second node sends a second Ethernet packet comprising the first timestamp, the first frame count, and a second timestamp to the first node, wherein the first Ethernet node uses the first timestamp, the frame count, and the second timestamp to synchronize the first clock with the second clock. 
         [0011]    These and other features will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings and claims. 
     
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0012]    For a more complete understanding of this disclosure, reference is now made to the following brief description, taken in connection with the accompanying drawings and detailed description, wherein like reference numerals represent like parts. 
           [0013]      FIG. 1  is an illustration of an embodiment of an Ethernet MAC frame. 
           [0014]      FIG. 2  is an illustration of an embodiment of an Ethernet data stream. 
           [0015]      FIG. 3  is an illustration of another embodiment of the Ethernet data stream. 
           [0016]      FIG. 4  is an illustration of an embodiment of a synchronization timestamp format. 
           [0017]      FIG. 5A  is an illustration of an embodiment of a process of establishing synchronized communication. 
           [0018]      FIG. 5B  is an illustration of an embodiment of a timeline for establishing synchronized communication. 
           [0019]      FIG. 6A  is an illustration of one embodiment of an H-TDM frame. 
           [0020]      FIG. 6B  is an illustration of another embodiment of the H-TDM frame. 
           [0021]      FIG. 7A  is an illustration of another embodiment of the H-TDM frame. 
           [0022]      FIG. 7B  is an illustration of another embodiment of the H-TDM frame. 
           [0023]      FIG. 8  is an illustration of another embodiment of a synchronization timestamp format. 
           [0024]      FIG. 9A  is an illustration of another embodiment of a process of establishing synchronized communication. 
           [0025]      FIG. 9B  is an illustration of another embodiment of a timeline for establishing synchronized communication. 
           [0026]      FIG. 10  is an illustration of an embodiment of a clocking architecture. 
           [0027]      FIG. 11  is an illustration of one embodiment of a general-purpose computer system suitable for implementing the several embodiments of the disclosure. 
       
    
    
     DETAILED DESCRIPTION 
       [0028]    It should be understood at the outset that although an illustrative implementation of one or more embodiments are provided below, the disclosed systems and/or methods may be implemented using any number of techniques, whether currently known or in existence. The disclosure should in no way be limited to the illustrative implementations, drawings, and techniques described below, including the exemplary designs and implementations illustrated and described herein, but may be modified within the scope of the appended claims along with their full scope of equivalents. 
         [0029]    Synchronization of nodes across a network has many practical applications. For example, it is preferable that all of the audio channels be synchronized to each other in sports stadiums or other large venues. In addition, at large venues there may be a video screen that provides visualizations along with the audio presentation. In this case, it may be important that not only the audio channels be all synchronized to each other but also to the video display. Another application of synchronization may be for multiple people to work remotely in real-time together. For example, each musician in a band may be remotely located from each other while having a recording session at a remote recording studio. In this example, the music produced by each musician may be synchronized together to be recorded at the remote recording studio. Many other applications that have already been envisioned and have yet to be envisioned are enabled through synchronized communication. 
         [0030]    The inclusion of clock synchronization data in the Ethernet packets has been previously addressed. For example, IEEE 1588 specifies that timestamps may be added to packets to communicate a timestamp between nodes in the network. However, including the timestamp within the packet creates the problem of accurately indicating the packet&#39;s transmission time, e.g. the time that the packet is sent. The transmission time may be inaccurate because the timestamp has to be processed and encapsulated within a packet, a process that delays the transmission of the packet in an unpredictable manner. Specifically, delays caused by the insertion of the timestamp into the packet and by implementing carrier sense multiple access with collision avoidance (CSMA/CA) may cause the timestamp to become stale. These delays are unpredictable in that they vary based on the packet, the node, and other network conditions. Similarly, upon receiving a timestamp at a downstream node, further delays may be incurred when the packet is buffered and/or processed to extract the timestamp. To compensate for these delays, IEEE 1588 specifies that a follow-up timestamp be communicated to a downstream node to indicate the precise time when the initial timestamp was communicated. Unfortunately, the prior art methods for clock synchronization are bandwidth limiting due to multiple packets being communicated from the upstream nodes to the downstream nodes. Further, the prior art methods for clock synchronization do not take into account internal processing delays at the downstream node. 
         [0031]    Disclosed herein are multiple operational modes that provide clock synchronization between nodes in an Ethernet network, which may be referred to herein as Huawei-Enhanced (HE) Ethernet operational modes. A first operational mode is frequency-synchronized communication mode, also referred to as a Huawei synchronized (H-Sync) operational mode. The H-Sync operational mode places a timestamp in the inter-packet gap (IPG) between two Ethernet packets. The timestamp may be used to indicate the start of a predefined periodic synchronization window that enables frequency-synchronized communication between two Ethernet nodes. The inclusion of the timestamp in the IPG may not be bandwidth limiting because the IPG is an idle period in standard Ethernet communication. Adding the timestamp to the IPG rather than to an Ethernet packet allows the nodes to process the timestamp without having to process an entire packet. 
         [0032]    A second operational mode is a frequency-synchronized and phase-aligned communication mode, also referred to as a Huawei Time Division Multiplexed (H-TDM) operational mode. The H-TDM operational mode defines an overlay synchronous timeslot scheme that multiplexes octet-sized timeslots of timestamp, control, and payload data within a predefined synchronization window. The payload data can include any of voice data, high priority data such as video data, and best-effort packet data. The overlay synchronous timeslot scheme enables deterministic transfer of high priority data without contention, and thus supports the stringent QoS requirements of the PSTN. The timestamp data contained in the overlay synchronous timeslot scheme includes a forward timestamp that establishes the start of the predefined synchronization window, which enable frequency-synchronized communication, and a loop-back timestamp that compensates for transmission delays, which enables phase-aligned communication between two Ethernet nodes. 
         [0033]      FIG. 1  illustrates one embodiment of an Ethernet packet  100 . The packet  100  begins with a preamble  104 , which may be about seven octets of a repeated pattern, such as “10101010.” The preamble  104  may allow a node&#39;s physical layer signaling (PLS) circuitry to reach steady-state synchronization with the packet&#39;s timing. The preamble  104  may be followed by a start of frame delimiter (SFD)  106 , which may be a single octet with the pattern “10101011,” and may be used to indicate the start of the packet  100 . The destination address (DA)  108  may specify the address of the destination node for which the packet  100  is intended, and may be about six octets. The source address (SA)  110  may specify the address of the source node from which the packet  100  originated, and may be about six octets. The packet  100  may contain a plurality of optional octets  112  that are used to associate the packet  100  with a type protocol identifier (TPID) and/or a virtual local area network identifier (VID). For example, up to about sixteen octets may be used for associating the packet  100  with a TPID and a VID, for example, as described in IEEE 802.1Q. 
         [0034]    The packet  100  continues with a length/type field  114 , which may specify the length of the payload and the Ethernet protocol being used, and may be about two octets. The payload  116  may be a variable-sized field that carries a data payload. Although the payload  116  may contain any amount of data, in specific embodiments the payload  116  may contain from about  42  octets to about  1 , 500  octets in standard packets, and may contain from about  9 , 000  octets to about  12 , 000  octets in jumbo packets. The frame check sequence (FCS)  118  may be used for error detection, and may be a four-octet field that contains a cyclic redundancy check (CRC) value calculated using the contents of the packet  100 . Although not part of the packet  100 , the IPG  102  may be data or idle characters that separate the packets  100 . The IPG  102  may contain about twelve octets of idle control characters, although any amount of data or idle characters may be used in the IPG  102 . 
         [0035]    As illustrated in  FIG. 2 , a synchronous timestamp (Sync)  202  may be inserted in the IPG  102  between two packets  204 . The Sync  202  may be used to synchronize an upstream node&#39;s clock with a downstream node&#39;s clock in the H-Sync operational mode. Specifically, the Sync  202  may be a four-octet packet that synchronizes the two clocks in frequency, but does not necessarily align the clocks in phase. The Sync  202  may also indicate the start of a synchronization window having a predetermined period, such as about  125  microseconds (μs). The Sync  202  need not be located in every IPG  102 , but in some embodiments, it may be advantageous to have at least one Sync  202  during every synchronization window. 
         [0036]    In some embodiments, there are advantages to inserting the timestamp in the IPG  102 . For example, the H-Sync timestamp does not affect the available bandwidth because the Sync  202  is located in the IPG  102 , which is an idle period in standard Ethernet communications. Further, communicating the timestamp in the IPG  102 , rather than within the packet  100 , allows the timestamp to be transmitted independent of the packet  100 . The independent transmission of the Sync  202  and the packet  100  ensures that the timestamp will not become stale, and allows the upstream and downstream nodes&#39; clocks to be synchronized without transmitting multiple timestamps from the upstream node to the downstream node. Similarly, upon receiving the timestamp at a downstream node, the timestamp may be extracted and processed without processing the packet  100 . 
         [0037]    Clock accuracy is a consideration when synchronizing clocks between Ethernet nodes. Most clocks have imperfect frequency sources that lead to imperfect clock accuracy. Currently, IEEE 802.3 requires that the accuracy of a free-running oscillator sourcing the frequency base for an Ethernet interface to be ±100 parts per million (ppm), where the ppm indicates how much offset an oscillator will have for a given frequency over a given time period. A clock with ±100 ppm accuracy has a free-running oscillator that may be ahead (+) by 100 ppm or behind (−) by 100 ppm, resulting in a possible 200 ppm range of accuracy for a two-way Ethernet communication. As an example, a transmitting Ethernet interface oscillator may be ahead by 100 ppm and a receiving Ethernet interface oscillator may be behind by 100 ppm. Thus, if each clock has a one-second frequency with ±100 ppm accuracy, they will each be offset by as much as about 8.64 seconds over the course of a day. To better support the operational modes described herein and future development, e.g. 100 Gigabit Ethernet, the accuracy requirement of a free-running oscillator sourcing the frequency base for an Ethernet interface may be increased to be ± about 20 ppm. Thus, a clock with a one-second frequency and ±20 ppm accuracy may reflect an offset of about 1.728 seconds over the course of a day. When the requirement for a free-running oscillator sourcing the frequency base for an Ethernet interface is ± about 20 ppm or better, the Ethernet interface may be said to be operating in a Huawei-Ethernet (H-Eth) operational mode. 
         [0038]      FIG. 3  illustrates an embodiment in which at least one Sync  202  is included in a synchronization window. The Sync  202  is used to mitigate the effects of clock inaccuracy, and may be included in each synchronization window. For example, if the synchronization period is about 125 μs, then the Sync  202  is communicated at least once about every 125 μs. A network node may maintain a timer that indicates when the next Sync  202  should be communicated to ensure communication of the Sync  202  at least once within every synchronization window. Within the synchronization window, the communication of packets  100  may proceed as normal. Although not illustrated in  FIG. 3 , in some embodiments there may be a plurality of Syncs  202  between two packets. For example, there may be two or more Syncs  202  between two packets when the Syncs  202  are communicated at least once every 125 μs and the time period between the end of one packet and the beginning of a subsequent packet is more than 125 μs. Persons of ordinary skill in the art will be aware of other instances when there will be a plurality of Syncs  202  between packets. 
         [0039]      FIG. 3  also illustrates the placement of the Sync  202  within the IPG  102 . Although the Sync  202  may be inserted anywhere within the IPG  102 , some specific locations may be preferred. For example, the Sync  202  may be inserted into the center of the IPG  102  such that an equal number of idle octets  302  are present before and after the Sync  202 . Specifically, if the IPG  102  is twelve octets long, then the Sync  202  may be inserted in the middle four octets, with four octets  302  preceding and four octets  302  following the Sync  202 . Another example is that the Sync  202  may be placed within the IPG  102  such that at least two idle octets are located between the Sync  202  and the previous or next packet  100 . Alternatively, the Sync  202  may be inserted at the beginning of the IPG  102 , at the end of the IPG  102 , or at any other location within the IPG  102 . 
         [0040]      FIG. 4  illustrates one embodiment of the Sync  202 . Specifically,  FIG. 4  illustrates a four octet Sync  202  where each row represents an octet and each column represents the position of the bits within each octet. The octets may be arranged in order of significance from top to bottom such that the least significant octet, octet  0 , is at the top and the most significant octet, octet  3 , is at the bottom. Similarly, the bits may be arranged in order of significance from left to right, such that the least significant bit (LSB) of each octet, bit  0 , is at the left and the most significant bit (MSB) of each octet, bit  7 , is at the right. As is common in Ethernet communications, data may be communicated from the LSB to the MSB and from least significant octet to most significant octet, such that the bits are communicated row-by-row from left to right starting with the top octet and ending with the bottom octet. 
         [0041]    The first three octets in the Sync  202  may be a twenty-four bit timestamp. Specifically, octet  0  may contain the LSBs of the twenty-four bit timestamp, which may be bits  00  through  07  of the timestamp located in bit  0  through bit  7  of octet  0 . Octet  1  may contain the next eight bits of the twenty-four bit timestamp, which may be bits  08  through  15  of the timestamp located in bit  0  through bit  7  of octet  1 . Octet  2  may contain the MSBs of the twenty-four bit timestamp, which may be bits  16  through  23  of the timestamp located in bit  0  through bit  7  of octet  2 . With twenty-four bits available for the timestamp, each bit may represent a timestamp resolution of about 0.01 nanoseconds (ns) for a total range of about 167.772 μs. When the synchronization windows have a period of about 125 μs, then each bit may represent a timestamp resolution as low as about 7.451 picoseconds (ps) and still cover the full range of the 125 μs window. In some embodiments, more or less bits may be used in the timestamp. For example, if sixteen bits were used in the timestamp, then the size of the Sync  202  would be reduced to three octets, and each bit would represent a timestamp resolution of about two nanoseconds (ns) for a range of about 131.072 μs. Persons of ordinary skill in the art will recognize the proper balance of the number of bits used in the timestamp and the timestamp resolution represented by each bit to cover a given timestamp range for a given network. 
         [0042]    The fourth octet in the Sync  202  may be control information. The control information may be used to initiate frequency-synchronized communication between an upstream node and a downstream node. The first five bits, bit  0  through bit  4 , may indicate the clock quality of the node that is initiating the synchronization request, e.g. the upstream node. The sixth bit, bit  5 , may indicate whether the Sync  202  is a request for frequency-synchronized communication or an acknowledgement of a previous request for frequency-synchronized communication. In embodiments, bit  5  is set to “0” when the Sync  202  is a request for frequency-synchronized communication, and bit  5  is set to “1” when the Sync  202  is an acknowledgement of a previous request for frequency-synchronized communication. The seventh bit, bit  6 , may indicate the operational mode that is being requested or acknowledged. In embodiments, bit  6  is set to “0” when the H-Sync operational mode is being requested or acknowledged, and bit  6  is set to “1” when the H-TDM operational mode is being requested or acknowledged. The last bit, bit  7 , may be used as a parity bit for verifying the integrity of the Sync  202 . 
         [0043]      FIG. 5A  illustrates an exemplary block diagram of the process for initiating the frequency-synchronized communication between two nodes. The process may begin when a synchronization request  506  is sent from an upstream node  502  to a downstream node  504  at time T 1 . The synchronization request  506  may have the format of the Sync  202  illustrated in  FIG. 4 , where the timestamp indicates time T 1 . The synchronization request  506  may also have bit  5  of octet  3  set to “0” and bit  6  of octet  3  set to “0” to indicate that the synchronization request  506  is a request for the H-Sync operational mode. Concurrently, the upstream node  502  may initiate its synchronization window. If the downstream node  504  supports the H-Sync operational mode, then the downstream node  504  will receive and process the synchronization request  506  at time T 2 . Specifically, the downstream node  504  may use the timestamp to create a synchronization window that is frequency-synchronized to a corresponding synchronization window in the upstream node  502 . Both synchronization windows may have a period of about  125  its. 
         [0044]    The downstream node  504  may synchronize its synchronization window with the timestamp using various methods. Specifically, the downstream node  504  may frequency-synchronize its synchronization window to a corresponding synchronization window in the upstream node  502  by setting its clock equal to the timestamp. For example, upon receiving the synchronization request  506  with the timestamp T 1 , the downstream node  504  may set its clock to time T 1 . Alternatively, the downstream node  504  may record an offset between its clock and the timestamp, which allows the downstream node  504  to be frequency-synchronized to multiple nodes. For example, in addition to being downstream from the upstream node  502 , the downstream node  504  may also be downstream from another node, node A. Specifically, the upstream node  502  and node A may be connected to different ports on the downstream node  504 . In this case, the downstream node  504  may maintain a first clock offset, thereby enabling frequency-synchronized communication with the upstream node  502 , and maintain a second clock offset, thereby enabling frequency-synchronized communication with upstream node A. Maintaining a separate offset for each port may be beneficial when the downstream node  502  communicates with a plurality of other network nodes via a plurality of ports. 
         [0045]    If the downstream node  504  supports the H-Sync operational mode, the downstream node  504  may send a synchronization acknowledgement  508  to the upstream node  502 . The synchronization acknowledgement  508  may have the format of the Sync  202  illustrated in  FIG. 4  with bit  5  of octet  3  set to “1” and with bit  6  of octet  3  set to “ 0 ,” thereby indicating that the synchronization acknowledgement  506  is an acknowledgement of the H-Sync operational mode. In addition, the synchronization acknowledgement  508  may contain the timestamp received from the upstream node  502 . Specifically, the timestamp in the synchronization acknowledgement  508  may be set to time T 1 . The inclusion of the original timestamp, e.g. T 1  from the synchronization request  506 , in the synchronization acknowledgement  508  allows the upstream node  502  to correlate the synchronization request  506  with the synchronization acknowledgement  508 . The upstream node  502  may then interpret the synchronization acknowledgement  508  as an indication that the H-Sync operational mode has been established. This timestamp loop-back from the downstream node  504  to the upstream node  502  allows the upstream node  502  and the downstream node  504  to be frequency synchronized, that is offset in time from each other by a consistent amount of time. 
         [0046]    The synchronization request  506  does not affect nodes that do not support the H-Sync operational mode. Specifically, if the downstream node  504  does not support the H-Sync operational mode, then the downstream node  504  will view the synchronization request  506  as random data in the IPG, and will ignore and/or discard the synchronization request  506 . If the upstream node  502  does not receive a response to the synchronization request within a predetermined amount of time, the upstream node  502  may determine that the H-Sync operational mode is not supported by the downstream node  504 . Because nodes that do not support the H-Sync operational mode ignore and/or discard the synchronization request  506 , backwards compatibility may be enabled such that the nodes that support the H-Sync operational mode may revert to standard Ethernet protocols to communicate with the nodes that do not support the H-Sync operational mode. 
         [0047]    In some instances, the downstream node  504  may send a second synchronization request, rather than the synchronization acknowledgement  508 , to the upstream node  502 . For example, the downstream node  504  may determine that it has a higher quality clock by comparing its clock quality to the indication of the upstream node&#39;s clock quality found in the first five bits of the fourth octet of the synchronization request  506 . In such a case, the downstream node  504  may initiate its own synchronization request  506  that contains the downstream node&#39;s higher clock quality in the first five bits of the fourth octet. Alternatively, the downstream node  504  may support another operational mode, such as the H-TDM operational mode. In such a case, the downstream node  504  may initiate its own synchronization request with bit  5  of octet  3  set to “0” and bit  6  of octet  3  set to “1,” so as to request the H-TDM operational mode. In either case, if the upstream node  502  does not support the requested feature, e.g. the higher clock quality or the H-TDM operational mode, the upstream node  502  will not acknowledge the downstream node&#39;s request. After a predetermined time of not receiving an acknowledgement to its request, the downstream node  504  will determine that the upstream node  502  does not support the requested feature, and will acknowledge the original synchronization request  506 . 
         [0048]      FIG. 5B  illustrates a timeline of one embodiment of the frequency-synchronized communication between the upstream node  502  and the downstream node  504 . Specifically,  FIG. 5B  contains a separate timeline for the upstream node  502  and the downstream node  504 , where the timelines represent time relative to the corresponding node. The time relative to the upstream node  502  is shown above the timeline for the upstream node  502  and indicated by T Ux , where x is an integer. Similarly, the time relative to the downstream node  504  is shown below the timeline for the downstream node  504  and indicated by T Dx , where x is an integer. An absolute time is shown between the two timelines and indicated by Tx, where x is an integer. The time relative to the upstream node  502  and the time relative to the downstream node  504  may not necessarily be equal to the absolute time. 
         [0049]    At time T 1 , the upstream node  502  may communicate the synchronization request  506  to the downstream node  504  and initiate a synchronization window. The synchronization request may include a timestamp that indicates the time that the synchronization request  506  was transmitted from the upstream node  502 . Because the timestamp was created by the upstream node  502 , the timestamp indicates the synchronization request transmission time relative to the upstream node  502 , e.g. time T U1 . Concurrent with the transmission of the synchronization request  506 , the upstream node  502  initiates an upstream synchronization window, U Sync  Window, with a predetermined period, such as 125 μs. Upon initiating the U Sync  Window, time in the upstream node  502  may be measured relative to start of the current U Sync  Window. Specifically, a new U sync  Window is initiated after every predetermined period, e.g. at each of times T U2 , T U3 , and T U4 . 
         [0050]    At time T 2 , the downstream node  504  receives the synchronization request  506  and performs various actions. When the synchronization request  506  is received, the downstream node  504  may synchronize its clock to the timestamp in the synchronization request  506 , e.g. time T U1 . For example, the downstream node clock may be reset such that time relative to the downstream node  504 , T D1 , is equal to T U1 . In addition to synchronizing its clock, the downstream node  504  may initiate a downstream synchronization window D Sync  Window, with the same predetermined period as the U Sync  Window. Upon initiating the D Sync  Window, time in the downstream node  504  may be measured relative to start of the current D Sync  Window. Specifically, a new D sync  Window is initiated after every predetermined period, e.g. at each of times T D2 , T D3 , and so forth. Thus, the D Sync  Windows are frequency-synchronized to the U Sync  Windows. Upon processing the synchronization request  506 , the downstream node  504  may transmit a synchronization acknowledgement  508  at time T 2 , thereby informing the upstream node  502  that the H-Sync operational mode has been successfully established. 
         [0051]    While the U Sync  Windows and the D Sync  Windows are frequency-synchronized, they are not necessarily phase-aligned. As shown in  FIG. 5B , there may be a transmission delay, D 1 , when communicating the synchronization request  506  from the upstream node  502  to the downstream node  504 . The transmission delay D 1  causes in the phase misalignment between the U Sync  Windows and the D Sync  Windows. For example, the first U Sync  Window may be started at time T 1 , whereas the first D Sync  Window may be started at time T 2 , which may be equal to T 1  plus D 1 . While a particular transmission delay D 1  may be depicted in  FIG. 5B , it may be contemplated that shorter or longer delays may occur between the upstream node  502  and the downstream node  504 . For example, the D Sync  Windows may be offset from the U Sync  Windows by less than a single synchronization period, by more than a single synchronization window, or by x synchronization periods, where x is a number such as an integer. 
         [0052]    The H-TDM operational mode is more complex than the H-Sync operational mode. Specifically, the H-TDM operational mode provides an overlay synchronous timeslot scheme that allows the H-TDM operational mode to transport various types of data over a standard Ethernet Physical Layer interface. The overlay synchronous timeslot scheme may support higher QoS levels than is possible with previous Ethernet solutions. Moreover, the H-TDM operational mode can be used to frequency-synchronize and phase-align two Ethernet nodes. Specifically, the H-TDM timestamp allows the upstream node and the downstream node to initiate, synchronize, and phase-align their synchronization windows, thereby increasing the precision of the communication between the upstream and downstream nodes. Further, the H-TDM operational mode is configured such that the integrity of the data is maintained when some of the data at the beginning or the end of the window is lost. 
         [0053]    The overlay synchronous timeslot scheme may allow the H-TDM frame to transport a variety of data types. When the synchronization window has a period of about 125 μs, each of the timeslots in the overlay synchronous timeslot scheme represents a single channel with about 64 kilobits per second (Kbps) of bandwidth. These channels provide sufficient bandwidth to carry a voice conversation compatible with the public switched telephone network (PSTN). Thus, voice channels that are carried in the overlay synchronous timeslot scheme in an H-TDM frame may be referred to as TDM traffic. The overlay synchronous timeslot scheme also provides octet-sized granularity for enabling communication of other traffic with stringent quality of service (QoS) requirements, referred to herein as High-Performance Flow (HPF) traffic. Examples of HPF traffic include video, audio, and other multimedia traffic. HPF traffic may be assigned multiple channels with single-octet granularity according with bandwidth requirements of the HPF traffic. In other words, each channel assigned to a HPF increases the bandwidth allocated to the HPF by 64 Kbps. For example, a low resolution streaming video HPF requiring about 256 Kbps of bandwidth may be assigned about four channels from the H-TDM frame. Similarly, a HPF requiring about 3.2 megabits per second (Mbps) of bandwidth may be assigned about fifty channels from the H-TDM frame. 
         [0054]      FIG. 6A  depicts one embodiment of the overlay synchronous timeslot scheme of the H-TDM frame. Specifically,  FIG. 6A  illustrates a 125 μs synchronization window containing an overlay synchronous timeslot scheme comprising a start of frame delimiter (SFD)  603 , a synchronization timestamp (Sync)  604 , a timeslot map (TS Map)  606 , and a payload  608 . The SFD  603  may delineate a beginning of the H-TDM frame, and may be a reserved Ethernet control symbol, such as the /K28.1/ control symbol. As one skilled in the art will recognize, the /K28.1/ control symbol includes a comma that may be used to enable 8 bit/10 bit (8 B/10 B) symbol synchronization when the overlay synchronous timeslot scheme is communicated on 8 B/10 B encoded media. In an embodiment, the SFD  603  may also specify the size of the overlay synchronous timeslot scheme. The Sync  604  follows the SFD  603 . As described below, the Sync  604  in the H-TDM operational mode differs from the Sync  202  in the H-Sync operational mode in that the Sync  604  may be used to initiate the synchronization windows, synchronize the synchronization windows, and phase-align the synchronization windows between two nodes. 
         [0055]    The overlay synchronous timeslot scheme may continue with the TS Map  606 , which may specify the type and location of the data in the payload  608 . In one embodiment, the individual timeslots for the payload  608  may be assigned to TDM, HPF, and BEP traffic according to a predefined pattern. For example, the first thousand timeslots may be assigned to TDM traffic, the next five thousand timeslots may be assigned to HPF traffic, and the next three thousand timeslots may be assigned to BEP traffic. In such an embodiment, the TS Map  606  may be omitted from the H-TDM frame if the nodes are aware of the predefined pattern. Alternatively, the TS Map  606  may indicate the allocation of each timeslot in the payload  608  as a TDM, a HPF, or a BEP timeslot. Using the TS Map  606 , TDM, HPF, and BEP traffic may be dynamically interleaved within the overlay synchronous timeslot scheme. A detailed description of the TS Map  606  and the dynamic interleaving of the TDM, HPF, and BEP traffic may be found in the aforementioned provisional patent applications. 
         [0056]    Some timeslots at the beginning and/or end of the synchronization window may be part of a guard interval  602 . The guard intervals  602  allow the H-TDM frame to float within the synchronization window. Specifically, the location of SFD  603  in relation to the start of the synchronization window may vary between synchronization windows. As such, the guard interval  602  at the beginning of the synchronization window may be the same or a different size than the guard interval  602  at the end of the synchronization window, and the size of the guard intervals  602  in one synchronization window may vary from the size of the guard intervals  602  in other synchronization windows. Such an embodiment may be advantageous because the integrity of the SFD  603 , Sync  604 , TS Map  606 , and the TDM or HPF data in the payload  608  is maintained if any of the data in the guard intervals  602  is dropped, corrupted, lost, or otherwise unreadable, for example, due to clock tolerances or other non-deterministic factors. In some embodiments, the guard interval  602  may transport low priority best-effort packet (BEP) data. Alternatively, the guard interval  602  may be zero-padded or may contain idle characters. 
         [0057]    Although the synchronization window may be any duration, there are particular advantages to using a synchronization window with a period of 125 μs. Specifically, synchronizing the overlay synchronous timeslot schemes to a 125 μs synchronization window enables the Ethernet nodes to be interoperable with the PSTN, SONET, SDH, and other TDM networks. As such, when the overlay synchronous timeslot scheme has a 125 μs window, SONET/SDH transport overhead may be added to the overlay synchronous timeslot scheme format.  FIG. 6B  illustrates an overlay synchronous timeslot scheme containing SONET/SDH transport overhead  610 . The SONET/SDH transport overhead  610  allows the data in the payload  608  to be efficiently mapped between Ethernet networks and the SONET/SDH networks used by the PSTN. The SONET/SDH transport overhead  610  is depicted as surrounding the Sync  604  because the Sync  604  may be inserted into undefined octets of the SONET/SDH transport overhead  610 . A detailed description of the mapping of the H-TDM frames between the Ethernet format and the SONET/SDH format may be found in the aforementioned provisional patent applications. 
         [0058]      FIG. 7A  illustrates a more detailed layout of the Sync  604  from  FIG. 6A .  FIG. 7A  contains three rows of information: an internal synchronization signal  702  that delineates the synchronization window, a timeline  704  that enumerates each timeslot, and a descriptor  706  that describes the data that may be contained within each timeslot. The internal synchronization signal  702  may correspond to the synchronization window established when initiating the H-Sync or H-TDM operational modes. Timeslots N,  0 , and all other timeslots prior to X represent the guard intervals  602  described above, and thus the descriptor  706  indicates that BEP traffic may be transported during these timeslots. At timeslot X, the SFD  603  may delineate the start of the H-TDM frame. The Sync  604  follows the SFD  603  and is shown in timeslots X+ 1  through X+ 14 . The first seven timeslots of the Sync  604 , timeslots X+ 1  through X+ 7 , provide a forward timestamp  708 , and the last seven timeslots of the Sync  604 , timeslots X+ 8  through X+ 14 , provide a loop-back timestamp  710 , each of which are described in detail below. The TS Map  606  may follow in timeslot X+ 15  and subsequent timeslots. In one embodiment, one or more idle octets or SONET/SDH transport overhead  610  octets may be inserted between timeslots X+ 7  and X+ 8 . Such octets enable efficient mapping of the Sync  604  to an SONET/SDH frame, such that the Sync  604  aligns with the columns of the SONET/SDH frame. 
         [0059]      FIG. 7B  illustrates a more detailed layout of the Sync  604  from  FIG. 6B . The information contained in  FIG. 7B  is similar to the information contained in  FIG. 7A . However, rather than immediately communicating the Sync  604 , the overlay synchronous timeslot scheme first communicates the SONET/SDH transport overhead  610 . When communicating the overlay synchronous timeslot scheme over 64 bit/66 bit (64 B/66 B) encoded media, there may be an offset between the SFD  603  and the beginning of the SONET/SDH transport overhead  610 , which may provide rapid and deterministic alignment to 64 B/66 B sync fields. As such, the overlay synchronous timeslot scheme may include a pointer  712  that follows the SFD  603  and points to the start of the SONET/SDH transport overhead  610 . BEP data may be communicated in the offset interval  714  between the pointer  712  and the start of the SONET/SDH transport overhead  610  at timeslot Y. Alternatively, idle data may be communicated in the offset interval  714 . The Sync  604  may be communicated within undefined octets of the SONET/SDH transport overhead  610 . As such, a first portion of the SONET/SDH transport overhead  610  may be communicated from timeslot Y to timeslot Z− 1 , and the Sync  604  may be communicated between timeslot Z through timeslot Z+ 13 . Starting at timeslot Z+ 14 , a second portion of the SONET/SDH transport overhead  610  may communicate the remainder of the SONET/SDH transport overhead  610 . 
         [0060]      FIG. 8  illustrates one embodiment of the forward timestamp  708 . Specifically,  FIG. 8  illustrates a seven-octet forward timestamp  708 , where each row represents an octet and each column represents the position of the bits within the octet. The octets may be arranged in order of significance from top to bottom such that the least significant octet, octet  0 , is at the top and the most significant octet, octet  6 , is at the bottom. Similarly, the bits may be arranged in order of significance from left to right, such that the LSB of each octet, bit  0 , is at the left and the MSB of each octet, bit  7 , is at the right. As is common in Ethernet communications, data may be communicated from the LSB to the MSB and from least significant octet to most significant octet, such that the bits are communicated row-by-row from left to right starting with the top octet and ending with the bottom octet. 
         [0061]    The first three octets of the forward timestamp  708  may specify a twenty-four bit timestamp, T-Sync. The layout and properties of the timestamp are similar to the timestamp in the Sync  202  described above. Also similar to the Sync  202 , the fourth octet, octet  3 , of the forward timestamp  708  may be a control octet that may be used to initiate or maintain synchronized communication between nodes. Specifically, the first five bits, bit  0  through bit  4 , indicate the quality of the clock of the node that is communicating the Sync  604 . While bits  5  and  6  of octet  3  are used to establish the H-TDM operational mode as described above, these bits may be left open once the H-TDM operational mode is established between the two nodes. Bit  7  of octet  3  may be used as a parity bit for verifying the integrity of the first four octets. 
         [0062]    In addition to the twenty-four bit clock used for the T-Sync and other timestamps, each node that supports the H-TDM operational mode may maintain a sixteen-bit clock that counts once at the end of each window. For example, with each bit representing 0.01 ns, a 125 μs synchronization window may be reached when the twenty-four bit digital clock reaches a value of “101111101011110000100000.” Each time the twenty-four bit digital clock reaches this value, it may reset, and the sixteen-bit digital clock may increment once. Thus, the nodes may use the sixteen-bit clock to identify the individual frames with a frame count timestamp, which is similar to an identification or serial number. 
         [0063]    The next two octets in the forward timestamp  708 , octet  4  and octet  5 , specify the frame count timestamp, T-FC. The T-FC may be used to account for multi-frame periods of delay in the communication between an upstream node and a downstream node as described below. With sixteen bits available for the T-FC, each bit may represent one 125 μs window for a total range of about 8.192 seconds. Octet  4  contains the LSBs of the sixteen-bit T-FC, which may be bits  00  through  07  located in bit  0  through bit  7 . Octet  5  contains the MSBs of the sixteen-bit T-FC, which may be bits  08  through  15  located in bit  0  through bit  7 . The last octet in the forward timestamp  708 , octet  6 , has the first seven bits, bit  0  through bit  6 , open, and uses the eighth bit, bit  7 , as a parity bit for verifying the integrity of the forward timestamp  708 . The loop-back timestamp  710  may be formatted similar to the forward timestamp  708 , but with different loop-back values for the timestamp and frame count as described below. 
         [0064]      FIG. 9A  illustrates an exemplary block diagram of the process for establishing the H-TDM operational mode. The process may begin when an upstream node  902  creates an upstream synchronization timestamp (U-Sync)  906  with the format of the Sync  604 . The upstream node  902  sets the U-Sync&#39;s forward timestamp  708  values equal to the relative time of the upstream node  902 . Specifically, the value of the upstream node&#39;s twenty-four bit clock at time T 1  may be inserted into the T-Sync field, and the value of the upstream node&#39;s sixteen-bit clock at time T 1  may be inserted into the T-FC field. Concurrently, the upstream node  902  may also internally record the T-Syncs associated with each T-FC, thereby creating a record of when each U-Sync  906  was transmitted. Zero values may be transmitted in the loop-back synchronization timestamp (L-Sync) and the loop-back frame count (L-FC) fields in the loop-back timestamp  710  on the first communication iteration of this process. The upstream node  902  may then transmit the U-Sync  906  to the downstream node  904  at time T 1 . A transmission delay, D 1 , may occur while the U-Sync  906  is being transported from the upstream node  902  to the downstream node  904 . 
         [0065]    At time T 2 , the downstream node  904  receives the U-Sync  906  and stores an internal timestamp indicating when the U-Sync  906  was received. Upon receiving the U-Sync  906 , the downstream node  904  processes the timestamp, which causes an internal processing delay, D 2 . As part of the processing, the downstream node  904  may create a downstream synchronization timestamp (D-Sync)  908  with a format similar to the Sync  604 . When creating the D-Sync  903 , the downstream node  904  may set the D-Sync&#39;s forward timestamp  708  values equal to the relative time of the downstream node  904 . Specifically, the value of the downstream node&#39;s twenty-four bit clock at time T 3  may be inserted into the T-Sync field, and the value of the downstream node&#39;s sixteen-bit clock at time T 3  may be inserted into the T-FC field. The downstream node  904  may also set the value of the L-Sync equal to the calculated internal processing delay, D 2 , which may be calculated by subtracting the internal timestamp that indicates when the U-Sync  906  was received from the internal timestamp at time T 3 . In addition, the downstream node  904  may set the value of the L-FC equal to the T-FC value in the U-Sync  906 . The downstream node  904  then transmits the D-Sync  908  to the upstream node  902  at time T 3 . A transmission delay, D 3 , occurs while the D-Sync  908  is being transported from the downstream node  904  to the upstream node  902 . In some embodiments, it may be assumed that there are symmetric transmission delays between the upstream node  902  and the downstream node  904 , such that the value of D 3  is equal to the value of D 1 . 
         [0066]    At time T 4 , the upstream node  902  receives the D-Sync  906  and calculates the transmission delay D 1 . As part of the calculation of the transmission delay D 1 , a total delay, D Total , is first calculated. The upstream node  902  can correlate the U-Sync  906  to the D-Sync  908  using the T-FC in the U-Sync  906  and the L-FC in the D-Sync  908 . Thus, the upstream node  902  can determine that the U-Sync  906  was transmitted at time T 1  and the corresponding D-Sync  908  was received at time T 4 . The total delay may be equal to the difference between time T 4  and T 1 , and may also be equal to the sum of the internal processing delay D 2  at the downstream node  904  and the two transport delays, D 1  and D 3 . This relationship is illustrated in equation 1: 
         [0000]        D   Total   =T 4 −T 1 =D 1 +D 2 +D 3.   (1) 
       Using the assumption that the value of D 3  is equal to the value of D 1 , the total delay is equal to twice the transmission delay D 1  plus the internal processing delay D 2 . This relationship is illustrated in equation 2: 
       [0067]        D   Total =(2* D 1)+ D 2.   (2) 
       To calculate the transmission delay D 1 , the internal processing delay D 2  that was received in the L-Sync of the D-Sync  906  may be subtracted from the calculated total delay and the result may be divided by two, as shown in equation 3: 
       [0068]        D 1=( D   Total   −D 2)/2.   (3) 
         [0069]    The upstream node  902  can use the transport delay D 1  to synchronize its clock with the downstream node&#39;s clock, if desired. Specifically, the upstream node  902  can use the transport delay D 1  and the forward timestamp  708  from the downstream node to synchronize the upstream node&#39;s clock with the downstream node&#39;s clock, and phase-align the upstream node&#39;s synchronization windows with the downstream node&#39;s synchronization windows. However, doing so does not inform the downstream node of the upstream node&#39;s processing delay and/or the existence of the frequency-synchronization and phase-alignment. Thus, the upstream node may send a second U-Sync  906  to the downstream node  904 . Specifically, at time T 5 , the upstream node  902  transmits the second U-Sync  906  with the forward timestamp  708  values equal to the relative time of the upstream node  902  at time T 5 , and the loop-back timestamp  710  equal to the calculated transmission delay D 1 . At time T 5 , the upstream node  902  may also store an internal timestamp indicating when the U-Sync  906  was transmitted for subsequent transmission delay calculations. Like before, the transmission delay D 1  occurs while the U-Sync  906  is being transported from the upstream node  902  to the downstream node  904 . 
         [0070]    At time T 6 , the downstream node  904  receives the U-Sync  906  and stores an internal timestamp indicating when the U-Sync  906  was received. Upon receiving the U-Sync  906 , the downstream node  904  internally processes the timestamp. As part of the internal processing, the downstream node&#39;s clock, D-Clock, may be reset to be equal to the value of the upstream node&#39;s clock  902 , U-Clock. This may be accomplished by setting the value of the downstream node clock  904  equal to the sum of the forward timestamp  708  and the loop-back timestamp received in the U-Sync  906 , as shown in equation 4: 
         [0000]        D -Clock= U -Clock= T 5 +D 1=forward timestamp+loop-back timestamp   (4) 
         [0071]    As mentioned above, the first communication iteration may have zero values for the L-Sync and the L-FC fields. In such a case, the downstream node  904  effectively sets the value of its clock equal to the forward timestamp  708 . Because the first communication iteration does not take into account the transmission delay, the downstream node  904  may be frequency-synchronized but may not be phase-aligned to the upstream node  902 , similar to the H-Sync operational mode. Alternatively, rather than resetting the downstream node clock  904 , an offset between the clock and the received timestamp may be recorded to enable the downstream node  904  to be frequency-synchronized and phase-aligned to multiple nodes. The process described above continues such that the upstream node  902  and the downstream node  904  adjust the frequency-synchronization and phase-alignment with every synchronization window when the upstream node  902  and the downstream node  904  operate in the H-TDM operational mode. 
         [0072]      FIG. 9B  illustrates a timeline of a successful initiation of frequency-synchronized and phase-aligned communication between the upstream node  902  and the downstream node  904 . Like  FIG. 5B ,  FIG. 9B  contains a separate timeline for each of the upstream node  902  and the downstream node  904 , where each timeline represents time relative to the corresponding node. Specifically, the time relative to the upstream node  902  may be shown above the timeline for the upstream node  902  and indicated by T Ux , where x may be an integer. The time relative to the downstream node  904  may be shown below the timeline for the downstream node  904  and indicated by T Dx  where x may be an integer. An absolute time may be shown between the two timelines and indicated by Tx, where x may be an integer. The time relative to the upstream node  902  and the time relative to the downstream node  904  may not necessarily be equal to the absolute time. 
         [0073]    At time T 1 , the upstream node  902  communicates the U-Sync  906  to the downstream node  904 . The U-Sync  906  includes a timestamp set to the relative time of the upstream node  902  when the U-Sync  906  is transmitted. In the exemplary timeline shown in  FIG. 9B , the forward timestamp  708  has the T-Sync set to the twenty-four bit clock time for T U1  and the T-FC set to the sixteen-bit clock time for T U1 . At time T 1 , concurrent with communicating the U-Sync  906 , the upstream node  902  stores the relative time T U1  for use in subsequently calculating a total delay as described above. 
         [0074]    As shown in the timeline for the downstream node  904 , there may be a transmission delay, D 1 , when communicating the U-Sync  906  from the upstream node  902  to the downstream node  904 . At time T 2 , the downstream node  904  receives the U-Sync  906 , processes the U-Sync  906 , and calculates the processing delay D 2 . At time T 3 , the downstream node  904  transmits the D-Sync  908 , which includes the forward timestamp  708  set to the relative time of the downstream node  904 . In the exemplary timeline shown in  FIG. 9B , the forward timestamp  708 , including the synchronization window timestamp and the frame count timestamp, may be set to time T D1 . The D-Sync  908  also includes a loop-back timestamp  710  with L-Sync set to the calculated internal processing delay D 2  and the L-FC set to the T-FC received in the U-Sync  906 . 
         [0075]    As shown in the timeline for the upstream node  902 , there may be a transmission delay D 3  when communicating the D-Sync  908  from the downstream node  904  to the upstream node  902 . At time T 4 , the upstream node  902  receives the D-Sync  908  and calculates the one-way transmission delay between the upstream node  902  and the downstream node  904 . At time T 5 , the upstream node transmits another U-Sync  906  that includes a timestamp set to the upstream node  902  relative time when the U-Sync  906  is transmitted. In the exemplary timeline shown in  FIG. 9B , the forward timestamp  708  has the T-Sync and the T-FC set to time T U2 . The U-Sync  906  also has the loop-back timestamp  710  with L-Sync and L-FC set to indicate the calculated delay D 1 . 
         [0076]    As shown in the timeline for the downstream node  904 , there may be a transmission delay D 1  when communicating the U-Sync  906  from the upstream node  902  to the downstream node  904 . At time T 6 , the downstream node  904  receives and processes the U-Sync  906 . As part of the processing, the downstream node  904  sets the downstream node clock equal to the sum of the forward timestamp  708  and the loop-back timestamp  710 . As shown in  FIG. 9B , such a method of resetting the downstream node  904  clock aligns the synchronization windows of the upstream node  902  and the downstream node  904  in frequency and phase. In particular, the downstream node clock may be reset at time T 6  such that the relative time of the downstream node, T D2 , may be equal to the relative time of the upstream node, T U3 , which may be equal to the sum of the forward timestamp  708  of the U-Sync  906 , T U2 , and the delay D 1 . 
         [0077]      FIG. 10  illustrates an exemplary functional block diagram of a clocking architecture for supporting the frequency-synchronized communication in the H-Sync operational mode and the frequency-synchronized and phase-aligned communication in the H-TDM operational mode. As shown in  FIG. 10 , a network node  1000  may have a Physical Layer receiver (PHY Rx) interface  1002 , a Physical Layer transmitter (PHY Tx) interface  1004 , and a Media Access Control Layer (MAC) processor  1006 . The PHY Rx interface  1002  receives data from a communication medium, and the PHY Tx interface  1004  transmits data to a communication medium. While a single PHY Rx interface  1002  and PHY Tx interface  1004  pair may be shown on the network node  1000 , it may be contemplated that a plurality of PHY Rx interface  1002  and PHY Tx interface  1004  pairs may be implemented on the network node  1000 . Each PHY Rx interface  1002  and PHY Tx interface  1004  pair may represent one port for supporting full-duplex communication with other network nodes. 
         [0078]    The MAC processor  1006  interprets and frames data received on communication link  1008  from the PHY Rx interface  1002  for use by other logic (not shown), e.g. switch fabric, on the network node  1000 . The MAC processor  1006  also transmits frames of data received from the other logic on the network node  1000 , and communicates the data on communication link  1010  to be transmitted by the PHY Tx interface  1004 . In an embodiment, the data processed by the MAC processor  1006  may be formatted in accordance with a standard Ethernet packet or in accordance with one of the frame formats illustrated in  FIGS. 6A and 6B . 
         [0079]    The network node  1000  may use a clock reference  1012  to establish an internal synchronization window clock  1014 . The clock reference  1012  may be an internal reference such as a voltage-controlled crystal oscillator or some other oscillator that provides a reference clock with an accuracy of ± about 20 ppm. The clock reference  1012  may also be an external reference such as a clock signal received from external logic. For example, if the network node  1000  was implemented as a line card, then the network node  1000  may receive the clock reference  1012  from a switching fabric with which the network node  1000  is connected. The clock reference  1012  may also be an external reference that may be a distributed clock signal such as a building integrated timing supply (BITS). 
         [0080]    As mentioned above, the internal synchronization window clock  1014  may be implemented as a twenty-four bit digital clock that measures time within a synchronization window. Each time the internal synchronization window clock  1014  reaches the end of the synchronization window, it may reset to begin tracking time in a subsequent synchronization window. The internal frame count  1016  may be implemented as a sixteen bit digital clock that may be incremented once at the end of each synchronization window. Together, the value of the internal synchronization window clock  1014  and the value of the internal frame count  1016  make up an internal timestamp that may be used as described below. 
         [0081]    When receiving data on the PHY Rx interface  1002 , a new data frame may be indicated with a start of frame delimiter (SFD). The network node  1000  may have a receiving (Rx) timestamp generator  1018  that may use the values of the internal synchronization window clock  1014  and the value of the internal frame count  1016  to generate an Rx timestamp that indicates when the new data frame was received. The value of the Rx timestamp may be used by a delay calculator  1038  and a timestamp comparator  1040  as described in detail below. Subsequent to the SFD, a Sync  604  may be received by the PHY RX interface  1002 . The Sync  604  includes a forward timestamp  708  containing a T-Sync value and a T-FC value, and a loop-back timestamp  710  containing an L-Sync value and an L-FC value. The internal node may extract each of these values with a T-FC Extractor  1020 , an L-FC Extractor  1022 , a T-Sync Extractor  1024 , and an L-Sync Extractor  1026 . 
         [0082]    As discussed in detail above, the downstream node  904  operating in the H-TDM operational mode may reset their clock with the sum of the forward timestamp  708  and the loop-back timestamp  710  received from the upstream node  902 . If the network node  1000  is operating as the downstream node  904 , then the values of the T-FC Extractor  1020  and the L-FC Extractor  1022  may be input to a frame count (FC) sum  1028 , thereby generating a sum of the two FC values. The sum value generated by the FC sum  1028  may be used to update the value of the internal frame count  1016  of the network node  1000 . Similarly, the values of the T-Sync Extractor  1024  and the L-Sync Extractor  1026  may be input to a synchronization (Sync) sum  1030 . The sum value generated by the Sync sum  1030  may be used to update the value of the internal synchronization window clock  1014  of the downstream node  904 . 
         [0083]    When the network node  1000  is operating in the H-Sync operational mode, the values of the L-FC Extractor  1022  and the L-Sync Extractor  1026  may be initialized to zero. As discussed above, the Sync  202  only communicates values that may be extracted by the T-FC Extractor  1020  and the T-Sync Extractor  1024 . As such, when the network node  1000  is operating as the downstream node  904  in the H-Sync operational mode, the internal frame count  1016  and the internal synchronization window  1014  may be updated with the values of the T-FC Extractor  1020  and the T-Sync Extractor  1024 , respectively. When the network node  1000  is not operating as the downstream node  904 , then the FC sum  1028  and the Sync sum  1030  may be disabled such that the internal frame count  1016  and the internal synchronization window clock  1014  are not updated. 
         [0084]    When transmitting data on the PHY Tx interface  1004 , a new data frame may be indicated with a SFD. The network node  1000  may have a transmitting (Tx) timestamp generator  1032  that uses the values of the internal synchronization window clock  1014  and the internal frame count  1016  to generate a Tx timestamp that indicates when a new data frame is transmitted. The value of the Tx timestamp may be input to a selector  1034  discussed in detail below. 
         [0085]    When the network node  1000  is operating as the upstream node  902  in the H-TDM operational mode, the value of the Tx timestamp may also be input to a Tx timestamp memory  1036 . When the upstream node transmits the U-Sync  906 , the timestamp that indicates when the transmission occurs may be stored at the upstream node  902 . When the upstream node  904  receives the D-Sync  908  from the downstream node  904 , the value of the L-FC in the D-Sync  908  may be used to reference the timestamp with which the D-Sync  908  corresponds. When the network node  1000  is operating as the upstream node  904  in the H-TDM operational mode, the value of the L-FC Extractor  1022  may be input to the Tx timestamp memory  1036  and used as a reference when reading the timestamp that indicates when the corresponding U-Sync  906  was transmitted. 
         [0086]    The value read from the Tx timestamp memory  1036  may be input to a delay calculator  1038 . The delay calculator  1038  also receives inputs from the Rx timestamp generator  1018  and the L-Sync Extractor  1026 . As part of calculating the one-way transmission delay, a total delay is first calculated. The delay calculator  1038  may calculate the total delay by subtracting the value of the Rx timestamp generator  1018  from the value read from the Tx timestamp memory  1036 . The value of the Rx timestamp generator  1018  corresponds to the time that the D-Sync  908  may be received at the network node  1000  when it is implemented as the upstream node  902 . The delay calculator may then calculate the two-way transmission delay by subtracting the value of the L-Sync Extractor  1026  from the result of the previous subtraction. The value of the L-Sync Extractor  1026  corresponds with the internal delay, D 2 , calculated by the downstream node  908 . The delay calculator  1038  may then determine the one-way transmission delay by dividing the result of the subtraction by two. The delay calculated by the delay calculator may be input to a selector  1042 , discussed in more detail below. 
         [0087]    When the network node  1000  is operating as the downstream node  904  in the H-TDM operational mode, the value of the Tx timestamp may also be input to a timestamp comparator  1040 . The timestamp comparator  1040  receives inputs from the Rx timestamp generator  1018  and the Tx timestamp generator  1032 . The downstream node  904  may then calculate the internal processing delay, D 2 , and the timestamp comparator  1040  outputs a difference between the Rx timestamp and the Tx timestamp to determine the internal processing delay, D 2 . The difference calculated by the timestamp comparator  1040  may be input to the selector  1042 . 
         [0088]    The selector  1042  may be used to select which values are input to an L-Sync Inserter  1048  and an L-FC Inserter  1050 . When the network node  1000  operates as the downstream node  904 , the selector  1042  may select the timestamp comparator  1040  output for the value in the L-Sync Inserter  1048 , and may select the T-FC Extractor  1020  value from communication link  1044  for the value of the L-FC Inserter  1050 . When the network node  1000  operates as the upstream node  904 , the selector  1042  may select the delay calculator  1038  output for the value of the L-Sync Inserter  1048  and the L-FC Inserter  1050 . 
         [0089]    Similarly, the selector  1034  may be used to select which values are input to a T-Sync Inserter  1052  and a T-FC Inserter  1054 . When the network node  1000  operates in the H-TDM operational mode, the selector  1034  may select the Tx timestamp generator  1032  output. When the network node  1000  operates as the downstream node  904  in the H-Sync operational mode, the selector  1034  may select the T-Sync Extractor  1024  value from the communication link  1046  for the value of the T-Sync Inserter  1052 . When the network node  1000  operates as the upstream node  902  in the H-Sync operational mode, the selector  1034  may select the Tx timestamp generator  1032  output for the value of the T-Sync Inserter  1052 . 
         [0090]    A Tx selector  1056  receives the values held in each of the T-Sync Inserter  1052 , T-FC Inserter  1054 , L-Sync Inserter  1048 , L-FC Inserter  1050 , and data communicated on communication link  1010 , and selects each in succession for transmission by the PHY Tx interface  1004 . In an embodiment, the Tx selector  1056  selects each of the values such that the PHY Tx interface  1004  transmits data according to one of the formats shown in  FIG. 6A  or  6 B. When the network node  1000  operates as the upstream node in the H-Sync operational mode, the Tx selector  1056  may select the communication link  1010  for transmitting standard Ethernet packets, and may select the value held in the T-Sync Inserter  1052  during the IPG  102  to transmit the Sync  202 . 
         [0091]    One skilled in the art will recognize that the network node  1000  may be implemented as one or a combination of a plurality of application specific integrated circuits (ASICs) or implemented using general purpose computing, described in detail below. In an embodiment, the synchronization functions of the network node  1000  may be implemented for each port of a line card or other network interface card. Implementing the synchronization functions at each port enables each port of a line card or other network interface card to be asynchronously synchronized to a plurality of different upstream and downstream network nodes at the same time. In other words, each port may have an internal synchronization window clock  1014  and internal frame count  1016  synchronized with another network node, but the internal synchronization window clock  1014  and internal frame count  1016  does not need to be synchronized on all ports. 
         [0092]    In another embodiment, the synchronization functions of the network node  1000  may be implemented to synchronize all ports of a line card or other network interface card. In this embodiment, all of the ports on the line card or other network interface card may be synchronized to a single upstream network node. In a further embodiment, the synchronization functions of the network node  1000  may be implemented on a switching fabric or other higher-level hardware to synchronize communication of one or more line cards or other network interface cards. 
         [0093]    The upstream node  502 , the downstream node  504 , the upstream node  902 , the downstream node  904 , and the network node  1000  described above may be implemented on any general-purpose computer with sufficient processing power, memory resources, and network throughput capability to handle the necessary workload placed upon it.  FIG. 11  illustrates a typical, general-purpose computer system suitable for implementing one or more embodiments disclosed herein. The computer system  1100  includes a processor  1102  (which may be referred to as a central processor unit or CPU) that may be in communication with memory devices including secondary storage  1104 , read only memory (ROM)  1106 , random access memory (RAM)  1108 , input/output (I/O)  1110  devices, and network connectivity devices  1112 . The processor may be implemented as one or more CPU chips. 
         [0094]    The secondary storage  1104  may be typically comprised of one or more disk drives or tape drives and may be used for non-volatile storage of data and as an over-flow data storage device if RAM  1108  may not be large enough to hold all working data. Secondary storage  1104  may be used to store programs that are loaded into RAM  1108  when such programs are selected for execution. The ROM  1106  may be used to store instructions and perhaps data, which are read during program execution. ROM  1106  may be a non-volatile memory device, which typically has a small memory capacity relative to the larger memory capacity of secondary storage. The RAM  1108  may be used to store volatile data and perhaps to store instructions. Access to both ROM  1106  and RAM  1108  may be typically faster than to secondary storage  1104 . 
         [0095]    I/O  1110  devices may include printers, video monitors, liquid crystal displays (LCDs), touch screen displays, keyboards, keypads, switches, dials, mice, track balls, voice recognizers, card readers, paper tape readers, or other well-known input devices. The network connectivity devices  1112  may take the form of modems, modem banks, Ethernet cards, universal serial bus (USB) interface cards, serial interfaces, token ring cards, fiber distributed data interface (FDDI) cards, wireless local area network (WLAN) cards, radio transceiver cards such as code division multiple access (CDMA) and/or global system for mobile communications (GSM) radio transceiver cards, and other well-known network devices. These network connectivity devices  1112  may enable the processor  1102  to communicate with an Internet or one or more intranets. With such a network connection, it may be contemplated that the processor  1102  might receive information from the network, or might output information to the network in the course of performing the above-described method steps. Such information, which may be often represented as a sequence of instructions to be executed using processor  1102 , may be received from and outputted to the network, for example, in the form of a computer data signal embodied in a carrier wave 
         [0096]    Such information, which may include data or instructions to be executed using processor  1102  for example, may be received from and outputted to the network, for example, in the form of a computer data base band signal or signal embodied in a carrier wave. The base band signal or signal embodied in the carrier wave generated by the network connectivity devices  1112  may propagate in or on the surface of electrical conductors, in coaxial cables, in waveguides, in optical media, for example optical fiber, or in the air or free space. The information contained in the base band signal or signal embedded in the carrier wave may be ordered according to different sequences, as may be desirable for either processing or generating the information or transmitting or receiving the information. The base band signal or signal embedded in the carrier wave, or other types of signals currently used or hereafter developed, referred to herein as the transmission medium, may be generated according to several methods well known to one skilled in the art. 
         [0097]    The processor  1102  executes instructions, codes, computer programs, scripts which it accesses from hard disk, floppy disk, optical disk (these various disk based systems may all be considered secondary storage  1104 ), ROM  1106 , RAM  1108 , or the network connectivity devices  1112 . 
         [0098]    While several embodiments have been provided in the present disclosure, it should be understood that the disclosed systems and methods might be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The present examples are to be considered as illustrative and not restrictive, and the intention is not to be limited to the details given herein. For example, the various elements or components may be combined or integrated in another system or certain features may be omitted, or not implemented. In addition, persons of ordinary skill in the art will appreciate that the term octet as used herein is synonymous with the term byte, and that the octets described herein do not necessarily have to contain eight bits. 
         [0099]    In addition, techniques, systems, subsystems, and methods described and illustrated in the various embodiments as discrete or separate may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present disclosure. Other items shown or discussed as coupled or directly coupled or communicating with each other may be indirectly coupled or communicating through some interface, device, or intermediate component whether electrically, mechanically, or otherwise. Other examples of changes, substitutions, and alterations are ascertainable by one skilled in the art and could be made without departing from the spirit and scope disclosed herein.