Abstract:
The invention relates to data communications, more particularly to a protocol encapsulation technique which captures the time between frames at a transmitting site for comparison with an idle counter at a receiving site. At the originating site, the time between successive frames in the native protocol is encoded in an interframe time field associated with each respective frame, before the time encoded frame is mapped into the protocol associated with the transport medium. At the destination site, the framing associated with the transport medium is removed to reveal the interframe time field. The value contained in this field is compared with the value of an idle counter running at the native protocol rate. When the idle count is greater than or equal to the encoded inter-frame time, the data contained in the frame is then delivered to the client.

Description:
FIELD OF THE INVENTION 
   The invention relates to data communications, more particularly to a method of signalling across a transport medium the idle time between data frames integral to a native protocol associated with a pair of communicating devices. 
   BACKGROUND TO THE INVENTION 
   Modern enterprises are often geographically dispersed, with servers, data storage facilities and other communications devices interconnected through a metropolitan or wide area network (MAN/WAN). Enterprises typically rely on three main types of MAN/WAN infrastructures:
         a) IP/Ethernet networks;   b) Dense wavelength division multiplexing (DWDM) networks; and   c) Synchronous optical networks (SONET).       

   Enterprises often require data transmission solutions that are able to support a variety of protocols, individually and concurrently. For example, a typical enterprise may need to transport Single Byte Command Code Sets Connection (SBCON) and Gigabit Ethernet (GbE) formatted data in support of particular applications. As depicted in  FIG. 1 , SONET/SDH network  2  connects multiplexers  4 ,  6  to SBCON and GbE to switches  8 ,  10  which in turn connect to client devices  12 ,  14 , performing storage functions. The multiplexers/demultiplexers  4 ,  6  such as Akara Corporation&#39;s Optical Utility Services Platform (OUSP)™ multiplex/demultiplex client data protocols such as Fibre Channel (FC), Fibre Connection (FICON), Gigabit Ethernet (GbE), Enterprise System Connection (ESCON) and Single Byte Command Code Sets Connection (SBCON) directly into, for example, a SONET/SDH OC-3, OC-12, or OC-48 payload and transports it natively using SONET/SDH network  2  between geographically dispersed data centres A and B. 
   It will be understood by those in the art that frame based protocols such as Ethernet and SBCON, use idles between consecutive frames to both fill quiet time and guarantee a minimum time between frames. Such minimum inter-frame times may be required, for example, to ensure that the receiving device can process the current frame before subsequent frames are transmitted, in order to avoid receive buffer overflow resulting in data loss. 
   It will also be understood by those skilled in the art that some frame based protocols such as the American National Standards Institute (ANSI) standard interface protocol SBCON, use minimum spacing between frames which is negotiated between communicating pairs of end systems. This is referred to as time-based pacing. The pacing parameters are negotiated during initialization between the communicating devices. Devices used to transport multiple pairs of communicating devices across a single transport medium, such as a MAN/WAN link, may have to preserve different sets of pacing parameters for each communicating pair of end systems. As will be also appreciated by those in the art, time based pacing is one flow control technique associated with the data link layer defined in the Open Systems Interconnection (OSI) Reference Model developed by the International Organization for Standardization (ISO). A sublayer of the data link layer, the logical link control sublayer, is responsible for, among other things, controlling frame synchronization, error checking and flow control. Time based pacing is one of two primary flow control techniques, the other being credit-based flow control, whereby a data link regulates the flow of traffic by sending credit tokens to sources. 
   Either participating in, or snooping, the initialization procedure to determine potentially many different pacing parameters can be quite complicated and introduce undesirable system costs as well as additional transport latency. In addition, maintaining pacing information across transport media which transport information at rates different than the native protocol rate increases the complexity of the implementation. 
   At the present time, known solutions to this problem either terminate the protocol prior to the transport medium or provide a transport medium that is synchronous and transparent to the protocol. Devices which terminate the protocol are very complex and therefore expensive, since they must be application aware. Such implementations can require device spoofing and protocol translation. Further, processing capacity and cost places limits on the number of communicating devices that can be supported. Latency also increases due to the protocol interactions at both the entry and exit points of the transport systems. 
   Devices which provide synchronous protocol transparency such as DWDM, eliminate the complexities of protocol termination but they prevent transmission at transport rates other than those of the native protocol. This significantly reduces the connectivity options available. 
   There is a need, therefore, for a simple, low cost and reliable method for preserving data link layer (i.e. flow control) pacing information between clients communicating natively at rates other than the transfer rate of the transport medium. 
   SUMMARY OF THE INVENTION 
   The invention provides a device for and method of preserving pacing information through a simple protocol encapsulation technique which captures the time between frames for comparison with an inter-frame time counter (e.g. implemented using an idle counter) at the receiving destination. At the originating site, as part of the encapsulation process, the time between successive frames in the native protocol is encoded in an inter-frame time field associated with each respective frame, before the encapsulated frame is mapped into the protocol associated with the transport medium. At the destination site, the framing associated with the transport medium is removed to reveal the inter-frame time field. The value contained in this field is compared with an inter-frame time represented by the value of an idle counter running at the native protocol rate. When the inter-frame time represented by the idle count is greater than or equal to the inter-frame time encoded the inter-frame time field, the data contained in the frame is then delivered to the client. 
   In accordance with one aspect of the invention there is provided a method of preserving data link layer pacing information across a transport medium comprising: receiving serial data at a deserializer from a client in a first protocol format at a first rate; recovering a clock signal from said received serial data; converting the received serial data to parallel data and inputting the parallel data to a FIFO queue; detecting the end of a preceding client frame at a receive controller and clearing an inter-frame time counter having an associated count; incrementing the inter-frame time counter until the start of a current client frame is received; detecting the start of the current client frame; inserting in the current client frame a start of frame delimiter followed by the associated count of the incremented inter-frame time counter; optionally sending the current client frame to a receive buffer; inserting an end of frame delimiter when the end of the current frame is detected; extracting the current client frame from the receive buffer to a mapper; and mapping the current client frame into a protocol format associated with the transport medium, and serializing the mapped frame for transmission across the transport medium at a second data rate. 
   In accordance with another aspect of the invention there is provided a method of preserving data link layer pacing information across a transport medium carrying client data, wherein, prior to transmission over the transport medium, each frame of the client data is encoded with an inter-frame time field indicating the time between the start of a current frame and the end of a preceding frame, the method comprising: receiving serial data into a deframer at a destination site connected to the transport medium; converting the serial data to parallel data; removing any framing associated with the transport medium; decoding the parallel data at a de-mapper and placing the parallel data in a transmit buffer; incrementing a transmitter idle counter; detecting a frame in a transmit buffer with a transmit controller; extracting a start of frame delimiter field and the inter-frame time field from the detected frame; comparing an inter-frame time represented by the incrementing transmitter idle counter to the inter-frame time field; if the time count captured in the transmitter idle counter is greater than or equal to the encoded time contained in the inter-frame time field, feeding the frame to a serializer and removing an end of frame delimiter; and serializing the frame and sending said frame to a client, wherein the client data is associated with a first protocol format and is transmitted at a first data rate; and wherein the transport medium is associated with a second protocol format and operates at a second data rate. 
   The advantages of the present invention are numerous. The invention provides a simple, reliable and low cost method for preserving pacing information among any number of device pairs communicating across a transport medium at any rate. Since the pacing information is signalled in time units rather than transmission units, the method works equally well with transport rates which are either greater or lesser than the native protocol. Further, when this method is used with a transport rate which is less than the protocol rate, bandwidth efficiency can be gained since many idles at the protocol rate can be communicated across the transport medium with a single inter-frame time parameter. In addition, the simplicity of this method can significantly reduce the cost and improve the reliability of networks which transport the protocols benefiting from this invention. Finally, the low latency and transparency offered by this invention at the protocol layers and above, can enable applications which were previously not suitable for economical transport over a MAN/WAN. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
     A better understanding of the invention will be obtained by considering the detailed description below, with reference to the following drawings in which: 
       FIG. 1  depicts a network diagram of the prior art in which the present invention could be implemented; 
       FIG. 2A  depicts the method of frame encapsulation integral to the present invention; 
       FIG. 2B  depicts a typical frame sequence in which the present invention is used; 
       FIG. 3A  depicts a block diagram of a receiver used in the present invention; 
       FIG. 3B  depicts a flow chart detailing the steps of the receiver operation; 
       FIG. 4A  depicts a block diagram of a transmitter used in the present invention; and 
       FIG. 4B  depicts a flow chart detailing the steps of the transmitter operation. 
   

   DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
   In essence, the invention describes a protocol encapsulation technique that can be used to signal across a transport medium the time between when a preceding frame was transmitted and a current frame was received. For the purposes of the description, the terms transmit and receive are used to describe data flow with respect to a client connected to a transport medium. For the purposes of illustration, an implementation using SONET as the transport medium is described, although it will be understood by those skilled in the art that the invention could also be used in any MAN/WAN network. 
   Referring to  FIG. 2A , a start delimiter  15  and an end delimiter  16  are inserted when the protocol frame  17  is received from the client at an originating end. These delimiters are removed prior to transmission to the client at the destination end, as will be explained. The delimiters  15 ,  16 , are used to signal frame boundaries during transmission over the transport medium. Integral to the invention, an inter-frame time field  18  is inserted, indicating the time between when the start of the present frame and the end of the previous frame were received (i.e. the inter-frame time). It will be understood by those skilled in the art that protocol frame  17  could include an SBCON frame, but the invention is applicable to any data link layer protocol based on pacing. Referring to  FIG. 2B  and as will be explained in more detail below, as protocol frames  17  arrive from the client over time T, the inter-frame time field  18  is encoded with the inter-frame time  19 . The encoded frames are then mapped to a protocol frame  20  associated with a transport medium over which the client data will flow. 
   The invention comprises receive and transmit components, both of which are contained in devices at the origination and destination points respectively. The receive operation is performed by a device at the originating entity, while the transmit operation is performed by a device at the destination entity. In the event that data was transmitted in the opposite direction, the receive and transmit operations would be reversed. A device such as the OUSP™ multiplexer/demultiplexer offered by AKARA corporation could be used to perform the receiver/transmitter functions, but the functionality offered by the invention is not restricted to use in such a device. 
   The receive operation will now be described in relation to  FIGS. 3A and 3B . At step  100 , a deserializer  24  receives serial data from a client located at an originating site. The deserializer recovers a clock signal and converts the receive data from serial to parallel. The clock signal is used simply to determine character boundaries used in the local parallel processor of the receiver (i.e. the serial data is converted to simplify local processing). At step  102 , the receive data is then flowed to an input FIFO memory  26  which serves as an elastic buffer queueing the parallel receive data. 
   When the end of a client frame is detected at step  104  by the receiver controller  28 , the inter-frame time counter (IFT)  30  is cleared and then increments until the start of the next client frame is detected or until it reaches a maximum count (step  106 ). The maximum count is equal to the largest minimum inter-frame gap defined for the protocol. The IFT  30  counts at a rate equal to the protocol transmit time for a single idle character. 
   When the start of the next client frame is detected at step  108 , the receive controller  28  modifies the newly received frame with a start of frame delimiter  15  followed by an inter-frame time field  18  containing the value of the IFT  30  (step  109 ). This encoded frame then flows into the receive buffer  32 . As evident from the drawing, the FIFO  26  and the IFT  30  are connected through multiplexer  31  to the receive buffer  32 . It will be understood by those skilled in the art that receive buffer  32  is useful to perform many functions including rate adaptation and burst absorption, but need not be employed where data is to be streamed e.g. when the transport rate is greater than or equal to the protocol rate. When the end of the client frame is detected, the end of frame delimiter  16  is inserted. The mapper  34  then extracts the encoded frame at step  110  from the receive buffer  32 . Where the transport medium is a SONET network, the mapper  34  then encodes the frame in order to maintain character synchronization. This is required since SONET frame boundaries are arbitrary with respect to the encapsulated frame boundaries. It should be noted that the mapper  34  extraction rate could be greater or less than the native protocol rate i.e. there is no synchronization between the client and the transport medium. The frame is then placed into SONET format at the framer  36  as described, for example, in U.S. application Ser. No. 09/728,970 entitled Flexible Multiplexer/Demultiplexer and Method For Transport of Optical Line Data to a Wide/Metro Area Link filed by Bisson et al. on Dec. 4, 2000, the contents of which are herein incorporated by reference. The frames are then serialized at step  112  for transmission over the MAN/WAN at the transport rate of the medium. 
   As indicated above, when the inter-frame time is determined between a preceding and current frame, the time is encoded on the current frame. It will also be understood by those skilled in the art that alternately, the inter-frame time could be encoded on the preceding frame. This alternate technique would also accommodate the preservation of pacing information between clients communicating natively at rates other than the transfer rate of the transport medium and is meant to be included within the scope of the invention. 
   Upon initialization or reset, an additional procedure is required because the first frame arriving in the data stream from the client has not been encoded with an inter-frame time count. In this case, a default inter-frame time is assigned to the IFT  30 . The IFT  30  then increments until the first frame is received. The value of the counter is then encoded in the frame. 
   The transmit operation performed at the destination end will now be described in relation to  FIGS. 4A and 4B . The de-framer block  38  receives the serial data stream from the MAN/WAN at step  114 , converts it to parallel data and removes the SONET framing. The encoded data is then sent to the de-mapper  40  at step  116 . At step  118 , the de-mapper then decodes the data and places the encapsulated frames into the transmit buffer  42 . The transmitter idle counter (TIC)  44  is cleared when the end of frame delimiter is extracted from the transmit buffer  42  or after a reset. At step  120 , the TIC  44  increments until it reaches a maximum count (or is cleared again). The maximum count is equal to the largest minimum inter-frame gap defined for the protocol. The TIC  44  counts at a rate equal to the protocol transmit time for a single idle character. 
   When the transmit controller  46  detects a frame in the transmit buffer  42  at step  122 , it extracts the start of frame delimiter  15  and the inter-frame time encapsulation field  18  at step  124 . These fields do not get placed into the output FIFO  48  since they do not form part of the client data. At step  126 , the transmit controller  46  then compares an inter-frame time represented by the TIC  44  count to the extracted IFT  30  count (i.e. the inter-frame time  19  encoded in the inter-frame time field  18  during the receive operation). When the TIC  44  count is greater than or equal to the IFT  30  count, the frame is taken from the transmit buffer  42  and fed to the serializer  50  through the output FIFO  48 . When the end of frame delimiter  16  is detected, it is removed and not placed in the output FIFO  48 . At step  130 , the serialized frames are sent to the client. 
   The output FIFO  48  and the idle generator  52  are connected through a multiplexer  54  to the serializer  50 . The idle generator  52  is enabled through the multiplexer whenever there is no frame data to be sent. 
   It should also be understood that the receive and transmit operations described in the previous sections allow the inter-frame time gap to be preserved perfectly from client source to destination when the entire system is operating synchronously. When the reference clocks at the client receiver and transmitter are not synchronized, as would be the case in most implementations, the inter-frame time reproduced at the transmitter may have to be periodically reduced by one idle time less than received to adjust the transmit rate. This is required to avoid potential overflows at the transmit buffer which can occur when the transmit reference clock is operating at a lower rate than the receive reference clock. To overcome this problem, a solution well known in the art is implemented—the fill level of the transmit buffer is monitored to determine when inter-frame time reduction is required. 
   It is also important to appreciate that the insertion of an inter-frame time field  22  integral to this invention does not disrupt the application of the Generic Framing Procedure (GFP) outlined by the International Telecommunications Union (ITU) in recommendation G.7041/Y.1303, the contents of which is herein incorporated by reference. This recommendation defines a framing procedure to delineate octet-aligned variable-length payloads from higher layer client signals for mapping into octet synchronous networks (e.g. SONET). GFP consists of both common and client-specific aspects. Common aspects apply to all GFP adapted traffic. Client-specific aspects include two modes of client signal adaptation:
         a) PDU-oriented, referred to as Frame-Mapped GFP, for use with PDU-oriented signals (such as IP/PPP or Ethernet); and   b) Block-code oriented, referred to as Transparent GFP, for use with constant bit rate streams (such as Fibre Channel or ESCON/SBCON).
 
The invention described can in principle be used with both of the mapping paradigms above, while allowing for deletion (i.e. non-mapping/transport) of inter-frame idles. From a practical standpoint, the present invention serves to enhance the useful scope of GFP.
       

   Embodiments of the invention may be implemented in any conventional computer programming language. For example, preferred embodiments may be implemented in a procedural programming language (e.g. “C”) or an object oriented language (e.g. “C++”). Alternative embodiments of the invention may be implemented as pre-programmed hardware elements, other related components, or as a combination of hardware and software components. 
   Embodiments can be implemented as a computer program product for use with a computer system. Such implementation may include a series of computer instructions fixed either on a tangible medium, such as a computer readable medium (e.g., a diskette, CD-ROM, ROM, or fixed disk) or transmittable to a computer system, via a modem or other interface device, such as a communications adapter connected to a network over a medium. The medium may be either a tangible medium (e.g., optical or electrical communications lines) or a medium implemented with wireless techniques (e.g., microwave, infrared or other transmission techniques). The series of computer instructions embodies all or part of the functionality previously described herein. Those skilled in the art should appreciate that such computer instructions can be written in a number of programming languages for use with many computer architectures or operating systems. Furthermore, such instructions may be stored in any memory device, such as semiconductor, magnetic, optical or other memory devices, and may be transmitted using any communications technology, such as optical, infrared, microwave, or other transmission technologies. It is expected that such a computer program product may be distributed as a removable medium with accompanying printed or electronic documentation (e.g., shrink wrapped software), preloaded with a computer system (e.g., on system ROM or fixed disk), or distributed from a server over the network (e.g., the Internet or World Wide Web). Of course, some embodiments of the invention may be implemented as a combination of both software (e.g., a computer program product) and hardware. Still other embodiments of the invention may be implemented as entirely hardware, or entirely software (e.g., a computer program product). 
   Although various exemplary embodiments of the invention have been disclosed, it should be apparent to those skilled in the art that various changes and modifications can be made which will achieve some of the advantages of the invention without departing from the true scope of the invention. 
   A person understanding this invention may now conceive of alternative structures and embodiments or variations of the above all of which are intended to fall within the scope of the invention as defined in the claims that follow.