Method and system for adjusting network interface metrics

An approach is provided for adjusting network interface metrics to optimize transmissions across a network. A measurement of performance of a network interface is made, wherein the network interface is configured to provide traffic across an optical network. A determination is made regarding whether the measured performance satisfies a predetermined threshold. A network metric value of the network interface is adjusted if the measured performance of the network interface satisfies the predetermined threshold.

BACKGROUND OF THE INVENTION

Telecommunications networks have developed from connection-oriented, circuit-switched (CO-CS) systems, e.g., such as the public switched telephone network (PSTN), utilizing constant bit-rate, predefined point-to-point connections to connectionless, packet-switched (CNLS) systems, such as the Internet, utilizing dynamically configured routes characterized by one or more communication channels divided into arbitrary numbers of variable bit-rate channels. With the increase in demand for broadband communications and services, telecommunications service providers are beginning to integrate long-distance, large-capacity optical communication networks with these traditional CO-CS and CNLS systems. Typically, these optical communication networks utilize multiplexing transport techniques, such as time-division multiplexing (TDM), wavelength-division multiplexing (WDM), and the like, for transmitting information over optical fibers. However, an increase in demand for more flexible, resilient transport is driving optical communication networks toward high-speed, large-capacity packet-switching transmission techniques.

Such optical communication networks can experience significant fluctuations in traffic due to many factors, such as increases or decreases in the number of customers allocated to use the network and/or components of the network, changes in the structure of the network that increase of decrease capacity of the network and/or of components of the network, and fluctuations in usage that can be somewhat predictable or cyclical in nature, as well as random or bursty in nature. Certain dynamic performance changes that affect data flow along network interfaces may not be visible by the routers and other devices used to direct traffic through the network, and therefore the paths utilized by the routers to transfer data may not be optimal.

Therefore, there is a need for an approach that provides packet-based networks with efficient techniques for monitoring and adjusting router interface selection in response to changes in network interface performance.

DESCRIPTION OF THE PREFERRED EMBODIMENT

A preferred apparatus, method, and software for adjusting network interface metrics are described. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the preferred embodiments of the invention. It is apparent, however, that the preferred embodiments may be practiced without these specific details or with an equivalent arrangement. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the preferred embodiments of the invention.

Although various exemplary embodiments are described with respect to packet-switched networks, such as an Internet Protocol (IP) network, it is contemplated that various exemplary embodiments are applicable to other transport environments, and technologies.

FIG. 1is a schematic diagram of a communication system capable of adjusting network interface metrics for rerouting traffic, according to an exemplary embodiment For the purposes of illustration, a communication system100employs a network management system101to adjust network interface metrics and selection, for example, when communication nodes103and105communicate via data communication network107(e.g., Internet Protocol (IP) network). The nodes103,105can be an end-user device or network (e.g., local area network (LAN)).

In the embodiment depicted inFIG. 1, the data network107utilizes an optical transport network; however, the system100described herein can be utilized with other communication networks; for example, these communication networks may correspond to suitable wired and/or wireless networks providing, for instance, a local area network (LAN), metropolitan area network (MAN), wide area network (WAN), or combination thereof. Moreover, the communication network may be a backbone network of a service provider. As such, the communication network may operate as an asynchronous transfer mode (ATM) network, frame relay network, integrated services digital network (ISDN), IP network, multiprotocol label switching (MPLS) network, or SONET network, as well as any other suitable network, or combination thereof.

By way of example, network107inFIG. 1is a packet-switched (e.g., IP based) network configured for the transport of information (e.g., data, voice, video, etc.) between one or more sources (e.g., node103or node105) and one or more destinations (e.g., node105or node103). To assist with traffic engineering, service providers negotiate and apportion network capacity on general or subscriber-specific bases through service level agreements (SLA). These agreements define various communication service parameters in terms of bandwidth allocations, latency, jitter, etc.

Units of data (e.g., blocks, cells, frames, packets, etc.) transmitted across a transport environment are typically “policed” according to one or more committed rates of service, such as a committed burst bandwidth. These committed rates of service are generally associated with particular connection(s), e.g., links, pathways, etc., or other network parameters, e.g., incoming/outgoing interface, destination/source node, machine access control address, etc.

In the SONET network configuration shown inFIG. 1, the node103utilizes edge device109to connect to the IP network107, and the node105utilizes edge device111to connect to the IP network107. As seen inFIG. 1, edge devices109and111access IP network107via one or more routers (e.g., routers,113,115,117,119, etc.) by way of connections. Each router113,115,117, and119uses a respective optical node, e.g., add/drop multiplexer (ADM)121,123,125, and127, to send the data along the SONET ring, which in this embodiment includes connections129,131,133, and135.

Data units (e.g., blocks, cells, frames, packets, etc.) transported over IP network107and, thereby, between edge devices109and111, may traverse one or more other connections and/or nodes of IP network107. At the outset of the communication between node103and node105, a determination will be made regarding which router of routers113,115,117, and119will be an ingress router and which router of routers113,115,117, and119will be an egress router. The edge devices109and111may include traffic shapers configured to delay metered traffic according to one or more deterministic constraints (or rates of service), such as a maximum burst length (or size), maximum burst rate, sustainable burst length, sustainable burst rate, etc. It is noted that these traffic variables may be defined in terms of, for example, bandwidth allocations. Traffic shaping functions may be implemented by shaper through one or more buffers that temporarily “hold” and/or “schedule” units of data for transmission so that traffic shaper can disperse traffic as bandwidth becomes available on an outgoing connection.

In addition to the above determinations and on a different system layer than the above determinations, traffic flowing along an IP network107is routed along different flow paths (i.e., between different network interfaces) in an attempt to provide optimal utilization of the network capabilities. Traditionally, such physical layer determinations of the optimal flow path were based on network cost and/or metrics values that are preassigned between nodes of the network based on factors such as physical distance between nodes and other factors that determine normal latency between nodes. However, if a failure occurred at any point along the flow path on the SONET ring, then the SONET ADMs would simply transparently reroute traffic along a different path (e.g., a protection path) to achieve the same final destination as originally planned, without taking into consideration the increase in network cost/metrics involved with such a rerouting. An alternative system might perform constant monitoring of latency between nodes and constantly change paths based on such measurements. However, such a system would not provide a stable network as changes such as congestion would fluctuate relatively constantly, and therefore the network components would be greatly burdened by such changes.

The network management system101provides for monitoring and adjusting router interface selection to optimize data flow without overburdening the network components. The network management system101is configured to obtain measurements relating to flow data, such as latency, packet loss, and/or jitter, on a periodic basis (e.g., using International Telecommunications Union (ITU) Y.1731). ITU Y.1731 specifies mechanisms relating to the network and service aspects of ETH (Ethernet) layer. If the measurements fail to meet a minimum specification, then the network management system101reroutes traffic to an alternate path. The network management system101thus provides a method of collecting measurements latency, packet loss, and/or jitter on a frequent basis across network interfaces with configurable network cost/metrics values, and when measurements fail to meet a minimum specification, then changes the network cost/metrics value to a different preset value, thereby causing the control plane to reroute traffic to different path. This different path can be a more optimal route, in terms of costs and performance. Alternatively, the desirability of this different path can be based on a predetermined criteria used in conjunction with a rule or policy. In certain embodiments, the above functions for adjusting router interface selection can be performed within the network components themselves.

As depicted inFIG. 2, the network management module101includes a monitoring module201, and analysis module203, and a control module205. The network management system101also includes or can access a database207. The monitoring module201is configured to measure flow data in between the various nodes of the SONET ring, such as latency, packet loss, and/or jitter, on a periodic basis, the length of which can be set by balancing the need for taking enough data points in order to ensure that needed adjustments are made to optimize data flow, with the burden that such measurements and adjustments will place upon the components of the network. Thus, the monitoring module201is responsible for taking measurements and collecting measurement data for storage in the database and/or analysis by the analysis module203. The analysis module203performs the analysis of the measurement data to determine if a change needs to be made in the flow path, as will be described in greater detail below. The control module205notifies the routers113,115,117, and/or119of any changes to adjust the rerouting of data flow to another path (e.g., more optimal route) when needed.

The system100addresses the problem of measuring latency, packet loss, and/or jitter in hardware such that the measurements reported can reliably used to make a determination about the quality of the network being tested as opposed to the load of central processing units (CPUs) involved in the testing. A traditional way to implement this kind of testing is with IP pings, which are typically not implemented in hardware, and require a CPU at the far end to respond, which decreases accuracy and could cause a protection switch if the CPU get busy. ITU Y.1731 specifies a standard way to do the types of testing required at Layer 2 (or L2) in a manner that is better suited to hardware implementation than typical IP testing.

Additionally, the system100addresses the problem of configuring network interfaces (e.g., IP interfaces) running over protected optical (e.g., SONET) facilities. In this example, it is assumed that a working (or primary) path of the SONET network is shorter than the secondary path (or protect path). Conventionally, when the SONET facility switches to the protect path, the IP network is not aware of the change, so traffic is likely to stay on a suboptimal path.

By way of illustration,FIG. 3Adepicts a SONET ring where, if router metrics are proportional to distance, then traffic from router113to router119would normally go via router115if D1+D2<D3+D4. Thus, if the distance between router113and router115(i.e., D1) is one hundred miles, the distance between router115and router119(i.e., D2) is one hundred miles, the distance between router113and router117(i.e., D3) is two hundred miles, and the distance between router117and router119(i.e., D4) is one hundred miles, then traffic from router113to router119would travel from router113along connection129to router115and then from router115along connection133to router119, for a total path of two hundred miles. Such a path would be considered the primary path or working path.

However, in a traditional system, if there was a fiber cut301along connection129between router113and router115, as shown inFIG. 3B, the SONET ADMs121,123,125, and127would transparently reroute traffic from router113to router115through ADM125and AMD127. Such a rerouting would direct traffic from ADM121along connection131as path P1(equal in distance to D3) to ADM125, from ADM125along connection135as path P2(equal in distance to D4) to ADM127, from ADM127along connection133as path P3(equal in distance to D2) to ADM123, and then from ADM123back along connection133as path P4(equal in distance to D2) to ADM127to router119. Such rerouting (or redirection) of traffic would increase the distance travelled by the data from router113to router115to four hundred miles and the total distance travelled by the data from router113to router119to five hundred miles. This traversal may be unnecessary if other paths that are less costly can be utilized.

The network management system101provides a way to notify the routers113-119and the network control plane associated with the ADMs121-127of the change (e.g., the fiber cut301, etc.) so that traffic can be rerouted to the shortest path or optimal path. Effectively, the network management system101provides a means in which to monitor the network, analyze whether changes have occurred, and control rerouting of the traffic when needed. Thus, the network management system101can be used to make router113aware of the failure along connection129at301, so that router113can reroute traffic through router117, as shown inFIG. 3C. Accordingly, such a rerouting would direct traffic from ADM121along connection131as path P1to ADM125, and then from ADM125along connection135as path P2to ADM127to router119. Therefore, such a rerouting would decrease the total distance travelled by the data from five hundred miles inFIG. 3Bto three hundred miles inFIG. 3C.

Thus, the network management system101includes a monitoring module201that takes real time measurements of latency, packet loss, and/or jitter, for example, using Y.1731 implemented in hardware. Also, an analysis module203provides real time determinations of the network's ability to meet the minimum required latency, packet loss, and/or jitter. If a pre-programmed condition is met (e.g., 2 or more out of last 100 packets lost, or average latency of last 5 packets exceeds 20 ms), then a control module205can take action to reroute traffic to a different path, for example, by changing the interface cost/metric value on a network trunk or by signaling to the end user that the link no longer meets the user's requirements. By using an L2 method of generating test frames and measuring network characteristics, the interval between test packets can be reduced as low as required for the desired performance without risk of overrunning a general purpose CPU that has many other tasks running at any given time. In addition, using standard test measurements allows one-way testing to be done for interoperability testing, and ensures consistent operation across multiple vendors and equipment types, thereby easing configuration and appropriate metric selection.

In one embodiment, as shown inFIG. 4, the monitoring module201of the network management system101measures performance of the various network interfaces, in step401. For example, the monitoring module201can take real time measurements of latency, packet loss, and/or jitter between the various nodes. The analysis module203then analyzes the measured data to determine if the performance measurements have triggered a threshold network cost/metrics value in step403. For example, the analysis module203makes a real time determination of a correct network cost/metric based on a set of preset values and a defined threshold or threshold value in order to determine whether an adjustment of the network cost/metrics value(s) of one or more network interfaces needs to be made. If no threshold has been triggered, then the current network cost/metrics values are the network interfaces are maintained, as shown in step405. However, if one or more threshold values are triggered, then the control module205adjusts the network cost/metrics value(s) of the network interface(s) by a predetermined amount based on the threshold triggered, as shown inFIG. 407. Thus, the control module205effectively instructs the network control plane (not shown) to reroute traffic around network degradations.

For example, in a scenario where costs are based on latency and distance, each network interface could be assigned two costs (a low value and a high value) and a latency threshold over which cost should be changed from the low value to the high value. Thus, in the configuration shown inFIG. 1, two metrics between router113and115could be set at one hundred miles (e.g., which equals the distance D1along connection129) and four hundred miles (e.g., which equals the distances D3+D4+D2along connections131,135, and133, respectively), with a latency threshold of, for example, 7 ms round trip if normal latencies are 2 ms via connection129and 8 ms via connections131,135, and133. Thus, the routers could take latency measurements every 50 ms, which can be monitored by the monitoring module201, and if the analysis module203determines that a predetermined number of measurement(s) (e.g., three measurements in a row) are over the threshold latency of 7 ms, then the control module205can notify the routers of the change in metrics between router113and router115from one hundred miles to four hundred miles. Thus, the increase in latency between router113and router115to 8 ms and increase in the metric to 400 miles would cause the router113to reroute the traffic through router117in the manner shown inFIG. 3C.

FIG. 5depicts a process of routing traffic through a network using the network management system described above. In step501, each network interface is assigned plural predetermined network cost/metrics values, for example, that can be related to a working path value (e.g., the low value discussed above) and a protect path value (e.g., the high value discussed above). Then, in step503, the initial primary path used to send data from an ingress router to an egress router is selected using the normal network cost/metrics values (e.g., the working path values) of the various network interfaces along the various possible paths. For example, the routers would select the primary path based upon the lowest possible combined network cost/metrics values of the various combinations of network interfaces.

In step505, the network management system101would perform periodic, real-time measurements of the performance (e.g., latency, packet loss, jitter, etc.) of the network interfaces. Then, in step507, the network management system101would determine whether performance of any of the network interfaces trigger a threshold. For example, the threshold could be triggered if a certain value of the performance feature being measured is reached, or the threshold could be triggered if a certain value of the performance feature being measured is reached a certain number of times during a certain period of time or a certain number of consecutive times, etc. If the threshold has not been triggered, then in step509the current selected path from the ingress router to the egress router is maintained and the process loops back to step505. If the threshold has been triggered, then the process proceeds to step511.

In step511, a new network cost/metrics value is selected for the measured network interface for which a threshold was triggered. For example, the triggered network interface could be reassigned to have a protect path value (e.g., the high value discussed above). Then, in step513, the path from the ingress router to the egress router would be recalculated using currently assigned network cost/metrics values, which includes any new network cost/metrics value(s). In step515, the data traffic is then rerouted to the recalculated path. For example, the routers are notified of the updated network cost/metrics values and would then select a new primary path based upon the lowest possible combined network cost/metrics values of the various combinations of network interfaces, and reroute traffic to the new primary path. The process would then loop back to step505, where the network management system101would again measure performance of the network interfaces and make adjustments based on the set thresholds. For example, the system101could drop the network cost/metrics value of the previously adjusted interface back from the protect path value to the working path value if the measured performance warranted such an adjustment based on the threshold triggered, recalculate the path, and then reroute traffic to the recalculated path.

The above system101could also be applied to routers interconnected across trunks of variable bandwidth, in which high packet loss could be used to increase bandwidth-based metrics to compensate for a decrease in trunk bandwidth (e.g., sub-rate GE trunk bandwidth changes from OC12to OC3due to a failure). Also, the system could implement multiple measurements into a two-dimensional matrix used for metric selection or to abstract the metric selection to one of a number, n, of possible metrics based on n-1thresholds (e.g., measurement intervals of less than M1, between M1and M2, or greater than M2with costs of C1, C2, and C3). In an exemplary embodiment, this measurement functionality would be implemented in hardware and not require a CPU at the far end to respond, which decreases accuracy and could cause a protection switch if the CPU get busy. Existing functionality supports sending traffic on an unused VLAN dedicated to measurement with testing in hardware or sending traffic to an IP address on an active VLAN, which requires CPU intervention. Once the measurement capability is in place, then packets could be sent at a custom interval (at a custom frame size) and once some condition is met (e.g., 2 out of last 100 packets lost, or average latency of last 5 packets exceeds 20 ms), then the system can notify the router to change the metric.

This approach of selecting a preset metric based on measurements is more stable than implementations that directly calculate metrics from measurements, since such measurements are likely to vary with traffic load causing directly calculated metrics to change when there are no underlying physical layer failures.

Users of the system101as described herein may include service providers and/or end-users with latency-sensitive or packet loss-sensitive applications or services in which there are multiple paths through the network and need accurate measurement capabilities to measure the quality of individual network link or end-to-end paths so that traffic can be rerouted if needed. For example, this arrangement can be beneficial to service providers and/or end-users with IP networks trunked across SONET/Ethernet networks with restoration capabilities in which the restored path has qualities that are inferior to the original path (e.g., higher latency or lower bandwidth).

FIG. 6illustrates computing hardware (e.g., computer system)600upon which an embodiment according to the invention can be implemented. The computer system600includes a bus601or other communication mechanism for communicating information and a processor603coupled to the bus601for processing information. The computer system600also includes main memory605, such as a random access memory (RAM) or other dynamic storage device, coupled to the bus601for storing information and instructions to be executed by the processor603. Main memory605can also be used for storing temporary variables or other intermediate information during execution of instructions by the processor603. The computer system600may further include a read only memory (ROM)607or other static storage device coupled to the bus601for storing static information and instructions for the processor603. A storage device609, such as a magnetic disk or optical disk, is coupled to the bus601for persistently storing information and instructions.

The computer system600may be coupled via the bus601to a display611, such as a cathode ray tube (CRT), liquid crystal display, active matrix display, or plasma display, for displaying information to a computer user. An input device613, such as a keyboard including alphanumeric and other keys, is coupled to the bus601for communicating information and command selections to the processor603. Another type of user input device is a cursor control615, such as a mouse, a trackball, or cursor direction keys, for communicating direction information and command selections to the processor603and for adjusting cursor movement on the display611.

According to an embodiment of the invention, the processes described herein are performed by the computer system600, in response to the processor603executing an arrangement of instructions contained in main memory605. Such instructions can be read into main memory605from another computer-readable medium, such as the storage device609. Execution of the arrangement of instructions contained in main memory605causes the processor603to perform the process steps described herein. One or more processors in a multi-processing arrangement may also be employed to execute the instructions contained in main memory605. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the embodiment of the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software.

The computer system600also includes a communication interface617coupled to bus601. The communication interface617provides a two-way data communication coupling to a network link619connected to a local network621. For example, the communication interface617may be a digital subscriber line (DSL) card or modem, an integrated services digital network (ISDN) card, a cable modem, a telephone modem, or any other communication interface to provide a data communication connection to a corresponding type of communication line. As another example, communication interface617may be a local area network (LAN) card (e.g. for Ethernet™ or an Asynchronous Transfer Model (ATM) network) to provide a data communication connection to a compatible LAN. Wireless links can also be implemented. In any such implementation, communication interface617sends and receives electrical, electromagnetic, or optical signals that carry digital data streams representing various types of information. Further, the communication interface617can include peripheral interface devices, such as a Universal Serial Bus (USB) interface, a PCMCIA (Personal Computer Memory Card International Association) interface, etc. Although a single communication interface617is depicted inFIG. 6, multiple communication interfaces can also be employed.

The network link619typically provides data communication through one or more networks to other data devices. For example, the network link619may provide a connection through local network621to a host computer623, which has connectivity to a network625(e.g. a wide area network (WAN) or the global packet data communication network now commonly referred to as the “Internet”) or to data equipment operated by a service provider. The local network621and the network625both use electrical, electromagnetic, or optical signals to convey information and instructions. The signals through the various networks and the signals on the network link619and through the communication interface617, which communicate digital data with the computer system600, are exemplary forms of carrier waves bearing the information and instructions.

The computer system600can send messages and receive data, including program code, through the network(s), the network link619, and the communication interface617. In the Internet example, a server (not shown) might transmit requested code belonging to an application program for implementing an embodiment of the invention through the network625, the local network621and the communication interface617. The processor603may execute the transmitted code while being received and/or store the code in the storage device609, or other non-volatile storage for later execution. In this manner, the computer system600may obtain application code in the form of a carrier wave.

While the invention has been described in connection with a number of embodiments and implementations, the invention is not so limited but covers various obvious modifications and equivalent arrangements.