Rate limiting for DTCP message transport

A network device may limit the rate at which control messages are forwarded to a destination device when forwarding traffic pursuant to dynamic flow capture (DFC). In one implementation, a system may receive filtering criteria associated with DFC of network traffic and passively filter incoming traffic based on the filtering criteria to obtain traffic that matches the filtering criteria. The system may transmit a rate limited version of control messages associated with the filtered traffic to a control device.

BACKGROUND OF THE INVENTION

A. Field of the Invention

The principles described herein relate generally to network traffic monitoring and, more particularly, to systems and methods that provide dynamic flow capture of network traffic.

B. Description of Related Art

Network devices, such as routers, receive data on physical media, such as optical fiber, analyze the data to determine its destination, and output the data on physical media in accordance with the destination. In a high traffic public network, such as the Internet, the routers that make up the network may be owned and operated by a number of different entities. An Internet Service Provider (ISP), for example, may operate a number of routers. The ISP may sell access to the network to end-users, such as consumers or businesses.

ISPs may desire or need to monitor traffic from certain ones of its customers. In some jurisdictions, the law may require that the ISP have the ability to monitor its traffic.

Passive traffic monitoring techniques are known by which the ISP (or other entity that controls a router) may set up filtering criteria within the router. When data matches the filtering criteria, a copy of the data is forwarded to one or more destinations. For example, a filter may be set up that specifies that all packets from a particular IP address be forwarded to a designated destination.

SUMMARY

One aspect is directed to a network device that may include logic to store filtering criteria that defines conditions by which network traffic is to be forwarded to a first destination device; logic to forward, to a second destination device, control messages relating to the network traffic that is forwarded to the first destination device; and logic to limit the maximum rate at which the forwarded control message are forwarded to the second destination device.

Another aspect is directed to a method that may include receiving filtering criteria associated with dynamic flow capture (DFC) of network traffic; passively filtering incoming traffic based on the filtering criteria to obtain traffic that matches the filtering criteria; transmitting the traffic that matches the filtering criteria to a first destination device; and transmitting a rate limited version of control messages associated with the DFC of the network traffic to a second destination device, the rate limited version of the control messages for the second destination device being performed on a per-destination device basis.

Another aspect is directed to a router device that may include a first physical interface card (PIC) configured to provide a physical interface to a network and a second PIC configured to monitor traffic received by the first PIC from the network. The router device may further include logic to store filtering criteria that defines conditions by which the monitored traffic is to be forwarded to a destination device and logic to limit a maximum rate at which control messages associated with the forwarded traffic are transmitted to a second destination device

DETAILED DESCRIPTION

The following detailed description of the invention refers to the accompanying drawings. The same reference numbers in different drawings may identify the same or similar elements. Also, the following detailed description does not limit the invention.

As described herein, network devices, such as routers, passively capture and forward traffic that meets filtering criteria to destination devices. The network devices may also forward control messages related to the filtering. The network devices may limit the rate at which the control messages are forwarded to a particular destination control device. The rate may be limited and/or determined on a per-destination basis. By limiting the rate at which a particular destination control device receives the control messages, overloading of the destination control devices may be avoided.

Exemplary System Overview

FIG. 1is a diagram of an exemplary system100in which concepts described herein may be implemented. System100may include multiple entities, such as a server120, a network140, end-users160-1through160-N (collectively referred to herein as end-users160), and an Internet Service Provider (ISP)180. Server120may include one or more computing devices designed to provide information or to otherwise interact with end-users160. Similarly, end-users160may each include one or more computing devices designed to interact with and obtain content from server120. End-users160may communicate with other end-users160and with server120via network140.

Network140may comprise a wide area network (WAN), such as the Internet, a private WAN, a telephone network, such as the Public Switched Telephone Network (PSTN), or a combination of networks. Network140may include a number of routers or other switching devices, such as edge routers137-1and137-2(collectively referred to as edge routers137), and core routers138-1and138-2(collectively referred to as core routers138).

Edge routers137may generally function to connect devices, such as end-users160to network140. In some implementations, multiple end-users160may first connect to an intermediate access device which may connect them to an edge router137. Core routers138may generally function to transmit data between other routers within network140. In addition to simply routing data, edge routers137and core routers138may support other “value added” functions, such as quality of service (QoS) features, specialized security functions, traffic accounting features, or traffic flow capture functions. In particular, regarding traffic flow capture, edge routers137and/or core routers138may implement dynamic flow capture (DFC) through which an administrator of the router (e.g., an ISP) can capture and forward traffic flows on the basis of dynamic filtering criteria. DFC will be discussed in more detail below.

ISP180may provide access to network140or portions of network140for server120and end-users160. ISP180may, for instance, own or manage routers, such as edge routers137and core routers138in network140. In some situations, such as when network140is the Internet, ISP180may control only a portion of the routers and connections in network140. In this situation, ISP180may, for example, participate in peering arrangements with other ISPs or other entities in which ISP180may send traffic to and receive traffic from routers or other network equipment owned by other ISPs.

One of ordinary skill in the art will appreciate that, in practice, system100may include other network devices. Additionally, although one server120, one ISP180, two-edge routers137, and two core routers138are shown inFIG. 1, it can be appreciated that system100may have more or fewer servers, ISPs, edge-routers, or core routers. In still other implementations, one or more components of system100may perform one or more of the tasks described as being performed by one or more other components of system100.

Exemplary Network Device Architecture

FIG. 2is a block diagram illustrating a high-level exemplary implementation of one of edge routers137or core routers138, referred to as router137/138. Router137/138may include packet forwarding engines (PFEs)201-1through201-M (collectively referred to as PFEs201), an internal switch fabric205, and a routing engine (RE)215. Router137/138may receive data from physical links, process the data to determine destination information, and transmit the data out on a link in accordance with the destination information.

RE215may perform high level management functions for router137/138. For example, RE215may communicate with other networks and systems connected to router137/138to exchange information regarding network topology. RE215may create routing tables based on the network topology information and forward the routing tables to PFEs201. PFEs201may use the routing tables to perform route lookup for incoming data. RE215may also perform other general control and monitoring functions for router137/138.

PFEs201may each connect to RE215via switch fabric205. Switch fabric205provides internal links between different PFEs201and RE215. In general, PFEs201receive data on ports connecting physical links that lead to network140. Each physical link could be one of many types of transport media, such as optical fiber or Ethernet cable. The data on the physical link may be formatted according to one of several protocols, such as the synchronous optical network (SONET) standard. PFEs201process the received data, determine the correct output port for the data, and transmit the data on the physical link corresponding to the determined output port.

Each PFE201may be associated with one or more physical interface cards (PICs)210-1through210-9(referred to collectively as PICs210). PICs210may provide low-level interfaces for the PFEs to the physical links. PICs210may receive and transmit packets from the physical links. PICs210may include media-specific logic that performs, for example, framing and checksum verification. Different types of PICs210may operate according to different transmission rates or physical media types, such as OC-192 and OC-48 transmission rates, and protocols or standards, such as the Synchronous Optical Networking (SONET) standard for data transmission over optical networks. PICs210may be “pluggable” in the sense that PICs may be removed or added to PFEs201as needed.

At least one of PICs210, such as PIC210-9, may be a special purpose PIC designed to implement DFC. PIC210-9may be a DFC PIC that is designed to monitor traffic received at one or more of the other PICs210. For example, a traffic flow received at PIC210-1may be forwarded to PIC210-9. At PIC210-9, packets in the traffic flow may be compared to filtering criteria, and packets that match the criteria may be forwarded to a specified destination device, such as a computing device at ISP180. The traffic monitoring operation of PIC210-9may be passive, meaning that monitored traffic is still forwarded on the way to its original destination.

Exemplary Computing Device Architecture

FIG. 3is a block diagram of an exemplary computing device300, which may correspond to server120, a computing device associated with end-user160, or a computing device associated with ISP180(not shown inFIG. 1). Device300may include a bus310, a processor320, a main memory330, a read only memory (ROM)340, a storage device350, an input device360, an output device370, and a communication interface380. Bus310may include a path that permits communication among the elements of the device.

Processor320may include a processor, microprocessor, or processing logic that may interpret and execute instructions. Main memory330may include a random access memory (RAM) or another type of dynamic storage device that may store information and instructions for execution by processor320. ROM340may include a ROM device or another type of static storage device that may store static information and instructions for use by processor320. Storage device350may include a magnetic and/or optical recording medium and its corresponding drive.

Input device360may include a mechanism that permits an operator to input information to the device, such as a keyboard, a mouse, a pen, voice recognition and/or biometric mechanisms, etc. Output device370may include a mechanism that outputs information to the operator, including a display, a printer, a speaker, etc. Communication interface380may include any transceiver-like mechanism that enables the device to communicate with other devices and/or systems.

Device300may perform operations in response to processor320executing software instructions contained in a computer-readable medium, such as memory330. A computer-readable medium may be defined as a physical or logical memory device.

The software instructions may be read into memory330from another computer-readable medium, such as data storage device350, or from another device via communication interface380. The software instructions contained in memory330may cause processor320to perform processes that will be described later. Alternatively, hardwired circuitry may be used in place of or in combination with software instructions to implement processes consistent with the principles described herein. Thus, implementations described herein are not limited to any specific combination of hardware circuitry and software.

Dynamic Flow Capture

As previously mentioned, one of PICs210, such as PIC210-9, may be a special purpose PIC designed to implement DFC.FIG. 4is a diagram illustrating exemplary components of a DFC PIC in additional detail. DFC PIC210-9may include monitor logic420, DFC control logic425, and rate limiter logic430.

Monitor logic420may store filtering criteria421from one or more control devices, where each filter criterion may define conditions by which traffic is to forwarded to a destination device. For example, a filter criterion may specify that any packets from a certain IP address and using the ftp (file transfer protocol) are to be forwarded to the destination device.

Monitor logic420may monitor input traffic, based on filtering criteria421, to determine whether packets within the input traffic match the filtering criteria. Packets that match the filtering criteria may be transmitted from DFC PIC210-9to a predetermined destination. The destination device may be, for example, an archiving server that is specified by a control device.

DFC control logic425may receive input Dynamic Tasking Control Protocol (DTCP) control messages from a control device. DTCP is a known message-based interface by which an authorized client may connect to a server, such as a router. DTCP allows for multiple clients to simultaneously control a single device. The DTCP control messages may be sent over UDP (User Datagram Protocol).

The DTCP control messages received by DFC control logic425may include messages from authorized control devices that setup, modify, or stop DFC sessions. The DTCP control messages may also include messages that request statistics or other information relating to a DFC session.

A DTCP control message may, for example, include filter criterion421for monitoring logic420. In this situation, DFC control logic425may update filtering criteria421to reflect the contents of the DTCP control message.

Rate limiter logic430may operate to limit the rate at which DTCP control messages are transmitted to destination devices. Because the output DTCP control messages may normally be sent at the maximum output rate of an I/O PIC (called “line rate”), it is possible that the output DTCP control messages may have a relatively high bandwidth. In contrast, the control device that receives the DTCP control messages for a particular DFC session (e.g., a particular filter criterion) may have a much lower input capacity. Accordingly, forwarding DTCP control message traffic at line rate may, in some situations, overwhelm the destination device.

In one implementation, rate limiter logic430may include a number of buffers435-1through453-N (collectively buffers435). Buffers435may be implemented using, for example, dedicated registers or physical memories for each buffer435or, alternatively, as logical queues within a single physical memory. Buffers435may be implemented on a per-destination control device basis. In other words, each buffer435may correspond to one control destination device and input messages from DFC control logic425that is associated with the control destination device may be queued into that buffer. Each buffer435may be a first-in-first-out (FIFO) queue.

Rate limiter logic430may use buffers435to limit the maximum rate at which messages are transmitted to each destination device. If, for instance, a burst of data is output by DFC control logic425that is above a threshold rate for the corresponding destination device, rate limiter logic430may temporarily store the additional traffic in one of buffers435. In this manner, rate limiter logic430may limit the maximum rate at which traffic is transmitted to any particular destination device.

The threshold rate used by rate limiter logic430may be determined based on the goal of ensuring that the destination devices will be able to handle the incoming traffic. In one implementation, the threshold rate used by rate limiter logic430may be determined on a per-destination basis. In other implementations, rate limiter logic430may use a single threshold rate for all destination devices. The threshold rate may be determined based on information that is known or assumed about a destination device. For example, if the destination device is a personal computer or workstation, the threshold rate may be determined based on the socket buffer size of the destination device, the speed of the processor of the destination device, or other parameters that are indicative of the capacity of the destination device to service incoming traffic. Heuristics-based techniques may be used to estimate the threshold rate. In an alternative implementation, the threshold rate may be determined through a trial and error based process.

FIG. 5is a flow chart illustrating exemplary operations that may be performed by a network device. The operations ofFIG. 5will be particularly illustrated with reference toFIG. 6, which is a diagram conceptually illustrating an example of the interaction of elements of system100consistent with the operations shown inFIG. 5.

A request to establish dynamic flow capture may be initially received by, for example, DFC PIC210-9(act501). The request may be associated with DFC filtering criteria (act501). The request may be received by DFC control logic425as a DTCP control message.

InFIG. 6, the network device performing dynamic flow capture will be assumed to be edge router137-2, although it can be appreciated that dynamic flow capture consistent with aspects described herein may be implemented on other routers or other network devices. Router137-2may include DFC PIC210-9. In this example, three other PICs (PICs210-1,210-2, and210-3) are shown as associated with router137-2. Each of PICs210-1through210-3may be input/output (I/O) PICs designed to receive and forward data from physical links. DFC PIC210-9may not directly communicate with external physical links. Instead, DFC PIC210-9may receive packets that were originally received by I/O PICs210-1through210-3, process the packets, and forward the packets back through router137-2to one or more of I/O PICs210-1through210-3.

As shown in the example ofFIG. 6, a request640is sent by an administrator at ISP180, such as by a user working at a computing device660-1(e.g., a personal computer). Request640may be sent to DFC control logic425as a message using DTCP and may be a request to begin dynamic flow capture for a particular packet flow or set of packet flows. Request640may include filtering criteria and an indication of a destination device to which packets that match the filter criteria are forwarded. The destination device may be a relatively high capacity destination server. In this example, assume that the destination device is device660-2and the filter criteria specifies that any packets from a certain IP address and using the ftp (file transfer protocol) are to be forwarded to the destination device.

Referring back toFIG. 5, received traffic that corresponds to traffic being monitored may be forwarded to the DFC PIC (FIG. 5, “Monitored Traffic”). At the DFC PIC, monitor logic420may compare the traffic to filtering criteria421to determine whether the traffic is traffic that should be forwarded to the designated destination device. If the traffic matches the filtering criteria, (act503), the matching traffic may be sent to the corresponding destination device (e.g., a high capacity server) (act504).

The control messages may be DTCP control messages. In addition to the traffic forwarded by monitor logic420, DFC PIC210-9may generate the output control messages. As shown inFIG. 6, these DTCP control messages may be sent to the computing device that initiated the DFC session, such as computing device660-1. In some implementations, the computing device that initiates a DFC session may specificy that a different computing device receive the subsequent DTCP control messages for the session.

As an example of control messages sent to a control device, DFC control logic425may send statistics relating to a DFC session and that are requested by the control device. Since the control messages are typically sent as DTCP control messages over UDP (a non-guaranteed delivery message protocol), if the bandwidth to the control device is too high, the control messages may get dropped, thus giving an inaccurate picture to the control device. Other types of control messages may include notifications to the control devices. With this type of control information it may also be important that the bandwidth of the control messages do not overwhelm the control device.

A rate-limited version of the DTCP control messages may be forwarded to the specified computing device (act502). As described above, the rate limiting may be performed by rate-limiter logic430.

Returning to the example ofFIG. 6, a traffic flow that is being monitored is illustrated by curves642. Assume that a user initiating at least some of this traffic is suspected of illegal activity and ISP180was instructed by a law enforcement agency to monitor this traffic. Traffic642is received by I/O PIC210-1and may be forwarded as normal through edge router137-2to I/O PIC210-3. As shown, a copy of traffic642is also sent to DFC PIC210-9. DFC PIC210-9may apply the filtering criteria defined for this traffic flow. The portions of the traffic flow that match the filtering criteria are then forwarded to destination device660-2as traffic644. DTCP control messages relating to this DFC session may be sent to destination device660-1. The control messages may be rate limited by rate limiter logic430to a bandwidth appropriate for destination device660-1. For example, destination device660-1may be a personal computer or workstation with limited capacity to service incoming traffic. Rate limiter logic430may ensure that destination device660-1is able to handle the control messages sent to it. This is particularly useful with DTCP control messages that use UDP for communication. Because message delivery via UDP is not guaranteed, the high potential bandwidth via which DFC PIC210-9may transmit control messages to destination device660-1may cause destination device660-1to potentially drop control messages that contain needed information. The per-device rate control implemented by rate limiter logic430of DFC PIC210-9, as described herein, can advantageously mitigate this issue to thus provide the information in a more controlled manner.

CONCLUSION

As described above, by rate limiting the output of control messages relating to a DFC session, the device receiving the control messages may be able to better handle high-bandwidth traffic bursts that would otherwise be forwarded to the device. This can be particularly important when the device has a limited capacity to handle traffic bursts (such as when the destination device is a personal computer) or is connected to the network via a connection with relatively limited bandwidth.

Although, in the above disclosure, routers were primarily described as performing DFC, one of ordinary skill in the art will appreciate that other network devices could perform this function. Further, although rate limiting was described above as being done on a per-destination device basis, the rate limit can be done on the basis of other physical or logical entities, such as an IP address or device associated with the control source for DFC requests (e.g., device660-1in the example ofFIG. 6).

For example, while a series of acts has been described with regard toFIG. 5, the order of the acts may be varied in other implementations consistent with the invention. Moreover, non-dependent acts may be implemented in parallel.

Further, certain portions of the invention may be implemented as “logic” or as a “component” that performs one or more functions. This logic or component may include hardware, such as an application specific integrated circuit or a field programmable gate array, software, or a combination of hardware and software.