Arrangement in an IP node for preserving security-based sequences by ordering IP packets according to quality of service requirements prior to encryption

A router has at least one outbound interface configured for establishing multiple IP-based secure connections (i.e., tunnels) with respective destinations based on transmission of encrypted data packets via the IP-based secure connections. The encrypted data packets are generated by a cryptographic module, where each encrypted packet successively output from the cryptographic module includes a corresponding successively-unique sequence number. The supply of data packets to the cryptographic module is controlled by a queue controller: the queue controller assigns, for each secure connection, a corresponding queuing module configured for outputting a group of data packets associated with the corresponding secure connection according to a corresponding assigned maximum output bandwidth. Each queuing module also is configured for reordering the corresponding group of data packets according to a determined quality of service policy and the corresponding assigned maximum output bandwidth.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to transport of Internet Protocol (IP) packets, requiring a guaranteed quality of service (QoS), via secure IP connections.

2. Description of the Related Art

The development of newer protocols for Internet Protocol (IP) networks has extended the capabilities of IP networks. For example, deployment of QoS policies in IP networks has enabled the reliable transport of time-sensitive media data, including audio, video, Voice over IP (VoIP), etc., based on prioritizing the transport of data packets.

In particular, data packets identified as associated with latency-sensitive data traffic (e.g., audio, video, VoIP, etc.) are assigned a higher priority than other lower-priority data packets associated with, for example, Simple Mail Transfer Protocol (SMTP) or User Datagram Protocol (UDP) applications. Hence, an outbound interface in a router may include a high-priority queue (for high priority packets) and a low-priority queue (for low priority packets), enabling the packets to be output by the outbound interface according to their priority and determined capacity: if a network interface driver (i.e., an executable software resource configured for controlling the outbound interface) detects backpressure (i.e., congestion), the network interface driver will reorder outbound traffic based on priority of the data packets (e.g., by avoiding outputting packets from the low-priority queue until the high-priority queue has been emptied). Hence many QoS techniques will reorder packets, causing packets to be output in a sequence that differs from the order the packets were supplied to the outbound interface.

Problems also exist in maintaining a guaranteed quality of service in cases were a destination host is limited in its downstream capacity. For example, a broadband service provider may limit the downstream bandwidth available to a broadband subscriber; hence, even though a headend router is capable of outputting multiple 1024 kbps or higher (e.g., 1.5 Mb/s) media streams on a high-speed interface (e.g., T1or higher), the destination router within the broadband network may be configured by the broadband service provider to limit the downstream bandwidth to a contracted rate of 1024 kbps.

Consequently, the destination router may drop downstream packets destined for the broadband subscriber if the amount of data traffic exceeds the contracted rate (e.g., 1024 kbps), for example in the case of a destination router receiving, for the broadband subscriber, a 1024 kbps media stream plus a burst of of SMTP packets. The result is reduced network efficiency due to the unnecessary waste of network resources utilized in generation, routing, and transmission of the data packets that ultimately were dropped by the destination router.

The development of secure IP connections involves IP packets passing through encrypted tunnels. In particular, secure IP tunnels have been used to establish virtual private networks (VPNs) between a local area network (e.g., a corporate LAN) and a remote node (e.g., a telecommuter's computer). In particular, a secure IP tunnel is established between the remote node (referred to as the VPN client) and a VPN server that separates the local area network via the wide area network.

The Internet Engineering Task Force (IETF) has published a Request for Comments (2401), by Kent et al., entitled “Security Architecture for the Internet Protocol” available on the IETF website at http://www.ietf.org/rfc/rfc2401.txt?number=2401, the disclosure of which is incorporated in its entirety herein by reference. The above-incorporated RFC 2401 discloses an architecture (referred to as IPSEC) for providing security services for IPv4 or IPv6 data packets at the IP layer, and uses a prescribed Authentication Header (AH) protocol and a prescribed Encapsulating Security Payload (ESP) protocol to provide traffic security. Both the AH protocol and the ESP protocol permit use of anti-replay services (i.e., replay protection), where sequence numbers are added by a transmitting node (e.g., a VPN server) to IP packets being output as a data stream onto an encrypted tunnel.

According to RFC 2401, when a security association (SA) is established between a sender and a receiver, their respective counters (Sequence Counter in the sender and Anti-Replay Window in the receiver) are set to zero: the first packet sent by the sender has a sequence number of “1”, the second packet sent by the sender has a sequence number of “2”, etc., such that each successive packet output by the sender onto that SA has a corresponding successive sequence number.

Hence, the receiver can expect the received data packets to have a respective contiguous sequence of sequence numbers. If the receiver detects a packet having a sequence number that is out of order relative to a previously received packet, the receiver determines the detected packet is an invalid packet and can discard the packet.

The receiver configured for implementing replay protection according to RFC 2401 also will drop packets that are received out-of-order: if the receiver has received packets according to the sequence numbers “1, 2, 3, 4, 100, 101, 5”, the receiver will drop the packet having the sequence number “5”, since it is out of order relative to the packets having the sequence numbers “100” and “101”.

As described above, many QoS techniques reorder packets. The IPSEC architecture, in contrast, requires packets to be received in order of the specified sequence numbers to ensure replay protection. Consequently, the inherent inconsistency between QoS techniques and the IPSEC architecture has caused unnecessary packet loss during past attempts at implementing IPSEC protocol and QoS policies on the same router interface.

In particular, attempts have been made to add IPSEC functionality to QoS-enabled routers in order to provide latency sensitive traffic (including voice and video) over Virtual Private Networks (VPN). Hence, voice and data packets must pass through encrypted tunnels. To date the voice and data packets have encountered IPSEC encryption and sequence number assignment prior to being passed to the outbound driver that performs the QoS functionality. Hence, any detection of congestion by the outbound driver causes reordering of packets such that the higher priority packets are at the front of the outbound queue.

Consequently, the decrypting peer, having detected an IPSEC sequence number that is out of order, drops the packets that were received out of order, even though the dropped packet is a valid, secure packet.

Although some encryption devices utilize queues before input to an encryption chip (i.e., integrated circuit), such queues have been used solely to prevent loss of data due to exceeding the input bandwidth of the encryption chip.

SUMMARY OF THE INVENTION

There is a need for an arrangement that enables latency sensitive traffic to be transported via encrypted tunnels, with guaranteed quality of service, without loss of packets due to reordering of packets bearing sequence numbers. There also is a need for an arrangement that enables transmission of data streams between secure tunnel endpoints with guaranteed of quality service, in a manner that minimizes packet loss between intermediate routers.

These and other needs are attained by the present invention, where a router has at least one outbound interface configured for establishing multiple IP-based secure connections (i.e., tunnels) with respective destinations based on transmission of encrypted data packets via the IP-based secure connections. The encrypted data packets are generated by a cryptographic module, where each encrypted packet successively output from the cryptographic module includes a corresponding successively-unique sequence number. The supply of data packets to the cryptographic module is controlled by a queue controller: the queue controller assigns, for each secure connection, a corresponding queuing module configured for outputting a group of data packets associated with the corresponding secure connection according to a corresponding assigned maximum output bandwidth. Each queuing module also is configured for reordering the corresponding group of data packets according to a determined quality of service policy and the corresponding assigned maximum output bandwidth.

Hence, the queue controller ensures that data packets are supplied to the cryptographic module in a manner that maintains quality of service policies for latency-sensitive traffic, while ensuring that the data packets are supplied in an order that ensures that quality of services policies implemented by the outbound interface have a minimal likelihood of reordering higher-priority packets. Hence, the queue controller ensures that the data flow remains below outbound interface congestion thresholds, minimizing the need for reordering packets in the outbound interface by QoS based queuing mechanisms. In addition, the queue controller can be configured to ensure that the assigned maximum output bandwidth corresponds to authorized bandwidth rates for a corresponding destination, minimizing the possibility that intermediate routers may drop the packets due to congestion on the downstream link to the destination.

One aspect of the present invention includes a method in a router having at least one outbound interface. The method includes establishing, on the outbound interface, a plurality of Internet Protocol (IP)-based secure connections with respective destinations based on receiving encrypted packets generated by a cryptographic module, each encrypted packet successively output from the cryptographic module having a corresponding successively-unique sequence number. The method also includes controlling supply of data packets to the cryptographic module. The supply of data packets is controlled by: (1) assigning, for each secure connection, a corresponding queuing module, (2) reordering, in each queuing module, a corresponding group of the data packets associated with the corresponding secure connection according to a determined quality of service policy and based on a corresponding assigned maximum output bandwidth for the corresponding queuing module, and (3) outputting to the cryptographic module the group of data packets, from each corresponding queuing module according to the corresponding assigned maximum output bandwidth, for generation of the encrypted packets. The method also includes outputting the encrypted packets from the cryptographic module to the one outbound interface for transport via their associated secure connections.

Another aspect of the present invention includes a router comprising a cryptographic module configured for successively outputting encrypted packets having respective successively-unique sequence numbers. The router also includes an outbound interface, and a queue controller. The outbound interface is configured for establishing a plurality of Internet Protocol (IP)-based secure connections with respective destinations based on receiving respective streams of the encrypted packets. The queue controller is configured for controlling supply of data packets to the cryptographic module, and also is configured for assigning, for each secure connection, a corresponding queuing module. Each queuing module configured for outputting to the cryptographic module a corresponding group of the data packets associated with the corresponding secure connection, and according to a corresponding assigned maximum output bandwidth for the corresponding queuing module, for generation of the corresponding stream of the encrypted packets. Each queuing module also is configured for selectively reordering the corresponding group of the data packets according to a determined quality of service policy and the corresponding assigned maximum output bandwidth.

Additional advantages and novel features of the invention will be set forth in part in the description which follows and in part will become apparent to those skilled in the art upon examination of the following or may be learned by practice of the invention. The advantages of the present invention may be realized and attained by means of instrumentalities and combinations particularly pointed out in the appended claims.

BEST MODE FOR CARRYING OUT THE INVENTION

FIGS. 1A and 1Bare diagrams illustrating routers12aand12b, respectively, configured for performing quality of service (QoS) based queuing for each secure connection and prior to encryption, according to an embodiment of the present invention.FIG. 1Aillustrates a router12ahaving multiple outbound interfaces22, andFIG. 1Billustrates a router12bhaving a single outbound interface22(e.g., a branch router having a LAN-based port and a WAN-based port). Hence, the disclosed embodiment can be implemented in a multiport router12aor a router12bhaving a single outbound interface22.

Each of the routers12aand12bare configured for establishing secure connections16with destination endpoints (i.e., tunnel endpoints) via a wide area network. For example,FIG. 1Aillustrates a virtual private network (VPN)10established between the router12aand destination endpoints14, via respective secure connections16, over a wide area packet switched network18such as the Internet. In particular, the router12aincludes multiple outbound interfaces22(e.g., OB1, OB2, OB3, etc.), each configured for establishing a plurality of IP-based secure connections16(i.e., tunnels) with the respective destination tunnel endpoints14via the Internet18. As illustrated inFIG. 1, the outbound interface22labeled “OB1” establishes the secure connections16labeled “S1”, “S2”, and “S3” with the destination tunnel endpoints14labeled “D1”, “D2”, and “D3”, respectively; the outbound interface22labeled “OB2” establishes the secure connections16labeled “S4” and “S5” with the destination tunnel endpoints14labeled “D4” and “D5”, respectively; and the outbound interface22labeled “OB3” establishes the secure connections “S6” and “S7” with the destination tunnel endpoints “D6” and “D7”, respectively.

The secure connections16are established by each outbound interface22based on receiving encrypted packets104generated by a cryptographic module20according to IPSEC protocol. Hence, the destination endpoints14could be implemented as a router, a gateway, or a host computer, but in any case the destination endpoints are configured for terminating the secure connection16established between the router12aor12band the destination endpoints14.

In particular, each router12aand12balso includes a cryptographic module20configured for outputting encrypted packets104according to the IPSEC protocol as specified in RFC 2401. Each encrypted packet successively output by the cryptographic module20has a corresponding successively-unique sequence number, enabling the destination tunnel endpoints14to implement anti-replay procedures according to RFC 2401. As illustrated inFIG. 1A, the cryptographic module20is configured for outputting: the encrypted packets associated with the secure connections16labeled “S1”, “S2”, and “S3” to the outbound interface22labeled “OB1”; the encrypted packets associated with the secure connections16labeled “S4” and “S5” to the outbound interface22labeled “OB2”; and the encrypted packets associated with the secure connections16labeled “S6” and “S7” to the outbound interface22labeled “OB3”.FIG. 1Billustrates that the router12bis configured for outputting the encrypted packets104associated with the secure connections16labeled “S1”, “S2”, and “S3” to the outbound interface22labeled “OB”.

Each outbound interface22also includes a quality of service (QoS) module26configured for implementing a prescribed quality of service procedures in the event that the corresponding outbound interface22encounters congestion. In particular, an outbound interface22may encounter congestion if the incoming rate of data packets to be transmitted exceeds the available bandwidth on the corresponding outbound link24. As illustrated inFIGS. 1A and 1B, the router12includes a routing core34configured for receiving IP packets from at least one inbound interface38, and outputting non-encrypted data streams32. The non-encrypted data streams32are illustrated inFIG. 1Aas “N1”, “N2”, and “N3” and are output by the routing core34to the outbound interfaces22labeled “OB1”, “OB2”, and “OB3”, respectively.

If congestion is detected in an outbound interface22, the corresponding quality of service module26is configured to prioritize packets to provide a guaranteed quality of service for latency-sensitive traffic. As described above, however, the prioritizing of packets by the QoS module26may cause reordering of the encrypted packets output by the cryptographic module20.

Concerns also arise in the case where a destination tunnel endpoint14, for example the destination host computer/gateway/router14labeled “D1” is limited to a contracted bandwidth rate of 1024 kbps: if the destination tunnel endpoint14labeled “D1” were to receive downstream video traffic that exceeds the contracted bandwidth rate, then certain downstream packets may be dropped by an access router configured for enforcing the contracted bandwidth rate.

According to the disclosed embodiment, the routers12aand12beach include a queue controller40configured for controlling supply of data packets (S) that require encryption. In particular, the queue controller40is configured for assigning, for each secure connection16, a corresponding queuing module, described in further detail below with respect toFIGS. 2-5. As described in further detail below, each queuing module is configured for outputting a group of data packets (e.g.,100a, illustrated in detail inFIGS. 2 and 5) associated with the corresponding secure connection according to a corresponding assigned maximum output bandwidth. Each queuing module also is configured for selectively reordering the corresponding group of data packets according to a determined quality of service policy and the corresponding assigned maximum output bandwidth.

Hence, the queue controller40ensures that the aggregate output (S′)102of all the secure connections14is less than the input bandwidth of the cryptographic module20. In addition, each queuing module can be configured to ensure that the packets destined for the corresponding secure connection16do not overwhelm the output bandwidth of the corresponding outbound interface22, and preferably the bandwidth allocated to the subscriber and enforced by the access router30.

FIG. 2is a diagram illustrating in detail the queue controller40, according to an embodiment of the present invention. The queue controller40includes a security association (SA) assignment module42configured for assigning, for each IPSEC-based secure connection16(i.e., each Security Association16), a corresponding queuing module44. As illustrated inFIG. 2, the SA assignment module42assigns the queue modules44a,44b,44c, and44dto the secure connections16labeled “S1”, “S2”, “S3”, and “S4”, respectively. The flow of packets for the flows associated with the secure connections16labeled “S1”, “S2”, “S3”, and “S4” are output by the routing core34.

The queue controller40also assigns a bandwidth controller46to each corresponding queuing module44. Each bandwidth controller46is controlling the output bandwidth for the corresponding queuing module (e.g,44a), hence the bandwidth utilized for the corresponding secure connection16. Preferably the prescribed threshold utilized by the bandwidth controller46is less than the threshold that would be utilized by the quality of service module26internal to the outbound interface22, in order to avoid reordering by the quality of service module26. Preferably the prescribed threshold utilized by the bandwidth controller46also is less than the input bandwidth limit of the cryptographic module20. Further, each bandwidth controller46can be configured such that the sum of bandwidth assigned among the secure connections16is less than thresholds for the outbound interface22or the IPSEC module20.

As illustrated inFIG. 2, the cryptographic chip20performs sequence number assignment (SEQ) to the encrypted packets before being output to the appropriate outbound interface22.

FIG. 3is a diagram illustrating in further detail an exemplary queue module44and the bandwidth controller46, according to an embodiment of the present invention. Each queue module44includes an input controller50, a packet queue52, and a connection-specific bandwidth controller46configured for outputting to the cryptographic module20the group of data packets associated with the corresponding secure connection16(e.g., “S1”). In particular, the connection-specific bandwidth controller46includes a congestion detector56, and output controller58, a maximum output bandwidth register60, and a bandwidth negotiator62.

The input controller50is configured for storing each received data packet that is associated with the corresponding secure connection16(e.g., “S1”) into one of a plurality of queues (e.g.,52a,52b, etc.) having respective identified priorities (e.g., “High”, “Low”, etc.), based on a corresponding identified priority for each packet. As recognized in the art, packets may be prioritized based on packet type (e.g., VoIP, video, TCP, UDP, SMTP, etc.), or some other identifier. Once the packets are stored in the queues52, the output controller58is configured for outputting the stored data packets to the cryptographic module20according to the corresponding assigned maximum output bandwidth specified in the register60. The maximum output bandwidth specified in the register60may be manually configured, or may be inserted by the bandwidth negotiation resource62. For example, the bandwidth negotiation resource62may utilize resource reservation protocol (RSVP) in order to communicate with the destination endpoint14to identify the maximum downstream bandwidth available to the destination endpoint14(e.g.,1024kbps).

The output controller58also can be configured for selectively reordering the data packets stored in the buffers52aand52b, relative to the sequence received by the input controller50, according to a determined quality of service policy in response to a detected congestion condition. The congestion condition may be generated the congestion detector56. The congestion detector56is configured for monitoring the congestion levels in the assigned outbound interface and the levels specified in the output bandwidth register60. Hence, if any of the above-described congestion levels are detected, the output controller58selectively reorders the stored data packets by outputting the data according to the priority queues, such that the high priority queue52awould be given priority over the data stored in the low priority queue.

Also note thatFIG. 3illustrates that each outbound interface22includes an executable driver resource70configured for controlling operations of the outbound interface22, including transfer of data from the cryptographic module20and onto the network link24. Each outbound interface22also includes an executable IPSEC resource72configured for establishing the secure connections16according to IPSEC protocol, and the QoS module26.

FIG. 4is a diagram illustrating the method of ordering packets for a given secure connection according to quality of service requirements prior to encryption, according to an embodiment of the present invention. The steps described herein with respect toFIG. 4can be implemented as executable code stored on a computer readable storage medium (e.g., floppy disk, hard disk, EEPROM, CD-ROM, etc.), or propagated via a computer readable transmission medium (e.g., fiber optic cable, electrically-conductive transmission line medium, wireless electromagnetic medium, etc.).

The method begins in step80, where the IPSEC resources72in each of the outbound interfaces22establish the respective secure connections (i.e., Security Associations (SAs)) according to IPSEC protocol. The SA assignment module assigns to each secure connection16a corresponding queuing module44in step82. Each queuing module44(e.g.,44a) determines in step84the corresponding assigned maximum output bandwidth to be used for the corresponding secure connection (e.g., S1). As described above, the assigned maximum output bandwidth may be obtained from a prior manual configuration, or based on the negotiation resource62obtaining the corresponding assigned maximum output bandwidth from the corresponding destination14.

The transmit data (S) that is to be encrypted is routed by the routing core34to the queuing controller40: the SA assignment module42forwards each data packet to the assigned queuing module44in step86based on the secure connection to be traversed by the data packet. The input controller50for the queuing module44stores each data packet in a selected queue (e.g.,52a,52b, etc.) based on a determined priority.

The output controller58for the queuing module44then prepares to output the stored data packets to the cryptographic module20in accordance with the maximum permitted output bandwidth specified in the corresponding register60: if in step88the output controller58for the corresponding queuing module44detects a congestion condition, the controller58reorders in step90the packets to be output based on priority, for example by outputting from the highest priority queue52a, minimizing the probability that high priority packets will be dropped. Note that congestion in step88can be detected, for example based on the congestion detector56, or based on the bandwidth controller detecting that levels in the queues52approaching congestion levels due to a higher rate of input data encountered by the input controller50. The output controller58outputs the data packets at or below the assigned maximum output bandwidth to the cryptographic module20in step92.

Note that the selective reordering based on congestion is optional: each queuing module (e.g.,44a) for a given secure connection16(e.g., S1) can be configured to reorder all packets according to priority, regardless of the presence of any congestion condition.

FIG. 5is a diagram illustrating the reordering of packets by queuing modules, according to an embodiment of the present invention. The bandwidth controllers46a,46b,46c, and46doutput respective streams of queued packets100a,100b,100c, and100d, for the respective secure tunnels S1, S2, S3, and S4. For example, the bandwidth controller46aoutputs the stream A1, A2, A3, A4, A5, etc.100afor the secure connection S1; the bandwidth controller46boutputs the stream B1, B2, B3, B4, B5, etc.100bfor the secure connection S2; the bandwidth controller46coutputs the stream C1, C2, C3, C4, C5, etc.100cfor the secure connection S3; and the bandwidth controller46doutputs the stream D1, D2, D3, D4, D5, etc.100dfor the secure connection S4. The streams100a,100b,100c, and100dare combined into a combined stream102and supplied to the cryptographic module20for encryption into encrypted packets104(e.g., A1′, B1′, C1′, D1′, etc.). The encrypted packets104are then supplied to the outbound interface22: for simplicity, assume that only the single outbound interface22ofFIG. 1Bis utilized.

Assume now that congestion encountered by the outbound interface22cause reordering of the packets, resulting in the reordered stream106output on the outbound link24. As illustrated inFIG. 5, the packets B1′ and B2′ associated with the stream100bhave been reordered relative to the other packets. However, the prior queuing by the queuing modules44on a per-secure tunnel basis ensures that the packets associated with the same secure tunnel remain in the appropriate order.

Hence, the encrypted packets (e.g.,100′a,100′b,100′c, and100′d) for a given secure tunnel (e.g., S1, S2, S3, and S4) arrive at the corresponding destination14(e.g., D1, D2, D3, and D4) in the appropriate order, minimizing the probability of dropping packets due to anti-replay protection mechanisms in the destination endpoints14.

According to the disclosed embodiment, reordering of encrypted packets having sequence numbers can be minimized, minimizing the unnecessary loss of of processor resources, cryptography engine resources, and bandwidth resources throughout the virtual private network10. Further, low priority packets can be dropped or delayed prior to encryption, optimizing resources within the router.