Patent ID: 8462781

Claim:
A packet deduplication system for reducing duplicate packets in a packet stream, comprising: a packet buffer configured to store a plurality of incoming packets from an input packet stream; a hash generator configured to apply at least one hash algorithm to at least a portion of the plurality of incoming packets to generate first and second hash values, the portion representing packets to be processed for deduplication, and the first and second hash values being generated for each packet processed for deduplication; a data storage system configured to store data associated with previously received packets; and a deduplication controller configured to use the first hash value for each packet to be processed to determine an address location for data stored in the data storage system, to compare the second hash value for each packet with data stored at the address location to determine if the packet matches a previously received packet, and to generate a control signal based at least in part upon the match determination, the control signal being applied to the packet buffer to determine whether the packet is deleted as a duplicate packet or passed from the packet buffer to an output packet stream as a non-duplicate packet; wherein the deduplication controller is further configured to store data in the data storage system at the address location determined by the first hash value for a packets determined not to match previously received packets, the data stored comprising the second hash value for the packet; wherein the first hash value is smaller than the second hash value.