Patent ID: 8965852
Filing Date: 2015-02-24
Classification: G06F

Abstract:
1. A method, comprising: receiving a request at a host to deduplicate a plurality of files, the host and a plurality of other hosts connected to a target system and a plurality of other target systems over a network, wherein the host communicates over the network with the target system by using a file level protocol, wherein the target system is a file server; determining at the host whether segments in the plurality of files have already been deduplicated at the target system in a deduplication system by comparing segment identifiers (segment IDs) for the segments to match segment IDs for segments already deduplicated, wherein segment IDs are evaluated by transmitting segment IDs to the target system to compare the segment IDs with entries in a deduplication dictionary; transmitting a first plurality of segments over a network to the target system by using the file level protocol, the first plurality of segments not already deduplicated at the target system, and not transmitting a second plurality of segments to the target system to reduce network bandwidth utilization, the second plurality of segments already deduplicated at the target system; modifying references and reference counts at the target system for the first plurality of segments and the second plurality of segments, wherein modifying references and reference counts includes modifying a filemap, dictionaries, and a datastore suitcase for both already deduplicated and not already deduplicated segments; wherein the dictionary is used to identify duplicate data segments and point to the location of the data segments, the datastore suitcase includes an index portion and a data portion to allow a system to perform a bulk read of the index portion to obtain offset data to allow parallel reads of large amounts of data in the data section and further includes three offset, reference count pairs which map to the data segments of the filemap file.