Patent ID: 7401080
Filing Date: 2008-07-15
Classification: G06F,Y10S

Abstract:
1. In a computing environment a method for performing time and memory efficient detection of duplicate files in a file system, the method comprising: performing a first scan of a plurality of files in a file system wherein the scan retrieves metadata from each file; retrieving file information from the metadata, the file information comprising a set of attributes for each file; performing a hash of at least one attribute of the set for each file into a hash value; storing the hash value in a hash vector; prior to storing the hash value in the hash vector, and upon determining that the hash vector already contains an entry corresponding to the hash value, incrementing a data value associated with the hash value, the data value representing an occurrence of at least some potentially duplicate files, said potentially duplicate files including at least one duplicate file; creating an entry in a duplicate file map that corresponds to said potentially duplicate files; discarding the hash vector; performing a second scan of the plurality of files to locate file information for at least some of the potentially duplicate files that correspond to the entry in the duplicate file map, including file information for said at least one duplicate file; and determining from the file information which files from among the groups of potentially duplicate files are actual duplicate files of at least one other file, including said at least one duplicate file.