Patent ID: 8171004

Claim:
A method, comprising: retrieving a hostname, the hostname being evaluated to determine if an address is associated with the hostname; detecting the address, the address being further processed to download a first file identified by the address if the first file is associated with the hostname or storing the address if the address points to another hostname; identifying a standardized portion of the first file by identifying a data set to be selected consistently from the first file, wherein the data set is identified based on a size and a location associated with the first file; running a first hashing algorithm against the standardized portion of the first file to generate a first hash value; determining whether the first hash value is the same as or substantially similar to another hash value associated with a standardized portion of a second file; in response to determining that the first hash value is the same as or substantially similar to the hash value associated with the standardized portion of the second file, running a second hashing algorithm different from the first hashing algorithm against the standardized portion of the first file to generate a second hash value different from the first hash value; and storing the second hash value and the address associated with the first file.