Patent ID: 8930648
Filing Date: 2015-01-06
Classification: G06F

Abstract:
1. A method, comprising: storing a global chunk data structure for a data storage cluster in a distributed manner among two or more of a plurality of metadata nodes of the data storage cluster, the global chunk data structure containing data fingerprints, wherein the two or more metadata nodes respectively maintain at least a portion of the global chunk data structure; recording fingerprints of new data written to the data storage cluster into a plurality of delta chunk data structures, wherein each of the delta chunk data structures is stored in a different one of the plurality of metadata nodes; requesting, by a metadata node of the plurality of metadata nodes, permission to lead the plurality of metadata nodes in a metadata epoch transition; in response to receiving permission from every other metadata node of the plurality of metadata nodes, updating a version of the global chunk data structure corresponding to a particular epoch to a consistent state based on versions of the delta chunk data structures corresponding to the particular epoch; and using the updated global chunk data structure to identify duplicate data in the data storage cluster.