Patent Document ID: 10162867
Application ID: 14994163
Patent Status: 1

Claim One:
1. A method of determining a number of distinct logical data units in a dataset, comprising: randomly selecting, from a dataset partitioned into a set of logical data units, a base subset comprising a specified number of the logical data units; randomly selecting from the dataset, a number of logical data units determined by a specified sampling ratio and a number of logical data units in the dataset, to serve as a sample subset of the set of logical data units, together with the logical data units in the base subset; calculating, by a processor, respective unit hash values for each of the logical data units in the sample subset; computing a first histogram indicating a duplication count in the sample subset, of each of the logical data units in the base subset, using the calculated unit hash values; deriving, from the first histogram, a second histogram indicating respective frequencies of duplication counts in the sample subset, of the logical data units in the base subset; deriving, a third histogram of predicted duplication counts in the set of logical data units, by performing an optimization method, with a target function that minimizes a distance between the second histogram and the result of applying a sampling transformation with the specified sampling ratio on candidate third histograms; and determining, based on the third histogram, a number of distinct logical data units in the dataset.