Patent ID: 11907077
Assignee: DELL PRODUCTS L.P.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 9:
10. A method of providing a dynamic prefetch process based on file locality in a deduplication system, comprising:
tracking a locality of data elements in a batched data stream, wherein the locality per batch corresponds to a number of different container IDs (CIDs) accessed by the data elements;
determining whether the locality within a batch exceeds a threshold separating acceptable versus non-acceptable distribution of data accesses by the data elements over a plurality of container IDs;
processing the data elements in a serial prefetch process for a batch that does not exceed the threshold;
dividing a batch into a plurality of smaller sub-batches for a batch that does exceed the threshold and executing the prefetch process in parallel for the sub-batches; and
determining a load on the system based on at least one of: number of streams, number of background jobs, and I/O response time, wherein a numerically higher threshold value indicates closer locality among data elements based on a lower number of CIDs accessed, and a numerically lower threshold value indicates a higher degree of distribution of the data elements among a greater number of CIDs accessed, the method further comprising determining a number of smaller sub-batches dividing the batch based on a locality a first batch and the load.