Patent ID: 8707320
Filing Date: 2014-04-22
Classification: G06F

Abstract:
1. A method of partitioning items in a data stream, the method comprising: receiving a data stream having a plurality of items of application data, wherein an amount of items in the data stream is at least one of unknown and undeclared; partitioning the received data stream into a plurality of data chunks including at least one item, wherein each data chunk has a data chunk size n number of items in the data chunk; assigning the plurality of data chunks having the same data chunk size n to a plurality of concurrent threads for processing on a multiple node processing system; and increasing the data chunk size n according to a sequence having at least one repeated power of two with occasional doubling up to a selected maximum data chunk size, wherein the sequence includes a selected number of the same powers of two before doubling; wherein the selected number is determined from a function f(x)=2 wherein the partitioning, assigning, and increasing are repeated until the plurality of items are consumed.