Patent ID: 8489967

Claim:
A method, in a data processing system, for processing streaming data at high sustained data rates, comprising: receiving a plurality of data elements, in the data processing system, over a plurality of non-sequential communication channels; writing the plurality of data elements directly to a file system of the data processing system in an unassembled manner; determining whether to perform a data scrubbing operation or not based on history information indicative of whether data elements in the plurality of data elements are being received in a substantially sequential manner; performing a data scrubbing operation, in response to a determination to perform data scrubbing, to determine if there are any missing data elements that are not present in the plurality of data elements written to the file system; and assembling the plurality of data elements into a plurality of data streams associated with the plurality of non-sequential communication channels in response to results of the data scrubbing indicating that there are no missing data elements.