Patent ID: 7730077

Claim:
A Programmable Streaming Data Processor (PSDP), arranged to perform primitive initial processing functions directly on a set of data comprising: a streaming data interface, arranged to receive non-field delineated data from a streaming data source; a streaming interface First In First Out (FIFO), arranged to temporarily store the streaming non-field delineated data from the streaming data interface; a data engine, arranged to receive the non-field delineated output data from the streaming interface FIFO, recognize the record and field structure of the non-field delineated data, determine field boundaries in the non-field delineated data, and process fields to select one or more fields to be assembled into output tuples, the data engine also containing logic arranged to determine whether an output tuple is to be selected for further processing by additional Job Processing Units (JPUs) and to assert a use or lose decision value according to that determination; a tuple generator, arranged to assemble fields into the output tuple and, if the use or lose decision value indicates that such output tuple is to be discarded, to prevent the output tuple from being transferred for further processing by the JPU; and an output FIFO device, arranged to temporarily store tuples prior to conditionally forwarding them to the JPU.