Patent ID: 8463928

Claim:
A method for managing data streams, comprising: receiving a data stream on at least one network interface card; performing operations on the data stream using a first process running a plurality of first threads for the at least one network interface card, the operations comprising: distributing at least some portions of the data stream among a plurality of buffers for processing by the plurality of first threads, processing the at least some portions of the data stream with a dedicated first process thread, wherein the dedicated first process thread is one of the plurality of first threads, and storing the at least some portions of the data stream that are processed in one of a plurality of dedicated shared memories; performing operations on the at least some portions of the data stream that are processed using at least one group of second multiple processes with at least one group of second threads comprising: calculating in-memory interval statistics and periodically saving the in-memory interval statistics to an interval file; processing the at least some portions of the data stream stored in the one of the plurality of dedicated shared memories with a dedicated second process thread, comprising: reading the at least some portions of the data stream from the shared memories, and calculating statistics on the at least some portions of the data stream read from the shared memories and periodically saving the statistics to the interval file, wherein the dedicated second process thread is one of the second threads; wherein the first process and the one or more groups of second multiple processes are independent and communicate via the shared memory; and wherein the plurality of first threads for the network interface cards are different than the at least one group of second threads.