Patent ID: 8291005

Claim:
A computer implemented method for providing consistency guarantees in a data stream processing engine, the computer implemented method comprising: attaching consistency tracking information that includes synchronization tokens to data streams coming into the data stream processing engine; propagating the consistency tracking information through a plurality of streaming operators that process the data streams within the data stream processing engine to form propagated consistency tracking information; responsive to an operator within the plurality of streaming operators receiving an event from an inbound data stream, determining whether the event is a synchronization token; responsive to determining that the event is a synchronization token, determining whether a set of synchronization tokens is complete, wherein the set of synchronization tokens includes the synchronization token and a corresponding synchronization token from each other inbound data stream to the operator; responsive to determining that the set of synchronization tokens is complete, creating a new synchronization token; pushing the new synchronization token into an output queue, wherein a presence of the new synchronization token within the output queue represents a consistent output state for all preceding processed events by the operator; and using the propagated consistency tracking information to detect a consistent state in an output stream.