Patent ID: 9325757
Filing Date: 2016-04-26
CPC Classification: G06F,H04L

Claim Text:
1. A method of achieving fault tolerance in a distributed stream processing system including a plurality of computer nodes organized as a directed acyclic graph, wherein at least a portion of the plurality of computer nodes performs the method comprising the steps of: managing a stream process within the distributed stream processing system including at least one operator executed by a computer node in the directed acyclic graph, the at least one operator communicatively associated with at least one downstream operator executed by a computer node in the directed acyclic graph; maintaining at least one data copy of a processing state of the at least one operator until the at least one data copy can be safely discarded; notifying the at least one operator when it is safe to discard at least one of the at least one data copy of the processing state; and using an identifier to denote the data copy of the processing state to be safely discarded, wherein the at least one operator forwards the at least one data copy in one or more batches and forwards the one or more batches to the at least one downstream operator having an equal or higher batch ID.