Patent ID: 8949801
Filing Date: 2015-02-03
Classification: G06F

Abstract:
1. A method for providing a failure recovery mechanism for a stream processing application comprising a plurality of components, the method comprising: receiving, by a compiler, source code for the stream processing application, wherein the source code defines a checkpointing frequency with which a checkpoint file is to be generated for each of the plurality of components, and wherein respective checkpointing frequencies defined for at least two of the plurality of components are different, and wherein different respective checkpointing frequencies are defined for the at least two of the plurality of components based on a parameter that differs between the at least two of the plurality of components, the parameter being at least one of: whether the at least two of the plurality of components are stateful or stateless, whether the at least two of the plurality of components are windowed stream operators, or sizes of respective states of the at least two of the plurality of components, wherein a stateful component is a component that operates on state variables, and wherein a stateless component is a component that does not operate on state variables; generating, by the compiler, a sequence of instructions for converting a state of the at least one of the plurality of components into a checkpoint file comprising a sequence of storable bits on a periodic basis, according to the checkpointing frequency defined for the at least one of the plurality of components; initiating execution of the stream processing application; and storing, during execution of the stream processing application, the checkpoint file at a location that is accessible after failure recovery.