A method for executing graph-based computations, the method including: accepting a specification of a computation graph in which data processing elements are joined by linking elements, wherein each of the linking elements is associated with a data flow from an output of an upstream data processing element to an input of a downstream data processing element; dividing the data processing elements into sets, (210, 211, 212) at least one of the sets including multiple of the data processing elements; assigning to each set a different computing resource; processing data according to the computation graph, including performing computations corresponding to the data processing elements using the assigned computing resources, and for at least one of the sets, performing computations corresponding to the data processing elements in the set according to a sequence defined by linking elements joining the data processing elements in the set; wherein one of the computing resources, which is assigned to one of the sets of data processing elements, passes work elements associated with a data flow from an output of an upstream data processing element to an input of a downstream data processing element in the same set as the upstream data processing element by calling a function associated with the downstream data processing element; wherein the data processing elements in a set form a connected directed subgraph of the computation graph; and in response to receiving a checkpoint control message for a data processing element in the subgraph that does not have an upstream element in the subgraph, calling functions associated with each of the other data processing elements in the subgraph to save state information for each data processing element in the subgraph in non-volatile storage before propagating the control message to a data processing element outside of the subgraph. accepting a specification of a computation graph in which data processing elements are joined by linking elements, wherein each of the linking elements is associated with a data flow from an output of an upstream data processing element to an input of a downstream data processing element; dividing the data processing elements into sets, (210, 211, 212) at least one of the sets including multiple of the data processing elements; assigning to each set a different computing resource; processing data according to the computation graph, including performing computations corresponding to the data processing elements using the assigned computing resources, and for at least one of the sets, performing computations corresponding to the data processing elements in the set according to a sequence defined by linking elements joining the data processing elements in the set; wherein one of the computing resources, which is assigned to one of the sets of data processing elements, passes work elements associated with a data flow from an output of an upstream data processing element to an input of a downstream data processing element in the same set as the upstream data processing element by calling a function associated with the downstream data processing element; wherein the data processing elements in a set form a connected directed subgraph of the computation graph; and in response to receiving a checkpoint control message for a data processing element in the subgraph that does not have an upstream element in the subgraph, calling functions associated with each of the other data processing elements in the subgraph to save state information for each data processing element in the subgraph in non-volatile storage before propagating the control message to a data processing element outside of the subgraph.