stratosphere/stratosphere — pull request #540 by tillrohrmann (2014-03-07 09:56:41+00:00)
Title: Reuse of already generated OptimizerNodes in the recursive descent step of GraphCreatingVisitor

I tried to tackle the problem of generating multiple OptimizerNodes for one and the same Operator during recursive descent steps of the GraphCreatingVisitor as they happen for the step function generation of iterations. This poses a problem, because flows with multiple sinks, where one of the inputs to the step function is outputted as well, will be wrongly recognised as an unconnected data flow. Moreover, in the case of an Operator which produces random results, the generated data of this Operator inside and outside of the iteration would be distinct and thus false.

My solution is to forward the generated OptimizerNodes to the recursive GraphCreatingVisitor. This implies, however, to consider the step function for the computation of the unclosed branch stack. Furthermore, the branch plans of the step function has to be taken account of as well when merging the branch plans of the iteration plan nodes. For the WorksetIterationPlanNode, I could only come up with a nasty hack to disable the mergeBranchPlanMaps of the DualInputPlanNode.

So far, all test cases succeed locally but I'm not sure whether I found all code positions requiring a change for this modification. Would be great if you could review it to make sure that I didn't forget anything.