rengolin/HPCC-Platform — comment on commit ae87d27245950268be53ac7d9fea3defb4d9d5af
File:  line 0
By rengolin:

Regarding discontinuous ranges... Some random thoughts...

If a consumer fails and has the power to re-try indefinitely, than you are right.

If upon failure, a consumer dies and gives the sub-range back for grabbing, than we need discontinuous ranges.

On the simplistic case where the data and the execution are static, the first case is appropriate. After the Nth attempt, just fail the whole job and contact the developer.

On the indefinite streaming case, the code might be wrong (compiled wrong, or wrong code, or outdated volatile code), and you don't want it to stop processing data. You just re-compile and you have a new consumer that could grab that failed sub-range.

If we need the same consumer to process it (because ranges are continuous), than we need to implement the hot-swap for volatile code first, which is also not a bad idea...

As you said, we need to see code implemented before start second-guessing ideas. I'll revert to continuous ranges and we deal with the problem later.