hpcc-systems/HPCC-Platform — pull request #5034 by AttilaVamos (2013-10-21 16:23:40+00:00)
Title: HPCC-10155 Prevent DFU Spray to output 100,000's of "CSV splitRecordSize(65536) at end of file" for hours.

This change implement buffer overruns monitoring. Every time when
getSplitRecordSize() reach at the end of the current buffer (and it is not
forced by processFullBuffer parameter) it counts the event and cumulate the
processed bytes. When the number of events exceeds the
MAX_NUMBER_OF_BUFFER_OVERRUN value it generates an exception to break this
possible wrong process.

This changes impacts both CCsvPartionier and CUtfPartitioner.

(This prevention is already implemented into corresponding QuickPartitioners)

Signed-off-by: Attila Vamos