hpcc-systems/HPCC-Platform — pull request #4383 by AttilaVamos (2013-05-15 09:24:41+00:00)
Title: HPCC-8477 - DFU server is CPU bound for the first phase of CSV files

These fix improves CSV file spray performance when the target is multiple
nodes.

Changes:

- In this particular case a new CCsvMatcher class introduced because the
original matcher was a bottleneck. If the Special characters of CSV file
(separator, terminator and quote) are doesn't change or change to only one
character long new character with a command line parameter, the the process
use CCsvMatcher. Any other case the old matcher used.

- The used blocksize increased to decrease IO overload.

- When the split size is bigger than the blocksize
CCsvPartitioner::getSplitRecordSize() function returns with the summarised
record size in the buffer instead of individual one until it reaches the last
part ( which is < blocksize). For the last part is switch back to returns
with individual record size.

Signed-off-by: Attila Vamos