Fixing out of memory errors during CSV data import


I have tried to import one CSV data file about 1GB into a VoltDB cluster with 2 nodes both having 16GB RAM and got out of memory errors.

After examining the details of CSV import process, I found that when the CSV data is not very clear (such as many duplicate records) and may cause many failures (such as CONSTRAINT VIOLATION). The failed queue used to hold individual rows for insertion will be soon fully filled too many requests that consume too much memory on the machine. 

This PR is intended to fix this problem. I also noticed that invalid rows are kept in memory until data insertion is done and then are flushed in to a file on disk. It also consumes too much memory space when there are too many invalid rows. So this PR makes modification on that behavior to progressively flush invalid rows into file.

Your concern is right. However, since the threshold for invoking the flushing is low now, the influence might not be so high to block network message processing. At least I did not observe such effect during my test. The threshold is low because the idea is to store as less as possible error info in `m_errorInfo` so it would not consume too much memory.

This might be an extreme case because normally there would be so many failed data rows. That is because my data (not generated by myself, obtained from a vendor) is not very clear.

I am not very sure that if I can change `m_errorInfo` to `LinkedBlockingQueue` since `m_errorInfo` not only keeps error info but also line number as key.