votinginfoproject/Metis — pull request #221 by tank157 (2014-08-05 15:52:45+00:00)
Title: Use different csv parser

Changed the csv parser from fast-csv to a more popular node-csv parser. This parser had an admittedly undocumented relaxed parsing mode that made it more resilient to some csv formatting issues. Without the relaxed mode turned on, it would error out on the WI feed I'm testing with, and would give line numbers, but the line numbers didn't correspond to any lines that had actual problems. So turning on the relaxed mode seemed better, and allowed it to process an unadulterated feed.

I also had to try turning on a mongoose aggregate mode that allows it to use disk for some processing, otherwise the unique id check was too large to hold in mongo's memory.

Assigning to @cap10morgan