liebke/incanter — pull request #95 by jstepien (2012-10-10 16:13:00+00:00)
Title: incanter.io/read-dataset reads on a per-line basis

`read-dataset` used `CSVReader.readAll` which parsed the whole file at once. In case of larger CSV files it caused some major headaches for the GC. This patch makes CSVReader work on a per-line basis by using `CSVReader.read` and fully parsing each line before moving to a next one.

A following benchmark was used to measure effects of this patch. 1k.csv is a 1k lines long dataset with 700 numerical features. It's around 1.7MB.

cat <