PredictionIO/PredictionIO — pull request #13 by leohackin (2014-07-29 12:24:28+00:00)
Title: Experimental BulkCsvUploader

Hi guys,

I faced on the last days the slow upload data problem when try to send my data to PredictionIO server and after some googling I found in the user forum discussions the experimental bulk upload tool. This was the path to solve this problem. :)

I'm starting with Recommendations, Mahout, PredictionIO and I have a little experience in Java/Scala.

Thinking about how to make this upload more easy, I implemented a extended version of the original bulkUploader to work with CSV files. The main idea is enable the user to upload items, users and U2Iactions in distinct files to specific Apps.

I'm not implemented the unit tests because not have any to the original toolsBulkUpload. I'm imagining that is because a experimental stuff.

In the tests, under the Amazon MarketPlace pre-built PredictionIO machine with 8GB of memory, I uploaded 3000000 u2iactions in 7,8 minutes.

I hope this help a little.