working on using a Spark or MapReduce job and Faunus to ingest data