One other comment on style -- we don't usually write the types of variables in methods when they can be inferred. Would probably be best to leave those out unless it's not clear to the reader what a type is (e.g. this line here is fine to make sure you get an RDD[Double]).