refined spark implementation
refinements to documentation