both spark and sparse implementations for testing