updates for spark