A similar commitJob api exists even for mapred api - addition of that would be sufficient (instead of moving to new hadoop api) : it would also mean fixing that spark impl