apache/spark — pull request #3849 by JoshRosen (2014-12-30 22:07:30+00:00)
Title: [SPARK-4014] Change TaskContext.attemptId to return attempt number instead of task ID

This patch modifies `TaskContext.attemptId` to return an attempt number, which conveys how many times a task has been attempted, instead of a taskId, which uniquely identifies a particular task attempt within a particular SparkContext. Prior to this change, it was impossible to determine whether a task was being re-attempted (or was a speculative copy), which made it difficult to write unit tests for tasks that fail on early attempts or speculative tasks that complete faster than original tasks.

I've introduced a new `TaskContext.taskId` field which returns the old value.

Most of this patch is fairly straightforward, but there is a bit of trickiness related to Mesos tasks: since there's no field in MesosTaskInfo to encode the attemptId, I packed it into the `data` field alongside the task binary.