Banno/samza-mesos — pull request #2 by zcox (2014-12-26 17:12:20+00:00)
Title: New task ID for failed tasks

Previously, when the Samza container Mesos task would fail, the Samza scheduler would run the exact same Mesos task again, with the same taskId. This replacement task would be killed by its executor immediately, and then the Samza scheduler would re-run this task again a 2nd time. This 2nd re-run would actually succeed and keep running.

I think by keeping the same taskId, the 1st replacement task would get assigned to the same executor, which was just in the process of cleaning up after the original task failed. This executor would then kill the 1st replacement task. Then I think the 2nd replacement task would be a fresh new executor and it would all work properly.

This PR essentially just gives a failed task a new taskId, and this seems to work better - the 1st replacement task runs properly. This is also essentially what Marathon does - failed tasks get new taskIds when they are re-run.