mesos/spark — pull request #428 by woggling (2013-01-30 18:03:22+00:00)
Title: Make ExecutorIDs include SlaveIDs when running Mesos

The recent change to identify spark executors by unique ids assumed that Mesos executor IDs would be suitable for this purpose. Unfortunately, Mesos executor IDs are only unique on a given slave.

An ideal fix would be to produce a Spark executor ID from the Mesos SlaveID and Mesos ExecutorID. Surprisingly, however, the SlaveID is not easily available from the executor (https://issues.apache.org/jira/browse/MESOS-342). As an alternative, this patch would have Spark specify the SlaveID string as the ExecutorID when launching Mesos tasks.

This is a rather hacky fix; some alternatives I'd appreciate opinions on:
- use the slave hostname (which Mesos does provide, but one may want to run multiple Mesos slaves on the same host); or
- serialize the SlaveID in the ExecutorInfo data field along with the system properties (but we would have to do some funny encoding or reserialize the system properties for each Slave ID we see)