mesos/spark — pull request #317 by woggling (2012-12-06 08:23:22+00:00)
Title: Implement block manager heartbeat

Implement heart beats for the BlockManagers. BlockManagers periodically send HeartBeat messages to the master, and the master periodically deregisters slaves which have not sent a heart beat in about 3 heart beat periods. The BlockManagerMaster will now indicate when a BlockManager updates a block or sends a heart beat but is not registered. In this case, the BlockManager will reregister and resend block updates for all its blocks.

Other changes made along the way:
- deregistering a BlockManager and forgetting about its blocks is implemented in BlockManagerMaster;
- the old "HeartBeat" message is renamed to "BlockUpdate";
- the BlockManagerMaster tracks the BlockManagerId for each hostname. If a new BlockManagerMaster registers for a hostname, the old one is deregistered. This tracking incidentally fixes a bug where notifyADeadHost() would try to remove a BlockManagerId that usually wouldn't exist;
- local-cluster[...] mode uses distinct hostnames -- 127.100.0. (which should be a valid name for localhost like all addresses in 127/8), which permits BlockManager to behave correctly in spite of multiple executors on the local node;