Properly deal with removed hosts in BlockManagerMaster