Project: mesos
Issue MESOS-3896: Add accounting for reservation slack in the allocator.
MESOS-XXX: Optimsistic accounter {code} class HierarchicalAllocatorProcess { struct Slave { ... struct Optimistic { Resources total; // The total allocation slack resources Resources allocated; // The allocated allocation slack resources }; Optimistic optimistic; }; } {code} MESOS-4146: flatten & allocationSlack for Optimistic Offer {code} class Resources { // Returns a Resources object with the same amount of each resource // type as these Resources, but with all Resource objects marked as // the specified `RevocableInfo::Type`; the other attribute is not // affected. Resources flatten(Resource::RevocableInfo::Type type); // Return a Resources object that: // - if role is given, the resources did not include role's reserved // resources. // - the resources's revocable type is `ALLOCATION_SLACK` // - the role of resources is set to "*" Resources allocationSlack(Option role = None()); } {code} MESOS-XXX: Allocate the allocation_slack resources to framework {code} void HierarchicalAllocatorProcess::allocate( const hashset& slaveIds_) { foreach slave; foreach role; foreach framework { Resource optimistic; if (framework.revocable) { Resources total = slaves[slaveId].optimistic.total.allocationSlack(role); optimistic = total - slaves[slaveId].optimistic.allocated; } ... offerable[frameworkId][slaveId] += resources + optimistic; ... slaves[slaveId].optimistic.allocated += optimistic; } } {code} Here's some consideration about `ALLOCATION_SLACK`: 1. 'Old' resources (available/total) did not include ALLOCATION_SLACK 2. After `Quota`, `remainingClusterResources.contains` should not check ALLOCATION_SLACK; if there no enough resources, master can still offer ALLOCATION_SALCK resources. 3. In sorter, it'll not include ALLOCATION_SLACK; as those resources are borrowed from other role/framework 4. If either normal resources or ALLOCATION_SLACK resources are allocable/!filtered, it can be offered to framework 5. Currently, allocator will assign all ALLOCATION_SALCK resources in slave to one framework MESOS-XXX: Update ALLOCATION_SLACK for dynamic reservation (updateAllocation) {code} void HierarchicalAllocatorProcess::updateAllocation( const FrameworkID& frameworkId, const SlaveID& slaveId, const vector& operations) { ... Try updatedOptimistic = slaves[slaveId].optimistic.total.apply(operations); CHECK_SOME(updatedTotal); slaves[slaveId].optimistic.total = updatedOptimistic.get().stateless().reserved().flatten(ALLOCATION_SLACK); ... } {code} MESOS-XXX: Add ALLOCATION_SLACK when slaver register/re-register (addSlave) {code} void HierarchicalAllocatorProcess::addSlave( const SlaveID& slaveId, const SlaveInfo& slaveInfo, const Option& unavailability, const Resources& total, const hashmap& used) { ... slaves[slaveId].optimistic.total = total.stateless().reserved().flatten(ALLOCATION_SLACK); ... } {code} No need to handle `removeSlave`, it'll all related info from `slaves` including `optimistic`. MESOS-XXX: return resources to allocator (recoverResources) {code} void HierarchicalAllocatorProcess::recoverResources( const FrameworkID& frameworkId, const SlaveID& slaveId, const Resources& resources, const Option& filters) { if (slaves.contains(slaveId)) { ... slaves[slaveId].optimistic.allocated -= resources.allocationSlack(); ... } } {code}
Story points: 13