Project: Volcano (Orchestration & Management / Scheduling & Orchestration)
File: queue-resource-reservation-design.md

@; Nov 3rd, 2020 As mentioned, Volcano should support resource reservation for specified queue. Requirement detail as follows: Support reserving specified resources for specified queue We only Consider non-preemption reservation. Support enable and disable resource reservation for specified queue dynamically without restarting Volcano. Support hard reservation resource specified and percentage reservation resource specified. The request cannot be more than the total resource amount of cluster at all dimensions. If `capability` is set, request must be no more than it at all dimensions. The resource amount reserved must be no less than request at all dimensions but should not exceed too much. An algorithm ensuring the reserved amount to the point is necessary. Support total node resource percentage of cluster as request. If reservation resource amount is also specified, it's important to decide which configuration is adopted. This feature is more useful on condition that resource specification of all nodes are almost the same. reservation needs gentle treatment to balance scheduling and reserving performance. Nodes locked by target job cannot be chosen as locked node. Nodes locked by another queue cannot be chosen as locked node. Try to lock nodes, whose total resource and nodes numbers is as less as possible on base of satisfying requirement, to avoid dramatic scheduling performance degradation. Malicious application for large amount of resource reservation will cause jobs in other queue to block. ``` apiVersion: scheduling.volcano.sh/v1beta1 kind: Queue metadata: name: q1 spec: reclaimable: true weight: 1 guarantee: // reservation key word policy: Best-Effort // preemption reservation or non-preemption reservation percentage: // locked nodes resource percentage in cluster dimensions: ["cpu", "memory, "gpu", "other-scalable-resource-type"...] value: 0.2 resource: // specified reserving resource cpu: 2c memory: 4G status: state: Open reservation: // reservation status key word nodes: [n1, n2] // locked nodes list resource: // total idle resource in locked nodes cpu: 1C memory: 2G ``` Option Field, options are "Best-Effort" and "Guaranteed" and "Best-Effort" by default. If "Best-Effort" is set, scheduler will reserve resource without evicting running workloads, while "Guaranteed" is opposite. Option Field if `resource` not specified, legal value range is [0, 1]. Locked node number will be as follows: ``` math.Floor(clusterNodeNumber * percentage) ``` It should be noted that nodes will be locked randomly util reaching the

If `resource` is also specified, choose the one with more reserved resources as the final result. Option Field if `percentage` not specified. List of reserving resource categories and amount. List of locked nodes. Total idle resource of locked nodes. You can judge whether reservation has met demand by watching this field. Sort all nodes combination which satisfy the queue request resource in all dimensions. and these situations will effect the order of the nodes combination. sum(nodes's resource) the sum of the combination nodes's resource is lower,the priority is higher. and the weight is 0.4 queue already used the more resources are used on the nodes that belong to the queue,the priority is higher. and the weight is 0.35 sum(nodes's number) the nums of the combination nodes is lower,the priority is higher. and the weight is 0.15 sum(nodes's idle resource) the more idle resources on the nodes,the priority is higher. and the weight is 0.1 sum : the sum of the combination nodes's resource. target: the target queue's request resource. used : the sum of used resources on the nodes. n : the nums of the combination nodes. idle : the sum idle of resources on the nodes. We hope the value of (sum-target) is as less as possible to avoid wasting of the cluster resources. and this should have the hightest weight. We condider the element 'used' will effect the locking nodes's efficiency and stable. When the above two conditions's score are almost close. we preffer the The least number of combination nodes as the best choice,cause the fewer nodes lokced,The smaller the impact on the cluster. The least important condition is the idle of the nodes,the more idle the nodes are, the more efficient it is to lock on. ``` 0.41/(sum-target)/[(sum-target)+used+n+idle] + 0.35used/[(sum-target)+used+n+idle] + 0.151/n/[(sum-target)+used+n+idle] + 0.1idle/[(sum-target)+used+n+idle] ``` schedule relock for every 5s or 10s. the worker in the queueController will find the best combination nodes for a queue. and update the queue's locked nodes,and caculate the idle resources on the locked nodes. Add a new worker in the queue_controller: reserveWorker. reserveWorker aims to take care of the queue which has the requirement of reservation ,including finding and updating the node details for the queue to lock, updating the queue reservation status. Add new Plugin node_reservation to implement algorithm detail above.