Cloud computing providers must handle customer workloads that wish to scale their use of resources , such as virtual machines, up and down over time.  Currently, this is often done using simple threshold policies to reserve large parts of each cluster . This leads to low average utilization of the cluster. In this paper, we propose more sophisticated Bayesian policies for controlling admission to a cluster and demonstrate that they significantly increase cluster utilization. We first introduce a model for the cluster admission problem and fit its parameters on a data trace from Microsoft Azure. We then design Bayesian cluster admission policies that estimate moments of each workload's distribution of future resource usage. Via simulations we show that , while estimating the first moments of workloads leads to a substantial improvement over the simple threshold policy , also taking the second moments into account yields another improvement in utilization . We then evaluate how much further this can be improved with learned or elicited prior information and how to incentivize users to provide this information.
Cloud computing providers must handle heterogeneous customer workloads for resources such as (virtual) CPU or GPU cores. This is particularly challenging if customers, who are already running a job on a cluster, scale their resource usage up and down over time. The provider therefore has to continuously decide whether she can add additional workloads to a given cluster or if doing so would impact existing workloads' ability to scale. Currently, this is often done using simple threshold policies to reserve large parts of each cluster , which leads to low average utilization of the cluster. In this paper, we propose more sophisticated Bayesian policies for controlling admission to a cluster and demonstrate that they significantly increase cluster utilization. We first introduce the cluster admission problem and formalize it as a constrained Partially Observable Markov Decision Problem (POMDP). We then fit the parameters of the POMDP on a data trace from Microsoft Azure. As it is infeasible to solve the POMDP optimally, we then systematically design heuristic Bayesian admission policies that estimate moments of each workload's distribution of future resource usage. Via simulations we show that our Bayesian admission policies lead to a substantial improvement over the simple threshold policy . We then evaluate how much further this can be improved with learned or elicited prior information and how to incentivize users to provide this information.
R
customer workloads that wish to scale their use of resources , such as virtual machines,
heterogeneous customer workloads for resources such as (virtual) CPU or GPU cores. This is particularly challenging if customers, who are already running a job on a cluster, scale their resource usage
A
The provider therefore has to continuously decide whether she can add additional workloads to a given cluster or if doing so would impact existing workloads' ability to scale.
R
. This
, which
D
a model for
R
fit its parameters
formalize it as a constrained Partially Observable Markov Decision Problem (POMDP). We then fit the parameters of the POMDP
R
We then design Bayesian cluster
As it is infeasible to solve the POMDP optimally, we then systematically design heuristic Bayesian
R
, while estimating the first moments of workloads leads
our Bayesian admission policies lead
D
, also taking the second moments into account yields another improvement in utilization