We consider learning-based variants of the c \mu rule -- a classic and well-studied scheduling policy -- in single and multi-server settings for multi-class queueing systems. In the single server setting, the c \mu rule is known to minimize the expected holding-cost (weighted queue-lengths summed both over classes and time ). We focus on the setting where the service rates \mu are unknown , and are interested in the holding-cost regret -- the difference in the expected holding-costs between that induced by a learning-based rule (that learns \mu) and that from the c \mu rule (which has knowledge of the service rates) over any fixed time horizon. We first show that empirically learning the service rates and then scheduling using these learned values results in a regret of holding-cost that does not depend on the time horizon . The key insight that allows such a constant regret bound is that a work-conserving scheduling policy in this setting allows explore-free learning, where no penalty is incurred for exploring and learning server rates. We next consider the multi-server setting. We show that in general, the c \mu rule is not stabilizing (i.e. there are stabilizable arrival and service rate parameters for which the multi-server c \mu rule results in unstable queues ) . We then characterize sufficient conditions for stability (and also concentrations on busy periods) . Using these results, we show that learning-based variants of the c\mu rule again result in a constant regret (i. e. does not depend on the time horizon). This result hinges on (i) the busy period concentrations of the multi-server c \mu rule, and that (ii) our learning-based rule is designed to dynamically explore server rates, but in such a manner that it eventually satisfies an explore-free condition .
We consider learning-based variants of the c \mu rule for scheduling in single and parallel server settings of multi-class queueing systems. In the single server setting, the c \mu rule is known to minimize the expected holding-cost (weighted queue-lengths summed over classes and a fixed time horizon ). We focus on the problem where the service rates \mu are unknown with the holding-cost regret (regret against the c \mu rule with known \mu) as our objective. We show that the greedy algorithm that uses empirically learned service rates results in a constant holding-cost regret (the regret is independent of the time horizon ). This free exploration can be explained in the single server setting by the fact that any work-conserving policy obtains the same number of samples in a busy cycle. In the parallel server setting, we show that the c \mu rule may result in unstable queues , even for arrival rates within the capacity region . We then present sufficient conditions for geometric ergodicity under the c \mu rule . Using these results, we propose an almost greedy algorithm that explores only when the number of samples falls below a threshold. We show that this algorithm delivers constant holding-cost regret because a free exploration condition is eventually satisfied .
R
-- a classic and well-studied scheduling policy --
for scheduling
R
multi-server settings for
parallel server settings of
D
both
R
time
a fixed time horizon
R
setting
problem
R
, and are interested in
with
R
-- the difference in the expected holding-costs between that induced by a learning-based rule (that learns \mu) and that from
(regret against
R
(which has knowledge of the service rates) over any fixed time horizon. We first show that empirically learning the service rates and then scheduling using these learned values
with known \mu) as our objective. We show that the greedy algorithm that uses empirically learned service rates
R
regret of holding-cost that does not depend on
constant holding-cost regret (the regret is independent of
R
. The key insight that allows such a constant regret bound is that a
). This free exploration can be explained in the single server setting by the fact that any
R
scheduling policy in this setting allows explore-free learning, where no penalty is incurred for exploring and learning server rates. We next consider the multi-server setting. We show that in general,
policy obtains the same number of samples in a busy cycle. In the parallel server setting, we show that
R
is not stabilizing (i.e. there are stabilizable arrival and service rate parameters for which the multi-server c \mu rule results
may result
R
)
, even for arrival rates within the capacity region
R
characterize
present
R
stability (and also concentrations on busy periods)
geometric ergodicity under the c \mu rule
R
show that learning-based variants of the c\mu rule again result in a constant regret (i. e. does not depend on the time horizon). This result hinges on (i) the busy period concentrations of the multi-server c \mu rule, and that (ii) our learning-based rule is designed to dynamically explore server rates, but in such a manner that it eventually satisfies an explore-free condition
propose an almost greedy algorithm that explores only when the number of samples falls below a threshold. We show that this algorithm delivers constant holding-cost regret because a free exploration condition is eventually satisfied