Scheduling jobs with stochastic holding costs

1 Introduction . We consider the following algorithmic question : given a list of jobs , each of which requires a certain number of time steps to be completed while incurring a random cost in every time step until finished , learn the relative priorities of jobs and make scheduling decisions of which job to process in each time step , with the objective of minimizing the expected total cumulative cost . Here , we need an algorithm that seamlessly integrates learning and scheduling . This question is motivated by several applications . Online social media platforms moderate content items , and here , jobs correspond to reviewing content items while the jobs have random costs driven by the number of accumulated views as views of harmful content items represent a community-integrity cost . Then the content review jobs are prioritized based on popularity of content items [ 18 ] . More generally , modern data processing platforms handle complex jobs whose characteristics are often unknown in advance [ 10 ] , but as a system learns more about the jobs ’ features , it may flexibly adjust scheduling decisions to serve jobs with high priority first . Another application is in optimizing energy consumption of servers in data centers , where a job waiting to be served uses energy-consuming resources [ 3 , 7 ] . In emergency medical departments , patients undergo triage while being treated , and schedules for serving patients are flexibly adjusted depending on their conditions [ 14 , 15 ] . Note that patients ’ conditions may get worse while waiting , which corresponds to holding costs in our problem . For aircraft maintenance , diagnosing the conditions of parts and applying the required measures to repair them are conducted in a combined way [ 1 ] . We model and study the problem as a single-server scheduling system where job holding costs are according to stochastic processes with independent and identically distributed increments with unknown mean values , under the assumption that their mean service times are known . Recent works , 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . e.g . [ 5 ] , started investigating queuing system control policies under uncertainty about jobs ’ mean service time parameters , but restricting to the setting where job holding costs are assumed to be deterministic ( linear ) functions of stochastic job waiting times . In our problem setting , job holding costs are stochastic in a different way in that job holding costs themselves are according to some stochastic processes . Under the aforementioned application scenarios , it is more natural to model a job ’ s holding costs by a stochastic process . As the first step towards understanding the setting of stochastic job holding costs , we consider a single-server scheduling with a fixed list of jobs . Since the statistical parameters quantifying the random holding costs of jobs are unknown to the decision-maker , the main challenge is to learn the parameters of jobs , thereby obtaining a nearaccurate priority ranking of jobs to find a scheduling policy minimizing the total accumulated cost . Problem formulation We consider a discrete-time scheduling with a single server and N jobs . The jobs , labelled by i = 1 , . . . , N , are present in the system from the beginning , and we assume no further job arrivals . Every job remaining in a time slot incurs a random holding cost at the beginning of the time step , and holding costs incurred by a job are according to a sub-Gaussian distribution and i.i.d . across time slots . Sub-Gaussian distributions allow for different parametric distributions , e.g . Bernoulli and Poisson , which are suitable for modeling jobs incurring random costs incrementally . The mean holding costs of jobs are given by c1 , . . . , cN whose values are a-priori unknown . The number of required time steps to complete job i is given deterministically by T/µi for each i , where T is a scaling parameter . Note that the larger the value of T the larger the number of per-job stochastic cost observations . The larger value of T does not necessarily mean that mean job service times are large in real time . In addition to the case of deterministic job service times , we also consider the case of stochastic job service times , assumed to be according to geometric distributions , which is a standard case studied in the context of queuing systems . We analyze the performance of our scheduling policy against the minimum ( expected ) cumulative holding cost that can be achieved when the jobs ’ mean holding cost values are fully known . The famous cµ rule , which sequentially processes jobs in the decreasing order of their ciµi values , is known to guarantee the minimum cumulative holding cost , so we use this as our benchmark . Assuming c1µ1 ≥ c2µ2 ≥ · · · ≥ cNµN , we can define the regret of a scheduling algorithm ϕ as R ( T ) : = C ( T ) − ∑ i∈ [ N ] ci ∑ j∈ [ i ] T/µj where C ( T ) is the expected cumulative holding cost of ϕ , i.e. , C ( T ) = ∑ i∈ [ N ] ciT ϕ i with T ϕ i denoting the completion time of job i and the second term on the right-hand side is the minimum expected cumulative holding cost , achievable by the cµ rule . Note that R ( T ) is a random variable , as it may depend on randomness in the algorithm ϕ . We will consider E [ R ( T ) ] , the expected regret of the algorithm . When ci ’ s and µi ’ s are constants , the minimum cumulative holding cost is O ( N2T ) . Here , our goal is to construct a policy whose expected regret is sublinear in T and subquadratic in N . Previous work [ 5 ] has focused on the case when the values of µ1 , . . . , µN are unknown while the values of c1 , . . . , cN are known . As the service time of a job is instantiated only after completing it , this work relies on assumptions that there is a fixed list of classes and multiple jobs may belong to the same class , which means that each µi can take one of a few values . In contrast , our paper is focused on addressing the challenge of uncertain costs . Our framework allows jobs to differ from each other ( equivalently , each job corresponds to a unique class ) , but to make learning distinct jobs possible , we assume that the factor T in the service times is large . We can think of the large T regime as the case where the jobs ’ features are observed frequently , as well as the setting of long job service times . Our contributions We develop our algorithm based on the empirical cµ rule , that is , the cµ rule applied with the current estimates of the mean holding costs . Since the ranking of jobs based on the ĉi , tµi values , where ĉi , t denotes the empirical mean of job i ’ s holding cost in time slot t , may change over time , it is natural to come up with two types of the empirical cµ rule , preemptive and nonpreemptive . Under the preemptive empirical cµ rule , the server selects a job for every single time slot . In contrast , under the nonpreemptive version , once a job is selected in a certain time slot , the server has to commit to serving this job until its completion , and then , it may select the next job based on the empirical cµ rule . The preemptive empirical cµ rule works well for instances with large gaps between the jobs ’ mean holding costs , whereas the nonpreemptive one is better for cases where the jobs ’ mean holding costs are close . However , it turns out that , under both the preemptive and nonpreemptive cases , the expected regret can grow linearly in T in the worst case . The preemptive case may result in undesired delays especially for jobs with similar values , while the nonpreemptive case may suffer from early commitment to a job with low priority . Our policy , the preemptive-then-nonpreemptive empirical cµ rule , is basically a combination of the preemptive and nonpreemptive empirical cµ rules . This variant of empirical cµ rule has a fixed length of preemption phase followed by nonpreemptive scheduling of jobs . The preemption period is long enough to separate jobs with large gaps in their mean holding costs , while it is not too long so that we can control delay costs from the preemption phase to be small , thereby avoiding undesired delays from continuous preemption and the risk of early commitment . In Section 3 , we give a theoretical analysis of our algorithm for the case of deterministic service times . We prove that the expected regret of our empirical cµ rule is sublinear in T and subquadratic in N . We also show that this is near-optimal by providing a lower bound on the expected regret of any algorithm that has the same scaling in T while there is a small gap in terms of the dependence on N . Furthermore , we give instance-dependent upper bounds that delineate how our algorithm performs depending on the gaps between ciµi values . In Section 4 , we consider the setting where the service time of each job is stochastic and geometrically distributed . Our analysis reveals that the expected regret of our algorithm for the case of geometrically distributed service times is also sublinear in T and subquadratic in N . Lastly , in Section 5 , we discuss results from numerical experiments that demonstrate our algorithm ’ s performance and support our theoretical claims . 2 Related work . The scheduling problem asking to minimize the sum of weighted completion times for a given set of jobs , with weights ci and processing times 1/µi , was studied in the seminal paper by Smith [ 13 ] , who showed that serving jobs in decreasing order of indices ciµi is optimal . This policy is often referred to as the Smith ’ s rule . This policy corresponds to the weighted shortest processing time first ( WSPT ) policy in the literature on machine scheduling . The Smith ’ s rule is also optimal for the objective of minimizing the expected sum of weighted completion times when job processing times are random with mean values 1/µi . We refer the reader to [ 11 ] for a comprehensive coverage of various results . Serving jobs by using ciµi as the priority index is known to be an optimal scheduling policy for multi-class , single-server queuing systems , with arbitrary job arrivals and random independent , geometrically distributed job processing times with mean values 1/µi [ 2 ] . This policy is commonly referred to as the cµ rule . A generalized cµ rule is known to be asymptotically optimal for convex job holding cost functions in a heavy-traffic limit , where the generalization corresponds to using a dynamic index defined as the product of the current marginal job holding cost and the mean processing time of a job [ 17 ] . This generalized cµ rule is also known to be asymptotically optimal in a heavy-traffic limit for multi-server queuing systems under a certain resource polling condition [ 9 ] . The work discussed above on the performance of Smith ’ s or cµ rule assumes that the values of marginal job holding costs and mean processing time parameters are known to the scheduler . Only some recent work considered the performance of these rules when some of these parameters are unknown . In the line of work on scheduling with testing [ 6 , 15 ] , marginal job holding costs and mean processing times of jobs are a-priori unknown , but their values for a job become known by testing this job . The question there is about how to allocate the single server to processing and testing , which can not be done simultaneously . The optimal policy combines testing the jobs up to certain time and serving the jobs according to the cµ rule policy . In [ 5 ] , a multi-class queuing system is considered under the assumption that mean job processing times are unknown to the scheduler . The authors established that using the empirical cµ rule in the single-server case guarantees a finite regret with respect to the cµ rule with known parameters as a benchmark . Similar result is established for the multi-server case by using the empirical cµ rule combined with an exploration mechanism . Unlike [ 5 ] , in our setting , each job corresponds to a unique class , so an exploration phase is required for each job . The preemption phase in our algorithm is for exploration within each job . Finally , our work is related to permutation or learning to rank problems , e.g . see [ 4 , 8 ] and the references therein , where the goal is to find a linear order of items based on some observed information about individual items , or relations among them . Indeed , the objective of our problem can be seen as finding a permutation π that minimizes the cost function ∑N i=1 icπ ( i ) , for the special case of identical mean processing time parameter values . For example , we may interpret ci as a measure of dissimilarity between item i and a reference item , and the goal is to sort items in decreasing order of these dissimilarity indices . The precise objective is defined for a sequential learning setting where irrevocable ranking decisions for items need to be made over time and the cost in each time step is the sum of dissimilarity indices of items which are still to be ranked .

Summary: The paper presents algorithms for the problem of ordering jobs on a machine so as to minimize the cumulative holding cost. The assumption is that the holding cost in each timestep follows a sub-Gaussian distribution (i.i.d. across timesteps), but the the mean values of these distributions for individuals jobs are unknown to the algorithm. The lengths of the jobs are known upfront, or in an extension considered later in the paper, are generated from geometric distributions whose mean values are known upfront to the algorithm. If holding costs and job lengths were both known, then the optimal strategy is to process jobs in decreasing order of the ratio of the holding cost and job length. One natural strategy would be to implement this same rule, but with estimated mean holding costs based on previous iterations. This has the advantage of not committing to any job upfront, and therefore, being able to adjust to estimation errors of the holding costs. But, this strategy has the downside that if the holding costs are similar, then it can process the jobs round robin thereby keeping all jobs in the queue longer and incurring a large cost. The alternative is to commit to a job that has the largest value of this (estimated) ratio, and this strategy works well when the ratios are close to each other but not when they are well-separated, since estimation errors can be costly in the latter case. The algorithm proposed in the paper is natural combination of these ideas: start with a learning/exploration phase where you use the former strategy thereby not committing to any job but learning their holding costs, and then in the exploitation phase, commit to the job with the largest (estimated) ratio and run it to completion. In addition to obtaining regret bounds for this algorithm, the paper also gives lower bounds on regret that can obtained by any algorithm, and does numerical experiments to evaluate the performance in practice.