# Analysis of the Optimal Resource Allocation for a Tandem Queueing System

Authors: Zaiming Liu; Wei Deng; Gang Chen
Journal: Mathematical Problems in Engineering
 (2017)

---

## Abstract

We study a controllable two-station tandem queueing system, where customers (jobs) must first be processed at upstream station and then the downstream station. A manager dynamically allocates the service resource to each station to adjust the service rate, leading to a tradeoff between the holding cost and resource cost. The goal of the manager is to find the optimal policy to minimize the long-run average costs. The problem is constructed as a Markov decision process (MDP). In this paper, we consider the model in which the resource cost and service rate functions are more general than linear. We derive the monotonicity of the optimal allocation policy by the quasiconvexity properties of the value function. Furthermore, we obtain the relationship between the two stations’ optimal policy and conditions under which the optimal policy is unique and has the bang-bang control property. Finally, we provide some numerical experiments to illustrate these results.


---

## Body

## 1. Introduction

Queueing systems where customers must be processed at each station in series from upstream station to downstream station are called tandem queueing system. As we all know, tandem queueing models have widespread applications in both service organizations and production factory in the sense that the system performance measures and optimization are of primary concerns, such as the control of semiconductor fabrication processes and broadband wireless networks, appointment scheduling in hospital, and production inventory system (see [1–3] and therein). Recently, this issue has attracted much attention and vast literatures have been studied, especially the dynamic resource allocation problems of the tandem queueing system. Most of them are forced on the models in two directions: admission control type and server resource allocation type. The admission control of the tandem queues have been widely studied (e.g., [4, 5]), while little work has appeared concerning the structure of the optimal resource allocation policy in the tandem queues.For many systems, service consists of two or more phases by one or more servers. A fundamental decision is how to allocate the resource (servers or workforces) owned by system to each station. This problem is a classic topic, which roots from Rosberg et al. [6] where the service rate in station 1 can be selected from a compact set and constant in station 2. Optimal control of a two-stage tandem queues system with only two flexible servers was discussed in Ahn et al. [7]. Arumugam et al. [8] considered inventory based allocation policies for flexible servers in serial systems. Smith and Barnes [9] analyzed the optimal server allocation in closed finite queueing networks. This question has been considered by the above authors for different cost or reward criterion but without considering the structure of the optimal policy. They just make a numerical experiment to get the optimal policy. In fact, it is more complex for the manager to obtain the detailed optimal policy in the practical application. The managers prefer more to make basal insight for the structure of the optimal policy. While for the study of the structure of optimal policy in single queue, many papers have investigated this issue. Iravani et al. [10] studied the optimal service scheduling in nonpreemptive finite-population queueing systems. The single-queue systems of the optimal resource allocation policy were considered by Yang et al. [11], who investigated the structural properties of the optimal resource allocation policy. Yang et al. [12] studied optimal resource allocation for parallel service facilities multiqueue systems with a shared server pool.For the optimal control of the tandem queueing system, some relative works are discussed. Weber and Stidham [13] considered a problem of optimal service rate control in queueing networks where the optimal policy has a monotone structure. Veatch and Wein [14] generalize the monotonicity results of [13] where the control policies were studied under the full information and service rate functions are linear in the service resource. Mayorga et al. [15] studied the problem of allocating flexible servers for a firm that operates a make-to-order serial production system. The max-min optimality of service rate control problem in closed queueing networks was studied by Xia and Shihada [16] where the cost functions are strictly convex (concave). Few studies, even among the most recent, have considered the structure of the optimal resource allocation policy in the tandem queues with nonlinear service rate and cost functions. In many logistic environments, however, the assumption of the linear resource cost and service rate is not appropriate. It is well known that if the service cost is linear, these problems have all-or-nothing (bang-bang) optimal policy (see [14]). Different from the works quoted above, in our model, the service resource cost and service rate are more general than linear in service resource. Recently, Xia et al. [17] investigated the optimal control of service rates of a tandem queue with power constraints and general cost function. They mainly derived some structures of the optimal policy, such as the bang-bang control policy and 3-element set policy for some special cases, while, in this paper, we study the structure of the optimal control policy for the case with general cost and service rate functions Moreover, some uninvestigated properties of the optimal policy are obtained in this paper. Using the theory of queueing system, we cast the optimal problem as a MDP. The theory of Markov, semi-Markov, and regenerative decision processes can be found in Morozov and Steyaert [18]. We mainly analyze the properties of the optimal policy under full information and partial information. Concretely, we first derived the properties (monotonicity and convexity property) of value function by the induction method and queueing theory (see [19]). Second, we provide insights into the optimal policy structure based on the properties of the value function and dynamic programming method (see [20–22]). Furthermore, we take Howard’s iteration procedure to obtain numerical results.The main contributions of this paper can be summarized as follows. First, to the best of our knowledge, our paper is the first to study the optimal resource allocation policy in the tandem queues with the general service rate and resource cost functions. Second, we get the monotone results of the optimal policy under the partial information based on the quasiconvexity property of the value function. Third, we derive the conditions under which the optimal policy is unique and the bang-bang control policy is established. This conclusion is totally new progress compared with all of the previous works in the literature. Furthermore, we derive the relationship between the two stations’ optimal policy. As far as we know, these are the most general results for the optimality of resource allocation in the tandem queueing system.The rest of the paper is organized as follows. In Section2, we introduce the model formulation in detail based on the controllable Markov decision problem. The characteristics of the optimization problem and the optimality equation are derived in Section 3. In Section 4, we present the structural properties of the optimal policy and main results of the paper. In Section 5, we give some numerical examples to provide the support for the results of the present model. Finally, some further discussions and conclusions are given in Section 6.


## 2. Model Description

We consider a tandem queueing system with two stations. Arrivals to the system at station 1 from outside follow a Poisson process with parameterλ and have exponentially distributed service requirement times at each station. After receiving service at station 1, customers join immediately to station 2 and receive service before leaving the system. A decision-maker can assign a number of service resources to each station. The service rate depends on the number of service resources assigned to the stations precisely. When a station has been allocated a resources, the service duration of the customer in station i is exponentially distributed with rate μi(a),i=1,2, which is strictly increasing in a. Without loss of generality, we assume that μi(0)=0,i=1,2. At any decision epoch, the decision-maker decides to choose the number of service resources to station 1 from a set A=[0,amax] and to station 2 from a set B=[0,bmax] at the same time. Each station has a single infinite-size FCFS queue. The interarrival and service times are assumed to be mutually independent. We assume that the stability condition λ<min⁡{μ1(amax),μ2(bmax)} holds. Figure 1 gives an illustration of the system.Figure 1
The controllable tandem queueing systems.We consider the following cost structure in the system. Our objective is to obtain dynamic resource allocation policy that minimizes the long-run average costs.(1) Resources Cost. When station i uses a resources, a cost of ci(a),i=1,2 is incurred by the system per unit time (ci(a) is a continuous function and strictly increasing in a. Without loss of generality, we assume that ci(0)=0).(2) Holding Cost. Holding costs are incurred at rates h1 and h2 per unit time for each customer in stations 1 and 2, respectively.LetXi(t) denote the number of customers at station i,i=1,2. The state of the system at time t can be described by X(t)=(X1(t),X2(t)). The system evolves as a continuous-time Markov process X(t)={(X1(t),X2(t)),t≥0}. We define the notations xi to classify the certain components of the vector state x∈E. Clearly, the system state space is E=x1,x2∣x1,x2∈N with N={0,1,2,…}. We consider the stationary Markov policy under which the system evolves as a continuous-time Markov chain. Moreover, in order to study the optimal policy in the ergodic Markov process, we assumed that the model is stable and conservative. The transition rate under a control action (a,b) is given by(1)qxya,b=λy=x+e1;μ1ay=x-e1+e2,x1>0;μ2by=x-e2,x2>0;0else,where(2)qxya,b≥0,y≠x,qxxa,b=-qxa,b=-∑y≠xqxya,b,qxa,b<∞.Here ei is the 2-dimensional vector with 1 in the ith coordinate and 0 elsewhere, i=1,2.The problem of the decision-maker is to choose an optimal dynamic policy based on the number of customers in each station that minimizes the long-run average costs. We formulate the service resource management problem as a Markov decision process. The set of decision epochs is composed of the set of all arrivals and service completions. The controllable system associated with a Markov process is a five-tuple(3)E,D=A,B,Qr,cia,hii=1,2,in which Q(r)=(qx,y(r))x,y∈E is the infinitesimal generator of the queueing system under the policy r. We consider the stationary Markov policy r:E→D with r=(a,b). Due to the Markov property of the queueing system, we know that the optimal policy depends only on the current state regardless of t. In our model we consider two situations: the decision with partial information and full information. Concretely, when the system state is x=(x1,x2), the manager makes an action as follows:(i)
Partial information: the action for station 1 (2) isa(x1)∈A (b(x2)∈B, resp.). That is the action of resource to station i only depends on the number of customers in station i.(ii)
Full information: the action for station 1 (2) isa(x1,x2)∈A (b(x1,x2)∈B, resp.). That is the action of resource to station i depends on the number of customers in both stations.


## 3. Optimization Problem and Optimality Equation

It is obvious that, under the stability conditionλ<min⁡{μ1(amax),μ2(bmax)}, the two-dimensional stochastic process X(t)={(X1(t),X2(t)),t≥0} is an ergodic continuous-time Markov chain for any fixed stationary policy r. As it is known from Tijms [23], the long-run average cost per unit of time for the policy r in our ergodic Markov process can be written in the following form:(4)gr=limt→∞⁡urx,tt=∑i=1∑j=1c1a+c2b+h1i+h2jπijr,in which ur(x,t) denotes the total expected costs up to time t when the system starts in state x=(i,j) and πij(r) denotes a stationary probability of the process under policy r=(a,b). The goal is to find a policy r∗ that minimizes the long-term average costs:(5)gr∗=minr⁡gr.Using the standard tools of uniformization and normalization, we construct a discrete-time equivalent of our original queueing system. Without loss of generality, we assume thatλ+μ1(amax)+μ2(bmax)=1. Now we consider a real-valued function v(x) which is defined on the state space. The relative value function v(x) can be regarded as the asymptotic difference in total costs that results from starting the process in state x instead of some reference state. As is shown in Puterman [24], the optimal policy r and the optimal average cost g are the solutions of the optimality equation:(6)Tvx=vx+g,where T is the dynamic programming operator acting on v defined as follows:(7)Tvx=λvx+e1+∑i=1,2Tivx+∑i=1,2hixi,in which(8)T1vx=mina∈A⁡μ1avx-e1+e2+μ1amax-μ1avx+c1a,T2vx=minb∈B⁡μ2bvx-e2+μ2bmax-μ2bvx+c2b.The first term in the expression Tv(x) models the arrivals of customers to station 1 from outside the system and the last one the customer holding cost. Similarly the first term in the expression T1v(x) corresponds to a customer who finished his service in station 1 and into station 2 and the second one the uniformization constant. The last one in T1v(x) is the resources cost in station 1. The first term in the expression T2v(x) corresponds to a customer who finished his service in station 2 and the second one the uniformization constant. The last one in T2v(x) is the resources cost in station 2.According to (4), we can solve another optimization problem: if ci≡0,hi=1,i=1,2, then (5) is equivalent to minimization of the mean number of customers in the queueing system. In this case, the optimal action would be always (amax,bmax) by intuition, which also satisfies the structure of the optimal policy in next section. In addition, the analysis method and structure in this section are held for both the partial and full information cases.


## 4. Structural Properties of the Optimal Policy

In this section, we focus on deriving the optimal policy. The properties of the optimal policy will provide basal insight for us, and this also helps one to find the optimal policy with less computational effort due to a reduction of the solution search space.In order to study the optimal policy, intuitively, the optimal equationTv(x)=v(x)+g should be solved. However it is hard to solve analytically in practice. It can be obtained by recursively defining vn+1=Tvn for arbitrary v0. We know that the actions converge to the optimal policy as n→∞. For existence and convergence of the solutions and optimal policy, we can see more details in the works of Aviv and Federgruen [25] and Sennott [26]. The backward recursion equation in our model is given by(9)vn+1x=λvnx+e1+∑i=1,2Tivnx+∑i=1,2hixi.For ease of notation, let thearg⁡Tiv(x),i=1,2 denote the set of optimal action for station i with state x=(x1,x2) in the partial information case in which the action is (a(x1),b(x2)), where a(x1)∈arg⁡T1vx,b(x2)∈arg⁡T2v(x).Using the optimality equation and recursive method, we can get some properties of the relative value function in the following lemma which will be used in the proof of the main results and the proofs of these properties are given in AppendixA.Lemma 1.
For the optimal value functionv(x) in this model, we have(i)
v(x+ei)≥v(x),i=1,2 for all x∈E;(ii)
if2h2≥h1, then v(x-e1+e2)≥v(x-e2) for all x∈E;(iii)
ifh1≥h2, then v(x)≥v(x-e1+e2) for all x∈E.As we know, at the decision epochs if the manager gets the full information about the system, he will make a decision based on the number of the customers in both stations. Weber and Stidham [13] and Veatch and Wein [14] used submodularity of the value function v(x)-v(x+e2-e1)-v(x-e2)+v(x-e1)≤0 to prove the main conclusion transition monotonicity for the full information case a(x)≤a(x-e2) and b(x)≤b(x+e2-e1). The optimal resource allocation policy has the switching function policy or region control policy type for the full information case [17]. However, the corresponding results for the partial information case are not studied. In this paper, we study the property of the optimal policy under partial information. Different from method in the full information case, we get some structure properties of the optimal policy by the quasiconvexity property of the relative value function and present the structure properties of the optimal policy in the following theorem.Theorem 2.
In our model under partial information, the optimal policy has the monotonicity properties, that is, for allx=(x1,x2)∈E:(i)
Ifb(x2+1)∈arg⁡T2v(x+e2) and b(x2)∈arg⁡T2v(x), then b(x2+1)≥b(x2).(ii)
Ifa(x1+1)∈arg⁡T1v(x+e1) and a(x1)∈arg⁡T1v(x), then a(x1+1)≥a(x1).The proof of the above theorem is based on the following property which shows the quasiconvexity properties of the relative value function. The proofs of Theorem2 and the following Lemma 3 are given in Appendix B.Lemma 3.
For the optimal value functionv(x1,x2) under partial information, we have(i)
v(x+e2)-2v(x)+v(x-e2)≥0, for all x=(x1,x2)∈E;(ii)
v(x+e1-e2)-2v(x)+v(x-e1+e2)≥0, for all x=(x1,x2)∈E.Based on the above properties of the value functions, we derive the relationship between the two stations’ optimal policy by analyzing the properties of the service rate and holding cost functions. The following theorem shows the conditions under which the optimal policy for station 1 is bigger than that in station 2.Theorem 4.
Assume thatamax=bmax and 2h2≥h1, if the condition holds c1(x)-c1(y)/c2(x)-c2(y)≥1 and μ2(x)-μ2(y)/μ1(x)-μ1(y)≥1 when x≥y. Then we have b≥a where a∈arg⁡T1vx,b∈arg⁡T2v(x).Proof.
Let(a∈arg⁡T1vx,b∈arg⁡T2v(x)) be an arbitrary optimal policy for stations 1 and 2 in state x, respectively. The proof is taken by contradiction method. Suppose that b<a; then we compare the policy (a,b) with the policy (b,a) (the assumption amax=bmax guarantees that we can always swap a and b). Hence, we have(10)Ta,bvnx-Tb,avnx=μ1avx-e1+e2+μ1amax-μ1avx+c1a+μ2bvx-e2+μ2bmax-μ2bvx+c2b-μ1bvx-e1+e2+μ1bmax-μ1bvx+c1b-μ2avx-e2+μ2amax-μ2avx+c2a=μ1a-μ1bvx-e1+e2-vx-μ2a-μ2bvx-e2-vx+c1a-c1b-c2a+c2b≥μ1a-μ1bvx-e1+e2-vx-e2+c1a-c1b-c2a+c2b≥0.The first equality is based on the definitions of operators T1 and T2. The second equality follows by rearranging the terms. The first inequality follows the conditions μ2(a)-μ2(b)/μ1(a)-μ1(b)≥1 when a≥b and Lemma 1 (i). The last inequality based on the conditions c1(a)-c1(b)/c2(a)-c2(b)≥1 when a≥b and Lemma 1 (ii). So that we obtain Ta,bvn(x)-Tb,avn(x)≥0, which implies that (a,b) is not an optimal policy for state x. Hence, we have b≥a.Remark 5.
From the above theorem we can conclude that under some conditions the optimal size of the service resources allocated to station 1 is less than that to station 2. We find that the optimal size of the resource allocated to each station depends on the resource cost variationc(a)-c(b) and the service rate variation μ(a)-μ(b) in each station. This condition seems to imply that when the same service resources are added to both station 1 and station 2, then the performance of station 2 is improved more than station 1 while the higher cost is incurred in station 1 than in station 2. So that it implies that the optimal policy satisfies the relationship b≥a.It is well known that if the service resource cost function is linear, then an all-or-nothing (bang-bang) control is optimal. Weber and Stidham [13] and Veatch and Wein [14] give a detailed conclusion for this issue. Actually it is not obvious whether the bang-bang control is also optimal, when the service resource cost and service rate functions are more general than linear in service resource. We are interested in the special structure of the optimal control policy in this model. In contrast to existing studies, the results in the following theorem are extension of the model with linear case. We are now ready to give some conditions under which the optimal policy is unique and has the bang-bang control property.Theorem 6.
(i) The optimal policy is unique if the following conditions hold: (1)m1(a)=c1′(a)/μ1′(a) and m2(b)=c2′(b)/μ2′(b) are monotonous on a∈A,b∈B. (2) ∃a∈(0,amax) and b∈(0,bmax) s.t., c1(a)/μ1(a)<c1(amax)/μ1(amax) and c2(b)/μ2(b)<c2(bmax)/μ2(bmax).
(ii) The optimal policy is a bang-bang control policy; that is,arg⁡T1v(x)={0,amax},arg⁡T2v(x)={0,bmax} if the functions c1(a)/μ1(a) and c2(b)/μ2(b) are strict decreasing for all a∈(0,amax),b∈(0,bmax).Proof.
We prove only conclusion for station 1, and the same proof can be applied to get conclusion for station 2. To prove part (i), we consider the optimal policya in station 1 service resource allocation. For the definition of the operator T1, we have the following minimization problem: (11)T1vx=mina∈A⁡μ1avx-e1+e2+μ1amax-μ1avx+c1a.Rearranging the terms of the first-order optimality condition of the above problem, we obtain (12)c1′aμ1′a=vx-vx-e1+e2.Because the allocation resource action a∈A=[0,amax], the optimal policy a in station 1 can be 0 or amax or satisfies the above equation. Since the function m1(a)=c1′(a)/μ1′(a) is monotonous on a∈A, there is at most one solution a solving the above equation. Next if the optimal policy is 0 or amax, we show that the action 0 and amax cannot be the optimal policy simultaneous for station 1. We take the contradiction method. Assume that the action 0 and amax be the optimal policy simultaneous for station 1 at state x. Then we have T1v(x)=T10v(x)=μ1(amax)v(x) and T1v(x)=T1amaxv(x)=μ1(amax)v(x-e1+e2)+c1(amax); that is,(13)vx-vx-e1+e2=c1amaxμ1amax.Because the action 0 is optimal policy, we have T1av(x)≥T10v(x) for every action a∈(0,amax), that is μ1(a)[v(x)-v(x-e1+e2)]≤c1(a). Taking the above equation into the inequality, we can get c1(a)/μ1(a)≥c1(amax)/μ1(amax) for all a∈(0,amax) which is contradicted against condition (2). Hence the optimal policy for station 1 is unique.
To prove part (ii), we consider the optimal policya in station 1 service resource allocation. We use the contradiction method and assume that there exists a state x∈E for which the optimal policy a∈arg⁡T1v(x) in station 1 satisfies a∈(0,amax). For any ε>0, we have(14)T1a+εvx-T1avx=μ1a+ε-μ1avx-e1+e2-vx+c1a+ε-c1a≥0, which implies that(15)vx-vx-e1+e2≤c1a+ε-c1aμ1a+ε-μ1a.Since the function c1(a)/μ1(a) is strict decreasing, we get c1(a+ε)-c1(a)/μ1(a+ε)-μ1(a)<c1(a)/μ1(a) so that v(x)-v(x-e1+e2)<c1(a)/μ1(a). Because the action a is the optimal policy for station 1 in state x, we have T1av(x)≤T10v(x); that is,(16)μ1avx-e1+e2+μ1amax-μ1avx+c1a≤μ1amaxvx.So we have v(x)-v(x-e1+e2)≥c1(a)/μ1(a) which is a contradiction with the above result v(x)-v(x-e1+e2)<c1(a)/μ1(a). Hence the optimal policy for station 1 is arg⁡T1v(x)={0,amax}; that is, the optimal policy is a bang-bang control policy.Remark 7.
From the above theorem we can conclude that, under some conditions, the optimal is a bang-bang control policy. We try to give intuitive interpretations to these conditions and results, which would help us to understand the theorem intuitively. For the conditions in Theorem4 (ii), it is clear that c1(a)/μ1(a) represents the expected service cost for one customer in station 1 under policy a. Since the function c1(a)/μ1(a) is strict decreasing for all a∈(0,amax) which yields that the policy a=1 is optimal for every customer service cost in station 1. For the total average cost of the system, we can regard it as the average cost per unit time since every customer must be processed in each station. While for the state x=(0,0), it is obvious that no service resource should be allocated in two stations.Remark 8.
By the proof of Theorem6, we know that the results in Theorem 6 are held for both partial and full information cases. In addition, the conditions in Theorem 6 are a bit complex. We give the corresponding looser conditions for Theorem 6 (1) as follows. The optimal policy is unique if the functions c1′(a)/μ1′(a) and c2′(b)/μ2′(b) are monotonous and c1(a)/μ1(a) and c2(b)/μ2(b) are nondecreasing on a∈A,b∈B, such as the case μi(a)=2a2,ci(a)=10a3,i=1,2.


## 5. Numerical Examples

For the full information case, the corresponding results and numerical example have been investigated in [14, 17]. In this section, we conduct numerical experiments under different parameter settings to demonstrate the main results obtained in this paper for the partial information case. On one hand, these examples provide direct insight into how the change of the system state may impact the optimal resource allocation parameters (a∗,b∗). On the other hand, the numerical experiments and Figures 2, 3, 4, and 5 provide the direct support for the results about the structure of the optimal resource allocation policy obtained in the above section. The following experiments are made for the case of A=B=[0,1]. As is shown in the figures, we can make the following observations.Figure 2
Optimal resource allocation policy(a∗,b∗) versus x1 for given x2=12.Figure 3
Optimal resource allocation policy(a∗,b∗) versus x2 for given x1=10.Figure 4
Optimal resource allocation policy(a∗,b∗) versus x1 for given x2=8.Figure 5
Optimal resource allocation policy(a∗,b∗) versus x2 for given x1=20.From Figures2 and 3, we present numerical results of the optimal policy for the case with λ=1.3,h1=1.5,h2=2,μi(a)=2a2,ci(a)=10a3,i=1,2. As can be seen from Figure 2, the optimal resource allocation policy a∗ increases as the number of customers in station 1 increases, which shows a staircase-like increasing pattern. This phenomenon is consistent with the results in Theorem 2, while the optimal policy b∗ for station 2 remains constant for varied value of x1. Meanwhile Figure 3 shows that the optimal resource allocation policy a∗ remains constant and b∗ also shows a staircase-like increasing pattern with the number of customers in station 2 increasing. Moreover, it is noted in these two figures that the line graph of the optimal policy a∗ for station 1 is always under the line graph of the optimal policy b∗ for station 2. This is easy to explain from the results in Theorem 4 that the conditions in this numerical experiment satisfy Theorem 4.In Figures4 and 5, we describe the characteristics of the optimal policy for the case with λ=1.1,h1=h2=1,μ1(a)=2a3,c1(a)=2a2,μ2(b)=3b2,c2(b)=4b3. From Figure 4, we find that the optimal policy for station 1 is a∗=0 if x1=0 otherwise a∗=1, which shows that the optimal policy for station 1 has a bang-bang control type, while the optimal policy b∗ for station 2 remains constant. As it is observed from Figure 5, the optimal policy a∗ for station 1 always equals 1, which also belongs to the bang-bang control policy. The optimal policy b∗ for station 2 shows a staircase-like increasing pattern with the number of customers in station 2 increasing. These figures provide a direct support for the results in Theorem 6 since the functions μ1(a)=2a3,c1(a)=2a2 in this numerical experiment follow the conditions in Theorem 6 (ii).


## 6. Conclusion

In this paper, we have analyzed the optimal resources allocation control policy of a tandem queueing system with the general service cost and service rate functions. Applying the queueing system and MDP theories, we not only give some traditional properties of the relative value function and optimal policy but also derive the conditions under which the optimal policy is unique and has a bang-bang control property, which has not been studied before our work. In particular, we have provided the relationship between two stations’ optimal policies, which can give the manager basal insight into the structure of optimal policy information to improve decision-making of the system.From the above results, there arise some interesting extensions of the model which we may study in the near future. One possible change is to consider the tandem queueing system with retrial or feedback customers which will make the model more useful in practical system. Another way to extend the model is to apply the semi-Markov decision processes to consider the queueing system in which the service time of a customer is a general distribution. Furthermore, in practice, the production systems are often likely to be burdened by mixed uncertainties of both randomness and fuzziness; the study of the optimal control of the tandem queueing system with fuzziness may provide more precise information to managers, which is also an interesting topic for future research.



---

*Source: 5964272-2017-11-28.xml*