At the time of writing, according to the World Health Organization, the ongoing COVID-19 pandemic, caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), had already resulted in more than thirty-two million cases infected and more than one million deaths worldwide; see [19] , and even worse, the pandemic seems to be "showing no clear signs of slowing down" in many countries. To this end, governments are persistently striving to control and slow down the spread of COVID-19; e.g., minimizing the contact rate by consulting non-pharmaceutical intervention mechanisms such as lock-downs and social distancing or relying on some pharmaceutical interventions such as enhancing medical treatments, and developing possible cure or remedies. With this motivation in mind, the focal point for this paper, as a preliminary work, is to understand how the disease spreads and how it would be possibly controlled.
The remainder of the paper is organized as follows: In Section 2 we describe the preliminaries involving the details of our modified SIRD-based epidemiological model. Then two problem formulations for epidemic control: Almost sure epidemic control problem and average epidemic control problem are stated. Subsequently, in Sections 3, detailed analyses on our epidemiological model are provided. In Sections 4 and 5, using a linear epidemic control policy, we provide sufficient conditions on feedback gain so that the epidemic can be properly controlled in either expected value sense or almost sure sense. Next, in Section 6, we describe a simple model parameter estimation approach and illustrate its use via numerical examples involving historical COVID-19 data from the United States, the country among those with a relatively high confirmed cases in the middle of 2020. The control performance is also discussed. Finally, in Section 7, we provide some concluding remarks and promising directions for future research.
In the sequel, we take k to be the index indicating the day number and u(k) ≥ 0 to be the corresponding abstract epidemic control policy implemented by governments. 1 We view that the large values of u(k) correspond to enhanced medical treatments or stringent non-pharmaceutical interventions such as mandating social distancing with wearing the masks. The smaller values of u(k) might correspond to diminished medical treatments or relaxation of the rules. In the analysis to follow, we assume that the epidemic control policy u(k) is causal ; i.e., it only depends on the past and current information that is available at hand-not the future.
For k = 0, 1, . . . , let S(k) be the number of susceptible cases at the kth day, I(k) be the infected cases at the kth day and R(k) be the recovered cases at the kth day, and D(k) be the deceased cases at the kth day. Take N (k) be the underlying human population at the kth day satisfying N (k) = S(k)+I(k)+R(k). As mentioned in Section 1, many infected individuals may go undetected or asymptomatic [16] and the effectiveness of a control policy u(k) may be uncertain at the time, we introduce two uncertainty quantities: The uncertain rate of infected cases which are undetected or asymptomatic, call it δ(k), and the uncertain effectiveness rate of control, call it v(k), in our model to follow. Now, with initial values S(0) := N (0) := S 0 , I(0) := I 0 > 0, and R(0) = D(0) = 0, we consider a discrete-time stochastic epidemiological model with uncertainties described by
where d I (k) is the death rate for infected cases satisfying d I (k) ∈ [0, d max ] with d max < 1 for all k. We assume that the sequence d I (k) are independent and identically distributed (i.i.d.) random
variables. In addition, we assume that both of δ(k) and v(k) are i.i.d. random variables with arbitrary distribution but known bounds 0 ≤ δ(k) ≤ δ max and 0 < v min ≤ v(k) ≤ v max ≤ 1, respectively. 2 If v(k) = 1, it corresponds to the case where the control policy is extremely effective.
On the other hand, if v(k) ≈ 0, it corresponds to an extremely ineffective case; i.e., the people are not disciplined and may be unwilling to follow the medical advise for masks or social distancing; see also a discussion in Section 3.5. In the sequel, we assume further that I 0 δ max < S 0 ; and v(k), δ(k), and d I (k) are mutually independent. For the sake of notational convenience, we also take the shorthand notations δ :
Remark on the Epidemiology Model. As a preliminary work, while we allow uncertainties on both infected cases I(k) and control u(k), our epidemiological model 1 assumes that u(·) can interact with the infected cases instantaneously without any time delay. The action with delay is left for future work; see also Section 7. It is also worth mentioning that, if one considers a linear control policy u(k) = KI(k) with constant K ≥ 0, then a basic reproduction ratio of sorts, call it R 0 (·), can be obtained as follows: For k ≥ 0,
where β(k) := δ(k)(S(k) + I(k)))/S(k) is the infectious rate and γ(k) := v(k)K is the recovery rate. If R 0 (·) > 1, the number of infected cases is expected to increase; if R 0 (·) < 1, then the infected cases decrease. Finally, we should mention that, in terms of the terminology in systems and control theory, the epidemiological dynamics described by equation (1) is indeed an uncertain linear time-varying system. This observation is useful in the following sections to follow.
In this subsection, we introduce the first epidemic control problem, which we call the almost sure epidemic control problem. That is, given the modified SIRD model 1, we seek a control policy u(·) which assures the following conditions hold:
(i) The ratio of infected cases converge to zero with probability one; i.e., Remarks. The results related to this almost sure epidemic control problem is discussed in Section 4 when a linear control policy of the form u(k) = KI(k) with pure gain K ≥ 0 for all k ≥ 0 is applied.
While almost sure epidemic control policy, if exists, can be a good candidate to mitigate the pandemic. However, in practice, some potential issues remain. For example, in some cases, the cost for implementing an almost sure epidemic control policy may be still too expensive in practice; see Remark 4.1.2 in Section 4. To hedge this issue, we now introduce our second epidemic control problem which we call the average epidemic control problem aimed at controlling the "expected" infected cases and "expected" deceased cases.
To address the issue raised in previous subsection, we consider the second epidemic control problem which we call the average epidemic control problem aimed at controlling the "expected" infected cases and "expected" deceased cases. Rigorously speaking, given the modified SIRD model 1, we seek a control policy u(·) which assures the following conditions hold: Remark. Similar to the previous remark, the results related to this average epidemic control problem is discussed in Section 5 when a linear control policy of the form u(k) = KI(k) with K ≥ 0 for all k ≥ 0 is applied.
In this section, to understand the contagion process and the evolution of the epidemic, we assume that there exists a control policy u(·) which assures that infected cases I(k), susceptible cases S(k), recovered cases R(k) and deceased cases D(k) are all nonnegative for all k with probability one. 3 The analytic expression for infected cases, susceptible cases, recovered cases, and deceased cases are provided. The results obtained in this section are useful for the subsequent sections to follow.
In the sequel, we use a shorthand notation
for integer k ≥ k 0 ≥ 0. We mention here that Φ(k, k 0 ) is a state transition function for infected cases from day k 0 to k. The reader is referred to [7] for detailed discussion on this topic. We may sometimes write Φ I (·, ·) instead of Φ(·, ·) to emphasize that the function is related to the infected cases. The following lemma characterizes some useful properties of the function. 
Proof. To prove part (i), we let f (k) := 1 + δ(k) − d I (k). Since δ(k) ∈ [0, δ max ] and d I (k) ≥ 0 for all k ≥ k 0 ≥ 0, we have 1 − d max ≤ f (k) ≤ 1 + δ max for all k with probability one. Note that the upper and lower bounds are nonnegative. Hence, a straightforward multiplication leads to the desired inequality and the proof of part (i) is complete. To prove part (ii), with the aid of the fact that δ(k) and d I (k) are i.i.d. in k and are mutually independent, we obtain
Having proved Lemma 3.1, we are now ready to characterize the infected cases I(k).
Given I 0 > 0, the infected cases at the kth day is described by
for k ≥ 1. Moreover, the expected infected cases satisfy
Proof. Since the infected cases dynamics (1) is a linear time-varying system, the solution I(k) and its corresponding proof are well-established; hence we omitted here and refer the reader to any standard textbook in system theory; e.g., see [7] .
To complete the proof, we must show that the expected infected cases can be characterized by the form described in the lemma. Using Lemma 3.1 and the fact that v(k) ≥ v min and are i.i.d., we observe that
Note that u(i), as assumed in Section 2, is causal; i.e., it only depends on the information available up to the ith day. In addition, according to equation (2) , the state transition function Φ I (k, i + 1) =
As seen later in the next section, if one adopts the linear feedback policy of the form u(k) = KI(k), then the solution I(k) described in Lemma 3.2 can be greatly simplified.
The following lemma gives a closed-form expression for the susceptible cases S(k).
With S(0) = S 0 > 0, the number of susceptible cases at the kth day is given by
Using Lemma 3.2, we obtain
In this subsection, we characterize the solution of recovered cases and some of its useful properties.
With r(0) = 0, the recovered cases at the kth day is given by
for k ≥ 1.
Proof. While the proof is simple, for the sake of completion, we provide a full proof here. We first recall that r(0) = 0 by assumption on initial condition. To complete the proof, we proceed a proof by induction. Begin by noting that for k = 1, the recursion on R(k) tells us that
To complete the proof, we now show the desired expected recovered cases and its bounds. For
which is desired.
It is readily seen that R(k) is increasing in k since u, v are nonnegative.
The number of deceased cases and its expectation are provided in this subsection. 
for k ≥ 1. Moreover, the expected number of death is bounded by
Proof. With D(0) = 0, we begin by recalling that the deaths dynamics (1) is given by D(k + 1) = D(k) + d I (k)I(k) for k ≥ 0. An almost identical proof as seen in Lemma 3.2 leads to
To complete the proof, we take expectation on equation 5 and, with the linearity of expectation and the fact 0 ≤ d I (k) ≤ d max , the desired inequality follows immediately.
If there are low or no medical supports, i.e., u(k) ≈ 0, or the medical treatments are lacking or extremely ineffective, due to the reasons such as the people are not disciplined or unwilling to follow the medical advise to wearing masks or social distancing; i.e., v(k) ≈ 0, then it is readily verified that the susceptible cases, infected cases, recovered cases, and deceased cases become
Consistent with the intuition, the equation (6) above tells us that the recovered cases R(k) are close to zero approximately. On the other hand, the infected cases I(k) and the deceased cases D(k) are both increasing monotonically; i.e., an "outbreak" is seen.
To mitigate the pandemic, in this paper, we consider a class of linear epidemic control policy of the form u(k) := KI(k) for k ≥ 0 where K ≥ 0 is the feedback gain which represents the degree of the epidemic control effort on providing medical treatment or mandating the non-pharmaceutical interventions. In the sequel, we sometimes call the linear epidemic control policy above as linear feedback policy or just linear policy for the sake of brevity.
As mentioned in Section 2, our objective of this section is to address the almost sure epidemic control problem using the linear control policy. We begin with discussing the control of infected cases in the almost sure sense.
The following lemma is useful for deriving one of our main results: Theorem 4.2. 
. Then the infected cases dynamics becomes
Therefore, the product of f (i) is also nonnegative which implies that I(k) ≥ 0 for all k with probability one. To complete the proof of part (i), we note that for any K ∈ [0, 1−dmax vmax ], the infected cases satisfy
To prove part (ii), an almost identical proof as seen in part (i) applies. That is, if K < (1 − d max )/v max , then I(k) > 0 for all k with probability one.
According to part (ii) of the lemma above, we see that if K < (1 − d max )/v max , then the infected cases I(k) is strictly positive for all k with probability one. Said another way, the infected cases can not be eradicated at any time. However, as seen in the next theorem, the ratio of the infected cases can go down to zero asymptotically for some K. 
then we have (i) lim k→∞ I(k)/I 0 = 0 with probability one.
(ii) The controlled infected cases are upper bounded uniformly; i.e., I(k) < I 0 for all k with probability one. If K = δmax vmin , then I(k) ≤ I 0 .
Proof. To prove (i), we observe that
as defined in the proof of Lemma 4.1. With the assumed inequalities
and feedback gain δmax vmin < K < 1−dmax vmax , it is readily verified that 0 < f (i) < 1 for all i = 0, 1, . . . , k − 1. To show the limit of the ratio I(k)/I 0 is zero with probability one, we write
We also note that the logarithmic function above is well-defined for K within the assumed range. Hence,
Now, since f (i) are i.i.d., by the strong law of large number (SLLN); e.g., see [10] , we have
as k → ∞ with probability one. Now, using the fact that the logarithmic function is strictly concave, Jensen's inequality yields
where the last inequality hold by the monotonicity of logarithmic function. Hence, log(1+δ−Kv) ≤ 0 for any K ≥ δ max /v min ≥ δ/v. Therefore, E[log f (0)] < 0, which implies that lim k→∞ I(k)/I 0 → 0 and the proof for part (i) is complete.
To prove (ii), we must show that the I(k) is upper bounded by the initial infected cases I 0 . As seen in the proof of part (i) of Lemma 4.1, we have, for all k,
where the last inequality holds by using the assumed fact that K ≥ δmax vmin .
In practice, since the medical resource is limited, it is natural to put additional constraints on the feedback gain such as K ∈ [0, L] with L ≥ 1 being the constant which corresponds to the maximum allowable medical resources to be used. Hence, other than the sufficient condition 7, for any δ max < vmin(1−dmax) vmax , one might require K to satisfy
However, one should note that the set K might be an empty set. To see this, we consider L = 1, v min = v max = 0.01 and δ max = 0.1 and d max = 0.01. Then it is readily verified that the condition δ max < vmin(1−dmax) vmax but K = (10, 99)∩[0, 1] = ∅. If K = ∅, it tells us that the infected cases does not converge to zero asymptotically and the epidemic is uncontrollable. If this is the case, one should put whatever they can to suppress the disease; e.g., by putting K := L for possible L ≥ 1.
On the other hand, given the fact that the higher feedback gain would cause a higher consumption on medical/economical resources, one should choose the lowest feedback gain which is nonzero. Hence, an immediate "sub-optimal" choice would simply be as follows: If δ max ≤ v min we take K * := δ max /v min with the aim that I(k) ≤ I 0 can be guaranteed at least. On the other hand, if δ max > v min , we take K * := L. Or, we can write it in a more compact way as follows:
provided that δ max < vmin(1−dmax) vmax where 1 {δmax≤Lvmin} and 1 {δmax>Lvmin} are the indicator function; see [11] for a detailed discussion on this topic. As seen later in Section 6, the K * above will be adopted to test the epidemic control performance using historical data.
In this subsection, the recovered cases R(k), susceptible cases S(k), and deceased cases D(k) under linear control policy are discussed. The analysis of recovered cases is simple. We begin by recalling Lemma 3.4 and write R(k) = k−1 i=0 v(i)u(i). Now, by taking linear control policy u(k) = KI(k) with gain 0 ≤ K ≤ (1 − d max )/v max , we have R(k) ≥ 0 for all k with probability one and
where the last inequality holds by using Lemma 4.1. 
for k = 1, 2, . . . , ⌊S 0 /(I 0 δ max )⌋ with probability one where ⌊z⌋ is the floor function satisfying ⌊z⌋ := max{n ∈ Z : n ≤ z}.
Proof. Similar to the proof of Lemma 3.3, recalling that S(k+1) = S(k)−δ(k)I(k), a straightforward inductive calculation leads to
Using Lemma 4.1, the desired form of solution follows immediately. To complete the proof, we note that
Thus, via a lengthy but straightforward calculations, it is readily verified that S(k) ≥ 0 for k ≤ ⌊S 0 /(I 0 δ max )⌋ with probability one.
Remark. While the lemma above tells us that S(k) ≥ 0 holds up to k ≤ ⌊S 0 /(I 0 δ max )⌋, we should note that the initial cases S 0 = N (0) are often far larger than the denominator I 0 δ max . Hence, without loss of generality, in the sequel, we deemed that S(k) ≥ 0 for sufficiently large k. The next lemma indicates that the deceased cases under the linear control policy. 
for all k ≥ 1 with probability one.
Proof. We begin by recalling that D(k) = k−1 i=0 d I (i)I(i) for all k ≥ 1. Hence, using Lemma 4.1, we have
Note here that the assumption K ≤ (1 − d max )/v max assures I(k) ≥ 0 for all k with probability one; hence the inequality above is well-defined. To complete the proof, with the aid of sum of geometric series, we have
which completes the proof.
With the aids of Theorem 4.2 and Lemma 4.4, we see that if we take linear feedback policy with constant gain; i.e., u(k) = KI(k) and assuming that δ max < v min ( such that infected cases I(k) ≤ M D for all k with probability one. Therefore, the almost sure epidemic control problem is solved. While the almost sure epidemic control policy, if exists, can be a good candidate to mitigate the pandemic, some potential issues remain in practice. That is, recalling Remark 4.1.2 in Section 4, it tells us that, in some cases, the almost sure epidemic control policy, while exist, may not be possible to implement. To address this, as mentioned in Section 2, we now move to our second epidemic control problem which we call the average epidemic control problem. In this setting, the aim now becomes to control the "expected" infected cases and "expected" deceased cases
In this section, we provide our results on control of epidemics in the sense of expected value. We begin with discussing the control of expected infected cases.
The lemma below provides an useful analytical expression for the expected number of infected cases. Proof. The proof is straightforward. We begin by noting that, for any linear policy u(k) = KI(k) with gain K ∈ [0, (1 − d max )/v max ], I(k) ≥ 0 for all k with probability one and I(k) = k−1 i=0 (1 + δ(i)−d I (i)−Kv(i))I 0 . Since δ(k), d I (k) and v(k) are i.i.d in k and are mutually independent, taking the expected value on the I(k) above yields
and the proof is complete.
With the aid of the lemma above, we are now ready to provide our second main result. Proof. The idea of the proof is similar to the one used in Theorem 4.2. However, for the sake of completeness, we provide our full proof here. We begin by assuming that δ < (1−dmax)v vmax and δ−dI v
/v max , it implies that I(k) > 0 for all k with probability one. Now, according to Lemma 5.1, we have
where
Note that
Hence, log(1 + δ − d I − Kv) < 0 for any K > (δ − d I )/v and the logarithmic function is welldefined for K within the assumed range. Therefore, we have log E[f (0)] < 0, which implies that lim k→∞ E[I(k)]/I 0 → 0 and the proof for part (i) is complete. To prove part (ii), we fix k and simply note that, with the assumed assumptions on δ and K and the fact that d I (·) ≥ 0, it is readily verified that E[I(k)] ≤ I 0 , which completes the proof of part (ii).
To prove part (iii), take K = (δ − d I )/v and substitute it back into the equation 12, we obtain E[I(k)] = I 0 for all k and the proof of part (iii) is complete.
Similar to Remark 4.1.2, to accommodate the practical considerations, we fix L ≥ 1 and require
Due to the limited available medical resources, it is reasonable to choose a "sub-optimal" K in the sense of minimizing the potential use of medical resources. For example, we let 1 {δ≤Lv} and 1 {δ>Lv} be the indicator functions and consider
provided that δ < v(1−dmax) vmax to be our choice of feedback gain for controlling of epidemics in the sense of expected value. Note that K * > δ−dI v for d I > 0; hence, the theorem above applies. This usage of K * will be also seen later in Section 6. 
where the last inequality holds by using Lemma 5.1. The next lemma states the expected susceptible cases. 
We begin by recalling that Lemma 4.3 tells us that S(k) ≥ 0 for stage k = 1, 2, . . . , ⌊S 0 /(δ max I 0 )⌋. Now using the facts that δ(k) are i.i.d. in k, and δ, d I and v are mutually independent, we have
Using the geometric series, we conclude
which completes the proof.
For k ≥ 1, any linear feedback control policy of the form u(k) = KI(k) with 0 ≤ K ≤ (1 − d max )/v max yields the expected deceased cases
Proof. The proof is similar to Lemma 4.4. Using Lemma 4.1 and the fact that v(k), d I (k), and δ(k) are independent, we have
It is easy to verify that d I (i) and (1 + δ(j) − d I (j) − Kv(j)) are independent for all j = 0, . . . , i − 1; hence, we have
Using the geometric series, the equality above reduces to
which is desired.
For u(k) = KI(k), one can readily verify that
With the aids of Theorem 5. 
such that expected deceased cases E[D(k)] ≤ C D for all k. Therefore, the average epidemic control problem is solved. In the next section to follow, we provide an illustrative example using historical COVID-19 data to demonstrate our epidemic control performance.
We now illustrate the application of our control methodology on the epidemiological model using historical data for year 2020 available in https://ourworldindata.org/coronavirus, which contain the number of daily confirmed cases, denoted by c(k), and daily reported deaths, denoted by d(k) for k = 0, 1, . . . , N − 1 for some fixed integer N .
To study the epidemic control performance, there are various way to estimate the uncertain parameters; e.g., one can consult "standard" approach such as minimizing the least-square estimation error to obtain the "optimal" parameters δ, d I and v; e.g., see [5, 6, 9] . However, for the sake of simplicity, we now provide a simple "mean-replacing" approach to estimate the uncertain rate of infected cases δ(·) and the death rate d I via the available data of confirmed cases and the number of reported deaths. That is, we begin by recalling that the deceased cases satisfy D(k + 1) = D(k) + d I (k)I(k). For k = 0, 1, . . . , N − 1, given the confirmed (infected) cases c(k) and reported deaths d(k) and taking c(k) := I(k) and d(k) := D(k), we obtain
Then, the estimate of death rate d I (k), call it d I , is defined by 4
Having found d I , we can now estimate the remaining two uncertain quantities δ(k) and v(k). To this end, assume that the adopted epidemic control policy is linear of the form u(k) = KI(k) with K := 1; i.e., we ideally assumed that government had put all available resources to control the pandemic. Now, setting c(k) := I(k), d I := d I (k), and replacing v(k) by its mean v, we obtain
With the aid of equation (16), the estimate of δ(k), call it δ, is given by
Having obtained the estimates δ and d I , we then ready to apply the epidemiological model and use it to compare it with the available data; see next subsections to follow.
Using the data from the start of the epidemy, over a horizon starting from March 1, 2020 to September 8, 2020, we obtain the estimates δ with δ max := δ ≈ 0.5135 and d I := d max ≈ 0.0449. In addition, we take L = 2 and assume that the effectiveness rate of control v(k) follows a uniform distribution with v min = 0.1 and v max = 0.2; i.e., 10% to 20% effectiveness rate on control. Observe that
Hence, Theorem 4.2 does not apply. However, we can still choose a suboptimal feedback gain discussed in Section 4.1.2; i.e.,
This means that, if the infected cases can not be well-controlled in almost sure sense, governments should put whatever they can to suppress it; here, we see a 100 · L% feedback gain is applied. The epidemic control performance is shown in Figure 1 where the black solid lines depict the confirmed cases c(k) and reported deaths d(k) from historical COVID-19 data in the United States. The other thinner lines with various colors depict the epidemic control performance, in terms of c(k) and d(k), under the linear policy u(k) = Kc(k) with K := L. Interestingly, the figure also tells us that if L = 2 is possible, it may yield, in average, a lower confirmed cases.
With the aid of equations 15 and 17, we obtain the estimates d I ≈ 0.002 and δ := δ ≈ 0.215. Similar to the previous example, we again assume that v(k) follows a uniform distribution with v min = 0.1 and v max = 0.2; hence the average v = 0.15. It is readily verified that our estimates satisfy Hence, Theorem 5.2 applies if we take
The control policy above tells us that the government may need to bring in extra medical resources to achieve K > 1. The corresponding epidemic control performance is shown in Figure 2 where the confirmed cases is on the top panel and the deaths is on the bottom panel. In the figure, we see a downward trend occurs on the confirmed cases and the saturated deaths as time increases. The Figure 3 shows, with y-axis in log-scale, the epidemic control performance comparison where the black solid lines are the reported confirmed cases (top) and reported deaths (bottom).
This preliminary work has been done in the urgency of the ongoing COVID-19 pandemic, with the mind of providing a simple yet explainable epidemiological model with a rigorous study on the effectiveness of a linear epidemic control policy class. Consistent with the existing literature on epidemic modeling and control, this paper considers a modified stochastic SIRD-based model. We analyzed the model and considered two epidemy control problems: One is the almost sure epidemic control problem and the other is the average epidemic control problem. Then, for both of two problems, with linear control policy, we show sufficient conditions on feedback gain so that the epidemy is deemed to be "well-controlled" in the sense that infected cases goes down to zero asymptotically, and both infected cases and deceased cases are upper bounded uniformly. Subsequently, we provide a simple data-driven parameter estimations and show some promising numerical results using historical COVID-19 data in the United States. Based on our work to date, two important directions immediately present themselves for future work which described in the next subsections to follow.
It is possible to extend our analysis to involve multi-population epidemics. To illustrate this, below we consider only the infected cases dynamics. Fix m populations, then for each population i = 1, . . . , m, we write
with limited medical resource
for some u max > 0 and the overall infected cases are
It is also possible to consider the case where the control policy is with delay effect; i.e., u(k −d) with delay timed; see also [13] for a discussion on handling the time delay considerations in a class of positive finance systems.
Another possible research direction would be to take the economic budget into play. As seen in Remark 4.1.2, the idea of minimizing the expenditure induced by implementing control policy may lead to a version of "optimal" choice for feedback gain K. In particular, one can even consider the budget dynamics and carry out the optimization. Specifically, let B(k) be the available medical budget at the kth day which satisfies B(k + 1) = B(k) + p(k)u(k)
where p(k) is the price to pay per control policy at each stage, which can be modeled as a random variables with finite support. Then, it is readily verified that
for k ≥ 1. From here, there are many possible directions to pursue. For example, one can consider a optimization problem which minimizes the expected budget cost and infected cases; i.e., one might seek to find a sequence of K which solve inf K E[βB(k) + γI(k)]
for some constants β and γ.