SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Balance Risk and Reward: A Batched-Bandit Strategy for Automated Phased Release
Yufan Li $^{1}$ , Jialiang Mao $^{2}$ , Iavor Bojinov $^{3}$
$^{1}$ Harvard University  $^{2}$ LinkedIn Corporation  $^{3}$ Harvard Business School yufan_li@g.harvard.edu, jimao@linkedin.com, ibojinov@hbs.com
Phased releases are a common strategy in the technology industry for gradually releasing new products or updates through a sequence of A/B tests in which the number of treated units gradually grows until full deployment or depreciation. Performing phased releases in a principled way requires selecting the proportion of units assigned to the new release in a way that balances the risk of an adverse effect with the need to iterate and learn from the experiment rapidly. In this paper, we formalize this problem and propose an algorithm that automatically determines the release percentage at each stage in the schedule, balancing the need to control risk while maximizing ramp-up speed. Our framework models the challenge as a constrained batched bandit problem that ensures that our pre-specified experimental budget is not depleted with high probability. Our proposed algorithm leverages an adaptive Bayesian approach in which the maximal number of units assigned to the treatment is determined by the posterior distribution, ensuring that the probability of depleting the remaining budget is low. Notably, our approach analytically solves the ramp sizes by inverting probability bounds, eliminating the need for challenging rare-event Monte Carlo simulation. It only requires computing means and variances of outcome subsets, making it highly efficient and parallelizable.
Phased release, also known as staged rollout, is a widely used strategy in the technology industry that involves gradually releasing a new product or update to larger audiences over time [17, 30]. For example, Apple's App Store offers a phased release option where application updates are released over a 7-day period on a fixed schedule [1]. Google Play Console provides a similar feature with more flexibility in the release schedule [16]. Typically, the audiences are randomly selected at each stage from the set of all customers, and so phased releases can be thought of as a sequence of A/B tests (or randomized experiments) in which the proportion of units assigned to the treatment group changes until either the product or update is fully launched or deprecated [26, 18, 3, 33, 6]. The process of combining phased releases with A/B tests is often called controlled rollout or iterative experiments and provides companies with an important mechanism to gather feedback on early product versions [30, 20, 5].
The key advantage of phased release is its ability to mitigate risks associated with launching a new product or update directly to all users. The potential impact of faulty features is limited by releasing the update first to a small percentage of the users (i.e., the treatment group). However, this risk-averse approach introduces an opportunity cost for slowly launching beneficial features, which quickly adds up for companies that release thousands of features yearly [34]. Therefore, when designing a phased release schedule, it is important to determine the release percentage (known as ramp schedule) at each stage that balances the need to control risk while maximizing the speed of ramp-up. This paper proposes an algorithm to address this challenge by automatically determining the release
percentage for the next phase based on observations from previous stages. Specifically, we frame the challenge as a budget-constrained batched bandit problem. For each batch, we aim to determine the assignment probabilities of newly arrived users while keeping the probability of depleting a pre-specified experimental budget, where the experiment's cost is the cumulative treatment effect that are not directly observed. Formally, we derive recursive relations that decompose the risk of ruin (depleting the budget) of a phased release to the individual stages in the sense that the risk of ruin of the entire experiment is controlled if stage-wise ruin probabilities are controlled. Our algorithm is Bayesian in the sense that it learns from past observations by computing the posteriors of a conjugate Gaussian model and uses these parameters to infer the remaining budget and other cost-related quantities. However, the algorithm is robust to misspecifications and works well even when underlying outcomes are far from Gaussian by law of large number and central limit theorem; nevertheless, in Appendix E, we provide an extension to non-Gaussian outcomes. Finally, the next stage's assignment probabilities are derived from the posterior distribution and the stage-wise risk tolerances. Notably, our approach solves ramp sizes analytically from inverting the ruin probability upper bounds, avoiding challenging rare-event Monte-Carlo simulation for budget depletion events and data imputation procedures for unobserved counterfactual outcomes.
1.1 Literature review
While many firms have guidelines on how to conduct a phased release process, these guidelines are often ad-hoc and qualitative, making it difficult to create executable ramp schedules. The SQR framework in [34] is the first attempt to address this problem by providing quantitative guidance. Our work differs significantly from SQR. Our algorithm adopts a fully Bayesian approach, enabling us to incorporate prior information on the risk of a feature in a probabilistic manner when initiating a ramp. Additionally, unlike SQR, our approach introduces a "shared budget" over the entire phased release, allowing the budget to be sequentially adjusted based on the observations from prior iterations. Finally, our algorithm is robust to modifications made to the treatment during experiments and different outcome models.
Our work is notably distinct from the risk-averse multiarmed bandit approaches considered in previous research [19, 14, 35, 28, 23, 10, 8]. In these approaches, the agent considers the expected variability in expected rewards to identify and avoid less predictable (and therefore risky) actions, without considering a budget constraint. A related literature focuses on batched and Bayesian variants of these methods in multi-stage clinical trials [4, 27, 21, 24, 2]. While this literature also aims to determine treatment assignment for each stage of the experiment, it differs from our setting in two key aspects: (i) the objective is to maximize treatment effect while balancing exploration of treatment arms, rather than rapidly ramping up experiments, and (ii) to the best of our knowledge, no clinical trials paper has addressed the imposition of a budget for potential adverse treatment effects. Hence, bandit approaches developed for clinical trials cannot be directly applied to our setting. To illustrate the difference, we present a numerical simulation of a Thompson sampling-based Bayesian bandit from [27] and highlight that budget spent and the aggressiveness of the ramp-up schedule depends on model tuning in a very unpredictable way, making the ramp-up schedule far from ideal. Another related literature is budgeted multiarmed bandits [32, 31, 9, 29, 12]. However, most budgeted bandit algorithms are developed for settings very different from ours and do not consider risk-of-ruin control or handle unobserved costs. Therefore, these algorithms cannot be directly applied to our specific scenario.
Notation. Let  $\mathbb{N}$  be the set of non-negative integers and  $\mathbb{N}_{+} := \mathbb{N} \setminus \{0\}$ . Let  $\mathbb{R}, \mathbb{R}_{+}$  denote the set of real numbers and positive real numbers respectively.  $[N] := 1, \ldots, N$  for  $N \in \mathbb{N}_{+}$ .  $\sigma(\cdot)$  is the generated  $\sigma$ -algebra.  $X \in \mathcal{F}$  if random variable  $X$  is measurable to  $\mathcal{F}$ .  $[X \mid \mathcal{F}]$  denotes a random variable with distribution  $\mathbb{P}(X \in \cdot \mid \mathcal{F})$  for a random variable  $X$  and  $\sigma$ -algebra  $\mathcal{F}$ .
2 Risk-of-ruin-constrained experiment and strategy overview
2.1 Risk-of-ruin-constrained experiment
Consider a scenario in which a single feature is released to a sequence of subpopulations  $\mathcal{N}_t$  consisting of  $N_{t}$  units at stages  $t = 1,\dots,T$ , where  $T$  is not necessarily fixed. At each stage, we randomly assign treatment to a group of units denoting the indexing set  $\mathcal{T}_t$ , while the control group with indexing
set  $\mathcal{C}_t$ . The size of the treatment group at stage  $t$  is  $|\mathcal{T}_t| = m_t$  and the size of the control group is  $|\mathcal{C}_t| = N_T - m_t$ .
Our paper adopts the Neyman-Rubin framework for causal inference [22, 25, 7], where the potential outcome of each unit  $i \in \mathcal{N}_t$  during experiment stage  $t$  under control and treatment are denoted by  $Y_{i,t}(0) \in \mathbb{R}$  and  $Y_{i,t}(1) \in \mathbb{R}$ , respectively<sup>1</sup>. Appendix E provides the extension to multivariate outcomes. The treatment assignment of unit  $i \in \mathcal{N}_t$  at stage  $t$  is denoted by  $W_{i,t}$ . Since each unit only receives a single treatment at each stage, we only observe  $Y_{i,t}(W_{i,t})$  during the experiment, not the counterfactual  $Y_{i,t}(1 - W_{i,t})$  (we are explicitly assuming that there is full compliance).
Let  $\mathcal{F}_t = \sigma\left((Y_{\mathcal{T}_k,k}(1))_{k\in [t]},(Y_{\mathcal{C}_k,k}(0))_{k\in [t]},(W_{i,k})_{i\in \mathcal{N},k\in [t]}\right)$  be the  $\sigma$ -algebra generated by the treatment assignment and the observed experiment outcome in the first  $t$  stages, with  $\mathcal{F}_0$  representing the trivial  $\sigma$ -algebra. In our setting, the experimenter aims to ramp up the experiment to the "max-power stage" (50% of the population placed in treatment) as quickly as possible while avoiding the risk of a large negative business impact (or cost). We define the cost of the experiment as the treatment effect on the treated. See generalized cost in Appendix E.
Definition 2.1 (Experiment cost). The cost of the experiment from stage  $t \in [T]$  is  $r_t := \sum_{i \in \mathcal{T}_t} Y_{i,t}(1) - Y_{i,t}(0)$ , where  $r_t = 0$  if  $\mathcal{T}_t = \emptyset$ . The cumulative cost is  $R_t := \sum_{k \in [t]} r_k$ .
Throughout,  $r_t < 0$  corresponds to a negative business impact; our goal is to control the experiment cost by setting a budget  $B < 0$  and imposing the cost constraint  $R_T > B$ . Since the outcomes are stochastic, we require this cost constraint to be satisfied with probability at least  $1 - \delta$  for some  $\delta \in [0,1)$  set before the experiment. Our goal is then to adaptively determine the size of the treatment group based on the observed data while satisfying our risk constraint. We refer to such an experiment as a risk-of-ruin-constrained (RRC) experiment.
Definition 2.2 (RRC experiment). Fix any  $B < 0$ ,  $\delta \in [0,1)$ . A  $(\delta, B)$ -RRC experiment running for  $T$  stages selects the size of  $\mathcal{T}_t$  before  $t$ -th stage of the experiment such that  $\mathbb{P}(R_T > B) \geq 1 - \delta$ .
2.2 Strategy overview
Our goal is to determine the number of users to assign to the treatment  $m_t \in [0, N_t / 2]$ , such that  $\sum_{i=1}^{T} m_t$  is maximized while ensuring that the experiment is  $(B, \delta)$ -RRC.
Our strategy is to decompose the overall constraint (i.e., that the experiment is  $(B, \delta)$ -RRC) into a sequence of stage-wise adaptive constraints using Theorem 3.1, below. We then sequentially maximize  $m_t$  under the stage-wise constraint for the  $t$ -th stage. Under the Gaussian model, the stage-wise constraints can be directly solved using a simple quadratic equation to obtain the maximum  $m_t$ .
Although our algorithm solves a relaxed version of the original optimization problem, our solution has crucial practical implications. Theorem 3.1 decomposes the  $(B,\delta)$ -RRC constraints in an adaptive fashion, i.e., if the feature turns out to be safe, the stage-wise constraints will relax in response, and the experiment will ramp up quickly.
3 Model and the algorithm
3.1 Decompose the risk of ruin
Our experimental design is based on the following theorem, which identifies a sequence of sufficient conditions for a sequential experiment to be  $(\delta, B)$ -RRC. We defer its proof to Appendix A.
Theorem 3.1. Fix  $B < 0$  and  $\delta \in [0,1)$ . For any stopping time  $T \geq 1$ , let  $(b_{t})_{t\in [T]}$  be a budget sequence, such that  $b_{t} \stackrel{(i)}{\geq} B, \forall t \in [T]$ , and  $(\Delta_t)_{t\in [T]}$  be a risk tolerance sequence, such that
<sup>1</sup>We are implicitly assuming that there is no interference between experimental units, that is, each unit's outcomes do not depend on any other unit's assignments [11].
$\Delta_t \in [0,1), \forall t \in [T]$  and  $1 - \prod_{t=1}^{T} (1 - \Delta_t) \stackrel{(ii)}{\leq} \delta$ . Then, if  $(\mathcal{T}_t)_{t \in [T]}$  is chosen such that for  $t = 1$
$$
\mathbb {P} \left(r _ {1} \leq b _ {1}\right) \stackrel {(i i i)} {\leq} \Delta_ {1} \tag {1}
$$
and for any  $t = 2,\dots,T$  almost surely,
$$
\left\{ \begin{array}{l l} \mathcal {T} _ {t} = \emptyset , & i f \mathbb {P} \left(R _ {t - 1} > B \mid \mathcal {F} _ {t - 1}\right) = 0 \\ \mathbb {P} \left(R _ {t} \leq b _ {t} \mid R _ {t - 1} > B, \mathcal {F} _ {t - 1}\right) \stackrel {(i v)} {\leq} \Delta_ {t}, & o t h e r w i s e \end{array} \right. \tag {2}
$$
then  $\mathbb{P}(R_T > B) \geq 1 - \delta$ . This inequality is tight when  $(i) - (iv)$  are all equalities and  $r_t \leq 0, \forall t \in [T]$  almost surely. Furthermore, if we set  $\mathcal{T}_t \gets \emptyset$ , (1), (2) always hold.
Recall,  $B < 0$  denotes the budget and  $\delta \in [0,1)$  is our risk tolerance that controls the risk of ruin (i.e., the probability of exceeding the budget); both need to be fixed a priori. In general, smaller  $\delta$  leads to more conservative experimentation and slower releases. The sequence  $(b_{t})_{t\geq 1}$  "rations" the budget: setting  $b_{t} < B$  at stage  $t$  reserves  $B - b_{t}$  budget for later stages, which may be beneficial when the released feature is expected to undergo modifications during the experiment [20]. To quickly scale the experiment, we can set  $b_{t} = B, \forall t$ . The sequence  $(\Delta_t)_{t\geq 1}$  distributes the overall tolerance  $\delta$  to individual stages by  $\delta = 1 - \prod_{i = t}^{T}(1 - \Delta_{t})$  allowing us to customize the tolerance for individual stages. If  $T$  is fixed a priori, we can uniformly distribute the tolerance by setting  $\Delta_t = 1 - (1 - \delta)^{1 / T}, \forall t\in [T]$ .
Theorem 3.1 breaks the risk constraint  $\mathbb{P}(R_T\leq B)\leq \delta$  into stage-wise constraints in the form of (1),(2). The idea is to control current-stage cumulative experiment cost  $R_{t}$  given past observations  $\mathcal{F}_t$  and determine the treatment assignment based on posterior inference of the remaining budget. In our setting, the goal is to maximize  $m_t = |\mathcal{T}_t|$  subject to (1),(2). Note that the first line in (2) stops the experiment when the model estimates that the budget is exhausted, while the second line sets the stage cost  $r_t$  below the remaining budget  $b_{t} - R_{t - 1}$  with high probability. We require assumptions on the data-generating model to derive an explicit algorithm, which we present in the next sections.
Generally, the total number of stages  $T$ , stage-wise budget and tolerance  $(b_{t})_{t\geq 1}$ ,  $(\Delta_t)_{t\geq 1}$  can be determined dynamically during the process. That is, we can define  $\Delta_t$  and  $b_{t}$  just before stage  $t$  as long as  $\prod_{r = 1}^{t}(1 - \Delta_{t})\leq 1 - \delta$  and  $b_{t}\geq B$ . If we plan to terminate the experiment after stage  $t$ , we can define  $\Delta_t$  such that  $\prod_{r = 1}^{t}(1 - \Delta_r) = 1 - \delta$  and  $b_{t}\geq B$  is attained in which case  $T = t$ . If is also possible to have  $T = +\infty$  and  $\prod_{t = 1}^{\infty}(1 - \Delta_t) = 1 - \delta$ . For example, choosing  $\Delta_t = (\gamma_\star /t)^2$  where  $\gamma_{\star}$  is the unique solution of  $\mathrm{sinc}(\gamma_\star) = 1 - \delta$  on [0, 1] (cf. [13, Eq. (1)]) satisfy our condition.
Note that the decomposition scheme in Theorem 3.1 is formulated such that  $\mathbb{P}(R_T\leq B)\leq \delta$  is tight if inequalities (i)-(iv) are tight. Practically, this means that the ramp-up schedules obtained through this approach are typically not overly conservative, unlike approaches that leverage a union-bound for risk decomposition. Finally, Theorem 3.1 holds for a general definition of the cost  $r_t\coloneqq r_t\left((Y_{i,t})_{i\in \mathcal{N}_t},\mathcal{T}_t\right)$  such that  $r_t = 0$  if  $\mathcal{T}_t = \emptyset$ , making it useful in other budgeted online problems beyond our setting.
3.2 Gaussian outcome model
For this subsection, make the following model assumptions on the outcomes distribution; appendix E provides the extension to general outcome models.
Definition 3.1 (Conjugate Gaussian outcomes). Let the unknown model parameters  $\mu_{\mathrm{true}}(0), \mu_{\mathrm{true}}(1) \in \mathbb{R}$  satisfy the prior  $\mu_{\mathrm{true}}(w) \sim N\left(\mu_0(w), \sigma_0(w)^2\right)$  for  $w = 0,1$  independently, where  $\mu_0(0), \mu_0(1) \in \mathbb{R}, \sigma_0(0)^2, \sigma_0(1)^2 \in \mathbb{R}_+$  are hyperparameters. The experiment outcome of unit  $i$  at stage  $t$  are distributed independently and identically as
$$
\left( \begin{array}{c} Y _ {i, t} (0) \\ Y _ {i, t} (1) \end{array} \right) \stackrel {\text {i i d}} {\sim} N \left(\left( \begin{array}{c} \mu_ {\text {t r u e}} (0) \\ \mu_ {\text {t r u e}} (1) \end{array} \right), \left( \begin{array}{c c} \sigma (0) ^ {2} & 0 \\ 0 & \sigma (1) ^ {2} \end{array} \right)\right) \tag {3}
$$
where  $\sigma (0)^2,\sigma (1)^2\in \mathbb{R}_+$  are hyperparameters.
The unknown parameters  $\mu_{\mathrm{true}}(0)$  and  $\mu_{\mathrm{true}}(1)$  represent the intrinsic quality of the feature before and after the update, as measured by a specific metric. If  $\mu_{\mathrm{true}}(1) - \mu_{\mathrm{true}}(0) < 0$ , the feature update is likely to have a negative business impact, i.e.,  $Y_{i,t}(1) - Y_{i,t}(0) < 0$ .
To derive the posterior distribution we need the following statistics. For  $t = 1, w = 0, 1$
$$
S _ {0} ^ {\mathcal {C}} (w) = s _ {0} ^ {\mathcal {C}} (w) = S _ {0} ^ {\mathcal {T}} (w) = s _ {0} ^ {\mathcal {T}} (w) = M _ {0} (w) = 0 \tag {4}
$$
$$
\mu_ {p, t = 1} (w) = \mu_ {0} (w), \quad \sigma_ {p, t = 1} (w) ^ {2} = \sigma_ {0} (w) ^ {2}.
$$
and for  $t\geq 2,w = 0,1$
$$
s _ {t} ^ {\mathcal {T}} (w) := \sum_ {i \in \mathcal {T} _ {t}} Y _ {i, t} (w), S _ {t} ^ {\mathcal {T}} (w) := \sum_ {r \in [ t ]} s _ {r} ^ {\mathcal {T}} (w), s _ {t} ^ {\mathcal {C}} (w) := \sum_ {i \in \mathcal {C} _ {t}} Y _ {i, t} (w), S _ {t} ^ {\mathcal {C}} (w) := \sum_ {r \in [ t ]} s _ {r} ^ {\mathcal {C}} (w) \tag {5a}
$$
$$
\mu_ {p, t} (w) := \frac {1}{\frac {1}{\sigma_ {0} (w) ^ {2}} + \frac {M _ {t - 1} (w)}{\sigma (w) ^ {2}}} \left(\frac {\mu_ {0} (w)}{\sigma_ {0} (w) ^ {2}} + \frac {\mathbb {I} (w = 0) S _ {t - 1} ^ {\mathcal {C}} (w) + \mathbb {I} (w = 1) S _ {t - 1} ^ {\mathcal {T}} (w)}{\sigma (w) ^ {2}}\right) \tag {5b}
$$
$$
\sigma_ {p, t} (w) ^ {2} = \left(\sigma_ {0} (w) ^ {- 2} + M _ {t - 1} ^ {(w)} \sigma (w) ^ {- 2}\right) ^ {- 1}, \tag {5c}
$$
where  $M_t^{(1)} \coloneqq \sum_{r \in [t]} m_r$  and  $M_t^{(0)} \coloneqq \sum_{r \in [t]} N_r - m_r$  are the cumulative number of users in the treatment and control groups up to stage  $t$ , respectively. In (5a),  $S_t^{\mathcal{T}}(w)$  and  $S_t^{\mathcal{C}}(w)$  represent the cumulative sum of outcomes for  $w = 0, 1$  in the treatment and control groups up to stage  $t$ , while  $s_t^{\mathcal{T}}(w)$  and  $s_t^{\mathcal{C}}(w)$  represent the sum of outcomes at stage  $t$ . In equation (5b),  $\mu_{p,t}(w)$  represents the posterior mean of  $\mu_{\mathrm{true}}(w)$ , while in equation (5c),  $\sigma_{p,t}(w)^2$  represents the posterior variance of  $\mu_{\mathrm{true}}(w)$ , for  $w = 0, 1$ . When lacking prior information, we suggest using a non-informative priors by setting  $\mu_0(0) = \mu_0(1) = 0$  and  $\sigma_0(0)^2, \sigma_0(1)^2$  sufficiently large.
The model parameters  $\sigma(0)^2$  and  $\sigma(1)^2$  at stage  $t \geq 2$  can be estimated using unbiased and consistent estimators. For  $w = 0, 1$  let
$$
\sigma (w) ^ {2} \leftarrow \frac {\sum_ {r \in [ t - 1 ]} \sum_ {i \in \mathcal {C} _ {r}} \left(Y _ {i , r} (w) - \frac {1}{M _ {t - 1} (w)} \left(\mathbb {I} (w = 0) S _ {t - 1} ^ {\mathcal {C}} (w) + \mathbb {I} (w = 1) S _ {t - 1} ^ {\mathcal {T}} (w)\right)\right) ^ {2}}{M _ {t - 1} (w) - 1}. \tag {6}
$$
For  $t = 1$ , some prior estimate can be used, either from a similar experiment or from a small-scale pretrial run.
3.3 An algorithm for the sample size in an RRC experiment
We now derive an explicit algorithm from Theorem 3.1 that to outputs  $(m_t)_{t\geq 1}$ , the treatment group size at stage  $t$ , such that, the experiment is  $(\delta ,B)$ -RRC. Recall that for an experiment to be  $(\delta ,B)$ -RRC, it suffices that (1), (2) holds for each  $t\geq 1$ . Under Definition 3.1, we have that (i)  $(Y_{i,t})_{i,t}$  are exchangeable random variables (ii) for any  $t\geq 2$ ,  $\mathbb{P}(S_{t - 1}^{\mathcal{T}}(0) < S_{t - 1}^{\mathcal{T}}(1) - B|\mathcal{F}_{t - 1}) > 0$  almost surely for any choice of  $m_{[t - 1]}$ . Combining these observations, we get that (1), (2) hold if for each  $t\geq 1$ ,
$$
\mathbb {P} \left(s _ {t} ^ {\mathcal {T}} (1) - S _ {t} ^ {\mathcal {T}} (0) \leq b _ {t} - S _ {t - 1} ^ {\mathcal {T}} (1) \mid S _ {t - 1} ^ {\mathcal {T}} (0) <   S _ {t - 1} ^ {\mathcal {T}} (1) - B, \mathcal {F} _ {t - 1}\right) \leq \Delta_ {t}. \tag {7}
$$
Lemma 3.2 provides an upper bound of the left hand size of (7); the proof is in Appendix B.
Lemma 3.2 (Stochastic domination). Assume the outcomes  $(Y_{i,t}(0), Y_{i,t}(1))_{i,t}$  are generated as in Definition 3.1. For any  $\geq 1$ , almost surely,
$$
\begin{array}{l} \mathbb {P} \left(s _ {t} ^ {\mathcal {T}} (1) - S _ {t} ^ {\mathcal {T}} (0) \leq b _ {t} - S _ {t - 1} ^ {\mathcal {T}} (1) \mid S _ {t - 1} ^ {\mathcal {T}} (0) <   S _ {t - 1} ^ {\mathcal {T}} (1) - B, \mathcal {F} _ {t - 1}\right) \tag {8} \\ \leq \mathbb {P} \left(s _ {t} ^ {\mathcal {T}} (1) - S _ {t} ^ {\mathcal {T}} (0) \leq b _ {t} - S _ {t - 1} ^ {\mathcal {T}} (1) \Big | \mathcal {F} _ {t - 1}\right). \\ \end{array}
$$
Using Lemma 3.2, for (1) and (2) to hold, it suffices to choose any  $m_t \in \mathbb{N}$  such that
$$
\mathbb {P} \left(s _ {t} ^ {\mathcal {T}} (1) - S _ {t} ^ {\mathcal {T}} (0) \leq b _ {t} - S _ {t - 1} ^ {\mathcal {T}} (1) \mid \mathcal {F} _ {t - 1}\right) \leq \Delta_ {t} \tag {9}
$$
and set  $m_t = 0$  if such  $m_t$  does not exist. From posterior-predictive formulas for the conjugate Gaussian model in Definition 3.1 (see (15e), (15f) in Appendix C), we have  $\left[s_t^\mathcal{T}(1) - S_t^\mathcal{T}(0)\right|_{\mathcal{F}_{t-1}} \sim N(\tilde{\mu}_t(m_t), \tilde{\sigma}_t^2(m_t))$ , where
$$
\tilde {\mu} _ {t} (m) := \mu_ {p, t} (1) \cdot m - \mu_ {p, t} (0) \cdot \left(m + M _ {t - 1} ^ {(1)}\right) \tag {10a}
$$
$$
\tilde {\sigma} _ {t} ^ {2} (m) := m ^ {2} \cdot \sigma_ {p, t} (1) ^ {2} + m \cdot \sigma (1) ^ {2} + \left(m + M _ {t - 1} ^ {(1)}\right) ^ {2} \cdot \sigma_ {p, t} (0) ^ {2} + \left(m + M _ {t - 1} ^ {(1)}\right) \cdot \sigma (0) ^ {2}. \tag {10b}
$$
Combining the above with (9) yields the following Lemma.
Lemma 3.3. Assume the outcomes  $(Y_{i,t}(0),Y_{i,t}(1))_{i,t}$  are generated as in Definition 3.1. For each  $t\geq 1$ , the inequality (9) holds if and only if
$$
\frac {b _ {t} - S _ {t - 1} ^ {\mathcal {T}} (1) - \tilde {\mu} _ {t} \left(m _ {t}\right)}{\tilde {\sigma} _ {t} \left(m _ {t}\right)} \leq q _ {t} := \Phi^ {- 1} \left(\Delta_ {t}\right) \tag {11}
$$
where  $\Phi^{-1}$  denotes inverse CDF of the standard normal distribution.
Replace the inequality in (11) with equality and square both sides gives us the quadratic equation  $A_{t} \cdot m_{t}^{2} + B_{t} \cdot m_{t} + C_{t} = 0$  where