SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Finally, we combine the last inequality with inequality (14) and sum these over  $k = 1,\dots ,K$
$$
\begin{array}{l} \sum_ {t = 1} ^ {\bar {t} _ {K}} \ell_ {t} (y _ {t}) - \ell_ {t} (y) \leq \sum_ {k = 1} ^ {K} \langle \tilde {g} _ {k}, y _ {t _ {k}} - y \rangle \\ \leq \sum_ {k = 1} ^ {K} \frac {1}{2 \eta_ {\bar {t} _ {k}}} \left(\| y _ {t _ {k}} - y \| _ {2} ^ {2} - \left\| y _ {t _ {k + 1}} - y \right\| _ {2} ^ {2}\right) + \sum_ {k = 1} ^ {K} \frac {\eta_ {\bar {t} _ {k}}}{2} \| \tilde {g} _ {k} \| _ {2} ^ {2} \\ = \frac {1}{2} \left(\frac {\left\| y _ {t _ {1}} - y \right\| _ {2} ^ {2}}{\eta_ {\tilde {t} _ {1}}} - \frac {\left\| y _ {t _ {K + 1}} - y \right\| _ {2} ^ {2}}{\eta_ {\tilde {t} _ {K}}} + \sum_ {k = 1} ^ {K - 1} \left(\frac {1}{\eta_ {\tilde {t} _ {k + 1}}} - \frac {1}{\eta_ {\tilde {t} _ {k}}}\right) \left\| y _ {t _ {k + 1}} - y \right\| _ {2} ^ {2}\right) \\ + \sum_ {k = 1} ^ {K} \frac {\eta_ {\bar {t} _ {k}}}{2} \| \tilde {g} _ {k} \| _ {2} ^ {2} \\ \leq \frac {1}{2} \left(\frac {D ^ {2}}{\eta_ {\bar {t} _ {1}}} + \sum_ {k = 1} ^ {K - 1} \left(\frac {1}{\eta_ {\bar {t} _ {k + 1}}} - \frac {1}{\eta_ {\bar {t} _ {k}}}\right) D ^ {2}\right) + \sum_ {k = 1} ^ {K} \frac {\eta_ {\bar {t} _ {k}}}{2} \| \tilde {g} _ {k} \| _ {2} ^ {2} \\ = \frac {D ^ {2}}{2 \eta_ {\bar {t} _ {K}}} + \sum_ {k = 1} ^ {K} \frac {\eta_ {\bar {t} _ {k}}}{2} \| \tilde {g} _ {k} \| _ {2} ^ {2}. \\ \end{array}
$$
Claim ii) is thereby proved by taking the supremum over  $y \in \mathcal{V}$ .
Proposition 19 gives us guidelines to design optimal update cycles and learning rates. First, this proposition states that it suffices to verify the feasibility constraint at the update periods to ensure that the whole sequence of order-up-to levels produced by COSD is feasible. On the other hand, to guarantee that this algorithm has a sublinear regret we will need to ensure that the cycles  $\mathcal{T}_k$  are not too long, i.e. that update periods are frequent enough, and then consider adequate learning rates.
Since OSD is an instance of COSD, we can recover classical regret bounds of OSD (see e.g. [8, Theorem 3.1] or [21, Theorem 2.13]) and in particular  $O(\sqrt{T})$  regret bounds, as a corollary of Proposition 19.
Corollary 20. Let assumptions 2.i) and 2.ii) be satisfied. Given positive non-increasing learning rates  $(\eta_t)_{t\in \mathbb{N}}$ , i.e.  $0 < \eta_{t + 1}\leq \eta_t$  for all  $t\in \mathbb{N}$ , the regret of OSD (see Algorithm 1) satisfies for all  $T\in \mathbb{N}$ ,
$$
R _ {T} \leq \frac {D ^ {2}}{2 \eta_ {T}} + \sum_ {t = 1} ^ {T} \frac {\eta_ {t}}{2} \| g _ {t} \| _ {2} ^ {2}.
$$
In particular, if  $\eta_t = \gamma D / (G\sqrt{t})$  with  $\gamma > 0$ , we have
$$
R _ {T} \leq \left(\frac {1}{2 \gamma} + \gamma\right) G D \sqrt {T}.
$$
Proof. The regret bound follows from claim ii) of Proposition 19. Indeed, by taking  $t_k = k$  for all  $k \in \mathbb{N}$ , COSD coincide with OSD, thus, for  $K = \bar{t}_K = T$  we obtain the desired regret bound.
A.3 Controlling the cycles' length
In this subsection we analyze COSD with eventually unbounded cycles as it is the case for the CUP or the MaxCOSD update strategy. We start by introducing an assumption on the cycles called geometric cycles which yields expected regret bounds and high probability regret bounds.
Assumption 21 (Geometric cycles). Let  $\mu \in (0,1]$ . An update strategy  $(t_k)_{k\in \mathbb{N}}$  has  $\mu$ -geometric cycles if there exists  $C_\mu \geq 1$  such that for any  $m\in \mathbb{N}$  and  $k\in \mathbb{N}$  we have:
$$
\mathbb {P} \left[ t _ {k + 1} - t _ {k} > m \right] \leq C _ {\mu} (1 - \mu) ^ {m}. \tag {15}
$$
The name geometric cycles is motivated by the fact that if  $\xi$  is a geometric random variable with parameter  $\mu$ , i.e.  $\xi \in \mathbb{N}$  and  $\mathbb{P}[\xi > m] = (1 - \mu)^m$  for all  $m \in \mathbb{N}$ , then Assumption 21 rewrites:
$\mathbb{P}\left[t_{k + 1} - t_k > m\right] \leq C_\mu \mathbb{P}\left[\xi > m\right]$  for all  $k, m \in \mathbb{N}$ . This property resembles sub-exponential concentration [29, Section 2.7]. This notion was inspired from a proof in [32] where the authors observe, under their assumptions, that CUP's cycles length are geometrically distributed with a parameter lower bounded by  $\mu$ . Assumption 21 generalizes this property.
The following proposition summarizes some important properties of geometric cycles.
Proposition 22. Consider an update strategy  $(t_k)_{k\in \mathbb{N}}$  with  $\mu$ -geometric cycles (see Assumption 21), then,
- For all  $k \in \mathbb{N}$ ,  $\mathbb{E}\left[t_{k + 1} - t_k\right] \leq C_\mu /\mu$  and  $\mathbb{E}\left[(t_{k + 1} - t_k)^2\right] \leq C_\mu (2 - \mu) / \mu^2 \leq 2C_\mu /\mu^2$ .
- For all  $K \in  \mathbb{N}$  and any confidence level  $\delta  \in  \left( {0,1}\right)$  we have with probability at least  $1 - \delta$  ,
$$
\sqrt {\sum_ {k = 1} ^ {K} \left(t _ {k + 1} - t _ {k}\right) ^ {2}} \leq \left(1 + \frac {\log \left(K C _ {\mu} / \delta\right)}{\mu}\right) \sqrt {K}.
$$
Proof. First, let us observe that if  $\mu = 1$  in Assumption 21 then,  $t_k = k$  for all  $k \in \mathbb{N}$  and one can easily check that all the claims are indeed verified. In the following we assume  $\mu \in (0,1)$ .
Let  $\xi$  be a geometric random variable of parameter  $\mu$ , that is,  $\xi \in \mathbb{N}$  and  $\mathbb{P}[\xi > m] = (1 - \mu)^m$  for all  $m \in \mathbb{N}$ . Then, it is well-known that  $\mathbb{E}[\xi] = 1 / \mu$  and  $\mathbb{E}[\xi^2] = (2 - \mu) / \mu^2$ .
We now prove the first claim by means of direct computations. For all  $k \in \mathbb{N}$ , we have:
$$
\mathbb {E} \left[ t _ {k + 1} - t _ {k} \right] = \sum_ {m = 0} ^ {+ \infty} \mathbb {P} \left[ t _ {k + 1} - t _ {k} > m \right] \leq \sum_ {m = 0} ^ {+ \infty} C _ {\mu} \mathbb {P} \left[ \xi > m \right] = C _ {\mu} \mathbb {E} \left[ \xi \right] = \frac {C _ {\mu}}{\mu}.
$$
Also, using the fact that for any integer  $a$  and real number  $b$ , we have,  $a > b$  if and only if  $a > \lfloor b \rfloor$ , we can upper bound  $\mathbb{E}\left[(t_{k + 1} - t_k)^2\right]$  as follows:
$$
\begin{array}{l} \mathbb {E} \left[ (t _ {k + 1} - t _ {k}) ^ {2} \right] = \sum_ {m = 0} ^ {+ \infty} \mathbb {P} \left[ (t _ {k + 1} - t _ {k}) > \sqrt {m} \right] = \sum_ {m = 0} ^ {+ \infty} \mathbb {P} \left[ (t _ {k + 1} - t _ {k}) > \lfloor \sqrt {m} \rfloor \right] \\ \leq \sum_ {m = 0} ^ {+ \infty} C _ {\mu} \mathbb {P} \left[ \xi > \lfloor \sqrt {m} \rfloor \right] = C _ {\mu} \mathbb {E} \left[ \xi^ {2} \right] = C _ {\mu} \frac {2 - \mu}{\mu^ {2}} \leq \frac {2 C _ {\mu}}{\mu^ {2}}. \\ \end{array}
$$
Finally, let us prove the second claim. Let  $K \in \mathbb{N}$  and  $\varepsilon > 0$ , it is classical to see that:
$$
\begin{array}{l} \mathbb {P} \left[ \sqrt {\sum_ {k = 1} ^ {K} (t _ {k + 1} - t _ {k}) ^ {2}} > \varepsilon \right] \leq \mathbb {P} \left[ \exists k \in [ K ], (t _ {k + 1} - t _ {k}) ^ {2} > \varepsilon^ {2} / K \right] \\ \leq \sum_ {k = 1} ^ {K} \mathbb {P} \left[ \left(t _ {k + 1} - t _ {k}\right) > \lfloor \varepsilon / \sqrt {K} \rfloor \right]. \\ \end{array}
$$
We use Assumption 21, which yields:
$$
\mathbb {P} \left[ \sqrt {\sum_ {k = 1} ^ {K} (t _ {k + 1} - t _ {k}) ^ {2}} > \varepsilon \right] \leq K C _ {\mu} (1 - \mu) ^ {\lfloor \varepsilon / \sqrt {K} \rfloor} \leq K C _ {\mu} (1 - \mu) ^ {(\varepsilon / \sqrt {K}) - 1}.
$$
Now for  $\delta \in (0,1)$  we plug  $\varepsilon = \left(1 + \frac{\log(\delta / (KC_{\mu}))}{\log(1 - \mu)}\right)\sqrt{K} > 0$  in the last result, to obtain:
$$
\sqrt {\sum_ {k = 1} ^ {K} \left(t _ {k + 1} - t _ {k}\right) ^ {2}} \leq \left(1 + \frac {\log (\delta / (K C _ {\mu}))}{\log (1 - \mu)}\right) \sqrt {K},
$$
with probability at least  $1 - \delta$ . We conclude by simply observing that  $-1 / \log (1 - \mu) \leq 1 / \mu$ .
Using these tools, we are now ready to provide strong regret bounds for COSD under the assumption of geometric cycles. We are going from now on to focus on adaptive learning rates (see Eq. 7) which will allow us to unlock high probability regret bounds.
Corollary 23. Consider an inventory problem satisfying Assumption 2 and COSD (see Algorithm 3) with adaptive learning rates (see Eq. (7)) and an update strategy with  $\mu$ -geometric cycles (see Assumption 21), then, the following regret bounds hold for all  $T \in \mathbb{N}$ ,
$$
\mathbb {E} \left[ R _ {T} \right] \leq \frac {\sqrt {2 C _ {\mu}}}{\mu} D G \left(\frac {1}{2 \gamma} + \gamma + 1\right) \sqrt {T},
$$
$$
R _ {T} \leq D G \left(\frac {1}{2 \gamma} + \gamma + 1\right) \left(1 + \frac {1}{\mu} \log \left(\frac {T C _ {\mu}}{\delta}\right)\right) \sqrt {T}.
$$
Finally,  $COSD$  is feasible if and only if for all  $k \in \{2, 3, \ldots\}$ ,  $y_{t_k} \succeq x_{t_k}$ .
Proof. Let  $T \in \mathbb{N}$  and  $K = \min \{k \geq 1, t_k \leq T\}$ . We start by bounding the regret  $R_T$  in terms of the regret at the end of  $K - 1^{th}$  cycle  $R_{\tilde{t}_{K-1}} = R_{t_{K-1}}$  and a remainder term as follows:
$$
R _ {T} \leq R _ {\bar {t} _ {K - 1}} + \sup _ {y \in \mathcal {Y}} \sum_ {t = t _ {K}} ^ {T} \ell_ {t} (y _ {t}) - \ell_ {t} (y) \leq R _ {\bar {t} _ {K - 1}} + \sup _ {y \in \mathcal {Y}} \sum_ {t = t _ {K}} ^ {T} \langle g _ {t}, y _ {t} - y \rangle .
$$
where we used  $g_{t} \in \partial \ell_{t}(y_{t})$ . Applying Cauchy-Schwartz inequality, Assumption 2 and  $T \leq t_{K + 1} - 1$  leads us to:
$$
R _ {T} \leq R _ {\tilde {t} _ {K - 1}} + D G \left(t _ {K + 1} - t _ {K}\right). \tag {16}
$$
Let us now bound  $R_{\tilde{t}_{K - 1}}$ . As a consequence of claim ii) of Proposition 19 and after substituting  $\eta_t$  by its value and applying Lemma 25 provided in Appendix B to  $f(x) = 1 / (2\sqrt{x})$ ,  $a_0 = 0$  and  $a_k = \left\| \sum_{t\in \mathcal{T}_k}g_t\right\| _2^2$  for  $k\in [K - 1]$  we obtain:
$$
R _ {\bar {t} _ {K - 1}} \leq D \left(\frac {1}{2 \gamma} + \gamma\right) \sqrt {\sum_ {k = 1} ^ {K - 1} \left\| \sum_ {t \in \mathcal {T} _ {k}} g _ {t} \right\| _ {2} ^ {2}} \leq D G \left(\frac {1}{2 \gamma} + \gamma\right) \sqrt {\sum_ {k = 1} ^ {K - 1} (t _ {k + 1} - t _ {k}) ^ {2}}.
$$
Combining this with inequality (16) leads us to:
$$
\begin{array}{l} R _ {T} \leq D G \left(\left(\frac {1}{2 \gamma} + \gamma\right) \sqrt {\sum_ {k = 1} ^ {K - 1} \left(t _ {k + 1} - t _ {k}\right) ^ {2}} + \left(t _ {K + 1} - t _ {K}\right)\right) \\ \leq D G \left(\frac {1}{2 \gamma} + \gamma + 1\right) \sqrt {\sum_ {k = 1} ^ {K} \left(t _ {k + 1} - t _ {k}\right) ^ {2}}. \\ \end{array}
$$
To obtain the expected regret bound, we start by noticing that  $K \leq T$  then taking the expectation in this last inequality, applying Jensen's inequality, then, Proposition 22 that bounds  $\mathbb{E}\left[(t_{k + 1} - t_k)^2\right] \leq 2C_\mu /\mu^2$  we end up with the desired bound. Finally, to obtain the high probability regret bound, we notice again that  $K \leq T$  and then apply the high probability bound of Proposition 22.
A.4 Proof of Theorem 12
In the following lemma we claim that under the assumptions of Theorem 12, MaxCOSD with the appropriate learning rates has geometric cycles (see Assumption 21).
Lemma 24. Consider an inventory problem and let assumptions 2, 10 and 11 hold. Then, MaxCOSD with learning rates defined in Eq. (7) and  $\gamma \in (0,\rho /D]$  has  $\mu$ -geometric cycles with  $C_{\mu} = 1$ .
Proof. First, notice that if  $\min_{i\in [n]}d_{t,i}\geq \rho$  then  $x_{t + 1}\preceq \hat{y}_{t + 1}$ , i.e.  $t + 1$  is an update period for MaxCOSD. This is a consequence of Lemma 15, which applies since we have:
$$
\| \hat {y} _ {t + 1} - y _ {t} \| = \left\| \operatorname {P r o j} _ {\mathcal {Y}} \left(\hat {y} _ {t _ {k}} - \eta_ {t} \sum_ {s = t _ {k}} ^ {t} g _ {s}\right) - \hat {y} _ {t _ {k}} \right\| \leq \eta_ {t} \left\| \sum_ {s = t _ {k}} ^ {t} g _ {s} \right\| _ {2} \leq \gamma D \leq \rho \leq \min  _ {i \in [ n ]} d _ {t, i}.
$$
Now let  $k \in \mathbb{N}$  and  $m \in \mathbb{N}$ . Using this initial observation we have:
$$
\begin{array}{l} \mathbb {P} \left[ t _ {k + 1} - t _ {k} > m \right] = \mathbb {P} \left[ x _ {t _ {k} + 1} \not \leq \hat {y} _ {t _ {k} + 1}, \dots , x _ {t _ {k} + m} \not \leq \hat {y} _ {t _ {k} + m} \right] \\ \leq \mathbb {P} \left[ \min  _ {i \in [ n ]} d _ {t _ {k}, i} <   \rho , \dots , \min  _ {i \in [ n ]} d _ {t _ {k} + m - 1, i} <   \rho \right] \\ = \sum_ {s \geq 1} \mathbb {P} \left[ t _ {k} = s, \min  _ {i \in [ n ]} d _ {s, i} <   \rho , \dots , \min  _ {i \in [ n ]} d _ {s + m - 1, i} <   \rho \right] \\ \end{array}
$$
The last step of this proof is showing that in fact,
$$
\mathbb {P} \left[ t _ {k} = s, \min  _ {i \in [ n ]} d _ {s, i} <   \rho , \dots , \min  _ {i \in [ n ]} d _ {s + m - 1, i} <   \rho \right] \leq \mathbb {P} \left[ t _ {k} = s \right] (1 - \mu) ^ {m}. \tag {17}
$$
We prove this by a simple induction over  $m \geq 1$ . Noticing that  $\{t_k = s\} \in \sigma(g_1, x_2, \ldots, g_{s-1}, x_s) \subset \sigma(\ell_1, d_1, \ldots, \ell_{s-1}, d_{s-1})$ , where the last inclusion comes from Assumption 11, and using the basic properties of conditional expectations we derive the inequality for  $m = 1$ :
$$
\begin{array}{l} \mathbb {P} \left[ t _ {k} = s, \min  _ {i \in [ n ]} d _ {s, i} <   \rho \right] = \mathbb {E} \left[ \mathbb {P} \left[ t _ {k} = s, \min  _ {i \in [ n ]} d _ {s, i} <   \rho | \ell_ {1}, d _ {1}, \dots , \ell_ {s - 1}, d _ {s - 1} \right] \right] \\ = \mathbb {E} \left[ \mathbb {1} _ {\{t _ {k} = s \}} \mathbb {P} \left[ \min  _ {i \in [ n ]} d _ {s, i} <   \rho | \ell_ {1}, d _ {1}, \dots , \ell_ {s - 1}, d _ {s - 1} \right] \right] \\ \leq \mathbb {P} \left[ t _ {k} = s \right] (1 - \mu), \\ \end{array}
$$
where the last inequality comes from Assumption 10. Assume now the relation (17) holds for  $m$ , let us prove in a similar way that it holds for  $m + 1$ .
$$
\begin{array}{l} \mathbb {P} \left[ t _ {k} = s, \min  _ {i \in [ n ]} d _ {s, i} <   \rho , \dots , \min  _ {i \in [ n ]} d _ {s + m, i} <   \rho \right] \\ = \mathbb {E} \left[ \mathbb {P} \left[ t _ {k} = s, \min  _ {i \in [ n ]} d _ {s, i} <   \rho , \dots , \min  _ {i \in [ n ]} d _ {s + m, i} <   \rho | \ell_ {1}, d _ {1}, \dots , \ell_ {s + m - 1}, d _ {s + m - 1} \right] \right] \\ = \mathbb {E} \left[ \mathbb {1} _ {\{t _ {k} = s \}} \mathbb {1} _ {\{\underset {i \in [ n ]} {\min } d _ {s, i} <   \rho \}} \dots \mathbb {1} _ {\{\underset {i \in [ n ]} {\min } d _ {s + m - 1, i} <   \rho \}} \mathbb {P} \left[ \underset {i \in [ n ]} {\min } d _ {s + m, i} <   \rho | \ell_ {1}, d _ {1}, \ldots , \ell_ {s + m - 1}, d _ {s + m - 1} \right] \right] \\ \leq \mathbb {P} \left[ t _ {k} = s, \min  _ {i \in [ n ]} d _ {s, i} <   \rho , \dots , \min  _ {i \in [ n ]} d _ {s + m - 1, i} <   \rho \right] (1 - \mu) \\ \leq \mathbb {P} \left[ t _ {k} = s \right] (1 - \mu) ^ {m + 1}. \\ \end{array}
$$
Summing the relations (17) over  $s \geq 1$  leads to the final bound:  $\mathbb{P}[t_{k+1} - t_k > m] \leq (1 - \mu)^m$ , which is our claim.
We are now ready to prove Theorem 12.
Proof of Theorem 12. By definition, MaxCOSD is always feasible (independently of the learning rates chosen). Lemma 24 ensures that when  $\gamma \in (0,\rho /D]$ , MaxCOSD with adaptive learning rates as defined in Eq. (7) has  $\mu$ -geometric cycles with  $C_{\mu} = 1$ . Thus, Corollary 23 applies and leads to the regret bounds we claimed.
B Other lemmas
Lemma 25 (Lemma 4.13 of [21]). Let  $a_0, a_1, \ldots, a_K$  be non-negative numbers and  $f: \mathbb{R}_+ \to \mathbb{R}_+$  a measurable non-increasing function, we have:
$$
\sum_ {k = 1} ^ {K} a _ {k} f \left(a _ {0} + \sum_ {m = 1} ^ {k} a _ {m}\right) \leq \int_ {a _ {0}} ^ {\sum_ {k = 0} ^ {K} a _ {k}} f (x) d x.
$$
Proof. Define  $s_k = \sum_{m=0}^k a_m$ . The following holds for all  $k \in [K]$ ,
$$
a _ {k} f \left(a _ {0} + \sum_ {m = 1} ^ {k} a _ {m}\right) = a _ {k} f \left(s _ {k}\right) = \int_ {s _ {k - 1}} ^ {s _ {k}} f \left(s _ {k}\right) d x \leq \int_ {s _ {k - 1}} ^ {s _ {k}} f (x) d x.
$$
Summing over  $k = 1,\ldots ,K$  leads to the desired bound.
Lemma 26. Consider an inventory problem with non-negative feasible set  $\mathcal{Y} \subset \mathbb{R}_+^n$ . Then, any non-decreasing sequence of order-up-to levels  $(y_t)_{t \in \mathbb{N}} \subset \mathcal{Y}$  (i.e., such that  $y_t \preceq y_{t+1}$  for all  $t \in \mathbb{N}$ ) is feasible. In particular, constant strategies are feasible too.
Proof. Consider a non-decreasing sequence of order-up-to levels  $(y_{t})_{t\in \mathbb{N}}\subset \mathcal{V}\subset \mathbb{R}_{+}^{n}$ . First of all,  $y_{1}$  is feasible since we impose  $x_{1} = 0$  and  $y_{1}\in \mathbb{R}_{+}^{n}$ . Then, for every  $t\in \mathbb{N}$ , we can write:
$$
y _ {t + 1} \succeq y _ {t} \succeq [ y _ {t} - d _ {t} ] ^ {+} \succeq x _ {t + 1}.
$$
The first inequality comes from the assumption that  $(y_{t})_{t\in \mathbb{N}}$  is non-decreasing, the second one from the monotonicity of the positive part and the fact that  $y_{t}\in \mathbb{R}_{+}^{n}$ , finally, the last inequality is the inventory dynamical constraint. Thus, the whole strategy  $(y_{t})_{t\in \mathbb{N}}$  is feasible.
Lemma 27. Consider an inventory problem that satisfies Assumption 2, then, the regret of any algorithm is bounded as follows:
$$
R _ {T} \leq D G T.
$$
Proof. Let  $y \in \mathcal{V}$ . For all  $t \in \mathbb{N}$ , we have:
$$
\ell_ {t} (y _ {t}) - \ell_ {t} (y) \leq \langle g _ {t}, y _ {t} - y \rangle \leq \| g _ {t} \| _ {2} \| y _ {t} - y \| _ {2} \leq G D,
$$
where we used the definition of the subgradient  $g_{t} \in \partial \ell_{t}(y_{t})$ , then, Cauchy-Schwartz inequality and finally Assumption 2. Summing these inequalities over  $t = 1, \dots, T$  and taking the supremum over  $y \in \mathcal{V}$  leads to the desired bound.
Lemma 28. Consider the newsvendor cost function  $c$  defined in (5). Let  $d \in \mathbb{R}^n$ . A vector  $g \in \mathbb{R}^n$  is a subgradient of the function of  $c(\cdot, d)$  at  $y \in \mathbb{R}^n$  if and only if for all  $i \in [n]$  we have:
$$
g _ {i} \in \left\{ \begin{array}{l l} \{h _ {i} \}, & i f y _ {i} > d _ {i} \\ [ - p _ {i}, h _ {i} ], & i f y _ {i} = d _ {i} \\ \{- p _ {i} \}, & i f y _ {i} <   d _ {i}. \end{array} \right.
$$
In particular, denoting  $s = \min \{y, d\}$ , the vector  $(h_i \mathbb{1}_{\{y_i > s_i\}} - p_i \mathbb{1}_{\{y_i = s_i\}})_{i \in [n]}$  is a subgradient of  $c(\cdot, d)$  at  $y$ .
C Postponed proofs
C.1 Proof of Proposition 13
Proof of Proposition 13. Formally, in the lost sales single-product newsvendor setting with observable demand over  $\mathcal{V} = [0,D]$ , a feasible deterministic algorithm is defined by a sequence of functions  $(Y_{t})_{t\in \mathbb{N}}$  of the form  $Y_{t}:\mathbb{R}_{+}^{t - 1}\to [0,D]$  satisfying  $Y_{t + 1}(d_1,\ldots ,d_t)\geq [Y_t(d_1,\ldots ,d_{t - 1}) - d_t]^+$  for all  $d_{1},\ldots ,d_{t}\in \mathbb{R}_{+}$ .
Let  $\tilde{d} \in (0, D]$ , and consider a constant demand sequence defined by  $\tilde{d}_t \coloneqq \tilde{d}$  at every period  $t \in \mathbb{N}$ . Consider now  $(\tilde{y}_t)_{t \in \mathbb{N}}$  the sequence of order-up-to levels generated by this algorithm when facing this constant demand, that is,  $\tilde{y}_t \coloneqq Y_t(\tilde{d}, \ldots, \tilde{d})$ . We will now distinguish two cases.
i) Consider the case where  $\tilde{y}_t = 0$  for all  $t\in \mathbb{N}$ . Taking  $y = \tilde{d}$  in the regret definition (3) yields:
$$
R _ {T} \geq \sum_ {t = 1} ^ {T} c (0, \tilde {d}) - \sum_ {t = 1} ^ {T} c (\tilde {d}, \tilde {d}) = T p \tilde {d} = \Omega (T).
$$
ii) Now, consider the case where there exists  $T_0 \geq 1$  such that  $\tilde{y}_1 = \dots = \tilde{y}_{T_0 - 1} = 0$  and  $\tilde{y}_{T_0} > 0$ . Consider a new demand sequence  $(d_t)_{t \in \mathbb{N}}$  defined as follows:  $d_1 = \dots = d_{T_0 - 1} = \tilde{d}$  and  $d_t = 0$  for  $t \geq T_0$ . Denote by  $(y_t)_{t \in \mathbb{N}}$  the sequence of order-up-to levels generated by the algorithm against the demand sequence  $(d_t)_{t \in \mathbb{N}}$ , that is,  $y_t = Y_t(d_1, \ldots, d_{t-1})$ .
Since the algorithm is deterministic, we also have:  $y_{t} = 0$  for  $t \leq T_0 - 1$ . Indeed, we have,  $y_{t} = Y_{t}(d_{1},\ldots ,d_{t - 1}) = Y_{t}(\tilde{d},\ldots ,\tilde{d}) = \tilde{y}_{t} = 0$ . On the other hand, we have for all  $t \geq T_0 + 1$ ,  $y_{t} \geq [y_{t - 1} - d_{t - 1}]^{+} = y_{t - 1}$ , thus,  $y_{t} \geq y_{T_0} = \tilde{y}_{T_0} > 0$  for all  $t \geq T_0$ .
Taking  $y = 0$  in the regret definition (3) we obtain for  $T \geq T_0$ ,
$$
R _ {T} \geq \sum_ {t = 1} ^ {T} c (y _ {t}, d _ {t}) - c (0, d _ {t}) = \sum_ {t = T _ {0}} ^ {T} c (y _ {t}, 0) = \sum_ {t = T _ {0}} ^ {T} h y _ {t} \geq (T - T _ {0} + 1) h \tilde {y} _ {T _ {0}} = \Omega (T).
$$
C.2 Proof of Proposition 14
Proof of Proposition 14. Take  $(d_t)_{t\in \mathbb{N}}\subset \mathbb{R}_{++}$  such that  $\sum_{t = 1}^{\infty}d_t < y_1$  and define  $C = y_{1} - \sum_{t = 1}^{\infty}d_{t} > 0$ . Notice that due to the feasibility constraint (1) and the lost sales dynamic we have for all  $t\in \mathbb{N}$ ,  $y_{t + 1}\geq x_{t + 1} = [y_t - d_t]^+\geq y_t - d_t$ . Thus, we have for every  $t\in \mathbb{N}$ ,  $y_{t} = y_{1} + \sum_{s = 1}^{t - 1}(y_{s + 1} - y_{s})\geq y_{1} - \sum_{s = 1}^{t - 1}d_{s}\geq C$ . Now take the losses  $\ell_t(y) = y$ , then,  $R_{T} = \sum_{t = 1}^{T}y_{t}\geq CT$  for all  $T\in \mathbb{N}$ .
C.3 Proof of Lemma 15
Proof of Lemma 15. Since  $[y - d]^+ = \max \{y - d, 0\}$  and  $y' \succeq 0$ , it is enough to show that  $y' \succeq y - d$ . The latter holds since, for any  $i \in [n]$ , we have  $y_i - y_i' \leq \|y' - y\|_2 \leq \min_{i \in [n]} d_i \leq d_i$ .
C.4 Proof of Theorem 17
Proof of Theorem 17. The regret bound follows from the classical analysis of OSD, see our Corollary 20 or the proof of [8, Theorem 3.1]. Thus, we only need to show the feasibility. For all  $t \in \mathbb{N}$ , we have,
$$
\left\| y _ {t + 1} - y _ {t} \right\| _ {2} = \left\| \operatorname {P r o j} _ {\mathcal {Y}} \left(y _ {t} - \eta_ {t} g _ {t}\right) - y _ {t} \right\| _ {2} \leq \eta_ {t} \| g _ {t} \| _ {2} = \frac {\gamma D}{G \sqrt {t}} \| g _ {t} \| _ {2} \leq \frac {\rho}{\sqrt {t}} \leq \rho , \tag {18}
$$
the first inequality is provided by the property of non-expansiveness of the Euclidean projection, the second inequality comes from  $\gamma \leq \rho /D$  and  $\| g_t\| _2\leq G$ , and the last inequality from  $\sqrt{t}\geq 1$ . Combining this result with Assumption 16, we obtain  $\| y_{t + 1} - y_t\| _2\leq \min_{i\in [n]}d_{t,i}$ . According to Lemma 15 and the inventory dynamical constraint (2) this guarantees feasibility.
D.1 On the relation between Online Inventory Optimization and Online Convex Optimization
In the usual Online Convex Optimization (OCO) framework introduced by [36] a decision-maker and an environment interact as follows: at every time period  $t \in \mathbb{N}$ , first, the decision-maker chooses a decision  $y_{t} \in \mathcal{V}$  and the environment chooses a loss function  $\ell_t$ , then, the decision-maker receives some feedback which usually consists of either the loss function itself  $\ell_t$  (full-information setting), the loss incurred  $\ell_t(y_t)$  (bandit setting) or a subgradient  $g_{t} \in \partial \ell_{t}(y_{t})$  (first-order feedback setting). The goal of the decision-maker is to minimize its cumulative loss incurred  $\sum_{t = 1}^{T}\ell_{t}(y_{t})$ .
OIO extend the OCO framework by adding the feasibility constraint (1). A naive solution to accommodate such constraints into the OCO framework is by adding to the losses the convex indicator of the feasibility constraint, that is, by considering the losses  $\tilde{\ell}_t(y) = \ell_t(y) + \chi_t(y)$ , where  $\chi_t(y)$  takes the value 0 if  $y\succeq x_{t}$  holds and  $+\infty$  otherwise. However, this is not satisfactory since by
doing so we also alter the regret (3) by imposing on the competitor  $y \in \mathcal{V}$  the feasibility constraints associated to the algorithm.
Many extensions of the OCO framework have been developed over the years. Of particular interest, those which include constraints of different forms like: