SlowGuess/NeurIPS_2023_OCR huggingface dataset:
MaxCOSD is inspired by CUP [32], which handles the feasibility constraint through cyclical updates. This approach differs from AIM [9] or DDM [25], where the feasibility is enforced through projections onto the constraint. Among the differences between MaxCOSD and CUP is the fact that their cycle definition differ: in CUP stockouts trigger updates, whereas MaxCOSD relies directly on the feasibility condition, making its updates more frequent. Also, we use adaptive learning rates inspired by AdaGrad-Norm learning rates [27, Theorem 2] which allows us to be adaptive to the constant  $G$  and obtain high probability regret bounds which are not available for CUP. Finally, and most importantly, the assumptions required by CUP are restrictive: i.i.d. demands, single-product, perishable dynamic and a strong form of a demand non-degeneracy. On the other hand, MaxCOSD performs well under much milder assumptions, which we introduce next.
It may happen that  $\eta_t$  is undefined due to a denominator that is zero in Eq. (7), in such cases set  $\eta_t = 0$ .
Assumption 10 (Uniformly probably positive demand). There exists  $\mu \in (0,1]$  and  $\rho > 0$  such that, for all  $t \in \mathbb{N}$ , almost surely,
$$
\mathbb {P} [ \forall i \in [ n ], d _ {t, i} \geq \rho | \ell_ {1}, d _ {1}, \dots , \ell_ {t - 1}, d _ {t - 1} ] \geq \mu . \tag {8}
$$
In simple settings we recover through Assumption 10 conditions that already appeared in the literature:
- In single-product i.i.d. newsvendor inventory problems, our assumption is equivalent to the existence of  $\rho > 0$  such that  $\mathbb{P}[d_1 \geq \rho] > 0$ , that is,  $\mathbb{P}[d_1 > 0] > 0$ , or equivalently  $\mathbb{E}[d_1] > 0$ . This is exactly the non-degeneracy assumption required by [9] in AIM.
- In its multi-product extension, [25] assumes also pairwise independence across products. If we rather require mutual independence across products, then, Assumption 10 rewrites  $\mathbb{P}[d_{1,1} > 0] \cdots \mathbb{P}[d_{1,n} > 0] > 0$ , thus, our assumption reduces to  $\mathbb{E}[d_{1,i}] > 0$  for all  $i \in [n]$  which is also required by [25] in their algorithm DDM.
- We recover an assumption made for CUP [32] by requiring  $\rho = D$  where  $\mathcal{V} = [0, D]$ , and where  $D$  is also assumed to be large enough (see Remark 6).
- If the demand is deterministic, then  $\mu = 1$  and Eq. (8) becomes  $d_{t,i} \geq \rho$  for all  $i \in [n]$ .
- If the demand is discrete, i.e.  $d_{t,i} \in \{0,1,\ldots\}$  for all  $t \in \mathbb{N}, i \in [n]$ , then, we can take  $\rho = 1$  and rewrite Eq. (8) as follows:  $\mathbb{P}[\exists i \in [n], d_{t,i} = 0 | \ell_1, d_1, \dots, \ell_{t-1}, d_{t-1}] \leq 1 - \mu$ .
In addition to Assumption 10 we also introduce a mild technical condition.
Assumption 11 (Deterministic dynamic and subgradient selection). Dynamics and subgradient selections are deterministic, see Eq. (4) and Eq. (6).
We can now state our main result: under this new set of assumptions MaxCOSD achieves an optimal  $O(\sqrt{T})$  regret bound both in expectation and in high probability.
Theorem 12. Consider an inventory problem, and let assumptions 2, 10 and 11 hold. Then, MaxCOSD (see Algorithm 2) run with  $y_{1} \in \mathcal{V}$  and  $\gamma > 0$  is feasible. Furthermore, when  $\gamma \in (0, \rho / D]$ , it enjoys the following regret bounds for all  $T \in \mathbb{N}$ ,
$$
\mathbb {E} \left[ R _ {T} \right] \leq \frac {\sqrt {2} G D}{\mu} \left(\frac {1}{2 \gamma} + \gamma + 1\right) \sqrt {T},
$$
and for any confidence level  $\delta \in (0,1)$  we have with probability at least  $1 - \delta$
$$
R _ {T} \leq G D \left(\frac {1}{2 \gamma} + \gamma + 1\right) \left(1 + \frac {1}{\mu} \log \left(\frac {T}{\delta}\right)\right) \sqrt {T}.
$$
The obtained regret scales in  $O(\sqrt{T})$ , which is similar to other methods in the literature (see Section 3.2). As for the scaling in the constants  $\mu$  and  $\rho$ , our rate scales like  $O\left(\frac{1}{\rho\mu}\right)$  and we do not know whether this can be improved or not. As a matter of comparison, CUP enjoys the same scaling (see [32, Theorem 2 & Remark 3 & Assumption 1]).
5 Non-degenerate demands are needed for stateful inventory problems
Throughout this paper, we have seen instances of OIO which can be solved with sublinear regret rates: stateless OIO (equivalent to OCO), and some stateful OIO. It must be noted that, contrary to OCO, solving those stateful OIO problems required a non-degeneracy assumption on the demand (see Assumption 10 and Section 3.2). We argue here that such an assumption is necessary for solving stateful OIO. Note that this idea is not new, and was already observed in the conclusion of [3]: "To control for the impact of overordering, demands must be bounded away from zero, at least in expectation." Our contribution is to make this observation formal.
Proposition 13. Given any feasible deterministic<sup>4</sup> algorithm for the single-product lost sales newsvendor inventory problem with observable demand over  $\mathcal{Y} = [0, D]$ , there exists a sequence of demands such that the regret is linear, i.e.  $R_{T} = \Theta(T)$ .
4In general, a deterministic algorithm is defined by fixed in advance functions  $Y_{t}:(\mathbb{R}^{n}\times \mathbb{R}^{n})^{t - 1}\to \mathcal{V}$  and outputs  $y_{t} = Y_{t}(g_{1},x_{2},\ldots ,g_{t - 1},x_{t})$  for all  $t\in \mathbb{N}$ .
Proposition 13 shows that Assumption 2 is not sufficient to reach sublinear regret in general inventory problems. This is totally unusual from an OCO perspective, and is a specificity of stateful OIO. Furthermore, the above result shows that what prevents us from reaching sublinear rates is not the limited feedback (demands and losses are observable in this example) but rather zero demands. This is why it is necessary to make an assumption preventing demands to be too small, in some sense. Note that one may think of imposing positive demands to circumvent this difficulty, but this is not sufficient. Indeed, a sequence of demands converging too fast to zero can also be problematic.
Proposition 14. Given any feasible algorithm for the single-product lost sales problem with observable demand over  $\mathcal{Y} = [0, D]$  such that  $y_{1} \in (0, D]$ , there exists a constant sequence of losses and a sequence of positive demands such that the regret is linear, i.e.  $R_{T} = \Theta(T)$ .
Let us now investigate why degenerated demand becomes a problem when going from stateless to stateful OIO. The main difference between the two is that the feasibility constraint is always trivially satisfied for stateless OIO (see Remark 5). Instead, for stateful OIO, we can show that the higher is the demand, the easier it is for the feasibility constraint to be satisfied.
Lemma 15. Let  $y, y'$ ,  $d \in \mathbb{R}_+^n$ . If  $\|y' - y\|_2 \leq \min_{i \in [n]} d_i$ , then,  $y' \succeq [y - d]^+$ . In particular, given an inventory problem and a time period  $t \in \mathbb{N}$ , taking  $y_{t+1} \in \mathcal{V} \subset \mathbb{R}_+^n$  such that  $\|y_{t+1} - y_t\|_2 \leq \min_{i \in [n]} d_{i,t}$  ensures that  $y_{t+1}$  is feasible, in the sense that  $x_{t+1} \preceq y_{t+1}$ .
The above lemma shows that if  $y_{t + 1}$  is taken close enough from the previous  $y_{t}$ , then the algorithm is feasible. The key point here is that "close enough" is controlled by the demand, meaning that when the demand is closer to zero there are less feasible choices for the manager. In such a case, we understand that it may be impossible to achieve sublinear regret, because the set of feasible choices could be too reduced.
The distance between two consecutive decisions can easily be controlled in methods based on subgradient descents, through their learning rates. This is why Lemma 15 is very helpful in the design of efficient feasible algorithms. It has been employed in the proof of our main result regarding MaxCOSD (Theorem 12). In the following, we further illustrate its usefulness by showing that OSD (see Algorithm 1) with adequate learning rates is feasible when the demand is uniformly positive.
Assumption 16 (Uniformly positive demand). There exists  $\rho >0$  such that for all  $t\in \mathbb{N}$ $i\in [n]$
$$
d _ {t, i} \geq \rho . \tag {9}
$$
Theorem 17. Consider an inventory problem, and let assumptions 2 and 16 hold. Then, OSD (see Algorithm 1) run with  $y_{1} \in \mathcal{Y}$  and  $\eta_t = \gamma D / (G\sqrt{t})$  where  $\gamma \in (0, \rho / D]$ , is feasible and satisfies for all  $T \in \mathbb{N}$  that  $R_T \leq (1 + 2\gamma)(2\gamma)^{-1}GD\sqrt{T}$ .
6 Numerical results
The goal of the following numerical experiments is to show the versatility and performances of MaxCOSD in various settings. Let us consider the following problems.
- Setting 1. Single-product lost sales inventory problem with i.i.d. demands drawn according to Poisson(1).
- Setting 2. Single-product perishable inventory problem with a lifetime of 2 periods and i.i.d. demands drawn according to Poisson(1).
- Setting 3. Multi-product lost sales inventory problem with  $n = 100$  and capacity constraints. Demands are i.i.d. and drawn independently across products according to Poisson  $(\lambda_{i})$  where the intensities  $\lambda_{i}$  have been drawn independently according to Uniform[1, 2].
- Setting 4. Multi-product lost sales inventory problem with  $n = 3049$  and capacity constraints. Demands are taken from the real-world dataset of the M5 competition [17].
- Setting 5. Multi-product lost sales inventory problem with  $n = 3049$  and box constraints. As in Setting 4 we considered demands from the M5 competition dataset [17].
The code is available at https://github.com/Califrais/newsvendorTester.
Figure 1: Regret in settings 1 to 5 (from left to right) as a function of the learning rate parameter  $\gamma$ .
We use the newsvendor loss in all the settings. In all the settings the cost parameters satisfy  $p_i / h_i = 200$  since this ratio is known to exceed 200 in many applications [10]. In settings 1, 2 and 3,  $h_i = 1$  and in settings 4 and 5,  $h_i$  and  $p_i$  are proportional to the real average selling costs.
In settings 1, 2, 3 and 4 we compare MaxCOSD against the following baselines: AIM [9] for Setting 1, CUP [32] for Setting 2, and DDM [25] for settings 3 and 4. In Setting 5, we ran a parallelized version of MaxCOSD, that is, one instance of MaxCOSD per product, and a parallelized version of AIM. Notice that in Settings 4 and 5 demands are not i.i.d. and thus, do not fit the assumptions of the baselines considered. Notice also that Theorem 12 requires MaxCOSD's learning rate to be small enough  $(\gamma \leq \rho / D)$ , which we do not try to enforce here. All the algorithms have been initialized with  $y_{1} = 0$ . Settings 1, 2 and 3 have been run 10 times, with different demand realizations generated through independent samples.
Figure 1 shows, for every setting, the regret obtained after  $T$  periods as a function of the learning rate parameter  $\gamma \in [10^{-5}, 10^{1}]$ . We picked  $T = 1969$  for all the settings because it corresponds to the number of periods available in our real-world dataset [17]. We see that MaxCOSD performs well compared to baselines whenever the number of handled products remains low (Settings 1,2,3,5). Instead, we see that MaxCOSD is less efficient when the number of products becomes large, in particular in the Setting 4. The performance of MaxCOSD in the large  $n$  regime can be explained by the fact that the cycles become longer, as it becomes less likely that the feasibility condition is satisfied. Indeed, we have seen in Lemma 15 that the larger the overall demand is, the easier feasibility holds. But when  $n$  grows,  $\min_{i} d_{i,t}$  becomes smaller, making the problem harder.
In this paper, we address Online Inventory Optimization problems by introducing MaxCOSD, the first algorithm which can be applied to a wide variety of real-world problems. More precisely, MaxCOSD enjoys an optimal  $O(\sqrt{T})$  regret without assuming the demand to be i.i.d., and can handle a large class of dynamics, including perishability models. We achieved this result by applying ideas and methods from online learning to OIO problems.
Still, there is a lot of space for improvements and future developments. First, we observed that for problems with a large number of products and capacity constraints, the empirical performance of MaxCOSD could be improved. To do so, one would need to better handle the feasibility constraint, by using for instance projections onto the feasibitly constraint, an idea already used by DDM, but with no theoretical guarantees so far in real-world scenarios. Second, we obtained regret bounds under minimal structural assumptions on the problem (convex lipschitz losses), but we could expect to obtain better rates by making stronger assumptions on the problem. One such assumption, which is classical in the Online Convex Optimization literature, is to assume further that the losses are strongly convex or exp-concave, typically leading to a logarithmic  $O(\log(T))$  regret. Another assumption,
which is more specific to the Online Inventory literature, is to make some regularity assumption directly on the demand, also leading to a logarithmic pseudo-regret in the newsvendor case [3, Subsection 2.3]. Finally, even if our model is quite versatile, it has its limits, and more work is needed to improve it. For instance, we have seen in Section 2.2 that we do not accommodate for outdating costs as they appear in [32]. Also, we do not handle discrete feasible sets for which two promising techniques have been applied in the literature: expert algorithms [15] and probabilistic rounding [9, Subsection 3.4]. We could also hope to further weaken our non-degeneracy Assumption 10, by making an hypothesis which applies independently to each product, without having to assume pairwise independence across products. We believe that an adequate adaptation of online convex optimization techniques to the online inventory framework will prove to be a successful strategy for overcoming those challenges.
We are thankful to Adeline Germanian and the anonymous reviewers for helpful discussions and suggestions. Our work has been funded by the LOPF LabCom ANR-20-LCV1-0005 grant, provided by the French "Agence Nationale de la Recherche".
[1] Kenneth J Arrow, Theodore Harris, and Jacob Marschak. Optimal inventory policy. *Econometrica: Journal of the Econometric Society*, pages 250–272, 1951.
[2] Masoud Badiei, Na Li, and Adam Wierman. Online convex optimization with ramp constraints. In 2015 54th IEEE Conference on Decision and Control (CDC), pages 6730-6736. IEEE, 2015.
[3] Omar Besbes and Alp Muharremoglu. On implications of demand censoring in the newsvendor problem. Management Science, 59(6):1407-1424, 2013.
[4] Apostolos N Burnetas and Craig E Smith. Adaptive ordering and pricing for perishable products. Operations Research, 48(3):436-443, 2000.
[5] Xi Chen, Stefanus Jasin, and Cong Shi. The Elements of Joint Learning and Optimization in Operations Management, volume 18. Springer Nature, 2022.
[6] Gregory A Godfrey and Warren B Powell. An adaptive, distribution-free algorithm for the newsvendor problem with censored demands, with applications to inventory and distribution. Management Science, 47 (8):1101-1112, 2001.
[7] Ford W Harris. How many parts to make at once. 1913.
[8] Elad Hazan et al. Introduction to online convex optimization. Foundations and Trends® in Optimization, 2 (3-4):157-325, 2016.
[9] Woonghee Tim Huh and Paat Rusmevichientong. A nonparametric asymptotic analysis of inventory planning with censored demand. Mathematics of Operations Research, 34(1):103-123, 2009.
[10] Woonghee Tim Huh, Ganesh Janakiraman, John A Muckstadt, and Paat Rusmevichientong. An adaptive algorithm for finding the optimal base-stock policy in lost sales inventory systems with censored demand. Mathematics of Operations Research, 34(2):397-416, 2009.
[11] Rodolphe Jenatton, Jim Huang, and Cedric Archambeau. Adaptive algorithms for online convex optimization with long-term constraints. In International Conference on Machine Learning, pages 402-411. PMLR, 2016.
[12] Retsef Levi, Robin O Roundy, and David B Shmoys. Provably near-optimal sampling-based policies for stochastic inventory control models. Mathematics of Operations Research, 32(4):821-839, 2007.
[13] Tatsiana Levina, Yuri Levin, Jeff McGill, Mikhail Nediak, and Vladimir Vovk. Weak aggregating algorithm for the distribution-free perishable inventory problem. Operations Research Letters, 38(6):516-521, 2010.
[14] Nikolaos Liakopoulos, Apostolos Destounis, Georgios Paschos, Thrasyvoulos Spyropoulos, and Panayotis Mertikopoulos. Cautious regret minimization: Online optimization with long-term budget constraints. In International Conference on Machine Learning, pages 3944-3952. PMLR, 2019.
A Analysis of MaxCOSD
The goal of this appendix is to study the MaxCOSD algorithm (see Algorithm 2) and prove Theorem 12. To do so, we start by introducing a generalized version of MaxCOSD named Cyclic Online Subgradient Descent (COSD) which has general update periods and learning rates (see Algorithm 3). In Proposition 19 we provide a general analysis of COSD which shows that it is sufficient to control the cycles' length to derive  $O(\sqrt{T})$  regret bounds. As a byproduct of this proposition we also derive the classical analysis of OSD (see Corollary 20). A way of ensuring that the cycles' lengths are efficiently controlled is through a probabilistic property (see Assumption 21) which is similar to sub-exponential concentration [29, Section 2.7]. Finally, using Lemma 15 and Assumptions 2, 10 and 11 we show that when  $\gamma \in (0, \rho / D]$  the cycles' length of MaxCOSD satisfy Assumption 21 which allows us to conclude with Theorem 12.
A.1 Design of COSD
The Cyclic Online Subgradient Descent (COSD) generalizes MaxCOSD by allowing arbitrary learning rates and update periods. Recall that update periods are allowed to be dynamically defined, for this reason we will refer to a sequence of update periods  $(t_k)_{k\in \mathbb{N}}$  as an update strategy which is formally defined below.
Definition 18 (Update strategy). An update strategy  $(t_k)_{k\in \mathbb{N}}\subset \mathbb{N}$  is a sequence of random variables such that,  $t_1 = 1$ ,  $t_k < t_{k + 1}$  for every  $k\in \mathbb{N}$ , and for every  $t\in \mathbb{N}$ ,  $k\in \mathbb{N}$ , the event  $\{t_k = t\}$  is observable by the manager at the beginning of period  $t$ , i.e. it belongs to  $\sigma (g_1,x_2,\ldots ,g_{t - 1},x_t)$ .
The pseudo-code of COSD is given in Algorithm 3.
Algorithm 3: COSD
1 Parameters: learning rates  $(\eta_{t})_{t\in \mathbb{N}}$  , update strategy  $(t_k)_{k\in \mathbb{N}}$  and  $y_{1}\in \mathcal{V}$    
2 Initialization   
3 Set  $t_1 = 1,k = 1$  .   
4 for  $t = 1,2,\ldots$  do   
5 Output  $y_{t}$    
6 Observe  $g_{t}\in \partial \ell_{t}(y_{t})$  and  $x_{t + 1}$  .   
7 if  $t_{k + 1} = t + 1$  then   
8 Set  $y_{t + 1} = \mathrm{Proj}_{\mathcal{Y}}\left(y_{t_k} - \eta_t\sum_{s = t_k}^t g_s\right)$    
9 Set  $k = k + 1$  .   
10 else   
11 Set  $y_{t + 1} = y_{t}$
By choosing appropriately the update strategy we recover the following algorithms:
- OSD. If updates are made at every period, i.e.  $t_k = k$ , we recover OSD. Its feasibility is not guaranteed.
- Minibatch OSD. Given a fixed minibatch size  $\tau \in \mathbb{N}$ , the updates period are defined offline via  $t_k = 1 + (k - 1)\tau$ .
- CUP. For every  $k \in \mathbb{N}$ ,  $t_k$  is defined as the periods where the inventory is empty, that is,
$$
t _ {1} = 1, \quad t _ {k + 1} = \inf  \left\{t \geq t _ {k} + 1: x _ {t} \preceq \mathbf {0} \right\} \text {f o r} k \in \mathbb {N}. \tag {10}
$$
This update strategy corresponds to that used by CUP [32]. One easily sees that for such update periods  $t_k$ , feasibility always holds.
- MaxCOSD. To recover MaxCOSD we define dynamically the update strategy as the periods where the candidate order-up-to levels  $(\hat{y}_t)_{t\in \mathbb{N}}$  are feasible. Formally, it writes:
$$
t _ {1} = 1, \quad t _ {k + 1} = \inf  \left\{t \geq t _ {k} + 1: x _ {t} \preceq \hat {y} _ {t} \right\} \text {f o r} k \in \mathbb {N}, \tag {11}
$$
where  $(\hat{y}_t)_{t\in \mathbb{N}}$  is defined by:
$$
\hat {y} _ {1} = y _ {1}, \quad \hat {y} _ {t + 1} = \operatorname {P r o j} _ {\mathcal {Y}} \left(\hat {y} _ {t _ {k}} - \eta_ {t} \sum_ {s = t _ {k}} ^ {t} g _ {s}\right), \tag {12}
$$
with  $k = \max \{j \geq 1 : t_j \leq t\}$  or equivalently  $t \in \mathcal{T}_k = \{t_k, \dots, t_{k+1} - 1\}$ . Notice that at update periods the implemented order-up-to levels coincide with the candidate order-up-to level, that is, we have  $y_{t_k} = \hat{y}_{t_k}$  for all  $k \in \mathbb{N}$ .
A.2 Analysis of COSD and OSD
The following proposition summarizes the main properties of COSD. For convenience, we will use the notation  $\bar{t}_k \coloneqq t_{k+1} - 1$  for the last period of the  $k^{th}$  cycle  $\mathcal{T}_k$ .
Proposition 19. Let assumptions 2.i) and 2.ii) be satisfied. Given any update strategy and any sequence of learning rates  $(\eta_t)_{t \in \mathbb{N}}$  such that:  $0 < \eta_{\bar{t}_{k + 1}} \leq \eta_{\bar{t}_k}$  for all  $k \in \mathbb{N}$ , COSD (see Algorithm 3) has the following properties:
i) it is feasible if and only if for all  $k\in \{2,3,\ldots \}$ $y_{t_k}\succeq x_{t_k}$
ii) for all  $K \in \mathbb{N}$ , the regret at the end of the  $K^{th}$  cycle satisfies,
$$
R _ {\bar {t} _ {K}} \leq \frac {D ^ {2}}{2 \eta_ {\bar {t} _ {K}}} + \frac {1}{2} \sum_ {k = 1} ^ {K} \eta_ {\bar {t} _ {k}} \left\| \sum_ {t \in \mathcal {T} _ {k}} g _ {t} \right\| _ {2} ^ {2}. \tag {13}
$$
Proof. Let us prove claim i). By definition, the algorithm produces feasible sequence of order-up-to levels if  $y_{t} \succeq x_{t}$  for all  $t \in \mathbb{N}$ . But for all  $t \in \{t_k + 1, \dots, \bar{t}_k\}$  for some  $k \in \mathbb{N}$ , we have necessarily  $y_{t} = y_{t-1} \succeq [y_{t-1} - d_{t-1}]^{+} \succeq x_{t}$  where the last inequality comes from (2). Thus we only need to check feasibility at the update periods, i.e. check that  $y_{t_k} \succeq x_{t_k}$  for all  $k \in \mathbb{N}$ . Furthermore, it is clear that the latter is verified for  $k = 1$  since  $x_{t_1} = x_1 = 0 \preceq y_{t_1}$ .
Now to prove the regret bound (claim ii)) we follow the lines of the classical analysis of OSD (see e.g. the proof of [8, Theorem 3.1] or that of [21, Theorem 2.13]) when run against the sequence of losses  $(\sum_{t\in \mathcal{T}_k}\ell_t)_{k\in \mathbb{N}}$  instead of  $(\ell_t)_{t\in \mathbb{N}}$ . For convenience, we will write  $\tilde{g}_k = \sum_{t\in \mathcal{T}_k}g_t$  for all  $k\in \mathbb{N}$ .
Let  $y \in \mathcal{Y}, K \in \mathbb{N}$  and  $k \in [K]$ , we start by bounding  $\sum_{t \in \mathcal{T}_k} \ell_t(y_t) - \ell_t(y)$ . By definition of the subgradient  $g_t \in \partial \ell_t(y_t)$  we have:
$$
\sum_ {t \in \mathcal {T} _ {k}} \ell_ {t} \left(y _ {t}\right) - \ell_ {t} (y) = \sum_ {t \in \mathcal {T} _ {k}} \ell_ {t} \left(y _ {t _ {k}}\right) - \ell_ {t} (y) \leq \langle \tilde {g} _ {k}, y _ {t _ {k}} - y \rangle . \tag {14}
$$
We rewrite this bound as follows:
$$
\langle \tilde {g} _ {k}, y _ {t _ {k}} - y \rangle = \frac {1}{2 \eta_ {\bar {t} _ {k}}} \left(\| y _ {t _ {k}} - y \| _ {2} ^ {2} + \eta_ {t _ {k}} ^ {2} \| \tilde {g} _ {k} \| _ {2} ^ {2} - \| (y _ {t _ {k}} - y) - \eta_ {\bar {t} _ {k}} \tilde {g} _ {k} \| _ {2} ^ {2}\right).
$$
By using the property of non-expansiveness of the Euclidean projection we have:
$$
\left\| y _ {t _ {k + 1}} - y \right\| _ {2} ^ {2} = \left\| \operatorname {P r o j} _ {\mathcal {Y}} \left(y _ {t _ {k}} - \eta_ {\bar {t} _ {k}} \tilde {g} _ {k}\right) - \operatorname {P r o j} _ {\mathcal {Y}} (y) \right\| _ {2} ^ {2} \leq \| \left(y _ {t _ {k}} - \eta_ {\bar {t} _ {k}} \tilde {g} _ {k}\right) - y \| _ {2} ^ {2}.
$$
Combining the last two results leads to:
$$
\langle \tilde {g} _ {k}, y _ {t _ {k}} - y \rangle \leq \frac {1}{2 \eta_ {\bar {t} _ {k}}} \left(\| y _ {t _ {k}} - y \| _ {2} ^ {2} + \eta_ {\bar {t} _ {k}} ^ {2} \| \tilde {g} _ {k} \| _ {2} ^ {2} - \left\| y _ {t _ {k + 1}} - y \right\| _ {2} ^ {2}\right).
$$