SlowGuess/NeurIPS_2023_OCR huggingface dataset:
By Hölder's inequality,
$$
F _ {t} \left(x _ {t}\right) - F _ {t + 1} \left(x _ {t + 1}\right) + \left\langle v _ {t}, x _ {t} \right\rangle \leq \left\| v _ {t} - \hat {v} _ {t} \right\| _ {x _ {t}, *} \left\| x _ {t} - x _ {t + 1} \right\| _ {x _ {t}} - \frac {1}{\eta_ {t - 1} M ^ {2}} \omega \left(M \left\| x _ {t} - x _ {t + 1} \right\| _ {x _ {t}}\right). \tag {9}
$$
By the Fenchel-Young inequality,
$$
\omega \left(M \| x _ {t} - x _ {t + 1} \| _ {x _ {t}}\right) + \omega_ {*} \left(\eta_ {t - 1} M \| v _ {t} - \hat {v} _ {t} \| _ {x _ {t}, *}\right) \geq \eta_ {t - 1} M ^ {2} \| x _ {t + 1} - x _ {t} \| _ {x _ {t}} \| v _ {t} - \hat {v} _ {t} \| _ {x _ {t}, *}. \tag {10}
$$
Combining (9) and (10) yields
$$
F _ {t} (x _ {t}) - F _ {t + 1} (x _ {t + 1}) + \langle v _ {t}, x _ {t} \rangle \leq \frac {1}{\eta_ {t - 1} M ^ {2}} \omega_ {*} (\eta_ {t - 1} M \| v _ {t} - \hat {v} _ {t} \| _ {x _ {t}, *}).
$$
which proves the second inequality. The third inequality in the lemma then follows from the assumption that  $\eta_{t - 1}\| v_t - \hat{v}_t\|_{x_t,*}\leq 1 / (2M)$  and Proposition C.3.
E Proofs in Section 4 (Smoothness Characterizations)
E.1 Proof of Lemma 4.3
We write
$$
\| \nabla f (x) \| _ {x, *} ^ {2} = \frac {\sum_ {i = 1} ^ {d} a (i) ^ {2} x (i) ^ {2}}{\left(\sum_ {i = 1} ^ {d} a (i) x (i)\right) ^ {2}} \leq 1.
$$
E.2 Proof of Lemma 4.6
Define  $r\coloneqq \| x - y\| _y$  . Consider the following two cases.
(i) Suppose that  $r \geq 1/2$ . By Lemma 4.1,
$$
\left\| x \odot \nabla f (x) - y \odot \nabla f (y) \right\| _ {2} \leq \sqrt {2} \leq 4 \cdot \frac {1}{2} \leq 4 \| x - y \| _ {y}.
$$
(ii) Suppose that  $r < 1 / 2$ . We write
$$
r ^ {2} = \| x - y \| _ {y} ^ {2} = \sum_ {i = 1} ^ {d} \left(1 - \frac {x (i)}{y (i)}\right) ^ {2} \geq \left(1 - \frac {x (i)}{y (i)}\right) ^ {2}, \quad \forall i \in [ d ],
$$
showing that
$$
0 <   1 - r \leq \frac {x (i)}{y (i)} \leq 1 + r, \quad \forall i \in [ d ].
$$
Then [10, Lemma 1],
$$
\frac {\langle a , y \rangle}{\langle a , x \rangle} = \frac {\sum_ {i} a (i) y (i)}{\sum_ {i} a (i) x (i)} \leq \max _ {i \in [ d ]} \frac {a (i) y (i)}{a (i) x (i)} = \max _ {i \in [ d ]} \frac {y (i)}{x (i)} \leq \frac {1}{1 - r};
$$
similarly,
$$
\frac {\langle a , y \rangle}{\langle a , x \rangle} \geq \frac {1}{1 + r}.
$$
We obtain
$$
\frac {- 2 r}{1 - r} = 1 - \frac {1 + r}{1 - r} \leq 1 - \frac {x (i) \langle a , y \rangle}{y (i) \langle a , x \rangle} \leq 1 - \frac {1 - r}{1 + r} = \frac {2 r}{1 + r}.
$$
Since  $r < 1 / 2$
$$
\left(1 - \frac {x (i) \langle a , y \rangle}{y (i) \langle a , x \rangle}\right) ^ {2} \leq \max  \left\{\left(\frac {- 2 r}{1 - r}\right) ^ {2}, \left(\frac {2 r}{1 + r}\right) ^ {2} \right\} = \frac {4 r ^ {2}}{(1 - r) ^ {2}} <   1 6 r ^ {2}.
$$
$$
\begin{array}{l} \| x \odot \nabla f (x) - y \odot \nabla f (y) \| _ {2} ^ {2} = \sum_ {i = 1} ^ {d} \left(\frac {a (i) x (i)}{\langle a , x \rangle} - \frac {a (i) y (i)}{\langle a , y \rangle}\right) ^ {2} \\ = \sum_ {i = 1} ^ {d} \left(\frac {a (i) y (i)}{\langle a , y \rangle}\right) ^ {2} \left(1 - \frac {x (i) \langle a , y \rangle}{y (i) \langle a , x \rangle}\right) ^ {2} \\ <   1 6 r ^ {2} \sum_ {i = 1} ^ {d} \left(\frac {a (i) y (i)}{\langle a , y \rangle}\right) ^ {2} \\ <   1 6 r ^ {2}, \\ \end{array}
$$
showing that  $\| x\odot \nabla f(x) - y\odot \nabla f(y)\| _2 <   4\| x - y\| _y$
Combining the two cases, we obtain  $\| x\odot \nabla f(x) - y\odot \nabla f(y)\| _2\leq 4\| x - y\| _y$ . Since  $x$  and  $y$  are symmetric, we also have  $\| x\odot \nabla f(x) - y\odot \nabla f(y)\| _2\leq 4\| x - y\| _x$ . This completes the proof.
E.3 Proof of Lemma 4.7
We will use the notion of relative smoothness.
Definition E.1 (Bauschke et al. [3], Lu et al. [21]). A function  $f$  is said to be  $L$ -smooth relative to a function  $h$  if the function  $Lh - f$  is convex.
Lemma E.2 (Bauschke et al. [3, Lemma 7]). Let  $f(x) = -\log \langle a,x\rangle$  for some non-zero  $a\in \mathbb{R}_+^d$ . Then,  $f$  is 1-smooth relative to the logarithmic barrier  $h$ .
Fix  $x\in \mathrm{ri}\Delta$  . By Lemma E.2, the function  $h - f$  is convex and hence
$$
h (x) - f (x) + \langle \nabla h (x) - \nabla f (x), v \rangle \leq h (x + v) - f (x + v), \quad \forall v \in \operatorname {r i} \Delta - x,
$$
where  $\mathrm{ri}\Delta - x \coloneqq \{u - x | u \in \mathrm{ri}\Delta\}$ . By Lemma 4.4, Theorem C.2, and Proposition C.3, if  $\| v \|_x \leq 1/2$ ,
$$
h (x + v) - h (x) - \langle \nabla h (x), v \rangle \leq \omega_ {*} (\| v \| _ {x}) \leq \| v \| _ {x} ^ {2}.
$$
Combining the two inequalities above yields
$$
\langle - \nabla f (x), v \rangle - \| v \| _ {x} ^ {2} \leq f (x) - f (x + v).
$$
Since  $f(x + v) \geq \min_{y \in \Delta} f(y) = 0$  (Assumption 1) and  $\langle e, v \rangle = 0$ , we write
$$
\langle - \nabla f (x) - \alpha e, v \rangle - \| v \| _ {x} ^ {2} \leq f (x), \quad \forall \alpha \in \mathbb {R}, v \in \operatorname {r i} \Delta - x \text {s u c h t h a t} \| v \| _ {x} \leq 1 / 2.
$$
The left-hand side is continuous in  $v$ , so the condition that  $v \in \operatorname{ri} \Delta - x$  can be relaxed to  $v \in \Delta - x := \{u - x | u \in \Delta\}$ . We get
$$
\langle - \nabla f (x) - \alpha e, v \rangle - \| v \| _ {x} ^ {2} \leq f (x), \quad \forall \alpha \in \mathbb {R}, v \in \Delta - x \text {s u c h t h a t} \| v \| _ {x} \leq 1 / 2. \tag {11}
$$
We show that choosing
$$
v = - c \nabla^ {- 2} h (x) (\nabla f (x) + \alpha_ {x} (\nabla f (x)) e), \tag {12}
$$
for any  $c\in (0,1 / 2]$  ensures that  $v\in \Delta -x$  and  $\| v\|_{x}\leq 1 / 2$
- First, we check whether  $\| v\| _x\leq 1 / 2$  . We write
$$
\begin{array}{l} \left\| v \right\| _ {x} = c \big \| \nabla^ {- 2} h (x) (\nabla f (x) + \alpha_ {x} (\nabla f (x)) e) \big \| _ {x} \\ = c \| \nabla f (x) + \alpha_ {x} (\nabla f (x)) e \| _ {x, *} \\ \leq c \| \nabla f (x) \| _ {x, *} \\ \leq c \\ \leq 1 / 2, \\ \end{array}
$$
where the last line follows from Remark 4.8 and Lemma 4.3.
- Then, we check whether  $v \in \Delta - x$  or, equivalently, whether  $v + x \in \Delta$ . By the definition of  $\alpha_{x}(\nabla f(x))$  (4), each entry of  $v$  is given by
$$
v _ {i} = - c x (i) ^ {2} \left(\nabla_ {i} f (x) - \frac {\sum_ {j} x (j) ^ {2} \nabla_ {j} f (x)}{\sum_ {j} x (j) ^ {2}}\right), \quad \forall i \in [ d ],
$$
where  $\nabla_j f(x)$  is the  $j$ -th entry of  $\nabla f(x)$ . Obviously,
$$
\sum_ {i = 1} ^ {d} v (i) = \sum_ {i = 1} ^ {d} - c x (i) ^ {2} \left(\nabla_ {i} f (x) - \frac {\sum_ {j} x (j) ^ {2} \nabla_ {j} f (x)}{\sum_ {j} x (j) ^ {2}}\right) = 0,
$$
so  $\sum_{i}x(i) + v(i) = 1$ . It remains to check whether  $x(i) + v(i)\geq 0$  for all  $i\in [d]$ . Notice that  $\nabla f(x)$  is entrywise negative. Then,
$$
\begin{array}{l} x (i) + v (i) = x (i) - c x (i) ^ {2} \left(\nabla_ {i} f (x) - \frac {\sum_ {j} x (j) ^ {2} \nabla_ {j} f (x)}{\sum_ {j} x (j) ^ {2}}\right) \\ \geq x (i) + c x (i) ^ {2} \frac {\sum_ {j} x (j) ^ {2} \nabla_ {j} f (x)}{\sum_ {j} x (j) ^ {2}}. \\ \end{array}
$$
To show that the lower bound is non-negative, we write
$$
\begin{array}{l} - x (i) ^ {2} \frac {\sum_ {j} x (j) ^ {2} \nabla_ {j} f (x)}{\sum_ {j} x (j) ^ {2}} = \frac {x (i) ^ {2} \sum_ {j} a (j) x (j) ^ {2}}{\sum_ {j} x (j) ^ {2} \langle a , x \rangle} \\ \leq \frac {x (i) ^ {2}}{\sum_ {j} x (j) ^ {2}} \max _ {j \in [ d ]} x (j) \\ = x (i) \cdot \max  _ {j} \frac {x (i) x (j)}{\sum_ {k} x (k) ^ {2}}. \\ \end{array}
$$
If  $i = j$ , then  $\max_j \frac{x(i)x(j)}{\sum_k x(k)^2} \leq 1$ ; otherwise,
$$
\max  _ {j \neq i} \frac {x (i) x (j)}{\sum_ {k} x (k) ^ {2}} \leq \max  _ {j \neq i} \frac {x (i) x (j)}{x (i) ^ {2} + x (j) ^ {2}} \leq \frac {1}{2}.
$$
$$
x (i) + v (i) \geq (1 - c) x (i) \geq \frac {1}{2} x (i) \geq 0, \quad \forall i \in [ d ].
$$
Plug the chosen  $v$  (12) into the inequality (11). We write
$$
\begin{array}{l} f (x) \geq \sup  _ {0 \leq c \leq 1 / 2} c \| \nabla f (x) + \alpha_ {x} (\nabla f (x)) e \| _ {x, *} ^ {2} - c ^ {2} \| \nabla f (x) + \alpha_ {x} (\nabla f (x)) e \| _ {x, *} ^ {2} \\ = \sup  _ {0 \leq c \leq 1 / 2} (c - c ^ {2}) \| \nabla f (x) + \alpha_ {x} (\nabla f (x)) e \| _ {x, *} ^ {2} \\ = \frac {1}{4} \| \nabla f (x) + \alpha_ {x} (\nabla f (x)) e \| _ {x, *} ^ {2}. \\ \end{array}
$$
Proof in Section 5 (LB-FTRL with Multiplicative-Gradient Optimism)
F.1 Proof of Theorem 5.1
We first check existence of a solution. For convenience, we omit the subscripts in  $p_{t+1}$  and  $\eta_t$  and write  $g$  for  $g_{1:t}$ .
Proposition F.1. Assume that  $\eta p\in [-1,0]^d$  . Define
$$
\lambda^ {\star} \in \underset {\lambda \in \operatorname {d o m} \psi} {\arg \min } \psi (\lambda), \quad \psi (\lambda) := \lambda - \sum_ {i = 1} ^ {d} (1 - \eta p (i)) \log (\lambda + \eta g (i)). \tag {13}
$$
Then,  $\lambda^{\star}$  exists and is unique. The pair  $(x_{t + 1},\hat{g}_{t + 1})$ , defined by
$$
x _ {t + 1} (i) := \frac {1 - \eta p (i)}{\lambda^ {\star} + \eta g (i)}, \quad \hat {g} _ {t + 1} (i) := p (i) \cdot \frac {\lambda^ {\star} + \eta g (i)}{1 - \eta p (i)}, \tag {14}
$$
solves the system of nonlinear equations (5).
We will use the following observation [27, Theorem 5.1.1].
Lemma F.2. The function  $\psi$  is 1-self-concordant.
Proof of Proposition F.1. By Lemma F.2 and the fact that  $\operatorname{dom} \psi$  does not contain any line, the minimizer  $\lambda^{\star}$  exists and is unique [27, Theorem 5.1.13]; since  $\eta p \in [-1,0]^d$  and  $\lambda \in \operatorname{dom} \psi$ , we have  $x_{t+1}(i) \geq 0$ ; moreover,
$$
\psi^ {\prime} (\lambda^ {\star}) = 1 - \sum_ {i = 1} ^ {d} \frac {1 - \eta p (i)}{\lambda^ {\star} + \eta g (i)} = 0.
$$
$$
\sum_ {i = 1} ^ {d} x _ {t + 1} (i) = 1 - \psi^ {\prime} (\lambda^ {\star}) = 1,
$$
showing that  $x_{t + 1}\in \Delta$
Then, we check whether  $(x_{t + 1},\hat{g}_{t + 1})$  satisfies the two equalities (5). It is obvious that  $x_{t + 1}\odot \hat{g}_{t + 1} = p$ . By the method of Lagrange multiplier, if there exists  $\lambda \in \mathbb{R}$  such that
$$
\eta g (i) + \eta \hat {g} _ {t + 1} (i) - \frac {1}{x _ {t + 1} (i)} + \lambda = 0, \quad \sum_ {i = 1} ^ {d} x _ {t + 1} (i) = 1, \quad x _ {t + 1} (i) \geq 0,
$$
then  $x_{t + 1}\in \arg \min_{x\in \Delta}\langle g,x\rangle +\langle \hat{g}_{t + 1},x\rangle +\eta^{-1}h(x)$ . It is easily checked that the above conditions are satisfied with  $\lambda = \lambda^{\star}$  and (14). The proposition follows.
We now analyze the time complexity. Nesterov [26, Section 7] [27, Appendix A.2] proved that when  $p = 0$ , Newton's method for solving (13) reaches the region of quadratic convergence of the intermediate Newton's method [27, Section 5.2.1] after  $O(\log d)$  iterations. A generalization for possibly non-zero  $p$  is provided in Proposition F.3 below. The proof essentially follows the approach of Nesterov [27, Appendix A.2]. It is worth noting that while the function  $\psi$  is self-concordant, directly applying existing results on self-concordant minimization by Newton's method [27, Section 5.2.1] does not yield the desired  $O(\log d)$  iteration complexity bound.
Starting with an initial iterate  $\lambda_0\in \mathbb{R}$ , Newton's method for the optimization problem (13) iterates as
$$
\lambda_ {t + 1} = \lambda_ {t} - \psi^ {\prime} (\lambda_ {t}) / \psi^ {\prime \prime} (\lambda_ {t}), \quad \forall t \in \{0 \} \cup \mathbb {N}. \tag {15}
$$
Proposition F.3. Assume that  $\eta p\in [-1,0]^d$ . Define  $i^{\star}\in \arg \min_{i\in [d]}(-g(i))$ . Then, Newton's method with the initial iterate
$$
\lambda_ {0} = 1 - \eta g (i ^ {\star})
$$
enters the region of quadratic convergence of the intermediate Newton's method [27, Section 5.2.1] after  $O(\log d)$  iterations.
Firstly, we will establish Lemma F.4 and Lemma F.5. These provide characterizations of  $\psi'$  and  $\lambda_t$  that are necessary for the proof of Proposition F.3.
Lemma F.4. The following hold.
(i)  $\psi^\prime$  is a strictly increasing and strictly concave function.
(ii)  $\psi^{\prime}(\lambda^{\star}) = 0$
(iii)  $\lambda_0\leq \lambda^{\star}$
Proof of Lemma F.4. The first item follows by a direct calculation:
$$
\psi^ {\prime \prime} (\lambda) = \sum_ {i = 1} ^ {d} \frac {1 - \eta p (i)}{(\lambda + \eta g (i)) ^ {2}} > 0, \quad \psi^ {\prime \prime \prime} (\lambda) = - \sum_ {i = 1} ^ {d} \frac {2 (1 - \eta p (i))}{(\lambda + \eta g (i)) ^ {3}} <   0.
$$
The second item has been verified in the proof of Proposition F.1. As for the third item, we write
$$
\psi^ {\prime} (\lambda_ {0}) = 1 - \sum_ {i = 1} ^ {d} \frac {1 - \eta p (i)}{\lambda_ {0} + \eta g (i)} \leq 1 - \frac {1 - \eta p (i ^ {\star})}{\lambda_ {0} + \eta g (i ^ {\star})} = \eta p (i ^ {\star}) \leq 0 = \psi^ {\prime} (\lambda^ {\star}).
$$
The third item then follows from  $\psi'(\lambda_0) \leq \psi'(\lambda^{\star})$  and the first item.
Lemma F.5. Let  $\{\lambda_t\}$  be the sequence of iterates of Newton's method (15). For any  $t \geq 0$ ,  $\psi'(\lambda_t) \leq 0$  and  $\lambda_t \leq \lambda^\star$ .
Proof of Lemma F.5. Recall that  $\mathrm{dom}\psi = (-g(i^{\star}),\infty)$ . We proceed by induction. Obviously,  $\lambda_0\in \mathrm{dom}\psi$ . By Lemma F.4, the lemma holds for  $t = 0$ . Assume that the lemma holds for some non-negative integer  $t$ . By the iteration rule of Newton's method (15) and the assumption that  $\psi^{\prime}(\lambda_t)\leq 0$
$$
\lambda_ {t + 1} = \lambda_ {t} - \frac {\psi^ {\prime} (\lambda_ {t})}{\psi^ {\prime \prime} (\lambda_ {t})} \geq \lambda_ {t},
$$
which ensures that  $\lambda_{t + 1}\in \mathrm{dom}\psi$ . By Lemma F.4 (i) and the iteration rule of Newton's method (15), we have
$$
\psi^ {\prime} (\lambda_ {t + 1}) \leq \psi^ {\prime} (\lambda_ {t}) + \psi^ {\prime \prime} (\lambda_ {t}) (\lambda_ {t + 1} - \lambda_ {t}) = \psi^ {\prime} (\lambda_ {t}) - \psi^ {\prime \prime} (\lambda_ {t}) \frac {\psi^ {\prime} (\lambda_ {t})}{\psi^ {\prime \prime} (\lambda_ {t})} = 0.
$$
By Lemma F.4 (i) and Lemma F.4 (ii),  $\lambda_{t + 1} \leq \lambda^{\star}$ . The lemma follows.
Proof of Proposition F.3. Define the Newton decrement as
$$
\delta (\lambda) := \frac {\left| \psi^ {\prime} \left(\lambda_ {t}\right) \right|}{\left| \psi^ {\prime} (\lambda) / \sqrt {\psi^ {\prime \prime} (\lambda)} \right|}.
$$
By Lemma F.2, the region of quadratic convergence is [27, Theorem 5.2.2]
$$
\mathcal {Q} := \{\lambda \in \operatorname {d o m} \psi : \delta (\lambda) <   1 / 2 \}.
$$
Define  $T \coloneqq \lceil (9 + 7\log_2d) / 4\rceil$ . By Lemma F.5,  $\psi^{\prime}(\lambda_t)\leq 0$  for all  $t\geq 0$ . If  $\psi^{\prime}(\lambda_{t}) = 0$  for some  $0\leq t\leq T$ , then Newton's method reaches  $\mathcal{Q}$  within  $T$  steps and the proposition follows. Now, let us assume that  $\psi^{\prime}(\lambda_t) < 0$  for all  $0\leq t\leq T$ . By the concavity of  $\psi^\prime$  (Lemma F.4 (i)),
$$
\psi^ {\prime} (\lambda_ {t - 1}) \leq \psi^ {\prime} (\lambda_ {t}) + \psi^ {\prime \prime} (\lambda_ {t}) (\lambda_ {t - 1} - \lambda_ {t}) = \psi^ {\prime} (\lambda_ {t}) + \frac {\psi^ {\prime \prime} (\lambda_ {t})}{\psi^ {\prime \prime} (\lambda_ {t - 1})} \psi^ {\prime} (\lambda_ {t - 1}).
$$
Divide both sides by  $\psi'(\lambda_{t-1})$ . By Lemma F.5 and the AM-GM inequality,
$$
1 \geq \frac {\psi^ {\prime} (\lambda_ {t})}{\psi^ {\prime} (\lambda_ {t - 1})} + \frac {\psi^ {\prime \prime} (\lambda_ {t})}{\psi^ {\prime \prime} (\lambda_ {t - 1})} \geq 2 \sqrt {\frac {\psi^ {\prime} (\lambda_ {t}) \psi^ {\prime \prime} (\lambda_ {t})}{\psi^ {\prime} (\lambda_ {t - 1}) \psi^ {\prime \prime} (\lambda_ {t - 1})}}.
$$
Then, for all  $1 \leq t \leq T + 1$ ,
$$
| \psi^ {\prime} (\lambda_ {t}) \psi^ {\prime \prime} (\lambda_ {t}) | \leq \frac {1}{4} | \psi^ {\prime} (\lambda_ {t - 1}) \psi^ {\prime \prime} (\lambda_ {t - 1}) |.
$$
This implies that for all  $0 \leq t \leq T + 1$ ,
$$
\left| \psi^ {\prime} \left(\lambda_ {t}\right) \psi^ {\prime \prime} \left(\lambda_ {t}\right) \right| \leq \frac {1}{4 ^ {t}} \left| \psi^ {\prime} \left(\lambda_ {0}\right) \psi^ {\prime \prime} \left(\lambda_ {0}\right) \right|,
$$
which further implies that
$$
\delta (\lambda_ {t}) ^ {2} = \frac {(\psi^ {\prime} (\lambda_ {t}) \psi^ {\prime \prime} (\lambda_ {t})) ^ {2}}{(\psi^ {\prime \prime} (\lambda_ {t})) ^ {3}} \leq \frac {1}{1 6 ^ {t}} \cdot \frac {(\psi^ {\prime} (\lambda_ {0}) \psi^ {\prime \prime} (\lambda_ {0})) ^ {2}}{(\psi^ {\prime \prime} (\lambda_ {t})) ^ {3}}.
$$
It remains to estimate  $\psi'(\lambda_0)$ ,  $\psi''(\lambda_0)$ , and  $\psi''(\lambda_t)$ . Given  $\eta p \in [-1, 0]^d$  and  $\lambda_0 + \eta g(i) \geq 1$ , through direct calculation,
$$
| \psi^ {\prime} (\lambda_ {0}) | = \sum_ {i = 1} ^ {d} \frac {1 - \eta p (i)}{\lambda_ {0} + \eta g (i)} - 1 \leq \sum_ {i = 1} ^ {d} (1 - \eta p (i)) - 1 <   2 d,
$$
$$
\psi^ {\prime \prime} (\lambda_ {0}) = \sum_ {i = 1} ^ {d} \frac {1 - \eta p (i)}{(\lambda_ {0} + \eta g (i)) ^ {2}} \leq \sum_ {i = 1} ^ {d} \frac {2}{1} = 2 d.
$$
$$
\psi^ {\prime} (\lambda_ {t}) = 1 - \sum_ {i = 1} ^ {d} \frac {1 - \eta p (i)}{\lambda_ {t} + \eta g (i)} \leq 0,
$$
by the Cauchy-Schwarz inequality,
$$
\psi^ {\prime \prime} (\lambda_ {t}) = \sum_ {i = 1} ^ {d} \frac {1 - \eta p (i)}{(\lambda_ {t} + \eta g (i)) ^ {2}} \geq \frac {\left(\sum_ {i = 1} ^ {d} \frac {1 - \eta p (i)}{\lambda_ {t} + \eta g (i)}\right) ^ {2}}{\sum_ {i = 1} ^ {d} (1 - \eta p (i))} \geq \frac {1}{\sum_ {i = 1} ^ {d} (1 - \eta p (i))} \geq \frac {1}{2 d}.
$$
Therefore, for  $0 \leq t \leq T + 1$ ,
$$
\delta \left(\lambda_ {t}\right) ^ {2} <   \frac {(2 d) ^ {7}}{1 6 ^ {t}}.
$$
In particular,  $\delta (\lambda_{T + 1}) < 1 / 2$ , i.e.,  $\lambda_{T + 1}\in \mathcal{Q}$ . This concludes the proof.
Proposition F.3 suggests that after  $O(\log d)$  iterations of Newton's method, we can switch to the intermediate Newton's method which exhibits quadratic convergence [27, Theorem 5.2.2]. Each iteration of both Newton's method and the intermediate Newton's method takes  $O(d)$  time. Therefore, it takes  $\tilde{O}(d\log d)$  time to approximate  $\lambda^{\star}$ . Given  $\lambda^{\star}$ , both  $x_{t+1}$  and  $\hat{g}_{t+1}$  can be computed in  $O(d)$  time, according to their respective definitions in Equation (14). Therefore, the overall time complexity of computing  $x_{t+1}$  and  $\hat{g}_{t+1}$  is  $\tilde{O}(d)$ .
F.2 Proof of Corollary 5.2
By the definition of local norms (3), we write
$$
\left\| g _ {t} - \hat {g} _ {t} + u _ {t} \right\| _ {x _ {t}, *} = \left\| \left(g _ {t} + u _ {t}\right) \odot x _ {t} - p _ {t} \odot x _ {t} \odot x _ {t} \right\| _ {2} = \left\| \left(g _ {t} + u _ {t}\right) \odot x _ {t} - p _ {t} \right\| _ {2}.
$$
By Lemma 4.4 and Theorem 3.2, if  $\eta_{t - 1}\| (g_t + u_t)\odot x_t - p_t\| _2\leq 1 / 2$  , then Algorithm 2 satisfies
$$
R _ {T} (x) \leq \frac {h (x)}{\eta_ {T}} + \sum_ {t = 1} ^ {T} \eta_ {t - 1} \| \left(g _ {t} + u _ {t}\right) \odot x _ {t} - p _ {t} \| _ {2} ^ {2}.
$$
For any  $x\in \Delta$  , define  $x^{\prime} = (1 - 1 / T)x + e / (dT)$  . Then,
$$
h \left(x ^ {\prime}\right) \leq - d \log d - \sum_ {i = 1} ^ {d} \log \left(\frac {1}{d T}\right) = d \log T, \quad \forall x \in \Delta
$$
and hence [22, Lemma 10]
$$
R _ {T} (x) \leq R _ {T} (x ^ {\prime}) + 2 \leq \frac {d \log T}{\eta_ {T}} + \sum_ {t = 1} ^ {T} \eta_ {t - 1} \| (g _ {t} + u _ {t}) \odot x _ {t} - p _ {t} \| _ {2} ^ {2} + 2.
$$
G Proof of Theorem 6.1 (Gradual-Variation Bound)
G.1 Proof of Theorem 6.1
We will use the following lemma, whose proof is postponed to the end of the section.
Lemma G.1. Let  $\{x_{t}\}$  be the iterates of Algorithm 2. If for all  $t\geq 1$
$$
\eta_ {t} \leq \frac {1}{6}, \quad \eta_ {t - 1} \| g _ {t} - \hat {g} _ {t} \| _ {x _ {t}, *} \leq \frac {1}{6}, \quad \left(1 - \frac {1}{6 \sqrt {d}}\right) \eta_ {t - 1} \leq \eta_ {t} \leq \eta_ {t - 1}, \quad a n d \quad \| \hat {g} _ {t} \| _ {x _ {t}, *} \leq 1, \tag {16}
$$
then  $\| x_{t + 1} - x_t\|_{x_t}\leq 1$
We start with checking the conditions in Lemma G.1.
It is obvious that  $\eta_t \leq 1 / (16\sqrt{2}) \leq 1 / 6$ .
- By the definition of the dual local norm (3), the definition of  $\hat{g}_t$ , and Lemma 4.3,
$$
\left\| \hat {g} _ {t} \right\| _ {x _ {t}, *} = \left\| \hat {g} _ {t} \odot x _ {t} \right\| _ {2} = \left\| g _ {t - 1} \odot x _ {t - 1} \right\| _ {2} = \left\| g _ {t - 1} \right\| _ {x _ {t - 1}, *} \leq 1, \quad \forall t \geq 2.
$$
- Then, by the triangle inequality and Lemma 4.3,
$$
\eta_ {t - 1} \| g _ {t} - \hat {g} _ {t} \| _ {x _ {t}, *} \leq \frac {1}{1 6 \sqrt {2}} \left(\| g _ {t} \| _ {x _ {t}, *} + \| \hat {g} _ {t} \| _ {x _ {t}, *}\right) \leq \frac {1}{8 \sqrt {2}} \leq \frac {1}{6}.
$$
- It is obvious that the sequence  $\{\eta_t\}$  is non-increasing. By the triangle inequality, Lemma 4.3 and the fact that  $\sqrt{a + b} \leq \sqrt{a} + \sqrt{b}$  for non-negative numbers  $a$  and  $b$ , we write
$$
\begin{array}{l} \frac {\eta_ {t}}{\eta_ {t - 1}} = \frac {\sqrt {5 1 2 d + 2 + V _ {t - 1}}}{\sqrt {5 1 2 d + 2 + V _ {t}}} \\ \geq \frac {\sqrt {5 1 2 d + 2 + V _ {t - 1}}}{\sqrt {5 1 2 d + 2 + V _ {t - 1} + 2}} \\ \geq \frac {\sqrt {5 1 2 d + 2 + V _ {t - 1}}}{\sqrt {5 1 2 d + 2 + V _ {t - 1}} + \sqrt {2}} \\ = 1 - \frac {\sqrt {2}}{\sqrt {5 1 2 d + 2 + V _ {t - 1}} + \sqrt {2}} \\ \geq 1 - \frac {1}{1 6 \sqrt {d}} \\ \geq 1 - \frac {1}{6 \sqrt {d}}. \\ \end{array}
$$
Then, Lemma G.1 implies that
$$
r _ {t} := \left\| x _ {t + 1} - x _ {t} \right\| _ {x _ {t}} \leq 1.
$$
By Proposition C.3,
$$
\omega \big (\left\| x _ {t} - x _ {t + 1} \right\| _ {x _ {t}} \big) \geq \frac {r _ {t} ^ {2}}{4}.
$$
By Corollary 5.2,
$$
R _ {T} \leq \frac {d \log T}{\eta_ {T}} + \sum_ {t = 1} ^ {T} \left(\langle g _ {t} - p _ {t} \oslash x _ {t}, x _ {t} - x _ {t + 1} \rangle - \frac {1}{4 \eta_ {t - 1}} r _ {t} ^ {2}\right) + 2.
$$
For  $t \geq 2$ , by Hölder's inequality and the definition of the dual local norm (3),
$$
\begin{array}{l} \left\langle g _ {t} - p _ {t} \otimes x _ {t}, x _ {t} - x _ {t + 1} \right\rangle \leq \left\| g _ {t} - p _ {t} \otimes x _ {t} \right\| _ {x _ {t}, *} r _ {t} \\ = \left\| x _ {t} \odot g _ {t} - p _ {t} \right\| _ {2} r _ {t} \\ = \left\| x _ {t} \odot g _ {t} - x _ {t - 1} \odot g _ {t - 1} \right\| _ {2} r _ {t}. \\ \end{array}
$$
$$
\begin{array}{l} \left\| x _ {t} \odot g _ {t} - x _ {t - 1} \odot g _ {t - 1} \right\| _ {2} r _ {t} \\ \leq \left\| x _ {t} \odot g _ {t} - x _ {t - 1} \odot \nabla f _ {t} \left(x _ {t - 1}\right) \right\| _ {2} r _ {t} + \left\| x _ {t - 1} \odot \nabla f _ {t} \left(x _ {t - 1}\right) - x _ {t - 1} \odot g _ {t - 1} \right\| _ {2} r _ {t}. \\ \end{array}
$$
We bound the two terms separately. By the AM-GM inequality and Lemma 4.6, the first term is bounded by
$$
\begin{array}{l} \left\| x _ {t} \odot g _ {t} - x _ {t - 1} \odot \nabla f _ {t} \left(x _ {t - 1}\right) \right\| _ {2} r _ {t} \leq 4 \eta_ {t - 1} \left\| x _ {t} \odot g _ {t} - x _ {t - 1} \odot \nabla f _ {t} \left(x _ {t - 1}\right) \right\| _ {2} ^ {2} + \frac {1}{1 6 \eta_ {t - 1}} r _ {t} ^ {2} \\ \leq 6 4 \eta_ {t - 1} r _ {t - 1} ^ {2} + \frac {1}{1 6 \eta_ {t - 1}} r _ {t} ^ {2}. \\ \end{array}
$$
By the AM-GM inequality, the second term is bounded by
$$
\begin{array}{l} \left\| x _ {t - 1} \odot \nabla f _ {t} \left(x _ {t - 1}\right) - x _ {t - 1} \odot g _ {t - 1} \right\| _ {2} r _ {t} = \left\| \nabla f _ {t} \left(x _ {t - 1}\right) - g _ {t - 1} \right\| _ {x _ {t - 1}, *} r _ {t} \\ \leq 4 \eta_ {t - 1} \| \nabla f _ {t} (x _ {t - 1}) - g _ {t - 1} \| _ {x _ {t - 1}, *} ^ {2} + \frac {1}{1 6 \eta_ {t - 1}} r _ {t} ^ {2}. \\ \end{array}
$$
Combine all inequalities above. We obtain
$$
\begin{array}{l} \left\langle g _ {t} - p _ {t} \otimes x _ {t}, x _ {t} - x _ {t + 1} \right\rangle - \frac {1}{4 \eta_ {t - 1}} r _ {t} ^ {2} \tag {17} \\ \leq 4 \eta_ {t - 1} \| \nabla f _ {t} (x _ {t - 1}) - g _ {t - 1} \| _ {x _ {t - 1}, *} ^ {2} + 6 4 \eta_ {t - 1} r _ {t - 1} ^ {2} - \frac {1}{8 \eta_ {t - 1}} r _ {t} ^ {2} \\ \end{array}
$$
For  $t = 1$ , by a similar argument,
$$
\left\langle g _ {1}, x _ {1} - x _ {2} \right\rangle - \frac {1}{4 \eta_ {0}} r _ {1} ^ {2} \leq r _ {1} - \frac {1}{4 \eta_ {0}} r _ {1} ^ {2} \leq \frac {1}{2} \left(4 \eta_ {0} + \frac {1}{4 \eta_ {0}} r _ {1} ^ {2}\right) - \frac {1}{4 \eta_ {0}} r _ {1} ^ {2} = 2 \eta_ {0} - \frac {1}{8 \eta_ {0}} r _ {1} ^ {2}. \tag {18}
$$