SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Now write  $\pmb{x}^{\star} = (\pmb{x}_1^{\star},\dots,\pmb{x}_p^{\star})$ . In addition to the previous equation, for  $l_{i + 1}\in S,\ldots ,l_{r - 1}\in$ $\mathcal{S}^{l_{i + 1},\dots,l_{r - 2}}$  and any  $l_{r}\in \mathcal{T}^{l_{i + 1},\dots,l_{r - 1}}$ , one has  $(g_r^{l_{i + 1},\dots,l_r})^\top x_r^\star +1\geq \epsilon$ , because  $\pmb{x}^{\star}$  is within the cube  $\mathcal{C}_d$  and at least at distance  $\epsilon$  from the constraints of the cube. Similarly as when  $l_{r}\in \mathcal{S}^{l_{i + 1},\dots,l_{r - 1}}$ , for any  $l_{r}\in \mathcal{T}^{l_{i + 1},\dots,l_{r - 1}}$  we denote by  $c^{l_{i + 1},\dots,l_r}$  the volumetric center of the polyhedron  $\mathcal{P}_{l_r}^{(r)}$  along the corresponding computation path, if  $l_{r}$  corresponded to an added constraints when  $c^{l_{i + 1},\dots,l_r}\notin \mathcal{C}_{k_r}$ . Otherwise, if  $l_{r}$  corresponded to the constraint  $\pmb {a} = \pm \pmb {e}_j$  of the initial cube, we pose  $\pmb{c}^{l_{i + 1},\dots,l_r} = -\pmb{a}$ . Now by construction, in both cases one has  $(g_r^{l_{i + 1},\dots,l_r})^\top c^{l_{i + 1},\dots,l_r}\leq -1$  (1.7 of Algorithm 1). Thus,
$$
\left(\boldsymbol {g} _ {r} ^ {l _ {i + 1}, \dots , l _ {r}}\right) ^ {\top} \left(\boldsymbol {x} _ {r} ^ {\star} - \boldsymbol {c} ^ {l _ {i + 1}, \dots , l _ {r}}\right) \geq \epsilon . \tag {6}
$$
Recalling Eq (4), we then sum all equations of the form Eq (5) and Eq (6) along the computation path, to obtain
$$
\begin{array}{l} (A):= \sum_{\substack{l_{i + 1}\in \mathcal{S},\ldots ,\\ l_{p}\in \mathcal{S}^{l_{i + 1},\ldots ,l_{p - 1}}}}\lambda^{l_{i + 1}}\dots \lambda^{l_{i + 1},\ldots ,l_{p}} \\ \cdot O _ {S} \left(\boldsymbol {x} _ {1}, \dots , \boldsymbol {x} _ {i}, \boldsymbol {c} ^ {l _ {i + 1}}, \dots , \boldsymbol {c} ^ {l _ {i + 1}, \dots , l _ {p}}\right) ^ {\top} \left(\boldsymbol {x} ^ {\star} - \left(\boldsymbol {x} _ {1}, \dots , \boldsymbol {x} _ {i}, \boldsymbol {c} ^ {l _ {i + 1}}, \dots , \boldsymbol {c} ^ {l _ {i + 1}, \dots , l _ {p}}\right)\right) \\ +\sum_{i + 1\leq r\leq p}\sum_{\substack{l_{i + 1}\in \mathcal{S},\ldots ,l_{r - 1}\in \mathcal{S}^{l_{i + 1},\ldots ,l_{r - 2}},\\ l_{r}\in \mathcal{T}^{l_{i + 1},\ldots ,l_{r - 1}}}}\lambda^{l_{i + 1}}\dots \lambda^{l_{i + 1},\ldots ,l_{r}}\cdot (\boldsymbol{g}_{r}^{l_{i + 1},\ldots ,l_{r}})^{\top}(\boldsymbol{x}_{r}^{\star} - \boldsymbol{c}^{l_{i + 1},\ldots ,l_{r}})\geq \epsilon . \\ \end{array}
$$
Now using the convention
$$
G (r, r, \boldsymbol {x} _ {1}, \dots , \boldsymbol {x} _ {i}, \boldsymbol {c} ^ {l _ {i + 1}}, \dots , \boldsymbol {c} ^ {l _ {i + 1}, \dots , l _ {r}}) := \boldsymbol {g} _ {r} ^ {l _ {i + 1}, \dots , l _ {r}}, \qquad l _ {r} \in \mathcal {T} ^ {l _ {i + 1}, \dots , l _ {r - 1}},
$$
for any  $l_{i + 1}\in \mathcal{S},\ldots ,l_{r - 1}\in \mathcal{S}^{l_{i + 1},\dots ,l_{r - 2}}$  , we can write
$$
\begin{array}{l} (A) = \sum_{r\leq i}G(i,r,\boldsymbol{x}_{1},\ldots ,\boldsymbol{x}_{i})^{\top}(\boldsymbol{x}_{r}^{\star} - \boldsymbol{x}_{r}) + \sum_{i + 1\leq r\leq p}\sum_{\substack{l_{i + 1}\in \mathcal{S},\ldots ,\\ l_{r - 1}\in \mathcal{S}^{l_{i + 1},\dots,l_{r - 2}}}}\lambda^{l_{i + 1}}\ldots \lambda^{l_{i + 1},\ldots ,l_{r - 1}} \\ \times \sum_ {l _ {r} \in (\mathcal {S} \cup \mathcal {T}) ^ {l _ {i + 1}, \dots , l _ {r - 1}}} \lambda^ {l _ {i + 1}, \dots , l _ {r}} G (r, r, \boldsymbol {x} _ {1}, \dots , \boldsymbol {x} _ {i}, \boldsymbol {c} ^ {l _ {i + 1}}, \dots , \boldsymbol {c} ^ {l _ {i + 1}, \dots , l _ {r}}) ^ {\top} (\boldsymbol {x} _ {r} ^ {\star} - \boldsymbol {c} ^ {l _ {i + 1}, \dots , l _ {r}}). \\ \end{array}
$$
We next relate the terms  $G$  to the output of ApproxOracle. For simplicity, let us write  $G = (\mathcal{P}_{l_1}^{(1)},\ldots ,\mathcal{P}_{l_i}^{(i)})$ , which by abuse of notation was assimilated to  $(x_{1},\dots,x_{i})$ . Recall that by construction and hypothesis, all points where the oracle was queried belong to  $\mathcal{C}_d$ , so that for instance
$\| \pmb{x}_r^\star - \pmb{c}^{l_{i+1},\dots,l_r}\| \leq 2\sqrt{k_r} \leq 2\sqrt{d}$  for any  $l_r \in S^{l_{i+1},\dots,l_{r-1}}$ . Using the above equations together with Eq (2) and Lemma A.2 gives
$$
\begin{array}{l} \epsilon \leq \sum_{r\leq i}\left[\mathsf{ApproxOracle}_{\delta ,\xi ,\mathcal{O}_{f}}\left(i,r,\boldsymbol {G}\right)^{\top}(\boldsymbol{x}_{r}^{\star} - \boldsymbol{x}_{r}) + \frac{8d^{3 / 2}}{\sigma_{min}}\xi \right] + \sum_{i + 1\leq r\leq p}\sum_{\substack{l_{i + 1}\in \mathcal{S},\ldots ,\\ l_{r - 1}\in \mathcal{S}^{l_{i + 1},\dots,l_{r - 2}}}} \\ \lambda^ {l _ {i + 1}} \dots \lambda^ {l _ {i + 1}, \dots , l _ {r - 1}} \sum_ {l _ {r} \in (\mathcal {S} \cup \mathcal {T}) ^ {l _ {i + 1}}, \dots , l _ {r - 1}} \lambda^ {l _ {i + 1}, \dots , l _ {r}} \left[ \left(\boldsymbol {g} _ {r} ^ {l _ {i + 1}, \dots , l _ {r}}\right) ^ {\top} \left(\boldsymbol {x} _ {r} ^ {\star} - \boldsymbol {c} ^ {l _ {i + 1}, \dots , l _ {r}}\right) + \frac {8 d ^ {3 / 2}}{\sigma_ {\min}} \xi \right] \\ \leq \frac {8 p d ^ {3 / 2}}{\sigma_ {\min }} \xi + (p - i) \delta + \sum_ {r \leq i} \text {A p p r o x O r a c l e} _ {\delta , \xi , \mathcal {O} _ {f}} (i, r, \boldsymbol {G}) ^ {\top} (\boldsymbol {x} _ {r} ^ {\star} - \boldsymbol {x} _ {r}) \\ \end{array}
$$
where in the second inequality, we used Eq (3). Using  $p \leq d$ , this ends the proof of the lemma.
We are now ready to show that Algorithm 4 is a valid algorithm for convex optimization.
Theorem A.1. Let  $\epsilon \in (0,1)$  and  $O_S: \mathcal{C}_d \to \mathbb{R}^d$  be a separation oracle such that the successful set contains a ball of radius  $\epsilon$ . Pose  $\delta = \frac{\epsilon}{4d}$  and  $\xi = \frac{\sigma_{min}\epsilon}{32d^{5/2}}$ . Next, let  $p \geq 1$  and  $k_1, \ldots, k_p \leq \left\lceil \frac{d}{p} \right\rceil$  such that  $k_1 + \ldots + k_p = d$ . With these parameters, Algorithm 4 finds a successful vector with  $(C\frac{d}{p}\ln \frac{d}{\epsilon})^p$  queries and using memory  $\mathcal{O}\left(\frac{d^2}{p}\ln \frac{d}{\epsilon}\right)$ , for some universal constant  $C > 0$ .
Proof. Suppose by contradiction that Algorithm 4 never queried a successful point. Then, with the chosen parameters, Lemma A.3 shows that, for any vector  $\pmb{x}^{\star} = (\pmb{x}_1^{\star},\dots,\pmb{x}_p^{\star})$  such that  $B_{d}(\pmb{x}^{\star},\epsilon)$  is within the successful set, with the same notations, one has
$$
\sum_ {r \leq i} \mathsf {A p p r o x O r a c l e} _ {\delta , \xi , O _ {S}} (i, r, \mathcal {P} _ {l _ {1}} ^ {(1)}, \ldots , \mathcal {P} _ {l _ {i}} ^ {(i)}) ^ {\top} (\pmb {x} _ {r} ^ {\star} - \pmb {x} _ {r}) \geq \epsilon - \frac {8 d ^ {5 / 2}}{\sigma_ {m i n}} \xi - d \delta \geq \frac {\epsilon}{2}.
$$
Now denote by  $(a_{t},b_{t})$  the constraints that were added at any time during the run of Algorithm 1 when using the oracle ApproxOracle with  $i = j = 1$ . The previous equation shows that for all such constraints,
$$
\boldsymbol {a} _ {t} ^ {\top} \boldsymbol {x} _ {1} ^ {\star} - b _ {t} \geq \boldsymbol {a} _ {t} ^ {\top} (\boldsymbol {x} _ {1} ^ {\star} - \omega_ {t}) - \xi \geq \frac {\epsilon}{2} - \xi ,
$$
where  $\omega_{t}$  is the volumetric center of the polyhedron at time  $t$  during Vaidya's method Algorithm 1. Now, since the algorithm terminated, by Lemma 4.1, we have that
$$
\min  _ {t} \left(\boldsymbol {a} _ {t} ^ {\top} \boldsymbol {x} _ {1} ^ {\star} - b _ {t}\right) \leq \delta .
$$
This is absurd since  $\delta +\xi <  \frac{\epsilon}{2}$  . This ends the proof that Algorithm 4 finds a successful vector.
We now estimate its oracle-complexity and memory usage. First, recall that a run of ApproxOracle of level  $i$  makes  $\mathcal{O}(k_{i+1} \ln \frac{1}{\delta})$  calls to level-  $(i+1)$  runs of ApproxOracle. As a result, the oracle-complexity  $Q_d(\epsilon; k_1, \ldots, k_p)$  satisfies
$$
Q _ {d} (\epsilon ; k _ {1}, \ldots , k _ {p}) = \left(C k _ {1} \ln \frac {1}{\delta}\right) \times \ldots \times \left(C k _ {p} \ln \frac {1}{\delta}\right) \leq \left(C ^ {\prime} \frac {d}{p} \log \frac {d}{\epsilon}\right) ^ {p}
$$
for some universal constants  $C, C' \geq 2$ .
We now turn to the memory of the algorithm. For each level  $i \in [p]$  of runs for ApproxOracle, we keep memory placements for
1. the value  $j^{(i)}$  of the corresponding call to ApproxOracle(i,  $j^{(i)}, \cdot$ ) (for 1.6-7 of Algorithm 3):  $\mathcal{O}(\ln d)$  bits,
2. the iteration number  $t^{(i)}$  during the run of Algorithm 1 or within Algorithm 2:  $\mathcal{O}(\ln (k_i\ln \frac{1}{\delta}))$  bits
3. the polyhedron constraints contained in the state of  $\mathcal{P}^{(i)}$ :  $\mathcal{O}(k_i\times k_i\ln \frac{1}{\xi})$  bits,
Table 1: Memory structure for Algorithm 4
i	1	...	p
j	j(1)	j(p)
Iteration index	t(1)	t(p)
Polyhedron	P(1) = (k1, a1, b1k2, a2, b2...km, am, bm)	P(p)
Computed dual variables	(k*(1), λ*(1)) = (k1*, λ1* k2*, λ2*) ...	(k*(p), λ*(p))
Working separation vector	u(1)	u(p)
4. potentially, already computed dual variables  $\lambda^{\star}$  and their corresponding vector of constraint indices  $k^{\star}$  (1.3 of Algorithm 2):  $\mathcal{O}(k_i\times \ln \frac{1}{\xi})$  bits,
5. the working vector  $\pmb{u}^{(i)}$  (updated 1.8 of Algorithm 2):  $\mathcal{O}(k_i\ln \frac{1}{\xi})$  bits.
The memory structure is summarized in Table 1.
We can then check that this memory is sufficient to run Algorithm 4. An important point is that for any run of ApproxOracle(i, j, ·), in Algorithm 2, after running Vaidya's method Algorithm 1 and storing the dual variables  $\lambda^{\star}$  and corresponding indices  $k^{\star}$  within their placements ( $k^{\star(i)}$ ,  $\lambda^{\star(i)}$ ) (1.1-3 of Algorithm 2), the iteration index  $t^{(i)}$  and polyhedron  $\mathcal{P}^{(i)}$  memory placements are reset and can be used again for the second run of Vaidya's method (1.4-10 of Algorithm 2). During this second run, the vector  $\pmb{u}$  is stored in its corresponding memory placement  $\pmb{u}^{(i)}$  and updated along the algorithm. Once this run is finished, the output of ApproxOracle(i, j, ·) is readily available in the placement  $\pmb{u}^{(i)}$ . For  $i = p$ , the algorithm does not need to wait for the output of a level- $(i + 1)$  computation and can directly use the  $j^{(p)}$ -th component of the returned separation vector from the oracle  $O_S$ . As a result, the number of bits of memory used throughout the algorithm is at most
$$
M = \sum_ {i = 1} ^ {p} \mathcal {O} \left(k _ {i} ^ {2} \ln \frac {1}{\xi}\right) = \mathcal {O} \left(\frac {d ^ {2}}{p} \ln \frac {d}{\epsilon}\right).
$$
This ends the proof of the theorem.
We can already give the useful range for  $p$  for our algorithms, which will also apply to the case with computational-memory constraints Appendix B.
Proof of Corollary 3.1. Suppose  $\epsilon \geq \frac{1}{d^d}$ . Then, for some  $p_{max} = \Theta\left(\frac{C\ln\frac{1}{\epsilon}}{2\ln d}\right) \leq d$ , the algorithm from Theorem 3.2 yields a  $\mathcal{O}\left(\frac{1}{\epsilon^2}\right)$  oracle-complexity. On the other hand, if  $\epsilon \leq \frac{1}{d^d}$ , we can take  $p_{max} = d$ , which gives an oracle-complexity  $\mathcal{O}\left((C\ln \frac{1}{\epsilon})^d\right)$ .
B Memory-constrained feasibility problem with computations
In the last section we gave the main ideas that allow reducing the storage memory. However, Algorithm 4 does not account for memory constraints in computations as per Definition 2.2. For instance, computing the volumetric center VolumetricCenter  $(\mathcal{P})$  already requires infinite memory for infinite precision. More importantly, even if one discretizes the queries, the necessary precision and computational power may be prohibitive with the classical Vaidya's method Algorithm 1. Even finding a feasible point in the polyhedron (let alone the volumetric center) using only the constraints is itself computationally intensive. There has been significant work to make Vaidya's method computationally tractable [48, 1, 2]. These works address the issue of computational tractability, but the memory issue
is still present. Indeed, the precision depends among other parameters on the condition number of the matrix  $\pmb{H}$  in order to compute the leverage scores  $\sigma_{i}$  for  $i \in [m]$ , which may not be well-conditioned. Second, to avoid memory overflow, we also need to ensure that the points queried have bounded norm, which is again not a priori guaranteed in the original version Algorithm 1.
To solve these issues and also give a computationally-efficient algorithm, the cutting-plane subroutine Algorithm 1 needs to be modified. In particular, the volumetric barrier needs to include regularization terms. Fortunately, these have already been studied in [27]. In a major breakthrough, this paper gave a cutting-plane algorithm with  $\mathcal{O}(d^3\ln^{\mathcal{O}(1)}\frac{d}{\epsilon})$  runtime complexity, improving over the seminal work from Vaidya and subsequent works which had  $\mathcal{O}(d^{1 + \omega}\ln^{\mathcal{O}(1)}\frac{d}{\epsilon})$  runtime complexity, where  $\mathcal{O}(d^{\omega})$  is the computational complexity of matrix multiplication. To achieve this result, they introduce various regularizing terms together with the logarithmic barrier. While the main motivation of [27] was computational complexity, as a side effect, these regularization terms also ensure that computations can be carried with efficient memory. We then use their method as a subroutine.
For the sake of exposition and conciseness, we describe a simplified version of their method, that is also deterministic. This comes at the expense of a suboptimal running time  $\mathcal{O}(d^{1 + \omega}\ln^{\mathcal{O}(1)}\frac{1}{\epsilon})$ . We recall that our main concern is in memory usage rather than achieving the optimal runtime. The main technicality of this section is to show that their simplified method is numerically stable, and we emphasize that the original algorithm could also be shown to be numerically stable with similar techniques, leading to a time improvement from  $\tilde{\mathcal{O}} (d^{1 + \omega})$  to  $\tilde{\mathcal{O}} (d^3)$ . The memory usage, however, would not be improved.
B.1 A memory-efficient Vaidya's method for computations, via [27]
Fix a polyhedron  $\mathcal{P} = \{\pmb{x} : \pmb{A}\pmb{x} \geq \pmb{b}\}$ . Using the same notations as for Vaidya's method in Section 4.1, we define the new leverage scores  $\psi(\pmb{x})_i = (\pmb{A}_x(\pmb{A}_x^\top\pmb{A}_x + \lambda\pmb{I})^{-1}\pmb{A}_x^\top)_{i,i}$  and  $\Psi(\pmb{x}) = \text{diag}(\psi(\pmb{x}))$ . Let  $\mu(\pmb{x}) = \min_i \psi(\pmb{x})_i$ . Last, let  $Q(\pmb{x}) = A_x^\top(c_e\pmb{I} + \Psi(x))\pmb{A}_x + \lambda\pmb{I}$ , where  $c_e > 0$  is a constant parameter to be defined. In [27], they consider minimizing the volumetric-analytic hybrid barrier function
$$
p (\pmb {x}) = - c _ {e} \sum_ {i = 1} ^ {m} \ln s _ {i} (\pmb {x}) + \frac {1}{2} \ln \det (\pmb {A} _ {x} ^ {\top} \pmb {A} _ {x} + \lambda \pmb {I}) + \frac {\lambda}{2} \| \pmb {x} \| _ {2} ^ {2}.
$$
We can check [27] that
$$
\nabla p (\boldsymbol {x}) = - \boldsymbol {A} _ {x} ^ {\top} \left(c _ {e} \cdot \mathbf {1} + \boldsymbol {\psi} (x)\right) + \lambda \boldsymbol {x},
$$
where  $\mathbf{1}$  is the vector of ones. The following procedure gives a way to minimize this function efficiently given a good starting point.
Input: Initial point  $\pmb{x}^{(0)}\in \mathcal{P} = \{\pmb {x}:\pmb {A}\pmb {x}\geq \pmb {b}\}$
Input: Number of iterations  $r > 0$
Given:  $\| \nabla p(\pmb{x}^{(0)})\|_{\pmb{Q}(\pmb{x}^{(0)})^{-1}}\leq \frac{1}{100}\sqrt{c_e + \mu(x^{(0)})} := \eta.$
1 for  $k = 1$  to  $r$  do
2 if  $\| \nabla p(\pmb{x}^{(k - 1)})\|_{\pmb{Q}(\pmb{x}^{(0)})^{-1}}\leq 2(1 - \frac{1}{64})^r\eta$  then Break; 3  $\pmb{x}^{(k)} = \pmb{x}^{(k - 1)} - \frac{1}{8}\pmb {Q}(\pmb{x}^{(0)})^{-1}\nabla p(\pmb{x}^{(k - 1)})$
Output:  $\pmb{x}^{(k)}$
$$
\text {A l g o r m} 5: \boldsymbol {x} ^ {(r)} = \operatorname {C e n t e r i n g} \left(\boldsymbol {x} ^ {(0)}, r\right)
$$
We then present their simplified cutting-plane method.
In both Algorithm 5 and Algorithm 6, notice that the updates require to compute in particular the leverage scores  $\psi(\boldsymbol{x})$ , which can be computed in  $\mathcal{O}(d^{\omega})$  time using their formula. To achieve the  $\mathcal{O}(d^{3}\ln^{\mathcal{O}(1)}\frac{1}{\epsilon})$  computational complexity, an amortized computational cost  $\mathcal{O}(d^{2})$  is needed. The algorithm from [27] achieves this through various careful techniques aiming to update estimates of these leverage scores. The above cutting-plane algorithm is exactly that of [27] when these estimates are always exact (i.e. recomputed at each iteration), which yields the  $d^{\omega - 2}$  overhead time complexity. In particular, the original proof of convergence and correctness of [27] directly applies to this simplified algorithm.
Input:  $\epsilon, \delta > 0$  and a separation oracle  $O: \mathcal{C}_d \to \mathbb{R}^d$  Check: Throughout the algorithm, if  $s_i(\pmb{x}^{(t)}) < 2\epsilon$  for some  $i$  then return  $(\mathcal{P}_t, \pmb{x}^{(t)})$  Initialize  $\pmb{x}^{(0)} = \mathbf{0}$  and  $\mathcal{P}_0 := \{(-1, e_i, -1), (-1, -e_i, -1), i \in [d]\}$  for  $t \geq 0$  do  
if  $\min_{i \in [m]} \psi(\pmb{x}^{(t)})_i \leq c_d$  then  
 $\mathcal{P}_{t+1} = \mathcal{P}_t \setminus \{(k_j, a_j, b_j)\}$  where  $j \in \arg \min_{i \in [m]} \psi(\pmb{x}^{(t)})_i$  else  
 $\pmb{x}^{(t)} \notin \mathcal{C}_d$  then  $\pmb{a} = -\text{sign}(x_i)\pmb{e}_i$  where  $i \in \arg \min_{j \in [d]} |x_j^{(t)}|$ ;  
 $\pmb{a} = O(\pmb{x}^{(t)})$ ;  
Let  $b = \pmb{a}^\top \pmb{x}^{(t)} - c_a^{-1/2} \sqrt{\pmb{a}^\top (\pmb{A}^\top S_{x^{(t)}}^{-2}\pmb{A} + \lambda I)^{-1}a}$ ;  
 $\mathcal{P}_{t+1} = \mathcal{P}_t \cup \{(t, a, b)\}$ $\pmb{x}^{(t+1)} = \text{Centering}(\pmb{x}^{(t)}, 200, c_\Delta)$   
end
Algorithm 6: An efficient cutting-plane method, simplified from [27]
It remains to check whether one can implement this algorithm with efficient memory, corresponding to checking this method's numerical stability.
Lemma B.1. Suppose that each iterate of the centering Algorithm 5,  $\| \nabla p(\pmb{x}^{(k-1)}) \|_{\pmb{Q}(\pmb{x}^{(0)})^{-1}}$  is computed up to precision  $(1 - \frac{1}{64})^r \eta$  (l.2), and  $\pmb{x}^{(k)}$  is computed up to an error  $\pmb{\zeta}^{(k)}$  with  $\| \pmb{\zeta}^{(k)} \|_{\pmb{Q}(\pmb{x}^{(0)})} \leq \frac{1}{2^{10}r} (1 - \frac{1}{64})^r \eta$  (l.3). Then, Algorithm 5 outputs  $\pmb{x}^{(k)}$  such that  $\| \nabla p(\pmb{x}^{(k)}) \|_{\pmb{Q}^{-1}(\pmb{x}^{(k)})} \leq 3 (1 - \frac{1}{64})^r \eta$  and all iterates computed during the procedure satisfy  $\| S_{x^{(0)}}^{-1} (\pmb{s}(\pmb{x}^{(t)}) - \pmb{s}(\pmb{x}^{(0)})) \|_2 \leq \frac{1}{10}$ .
Proof. As mentioned above, without computation errors, the result from [27] would apply directly. Here, we simply adapt the proof to the case with computational errors to show that it still applies. Denote  $\mathbf{Q} = \mathbf{Q}(\mathbf{x}^{(0)})$  for convenience. Let  $\eta = \frac{1}{100}\sqrt{c_e + \mu(\mathbf{x}^{(0)})}$ . We prove by induction that  $\| \mathbf{x}^{(t)} - \mathbf{x}^{(0)}\|_{\mathbf{Q}} \leq 9\eta$ ,  $\| \nabla p(\mathbf{x}^{(t)})\|_{\mathbf{Q}^{-1}} \leq (1 - \frac{1}{64})^t\eta$  for all  $t \leq r$ . For a given iteration  $t$ , denote  $\tilde{\mathbf{x}}^{(t+1)} = \mathbf{x}^{(k-1)} - \frac{1}{8}\mathbf{Q}^{-1}\nabla p(\mathbf{x}^{(k-1)})$  the result of the exact computation. The same arguments as in the original proof give  $\| \tilde{\mathbf{x}}^{(t+1)} - \mathbf{x}^{(0)}\|_{\mathbf{Q}} \leq 9\eta$ , and
$$
\| \nabla p (\tilde {\boldsymbol {x}} ^ {(t + 1)}) \| _ {\boldsymbol {Q} ^ {- 1}} \leq \left(1 - \frac {1}{3 2}\right) \| \nabla p (\boldsymbol {x} ^ {(t)}) \| _ {\boldsymbol {Q} ^ {- 1}}.
$$
Now because  $\| \tilde{\boldsymbol{x}}^{(t + 1)} - \boldsymbol{x}^{(t + 1)} \|_{\boldsymbol{Q}} \leq \eta$ , we have  $\| \tilde{\boldsymbol{x}}^{(t + 1)} - \boldsymbol{x}^{(0)} \|_{\boldsymbol{Q}}, \| \boldsymbol{x}^{(t + 1)} - \boldsymbol{x}^{(0)} \|_{\boldsymbol{Q}} \leq 10\eta$ , so that [27, Lemma 11] gives  $\nabla^2 p(\boldsymbol{y}(u)) \preceq 8\boldsymbol{Q}(\boldsymbol{y}(u)) \preceq 16\boldsymbol{Q}$ , where  $\boldsymbol{y}(u) = \boldsymbol{x}^{(t + 1)} + u(\tilde{\boldsymbol{x}}^{(t + 1)} - \boldsymbol{x}^{(t + 1)}$  for  $u \in [0,1]$ . Thus,
$$
\begin{array}{l} \| \nabla p (\tilde {\boldsymbol {x}} ^ {(t + 1)}) - \nabla p (\boldsymbol {x} ^ {(t + 1)}) \| _ {\boldsymbol {Q} ^ {- 1}} \leq \left\| \int_ {0} ^ {1} \nabla^ {2} p (\boldsymbol {y} (u)) (\tilde {\boldsymbol {x}} ^ {(t + 1)} - \boldsymbol {x} ^ {(t + 1)}) \right\| _ {\boldsymbol {Q} ^ {- 1}} \\ \leq 1 6 \| \tilde {\boldsymbol {x}} ^ {(t + 1)} - \boldsymbol {x} ^ {(t + 1)} \| _ {\boldsymbol {Q}}. \\ \end{array}
$$
Now by construction of the procedure, if the algorithm performed iteration  $t + 1$ , we have  $\| \nabla p(\pmb{x}^{(t)}) \|_{\pmb{Q}^{-1}} \geq (1 - \frac{1}{64})^r \eta$ . Combining this with the fact that  $\| \tilde{\pmb{x}}^{(t + 1)} - \pmb{x}^{(t + 1)} \|_{\pmb{Q}} \leq \frac{1}{2^{10}r} (1 - \frac{1}{64})^r \eta$ , obtain
$$
\begin{array}{l} \| \nabla p (\boldsymbol {x} ^ {(t + 1)}) \| _ {\boldsymbol {Q} ^ {- 1}} \leq \| \nabla p (\tilde {\boldsymbol {x}} ^ {(t + 1)}) - \nabla p (\boldsymbol {x} ^ {(t + 1)}) \| _ {\boldsymbol {Q} ^ {- 1}} + \| \nabla p (\tilde {\boldsymbol {x}} ^ {(t + 1)}) \| _ {\boldsymbol {Q} ^ {- 1}} \\ \leq \left(1 - \frac {1}{6 4}\right) \| \nabla p (\boldsymbol {x} ^ {(t)}) \| _ {\boldsymbol {Q} ^ {- 1}}. \\ \end{array}
$$
We now write
$$
\begin{array}{l} \| \boldsymbol {x} ^ {(t + 1)} - \boldsymbol {x} ^ {(0)} \| _ {\boldsymbol {Q}} \leq \sum_ {k = 0} ^ {t} \| \tilde {\boldsymbol {x}} ^ {(k + 1)} - \boldsymbol {x} ^ {(k + 1)} \| _ {\boldsymbol {Q}} + \frac {1}{8} \| \boldsymbol {Q} ^ {- 1} \nabla p (\boldsymbol {x} ^ {(k)}) \| _ {\boldsymbol {Q}} \\ \leq \eta + \frac {1}{8} \sum_ {i = 0} ^ {\infty} \left(1 - \frac {1}{6 4}\right) ^ {i} \eta \leq 9 \eta . \\ \end{array}
$$
The induction is now complete. When the algorithm stops, either the  $r$  steps were performed, in which case the induction already shows that  $\| \nabla p(\pmb{x}^{(r)})\|_{Q^{-1}} \leq (1 - \frac{1}{64})^r\eta$ . Otherwise, if the algorithm terminates at iteration  $k$ , because  $\| \nabla p(\pmb{x}^{(k)})\|_{Q^{-1}}$  was computed to precision  $(1 - \frac{1}{64})^r\eta$ , we have (see 1.2 of Algorithm 5)
$$
\| \nabla p (\boldsymbol {x} ^ {(k)}) \| _ {\boldsymbol {Q} ^ {- 1}} \leq 2 \left(1 - \frac {1}{6 4}\right) ^ {r} \eta + \left(1 - \frac {1}{6 4}\right) ^ {r} \eta = 3 \left(1 - \frac {1}{6 4}\right) ^ {r} \eta .
$$
The same argument as in the original proof shows that at each iteration  $t$ ,
$$
\| \boldsymbol {S} _ {\boldsymbol {x} ^ {(0)}} ^ {- 1} (\boldsymbol {s} (\boldsymbol {x} ^ {(t)}) - \boldsymbol {s} (\boldsymbol {x} ^ {(0)})) \| _ {2} = \| \boldsymbol {x} ^ {(t)} - \boldsymbol {x} ^ {(0)} \| _ {\boldsymbol {A} ^ {\top} \boldsymbol {S} _ {\boldsymbol {x} ^ {(0)}} ^ {- 2} \boldsymbol {A}} \leq \frac {\| \boldsymbol {x} ^ {(t)} - \boldsymbol {x} ^ {(0)} \| _ {\boldsymbol {Q}}}{\sqrt {\mu (\boldsymbol {x} ^ {(0)}) + c _ {e}}} \leq \frac {1}{1 0}.
$$
This ends the proof of the lemma.
Because of rounding errors, Lemma B.1 has an extra factor 3 compared to the original guarantee in [27, Lemma 14]. To achieve the same guarantee, it suffices to perform  $70 \geq \ln(3) / \ln(1 / (1 - \frac{1}{64}))$  additional centering procedures at most. hence, instead of performing 200 centering procedures during the cutting plane method, we perform 270 (1.10 of Algorithm 6). We next turn to the numerical stability of the main Algorithm 6.