SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Analysis and proof techniques. The proof of Theorem 3.2, which is given in Appendix F, is appealing in its simplicity and modularity. The crux of the proof is to show that the representation learning guarantee in (6) is strong enough to ensure that the downstream spanner computation in RobustSpanner succeeds. It is straightforward to show that spanner computation would succeed if we had access to an estimated representation that  $\hat{\phi}^{(h)}$  that approximates  $\phi_h^*$  point-wise (i.e., uniformly for all  $(x,a)$  pairs), but the key challenge is that the guarantee in (6) only holds on average under the roll-in distribution  $\mathsf{unif}(\Psi^{(h)})$ . Prior works that make use of the same representation learning objective (BRIEE [49] and MOFFLE [35]) do not make use of spanners; instead, they appeal to exploration strategies based on elliptic bonuses, addressing the issue of approximation errors through additional assumptions (non-negativity of the factorization for MOFFLE, and Block MDP structure for BRIEE). Perhaps the most important observation in our proof is that barycentric spanners are robust to the average-case approximation error guarantee in (6) as-is, without additional structural assumptions. Intuitively, this benefit seems to arise from the fact that the spanner property only concerns the first moment of the feature map  $\phi^{\star}$ , while algorithms based on elliptic bonuses require approximation guarantees for the second moment; understanding this issue more deeply is an interesting question for future work. Another useful feature of our proof is to show that the notion of reachability in Assumption 2.1, which generalizes and extends all previous reachability conditions in the Low-Rank MDP and Block MDP literature [46, 5, 13, 33, 1, 35, 31], is sufficient to build a policy cover. We anticipate that this observation will find broader use.
We thank Noah Golowich, Dhruv Rohatgi, and Ayush Sekhari for several helpful discussions. ZM and AR acknowledge support from the ONR through awards N00014-20-1-2336 and N00014-20-1-2394, and ARO through award W911NF-21-1-0328. AB acknowledges support from the National Science Foundation Graduate Research Fellowship under Grant No. 1122374.
[40] M. Uehara, X. Zhang, and W. Sun. Representation learning for online and offline RL in low-rank mdps. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022, 2022.
[41] R. Wang, S. S. Du, L. Yang, and R. R. Salakhutdinov. On reward-free reinforcement learning with linear function approximation. Advances in neural information processing systems, 33: 17816-17826, 2020.
[42] R. Wang, R. Salakhutdinov, and L. F. Yang. Provably efficient reinforcement learning with general value function approximation. arXiv preprint arXiv:2005.10804, 2020.
[43] T. Xie, D. J. Foster, Y. Bai, N. Jiang, and S. M. Kakade. The role of coverage in online reinforcement learning. arXiv preprint arXiv:2210.04157, 2022.
[44] L. Yang and M. Wang. Sample-optimal parametric q-learning using linearly additive features. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 6995-7004. PMLR, 2019.
[45] H. Yao, C. Szepesvári, B. Á. Pires, and X. Zhang. Pseudo-mdps and factored linear action models. In 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2014, Orlando, FL, USA, December 9-12, 2014, pages 1-9. IEEE, 2014.
[46] A. Zanette, A. Lazaric, M. J. Kochenderfer, and E. Brunskill. Provably efficient reward-agnostic navigation with linear value iteration. Advances in Neural Information Processing Systems, 33: 11756–11766, 2020.
[47] T. Zhang. Feel-good thompson sampling for contextual bandits and reinforcement learning. arXiv preprint arXiv:2110.00871, 2021.
[48] T. Zhang, T. Ren, M. Yang, J. Gonzalez, D. Schuurmans, and B. Dai. Making linear mdps practical via contrastive representation learning. In International Conference on Machine Learning, pages 26447-26466. PMLR, 2022.
[49] X. Zhang, Y. Song, M. Uehara, M. Wang, A. Agarwal, and W. Sun. Efficient reinforcement learning in block mdps: A model-free representation learning approach. In International Conference on Machine Learning, pages 26517-26547. PMLR, 2022.
A Additional Related Work 14
B Omitted Algorithms 15
C Generic Guarantee for RobustSpanner 18
D Generic Guarantee for PSDP 20
E Guarantee for RepLearn 23
F Analysis 26
F.1 Proof Strategy 26
F.2 Guarantee for PSDP as a Subroutine for RobustSpanner 27
F.3 Guarantee for RobustSpanner as a Subroutine for SpanRL 28
F.4 Guarantee for RepLearn as a Subroutine for SpanRL 29
F.5 Concluding the Proof of Theorem 3.2 29
F.6 Proof of Lemma 3.1 32
G Application to Reward-Based RL 32
H Properties of Reachability Assumption 33
H.1 Comparison to Latent Variable Model 33
H.2 Relation to Other Reachability Assumptions 35
In this section, we discuss relevant related work not already covered.
Block MDPs. A particularly well-studied special case low-rank MDPs with the latent variable assumed in Modi et al. [35] (defined in Definition H.1) is the Block MDP (BMDP) model Du et al. [13], Misra et al. [32], Zhang et al. [49], Mhammedi et al. [31]. For this setting, Du et al. [13], Misra et al. [32] provide algorithms that conduct exploration in a provably oracle-efficient manner under a reachability assumption. This reachability assumption was removed by subsequent work of Zhang et al. [49] (with a suboptimal rate) and Mhammedi et al. [31] (with optimal error dependence), but the analysis in these works is tailored to the BMDP model.
Barycentric spanners. Huang et al. [20] consider a variant of the Low-Rank MDP framework in which we are given a class  $\Upsilon$  that realizes the next-state feature map  $\mu^{\star}$ , but do not have access to a class  $\Phi$  for the feature map  $\phi^{\star}$ , which is unknown. Their algorithm, like SpanRL, is based on barycentric spanners, though the algorithm design considerations and analysis are significantly different. Notably, their algorithm is not computationally efficient, and their analysis takes advantage of the fact that realizability of  $\mu^{\star}$  facilitates estimation of the occupancies  $\{d^{\pi}(\cdot)\}_{\pi \in \Pi_{M}}$  in  $\ell_1$ -error. Barycentric spanners were also in the work of Golowich et al. [19] for reinforcement learning in Partially Observable MDPs (POMDPs). Their analysis is substantially different from ours, and their algorithm appeals to the barycentric spanner computation approach in Awerbuch and Kleinberg [6] in an off-the-shelf fashion.
B Omitted Algorithms
Algorithm 2 RobustSpanner: Barycentric Spanner via Approximate Linear Optimization
Require: Approximate linear optimization subroutine LinOpt:  $\mathbb{R}^d\to \mathcal{Z}$  /See Section 3.2 \*/ Approximate index-to-vector subroutine LinEst:  $\mathcal{Z}\rightarrow \mathbb{R}^d$  Parameters  $C,\varepsilon >0$    
1: Set  $W = (w_{1},\ldots ,w_{d}) = (e_{1},\ldots ,e_{d})$    
2: for  $i = 1,\dots ,d$  do   
3: Set  $\theta_{i} = (\operatorname *{det}(e_{j},W_{-i}))_{j\in [d]}\in \mathbb{R}^{d}$  //  $W_{-i}$  is defined to be  $W$  without the ith column   
4: Set  $z_{i}^{+} = \mathrm{LinOpt}(\theta_{i} / \| \theta_{i}\|)$  and  $w_{i}^{+} = \mathrm{LinEst}(z_{i}^{+})$    
5: Set  $z_{i}^{-} = \mathrm{LinOpt}(-\theta_{i} / \| \theta_{i}\|)$  and  $w_{i}^{-} = \mathrm{LinEst}(z_{i}^{-})$    
6: if  $\theta_i^\top w_i^+\geq -\theta_i^\top w_i^-$  then   
7: Set  $\widetilde{w}_i = w_i^+,z_i = z_i^+$  , and  $w_{i} = \widetilde{w}_{i} + \varepsilon \theta_{i} / \| \theta_{i}\|$    
8: else   
9: Set  $\widetilde{w}_i = w_i^{-},z_i = z_i^{-}$  , and  $w_{i} = \widetilde{w}_{i} - \varepsilon \theta_{i} / \| \theta_{i}\|$    
10: for  $n = 1,2,\ldots$  do   
11: Set  $i = 1$    
12: while  $i\leq d$  do   
13: Set  $\theta_{i} = (\operatorname *{det}(e_{j},W_{-i}))_{j\in [d]}\in \mathbb{R}^{d}$    
14: Set  $z_{i}^{+} = \mathrm{LinOpt}(\theta_{i} / \| \theta_{i}\|)$  and  $w_{i}^{+} = \mathrm{LinEst}(z_{i}^{+})$    
15: Set  $z_{i}^{-} = \mathrm{LinOpt}(-\theta_{i} / \| \theta_{i}\|)$  and  $w_{i}^{-} = \mathrm{LinEst}(z_{i}^{-})$  .   
16: if  $\theta_i^\top w_i^+ +\varepsilon \cdot \| \theta_i\| \geq C\cdot |\operatorname *{det}(w_i,W_{-i})|$  then   
17: Set  $\widetilde{w}_i = w_i^+,z_i = z_i^+$  , and  $w_{i} = \widetilde{w}_{i} + \varepsilon \cdot \theta_{i} / \| \theta_{i}\|$    
18: break   
19: else if  $-\theta_i^\top w_i^{-} + \varepsilon \cdot \| \theta_i\| \geq C\cdot |\operatorname *{det}(w_i,W_{-i})|$  then   
20: Set  $\widetilde{w}_i = w_i^{-},z_i = z_i^{-}$  , and  $w_{i} = \widetilde{w}_{i} - \varepsilon \cdot \theta_{i} / \| \theta_{i}\|$    
21: break   
22: Set  $i = i + 1$    
23: if  $i = d + 1$  then   
24: break   
25: Return:  $(z_1,\dots ,z_d)$
Algorithm 3 PSDP  $(h,r_{1:h},\mathcal{G},\Psi^{(1:h)},n)$ : Policy Search by Dynamic Programming (variant of [8])
Require:
- Target layer  $h \in [H]$ .
- Reward functions  $r_{1:h}$ .
- Function class  $\mathcal{G}$ .
Policy covers  $\Psi^{(1)},\ldots ,\Psi^{(h)}$
Number of samples  $n\in \mathbb{N}$
1: for  $t = h, \dots, 1$  do
2:  $\mathcal{D}^{(t)}\gets \emptyset$
3: for  $n$  times do
4: Sample  $(\pmb{x}_t,\pmb{a}_t,\sum_{\ell = t}^h r_\ell (\pmb{x}_\ell ,\pmb{a}_\ell))\sim \mathrm{unif}(\Psi^{(t)})\circ_t\pi_{\mathrm{unif}}\circ_{t + 1}\hat{\pi}^{(t + 1)}$
5: Update dataset:  $\mathcal{D}^{(t)}\gets \mathcal{D}^{(t)}\cup \left\{\left(\boldsymbol{x}_t,\boldsymbol {a}_t,\sum_{\ell = t}^h r_\ell (\boldsymbol{x}_\ell ,\boldsymbol {a}_\ell)\right)\right\}$
6: Solve regression:
$$
g^{(t)}\leftarrow \operatorname *{arg  min}_{g\in \mathcal{G}}\sum_{(x,a,R)\in \mathcal{D}^{(t)}}(g(x,a) - R)^{2}.
$$
7: Define  $\hat{\pi}^{(t)}\in \Pi_{\mathbb{M}}^{t:h}$  via
$$
\hat {\pi} ^ {(t)} (x) = \left\{ \begin{array}{l l} \arg \max  _ {a \in \mathcal {A}} g ^ {(t)} (x, a), & x \in \mathcal {X} _ {t}, \\ \hat {\pi} ^ {(t + 1)} (x), & x \in \mathcal {X} _ {\ell}, \ell \in [ t + 1.. h ]. \end{array} \right. \tag {8}
$$
8: Return: Near-optimal policy  $\hat{\pi}^{(1)}\in \Pi_{M}$
Algorithm 4 EstVec(h,F,π,n): Estimate  $\mathbb{E}^{\pi}[F(\pmb{x}_h,\pmb{a}_h)]$  for policy  $\pi$  and function  $F:\mathcal{X}\times \mathcal{A}\to \mathbb{R}^{d}$
- Target layer  $h \in [H]$ .
- Vector-valued function  $F: \mathcal{X} \times \mathcal{A} \to \mathbb{R}^d$ .
Policy  $\pi \in \Pi_{\mathbb{M}}$
Number of samples  $n\in \mathbb{N}$
1:  $\mathcal{D}\gets \varnothing$
2: for  $n$  times do
3: Sample  $(\pmb{x}_h, \pmb{a}_h) \sim \pi$
4: Update dataset:  $\mathcal{D}\gets \mathcal{D}\cup \{(\pmb {x}_h,\pmb {a}_h)\} .$
5: Return:  $\bar{F} = \frac{1}{n}\sum_{(x,a)\in \mathcal{D}}F(x,a)$
Algorithm 5 RepLearn  $(h, \mathcal{F}, \Phi, \Psi, n)$ : Representation Learning for Low-Rank MDPs [35, 49].
Require:
- Target layer  $h \in [H]$ .
- Discriminator class  $\mathcal{F}$ .
- Feature class  $\Phi$ .
- A set of policies  $\Psi$ .
- Number of samples  $n \in \mathbb{N}$ .
1: Set  $\tilde{\varepsilon} = 100\frac{d\log n + \log(|\Phi| / \delta)}{n}, \varepsilon_1 = 16\sqrt{2} d^{3/2}\tilde{\varepsilon}^{1/2}$ .
2: Initialize  $\lambda = \sqrt{n}, T = \sqrt{\frac{d}{2\tilde{\varepsilon}}}$ , and  $\xi = \frac{3}{2}\varepsilon_1 + \tilde{\varepsilon} + \frac{2\lambda d}{n}, \phi^{(0)} \in \Phi$ .
3:  $\mathcal{D} \gets \emptyset$ .
4: for  $n$  times do
5: Sample  $(x_h, a_h, x_{h+1}) \sim \text{unif}(\Psi) \circ_h \pi_{\text{unif}}$ .
6: Update dataset:  $\mathcal{D} \gets \mathcal{D} \cup \{(x_h, a_h, x_{h+1})\}$ .
7: Define  $\mathcal{L}_{\mathcal{D}}(\phi, w, f) = \sum_{(x, a, x') \in \mathcal{D}} (\phi(x, a)^{\top}w - f(x'))^2 + \lambda \|w\|^2$ .
8: for  $t = 0, \ldots, T-1$  do
/* Discriminator selection */
9: Solve
\[ f^{(t)} \in \arg \max_{f \in \mathcal{F}} \widehat{\Delta}(f), \text{ where } \widehat{\Delta}(f) := \max_{\tilde{\phi} \in \Phi} \left\{ \min_{w} \mathcal{L}_{\mathcal{D}}(\phi^{(t)}, w, f) - \min_{\tilde{w}} \mathcal{L}_{\mathcal{D}}(\tilde{\phi}, \tilde{w}, f) \right\}.
10: if  $\widehat{\Delta}(f^{(t)}) \leq \xi$  then
11: Return  $\hat{\phi} = \phi^{(t)}$ .
/* Feature selection via least-squares minimization */
12: Solve
\[ \phi^{(t+1)} \in \arg \min_{\phi \in \Phi} \min_{w_0:t} \sum_{i=0}^{t} \mathcal{L}_{\mathcal{D}}(\phi, w_i, f^{(i)}).
C Generic Guarantee for RobustSpanner
In this section, we give a generic guarantee for the RobustSpanner algorithm when invoked with oracles LinOpt and LinEst satisfying the following assumption.
Assumption C.1 (LinOpt and LinEst as approximate Linear Optimization Oracles). For some abstract set  $\mathcal{Z}$  and a collection of vectors  $\{w^z\in \mathbb{R}^d\mid z\in \mathcal{Z}\}$  indexed by elements in  $\mathcal{Z}$ , there exists  $\varepsilon^{\prime} > 0$  such that for any  $\theta \in \mathbb{R}^d\setminus \{0\}$  and  $z\in \mathcal{Z}$ , the outputs  $\hat{z}_{\theta}\coloneqq \mathrm{LinOpt}(\theta /\| \theta \|)$  and  $\hat{w}_z\coloneqq \mathrm{LinEst}(z)$  satisfy
$$
\sup _ {z \in \mathcal {Z}} \theta^ {\top} w ^ {z} \leq \theta^ {\top} w ^ {\hat {z} _ {\theta}} + \varepsilon^ {\prime} \cdot \| \theta \|, \quad a n d \quad \| \hat {w} _ {z} - w ^ {z} \| \leq \varepsilon^ {\prime}.
$$
Letting  $\mathcal{W} \coloneqq \{w^z \mid z \in \mathcal{Z}\}$  and assuming that  $\mathcal{W} \subseteq \mathcal{B}(1)$ , the next theorem bounds the number of iterations of RobustSpanner(LinOpt(\cdot), LinEst(\cdot),  $\cdot, \cdot$ ) under Assumption C.1, and shows that the output is an approximate barycentric spanner for  $\mathcal{W}$  (Definition 3.1). Our result extends those of Awerbuch and Kleinberg [6], in that it only requires an approximate linear optimization oracle, which is potentially of independent interest.
Proposition C.1. Fix  $C > 1$  and  $\varepsilon \in (0,1)$  and suppose that  $\{w^z\mid z\in \mathcal{Z}\} \subseteq \mathcal{B}(1)$ . If RobustSpanner (Algorithm 2) is run with parameters  $C,\varepsilon >0$  and oracles LinOpt, LinEst satisfying Assumption C.1 with  $\varepsilon^{\prime} = \varepsilon /2$ , then it terminates after  $d + \left[\frac{d}{2}\log_{C}\frac{100d}{\varepsilon^{2}}\right]$  iterations, and requires at most twice that many calls to each of LinOpt and LinEst. Furthermore, the output  $z_{1:d}$  has the property that for all  $z\in \mathcal{Z}$ , there exist  $\beta_{1},\ldots ,\beta_{d}\in [-C,C]$ , such that
$$
\left\| w ^ {z} - \sum_ {i = 1} ^ {d} \beta_ {i} w ^ {z _ {i}} \right\| \leq \frac {3 C d \cdot \varepsilon}{2}.
$$
Proof of Proposition C.1. The proof will follows similar steps to those in Awerbuch and Kleinberg [6, Lemma 2.6], with modifications to account for the fact that linear optimization over the set  $\mathcal{W} \coloneqq \{w^z \mid z \in \mathcal{Z}\}$  is only performed approximately.
Part I: Bounding the number of iterations. In Algorithm 2, there are two loops, both of which require two calls to LinOpt and LinEst per iteration. As the first loop has exactly  $d$  iterations, it suffices to bound the number of iterations in the second loop.
Let  $M^{(i)} \coloneqq (w_1, \ldots, w_i, e_{i+1}, \ldots, e_d)$  be the matrix whose columns are the vectors at end of the  $i$ th iteration of the first loop (Line 2) of Algorithm 2; note that columns  $i+1$  through  $d$  are unchanged at this point in the algorithm. For  $i \in [d]$ , we define  $\ell_i(w) \coloneqq \det(w, M_{-i}^{(i)})$  and  $\theta_i \coloneqq (\det(e_j, M_{-i}^{(i)}))_{j \in [d]} \in \mathbb{R}^d$ , where we recall that for any matrix  $A$ , the matrix  $A_{-i}$  is defined as the result of removing the  $i$ th column from  $A$ . Note that  $\ell_i$  is linear in  $w$ , and in particular
$$
\ell_ {i} (w) := w ^ {\top} \theta_ {i}.
$$
Let  $W^{(0)} \coloneqq M^{(d)} = (w_1, \ldots, w_d)$ , and let  $W^{(j)}$  denote the resulting matrix after  $j$  iterations of the second loop (Line 10) of Algorithm 2. We will show that for any  $J \geq 1$ ,
$$
\det  \left(W ^ {(J)}\right) \leq \det  \left(W ^ {(0)}\right) \cdot \left(\frac {1 0 0 d}{\varepsilon^ {2}}\right) ^ {\frac {d}{2}}. \tag {9}
$$
By construction of the loop, we have  $\operatorname{det}(W^{(j)}) \geq C \cdot \operatorname{det}(W^{(j-1)})$  for each  $j \in [J]$ , and thus  $\operatorname{det}(W^{(J)}) \geq \operatorname{det}(W^{(0)}) \cdot C^J$ . Combining these two facts will establish the bound on the iteration complexity. We now prove (9).
Let  $u_{i} = e_{i}^{\top}\big(M^{(i)}\big)^{-1}$  (note that  $u_{i}$  is a row vector) and let  $U$  denote the matrix whose  $i$ th row is  $u_{i}$ . We observe that for all  $w \in \mathbb{R}^{d}$ ,
$$
u _ {i} w = \frac {\ell_ {i} (w)}{\ell_ {i} (w _ {i})},
$$
where we note that  $\ell_i(w_i) \neq 0$  by construction; indeed, the columns of  $M^{(i)}$  are a basis for  $\mathbb{R}^d$  because  $\operatorname*{det}(M^{(i)}) \neq 0$ , and the equality holds on the columns, so the two linear functions must be equal. Now, since Assumption C.1 holds with  $\varepsilon' = \varepsilon / 2$ , we have
$$
\theta_ {i} ^ {\top} w _ {i} ^ {+} \geq \sup  _ {z \in \mathcal {Z}} \theta_ {i} ^ {\top} w ^ {z} - \frac {\varepsilon}{2} \| \theta_ {i} \|, \quad \text {a n d} \quad \theta_ {i} ^ {\top} w _ {i} ^ {-} \leq \inf  _ {z \in \mathcal {Z}} \theta_ {i} ^ {\top} w ^ {z} + \frac {\varepsilon}{2} \| \theta_ {i} \|, \tag {10}
$$
where  $w_{i}^{\pm} = \mathrm{LinOpt}(z_{i}^{\pm})$ . We will now show that
$$
\ell_ {i} \left(w _ {i}\right) \geq \frac {\varepsilon}{2} \cdot \| \theta_ {i} \|. \tag {11}
$$
There are two cases. First, suppose that  $\theta_i^\top w_i^+ \geq -\theta_i^\top w_i^-$ , corresponding to the conditional in Line 6 of Algorithm 2 being satisfied. Combining this with (10), we have
$$
\begin{array}{l} \theta_ {i} ^ {\top} w _ {i} ^ {+} \geq \left(\sup  _ {z \in \mathcal {Z}} \theta_ {i} ^ {\top} w ^ {z} - \frac {\varepsilon}{2} \| \theta_ {i} \|\right) \vee (- \theta_ {i} ^ {\top} w _ {i} ^ {-}), \\ \geq \left(\sup  _ {z \in \mathcal {Z}} \theta_ {i} ^ {\top} w ^ {z} - \frac {\varepsilon}{2} \| \theta_ {i} \|\right) \vee \left(\sup  _ {z \in \mathcal {Z}} - \theta_ {i} ^ {\top} w ^ {z} - \frac {\varepsilon}{2} \| \theta_ {i} \|\right), \quad (\text {b y}) \\ = \left(\sup  _ {z \in \mathcal {Z}} \theta_ {i} ^ {\top} w ^ {z}\right) \vee \left(\sup  _ {z \in \mathcal {Z}} - \theta_ {i} ^ {\top} w ^ {z}\right) - \frac {\varepsilon}{2} \| \theta_ {i} \|, \\ \geq - \frac {\varepsilon}{2} \| \theta_ {i} \|. \tag {12} \\ \end{array}
$$
Because the conditional is satisfied,  $w_{i} = w_{i}^{+} + \varepsilon \cdot \frac{\theta_{i}}{\|\theta_{i}\|}$ , and so by plugging this into (12), we have
$$
\ell_ {i} (w _ {i}) = \theta_ {i} ^ {\top} w _ {i} \geq \frac {\varepsilon}{2} \cdot \| \theta_ {i} \|.
$$
The case that  $\theta_i^\top w_i^+ \leq -\theta_i^\top w_i^-$  is essentially identical, establishing (11). Now, recall that  $\mathcal{W} := \{w^z \mid z \in \mathcal{Z}\}$  and let  $\mathcal{W} \oplus \mathcal{B}\left(\frac{3\varepsilon}{2}\right) := \{w + b \mid w \in \mathcal{W}$  and  $b \in \mathcal{B}\left(\frac{3\varepsilon}{2}\right)\}$  denote the Minkowski sum with  $\mathcal{B}\left(\frac{3\varepsilon}{2}\right)$ . By Cauchy-Schwarz, it holds that for all  $w' := w + b \in \mathcal{W} \oplus \mathcal{B}\left(\frac{3\varepsilon}{2}\right)$ ,
$$
\ell_ {i} \left(w ^ {\prime}\right) = \theta_ {i} ^ {\top} w ^ {\prime} = \theta_ {i} ^ {\top} w + \theta_ {i} ^ {\top} b \leq \left(1 + \frac {3 \varepsilon}{2}\right) \cdot \| \theta_ {i} \|,
$$
where we used that  $\mathcal{W} \subseteq \mathcal{B}(1)$  (by assumption). Thus, for any  $w' \in \mathcal{W} \oplus \mathcal{B}\left(\frac{3\varepsilon}{2}\right)$ , we have
$$
\left| u _ {i} w ^ {\prime} \right| = \frac {\ell_ {i} \left(w ^ {\prime}\right)}{\ell_ {i} \left(w _ {i}\right)} \leq 1 + \frac {3 \varepsilon}{2}.
$$
We now observe that by construction and the fact that Assumption C.1 holds with  $\varepsilon' = \varepsilon / 2$ , the  $k$ th column  $w_k'$  of  $W^{(J)}$  belongs to  $\mathcal{W} \oplus \mathcal{B}\left(\frac{3\varepsilon}{2}\right)$ , for any  $k \in [d]$ . Thus, the  $(i, k)$  entry  $u_i w_k'$  of  $U W^{(J)}$  satisfies  $u_i w_k' \in \left[-1 - \frac{3\varepsilon}{2}, 1 + \frac{3\varepsilon}{2}\right]$ , and so the columns of  $U W^{(J)}$  have Euclidean norm at most  $\frac{10\sqrt{d}}{\varepsilon}$ . Since the magnitude of the determinant of a matrix is upper bounded by the product of the Euclidean norms of its columns, it holds that  $|\operatorname*{det}(U W^{(J)})| \leq \left(\frac{100d}{\varepsilon^2}\right)^{\frac{d}{2}}$ .
On the other hand, again by construction, we see that the columns  $w_{1}, \ldots, w_{d}$  of  $W^{(0)}$  satisfy  $u_{i}w_{j} = 0$ , for  $j < i$ , and  $u_{i}w_{i} = 1$ . Thus,  $UW^{(0)}$  is an upper-triangular matrix with 1s on the diagonal, and hence has determinant 1. Because determinants are multiplicative, this implies that  $\det(U) \neq 0$ . We now compute:
$$
\left| \det  \left(W ^ {(J)}\right) \right| = \frac {\left| \det  \left(U W ^ {(J)}\right) \right|}{\left| \det  (U) \right|} = \frac {\left| \det  \left(U W ^ {(J)}\right) \right|}{\left| \det  \left(U W ^ {(0)}\right) \right|} \leq \left(\frac {1 0 0 d}{\varepsilon^ {2}}\right) ^ {\frac {d}{2}}.
$$
Thus, the upper bound on  $|\operatorname{det}(W^{(J)})|$  holds and the claim is proven. Therefore, we have
$$
C ^ {J} \leq \left(\frac {1 0 0 d}{\varepsilon^ {2}}\right) ^ {\frac {d}{2}},
$$
and so  $J \leq \left\lceil \frac{d}{2} \log_{C}\left(\frac{100d}{\varepsilon^{2}}\right) \right\rceil$ .
Part II: Spanner property for the output. Having shown that the algorithm terminates, we now show that the result is an approximate barycentric spanner for  $\mathcal{W}$ . Let  $W\coloneqq (w_{1},\ldots ,w_{d})$  be the matrix at termination of the algorithm. By definition, if the second loop (Line 10) has terminated, then for all  $i\in [d]$ ,
$$
\max  \left(\theta_ {i} ^ {\top} w _ {i} ^ {+}, - \theta_ {i} ^ {\top} w _ {i} ^ {-}\right) + \varepsilon \cdot \| \theta_ {i} \| \leq C \cdot | \det  (w _ {i}, W _ {- i}) |,
$$
where  $\theta_{i} = (\operatorname*{det}(e_{j},W_{-i}))_{j\in [d]}\in \mathbb{R}^{d}$ . On the other hand, by Assumption C.1, (10) holds, and so
$$
\begin{array}{l} \forall z \in \mathcal {Z}, \forall i \in [ d ], \quad | \det  (w ^ {z}, W _ {- i}) | = | \theta_ {i} ^ {\top} w ^ {z} | \leq \max  \left(\theta_ {i} ^ {\top} w _ {i} ^ {+}, - \theta_ {i} ^ {\top} w _ {i} ^ {-}\right) + \varepsilon \cdot \| \theta_ {i} \|, \\ \leq C \cdot | \det  (w _ {i}, W _ {- i}) |. \tag {13} \\ \end{array}
$$
Now, fix  $z \in \mathcal{Z}$ . Since  $\operatorname*{det}(W) \neq 0$ , there exist  $\beta_{1:d} \in \mathbb{R}$  such that  $w^{z} = \sum_{i=1}^{d} \beta_{i} w_{i}$ . By plugging this into (13) and using the linearity of the determinant, we have
$$
\forall i \in [ d ], \quad C \cdot | \det  (w _ {i}, W _ {- i}) | \geq | \det  (w ^ {z}, W _ {- i}) | = \left| \sum_ {j = 1} ^ {d} \beta_ {i} \det  (w _ {j}, W _ {- i}) \right| = | \beta_ {i} | \cdot | \det  (w _ {i}, W _ {- i}) |.
$$
Therefore,  $|\beta_i| \leq C$ , for all  $i \in [d]$ . Now, by definition of  $w_{1:d}$  and  $\widetilde{w}_{1:d}$ , for all  $i \in [d]$ , we have that  $\| w_i - \widetilde{w}_i \| \leq \varepsilon$ . Furthermore, by Assumption C.1, we also have that  $\| \widetilde{w}_i - w^{z_i} \| \leq \varepsilon / 2$ . Therefore, by the triangle inequality, we have
$$
\left\| w ^ {z} - \sum_ {i = 1} ^ {d} \beta_ {i} w ^ {z _ {i}} \right\| \leq \left\| w ^ {z} - \sum_ {i = 1} ^ {d} \beta_ {i} w _ {i} \right\| + \sum_ {i = 1} ^ {d} | \beta_ {i} | \| \widetilde {w} _ {i} - w ^ {z _ {i}} \| + \sum_ {i = 1} ^ {d} | \beta_ {i} | \| \widetilde {w} _ {i} - w _ {i} \| \leq 3 d C \varepsilon / 2.
$$
D Generic Guarantee for PSDP