SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Recovery of structured matrices. In addition to directly spectrally approximating and solving in matrices which are well-approximated by preconditioners with diagonal or combinatorial structure, our framework also yields solvers for new families of matrices. We show that our preconditioning techniques can be used in conjunction with properties of graph-structured matrices to provide solvers and spectral approximations for inverse  $M$ -matrices and Laplacian pseudoinverses. Recovering Laplacians from their pseudoinverses and solving linear systems in the Laplacian pseudoinverse arise when trying to fit a graph to data or recover a graph from effective resistances, a natural distance measure (see [HMMT18] for motivation and discussion of related problems). More broadly, the problem of solving linear systems in inverse symmetric  $M$ -matrices is prevalent and corresponds to statistical inference problems involving distributions that are multivariate totally positive of order 2  $(\mathrm{MTP}_2)$  [KR83, SH14,  $\mathrm{FLS}^{+}17$ ]. Our main results are the following.
Theorem 4 (M-matrix recovery and inverse M-matrix solver). Let  $\mathbf{M}$  be the inverse of an unknown invertible symmetric  $M$ -matrix, let  $\kappa$  upper bound its condition number, and let  $U$  be the multiplicative range of  $\mathbf{M} \mathbb{1}$ . For any  $\delta \in (0,1)$  and  $\epsilon > 0$ , there is an algorithm recovering a  $(1 + \epsilon)$ -spectral approximation to  $\mathbf{M}^{-1}$  in time  $O(n^{2} \cdot \mathrm{poly}(\frac{\log \frac{n\kappa U}{\delta}}{\epsilon}))$ . Consequently, there is an algorithm for solving linear systems in  $\mathbf{M}$  to  $\epsilon$ -relative accuracy with probability  $\geq 1 - \delta$ , in time  $O(n^{2} \cdot \mathrm{polylog}(\frac{n\kappa U}{\delta\epsilon}))$ .
Theorem 5 (Laplacian recovery and Laplacian pseudoinverse solver). Let  $\mathbf{M}$  be the pseudoinverse of unknown Laplacian  $\mathbf{L}$ , and that  $\mathbf{L}$  corresponds to a graph with edge weights between  $w_{\mathrm{min}}$  and  $w_{\mathrm{max}}$ , with  $\frac{w_{\mathrm{max}}}{w_{\mathrm{min}}} \leq U$ . For any  $\delta \in (0,1)$  and  $\epsilon > 0$ , there is an algorithm recovering a Laplacian  $\mathbf{L}'$  with  $\mathbf{M}^{\dagger} \preceq \mathbf{L}' \preceq (1 + \epsilon)\mathbf{M}^{\dagger}$  in time  $O(n^{2} \cdot \text{poly}(\frac{\log \frac{nU}{\delta}}{\epsilon}))$ . Consequently, there is an algorithm for solving linear systems in  $\mathbf{M}$  to  $\epsilon$ -relative accuracy with probability  $\geq 1 - \delta$ , in time  $O(n^{2} \cdot \text{polylog}(\frac{nU}{\delta\epsilon}))$ .
See (6) and the following discussion for the definition of solving to relative accuracy.
${}^{10}\mathbf{M}\mathbb{1}$  is entrywise positive, as shown in Appendix B of the supplement.
Theorems 4 and 5 are perhaps a surprising demonstration of the utility of our techniques: just because a matrix family is well-approximated by structured preconditioners, it is not a priori clear that their inverses also are. However, we show that by applying recursive preconditioning tools in conjunction with our recovery methods, we can obtain analogous results for these inverse families. These results add to the extensive list of combinatorially-structured matrix families admitting efficient linear algebra primitives. We view our approach as a proof-of-concept of further implications in designing near-linear time system solvers for structured families via algorithms for (1).
Similarly to our results in Section 2, our results on solving matrix-dictionary recovery for graph-structured matrices (Theorems 3, 4, and 5) are the first we are aware of with runtimes improving upon black-box generic algorithms. In particular, for key matrices in each of these cases (e.g., constant-factor spectral approximations of Laplacians, inverse M-matrices, and Laplacian pseudoinverses) we obtain  $\widetilde{O}(n^2)$  time algorithms for solving linear systems in these matrices to inverse polynomial accuracy. This runtime is near-linear when the input is dense and in each case when the input is dense the state-of-the-art prior methods were to run general linear system solvers using  $O(n^{\omega})$  time.
4 Matrix-dictionary recovery: a general preconditioning framework
Our general strategy for matrix-dictionary recovery, i.e., recovering preconditioners in the sense of (1), is via applications of a new custom approximate solver we develop for a family of structured SDPs. SDPs are fundamental optimization problems that have been the source of extensive study for decades [VB96], with numerous applications across operations research and theoretical computer science [GW95], statistical modeling [WSV00, GM12], and machine learning [RSL18]. Though there have been recent advances in solving general SDPs (e.g.,  $\mathrm{[JKL^{+}20}$ $\mathrm{HJS^{+}22]}$  and references therein), the current state-of-the-art solvers have superlinear runtimes, prohibitive in large-scale applications. Consequently, there has been extensive research on designing faster approximate SDP solvers under different assumptions [KV05, WK06, AK07, BBN13, GHM15, AL17, CDST19].
We now provide context for our solver for structured "matrix-dictionary approximation" SDPs, state our algorithm and its guarantees, and summarize how it is used to obtain Theorems 1, 2, 3, 4, and 5.
Positive SDPs. One prominent class of structured SDPs are what we refer to as positive SDPs, namely SDPs in which the cost and constraint matrices are all positive semidefinite (PSD), a type of structure present in many important applications [GW95, ARV09, JJJW11, LS17, CG18, CDG19, CFB19, CMY20], including those in this paper. Positive SDPs generalize positive linear programming, itself a well-studied problem over the past several decades [LN93, PST95, You01, MRWZ16, AO19]. It was recently shown that a prominent special case of positive SDPs known as packing SDPs can be solved in nearly-linear time [ALO16, PTZ16], a fact that has had numerous applications in robust learning and estimation [CG18, CDG19, CFB19, CMY20] as well as in combinatorial optimization [LS17]. However, extending known packing SDP solvers to broader classes of positive SDPs, e.g. mixed packing-covering SDPs has been elusive [JY12, JLL $^{+}$ 20], and is a key open problem in the algorithmic theory of structured optimization. $^{11}$  The mixed packing-covering SDP problem is parameterized by "packing" and "covering" matrices  $\{\mathbf{P}_i\}_{i\in [n]}, \mathbf{P}, \{\mathbf{C}_i\}_{i\in [n]}, \mathbf{C} \in \mathbb{S}_{\geq 0}^d$ , and asks to find the smallest  $\mu > 0$  such that there exists  $w \in \mathbb{R}_{\geq 0}^n$  with  $\sum_{i\in [n]} w_i \mathbf{P}_i \preceq \mu \mathbf{P}$  (packing into  $\mu \mathbf{P}$ ) and  $\sum_{i\in [n]} w_i \mathbf{C}_i \succeq \mathbf{C}$  (covering  $\mathbf{C}$ ). Redefining  $\mathbf{P}_i \gets \frac{1}{\mu} \mathbf{P}^{-\frac{1}{2}} \mathbf{P}_i \mathbf{P}^{-\frac{1}{2}}$ ,  $\mathbf{C}_i \gets \mathbf{C}^{-\frac{1}{2}} \mathbf{C}_i \mathbf{C}^{-\frac{1}{2}}$  for all  $i \in [n]$ , (a slight strengthening of) this problem is equivalent to finding  $w \in \mathbb{R}_{\geq 0}^n$  such that
$$
\sum_ {i \in [ n ]} w _ {i} \mathbf {P} _ {i} \preceq \mathbf {I} \preceq \sum_ {i \in [ n ]} w _ {i} \mathbf {C} _ {i}, \tag {2}
$$
or refuting its existence. This was studied by [JY12, JLL $^{+}$ 20], and an important open problem in structured convex programming is designing a "width-independent" solver for testing feasibility of (2) up to a  $1 + \epsilon$  factor (i.e. testing whether (2) is approximately feasible with an iteration count polynomial in  $\epsilon^{-1}$  and polylogarithmic in other parameters). Such solvers have remained elusive beyond pure packing SDPs [ALO16, PTZ16, JLT20], even for basic extensions such as pure covering.
Matrix-dictionary approximation SDPs. In Theorem 6, we develop our main meta-algorithm, an efficient solver for specializations of (2) where the packing and covering matrices  $\{\mathbf{P}_i\}_{i\in [n]}$ ,  $\{\mathbf{C}_i\}_{i\in [n]}$
11A faster solver for general positive (mixed packing-covering) SDPs was claimed in  $\mathrm{[JLL^{+}20]}$ , but an error was later discovered in that work, as is recorded in the most recent arXiv version  $\mathrm{[JLL^{+}21]}$ .
are multiples of each other. As we will see, this family of structured SDPs, which we call matrix-dictionary approximation SDPs, is highly effective for capturing the forms of approximation required by (1). All our aforementioned preconditioning results follow via careful applications of our matrix-dictionary approximation SDP solver in Theorem 6 (and a generalization of it in Theorem 7).
Specifically, we develop efficient algorithms for the following main meta-problem we study. Given a set of matrices (a "matrix-dictionary")  $\{\mathbf{M}_i\}_{i\in [n]}\in \mathbb{S}_{\succeq 0}^d$ , a constraint matrix  $\mathbf{B}\in \mathbb{S}_{\succ 0}^{d}$ , a tolerance parameter  $\epsilon \in (0,1)$ , and  $\kappa^{\star}\geq 1$  such that there exists  $w^{\star}\in \mathbb{R}_{\geq 0}^{n}$  with
$$
\mathbf {B} \preceq \sum_ {i \in [ n ]} w _ {i} ^ {\star} \mathbf {M} _ {i} \preceq \kappa^ {\star} \mathbf {B}, \tag {3}
$$
the goal of matrix-dictionary approximation is to return weights  $w \in \mathbb{R}_{\geq 0}^{n}$  such that
$$
\mathbf {B} \preceq \sum_ {i \in [ n ]} w _ {i} \mathbf {M} _ {i} \preceq (1 + \epsilon) \kappa^ {\star} \mathbf {B}. \tag {4}
$$
When  $\mathbf{B} = \mathbf{I}$ , the problem in (3), (4) is a special case of (2) where each  $\mathbf{M}_i = \mathbf{C}_i = \kappa^{\star}\mathbf{P}_i$ ; we call this the isotropic case. We further handle general  $\mathbf{B}$ , and demonstrate that our formulation captures many interesting applications. We refer to the problem in (3), (4) as matrix-dictionary recovery.
Our results: matrix-dictionary recovery. Our results concerning (3) and (4) assume that the matrix-dictionary  $\{\mathbf{M}_i\}_{i\in [n]}$  is "simple" in two respects. First, we assume that we have explicit factorizations
$$
\mathbf {M} _ {i} = \mathbf {V} _ {i} \mathbf {V} _ {i} ^ {\top}, \mathbf {V} _ {i} \in \mathbb {R} ^ {d \times m}. \tag {5}
$$
Our applications in Sections 2 and 3 satisfy this assumption with  $m = 1$ . Second, denoting  $\mathcal{M}(w) \coloneqq \sum_{i \in [n]} w_i \mathbf{M}_i$ , we assume we can approximately solve systems in  $\mathcal{M}(w) + \lambda \mathbf{I}$  for any  $w \in \mathbb{R}_{\geq 0}^n$  and  $\lambda \geq 0$ . Concretely, for any  $\epsilon > 0$ , we assume there is a linear operator  $\widetilde{\mathcal{M}}_{w,\lambda,\epsilon}$  which we can compute and apply in  $\mathcal{T}_{\mathcal{M}}^{\mathrm{sol}} \cdot \log \frac{1}{\epsilon}$  time, and that  $\widetilde{\mathcal{M}}_{w,\lambda,\epsilon} \approx (\mathcal{M}(w) + \lambda \mathbf{I})^{-1}$  in that:
$$
\left\| \widetilde {\mathcal {M}} _ {w, \lambda , \epsilon} v - (\mathcal {M} (w) + \lambda \mathbf {I}) ^ {- 1} v \right\| _ {2} \leq \epsilon \left\| (\mathcal {M} (w) + \lambda \mathbf {I}) ^ {- 1} v \right\| _ {2} \text {f o r a l l} v \in \mathbb {R} ^ {d}. \tag {6}
$$
In this case, we say "we can solve in  $\mathcal{M}$  to  $\epsilon$ -relative accuracy in  $\mathcal{T}_{\mathcal{M}}^{\mathrm{sol}} \cdot \log \frac{1}{\epsilon}$  time." If  $\mathcal{M}$  is a single matrix  $\mathbf{M}$ , we say "we can solve in  $\mathbf{M}$  to  $\epsilon$ -relative accuracy in  $\mathcal{T}_{\mathbf{M}}^{\mathrm{sol}} \cdot \log \frac{1}{\epsilon}$  time." Notably, for the matrix-dictionaries in our applications, e.g., diagonal 1-sparse matrices or edge Laplacians, such access to  $\{\mathbf{M}_i\}_{i \in [n]}$  exists so we obtain end-to-end efficient algorithms. Ideally (for near-linear time algorithms),  $\mathcal{T}_{\mathbf{M}}^{\mathrm{sol}}$  is roughly the total sparsity of  $\{\mathbf{M}_i\}_{i \in [n]}$ , which holds in all our applications.
Under these assumptions, we give the following novel meta-solvers for matrix-dictionary recovery. $^{13}$
Theorem 6 (Matrix dictionary recovery, isotropic case). Given matrices  $\{\mathbf{M}_i\}_{i\in [n]}$  with explicit factorizations (5), such that (3) is feasible for  $\mathbf{B} = \mathbf{I}$  and some  $\kappa^{\star}\geq 1$ , we can return weights  $w\in \mathbb{R}_{\geq 0}^{n}$  satisfying (4) with probability  $\geq 1 - \delta$  in time
$$
O \left(\mathcal {T} _ {\mathrm {m v}} \left(\left\{\mathbf {V} _ {i} \right\} _ {i \in [ n ]}\right) \cdot \left(\kappa^ {\star}\right) ^ {1. 5} \cdot \operatorname {p o l y} \left(\frac {\log \frac {m n d \kappa^ {\star}}{\delta}}{\epsilon}\right)\right).
$$
Here  $\mathcal{T}_{\mathrm{mv}}(\{\mathbf{V}_i\}_{i\in [n]})$  denotes the computational complexity of multiplying an arbitrary vector by all matrices in  $\{\mathbf{V}_i\}_{i\in [n]}$ . Notably, in the isotropic case  $\mathbf{B} = \mathbf{I}$ , Theorem 6 does not require solvers in the sense of (6). We next state our solver which handles the case of general  $\mathbf{B}$ , under access to (6).
<sup>12</sup>We use this notation because, if  $\mathcal{T}_{\mathcal{M}}^{\mathrm{sol}}$  is the complexity of solving the system to constant error  $c < 1$ , then we can use an iterative refinement procedure to solve the system to accuracy  $\epsilon$  in time  $\mathcal{T}_{\mathcal{M}}^{\mathrm{sol}} \cdot \log \frac{1}{\epsilon}$  for any  $\epsilon > 0$ .
<sup>13</sup>We did not heavily optimize logarithmic factors and  $\epsilon^{-1}$  dependences; for many applications (notably Theorems 1, 2),  $\epsilon$  is a constant, so our runtimes are near-linear for a natural representation of the problem under the assumption (5). In several applications (e.g., Theorems 1 and 2) the most important parameter is the "relative condition number"  $\kappa^{\star}$  in the promise (3), so we primarily optimized for the dependence on  $\kappa^{\star}$ .
Theorem 7 (Matrix dictionary recovery, general case). Given matrices  $\{\mathbf{M}_i\}_{i\in [n]}$  with explicit factorizations (5), such that (3) is feasible for some  $\kappa^{\star} \geq 1$  and we can solve in  $\mathcal{M}$  to  $\epsilon$  relative accuracy in  $\mathcal{T}_{\mathcal{M}}^{\mathrm{sol}} \cdot \log \frac{1}{\epsilon}$  time, and  $\mathbf{B}$  satisfying
$$
\mathbf {B} \preceq \mathcal {M} (\mathbb {1}) \preceq \alpha \mathbf {B} a n d \mathbf {I} \preceq \mathbf {B} \preceq \beta \mathbf {I}, \tag {7}
$$
we can return weights  $w \in \mathbb{R}_{\geq 0}^{n}$  satisfying (4) with probability  $\geq 1 - \delta$  in time
$$
O \left(\mathcal {T} _ {\mathrm {t o t}} \cdot (\kappa^ {\star}) ^ {2} \cdot \operatorname {p o l y} \left(\frac {\log \frac {m n d \kappa^ {\star} \alpha \beta}{\delta}}{\epsilon}\right)\right), w h e r e \mathcal {T} _ {\mathrm {t o t}} := \mathcal {T} _ {\mathrm {m v}} \left(\left\{\mathbf {V} _ {i} \right\} _ {i \in [ n ]} \cup \left\{\mathbf {B} \right\}\right) + \mathcal {T} _ {\mathcal {M}} ^ {\mathrm {s o l}}.
$$
The first condition in (7) is no more general than assuming we have a "warm start" reweighting  $w_0 \in \mathbb{R}_{\geq 0}^n$  (not necessarily  $\mathbb{1}$ ) satisfying  $\mathbf{B} \preceq \sum_{i \in [n]} [w_0]_i \mathbf{M}_i \preceq \alpha \mathbf{B}$ , by exploiting scale invariance of the problem and setting  $\mathbf{M}_i \gets [w_0]_i \mathbf{M}_i$ . The second bound in (7) is equivalent to  $\kappa(\mathbf{B}) \leq \beta$  up to constant factors, since given a bound  $\beta$ , we can use the power method to shift the scale of  $\mathbf{B}$  so it is spectrally larger than  $\mathbf{I}$ . The operation requires just a logarithmic number of matrix vector multiplications with  $\mathbf{B}$ , which does not impact the runtime in Theorem 7.
Proof sketches of Theorems 6 and 7. We defer full proofs of Theorems 6 and 7 to Section 3 of the supplement, but overview our techniques here. Our main workhorse is the following Algorithm 1, which solves a decision variant of the isotropic matrix-dictionary recovery problem (i.e.,  $\mathbf{B} = \mathbf{I}$ ), leveraging any subroutine  $\mathcal{A}_{\mathrm{pack}}$  for solving pure packing instances of (2) from the literature. $^{14}$
We define our approximation notions (used in Lines 6 and 10 of Algorithm 1) in Section 2 of the supplement. In Section 3.1 of the supplement, we analyze correctness of Algorithm 1 using regret bounds for the matrix multiplicative weights framework [ZLO15], which Lines 4-12 are an instance of, and demonstrate tolerance to the stated approximations. Our proof shows that Algorithm 1 meets its output guarantees, which directly implies a solver for the matrix-dictionary recovery problem (3), (4) in the isotropic case  $\mathbf{B} = \mathbf{I}$ , provided we can efficiently implement the algorithm's steps. $^{15}$
By carefully combining polynomial approximations to the exponential, Johnson-Lindenstrauss sketches, and the power method, we obtain the needed approximations in Lines 6 and 10 of Algorithm 1, which combined with our correctness proof yields Theorem 6. Our proof of Theorem 7 in Section 3.2 of the supplement builds upon Theorem 6 and recursive preconditioning, based on the observation that if we could efficiently apply  $\mathbf{B}^{-\frac{1}{2}}$ , setting  $\mathbf{M}_i \gets \mathbf{B}^{-\frac{1}{2}}\mathbf{M}_i\mathbf{B}^{-\frac{1}{2}}$  reduces to the isotropic case. We show that at a  $\sqrt{\kappa^{\star}}$  overhead, we can use (6) to efficiently simulate  $(\mathbf{B} + \lambda \mathbf{I})^{-\frac{1}{2}}$  for  $\lambda$  values which are recursively halved, allowing use of Theorem 6 for each recursive call. This general type of adaptive regularization strategy, which we term a homotopy method, is reminiscent of techniques used by other recent works in the literature on numerical linear algebra and structured continuous optimization, such as [LMP13, KLM+14, BCLL18, AKPS19].
Preconditioning applications. Formal proofs of our applications in Section 2 are given in Section 5 of the supplement. Theorem 2 follows immediately from Theorem 6 with the dictionary  $\mathbf{M}_i = a_i a_i^\top$ , where  $\{a_i\}_{i \in [n]}$  are rows of  $\mathbf{A}$ , which satisfies (5) with  $m = 1$ . Specifically, note for  $w \in \mathbb{R}_{\geq 0}^n$ ,
$$
\mathcal {M} (w) = \sum_ {i \in [ n ]} w _ {i} \mathbf {M} _ {i} = \sum_ {i \in [ n ]} w _ {i} a _ {i} a _ {i} ^ {\top} = \mathbf {A} ^ {\top} \mathbf {W A}.
$$
A result analogous to Theorem 1, but depending quadratically on  $\kappa_o^\star (\mathbf{K})$ , follows from applying Theorem 7 with  $n = d$ ,  $\mathbf{M}_i = e_i e_i^\top$ ,  $\kappa = \kappa_o^\star (\mathbf{K})$ , and  $\mathbf{B} = \frac{1}{\kappa}\mathbf{K}$  (i.e., using the dictionary of 1-sparse diagonal matrices to approximate  $\mathbf{K}$ , which satisfies (5) with  $m = 1$  and (6) with  $\mathcal{T}_{\mathcal{M}}^{\mathrm{sol}} = d$ ). Compared to inner scaling, this application exchanges the role of the dictionary and the constraint matrix. Theorem 1 goes beyond this black-box use via a homotopy method for simulating access to matrix square roots (to reduce to the isotropic case), yielding an improved  $(\kappa_o^\star (\mathbf{K}))^{1.5}$  dependence.
<sup>14</sup>For simplicity, we assume each matrix dictionary element's top eigenvalue is in a bounded range. We explicitly bound the cost of achieving this in applications (via rescaling by a constant-factor approximation to the top eigenvalue of each matrix using the power method, see Fact 3 of the supplement), and this does not dominate the runtime. The runtime bottleneck in all our applications is the cost of approximate packing SDP oracles in Line 7; this is an active research area and improvements therein would also reflect in our algorithm's runtime.
<sup>15</sup>We show how to search for  $\kappa^{\star}$  in (3) using an incremental search over a small number of calls to Algorithm 1.
Algorithm 1: DecideStructuredMPC( $\{\mathbf{M}_i\}_{i \in [n]}, \kappa, \mathcal{A}_{\mathrm{pack}}, \delta, \epsilon$ )
1. Input:  $\{\mathbf{M}_i\}_{i\in [n]}\in \mathbb{S}_{\preceq \mathbf{0}}^d$  with  $\lambda_{\mathrm{max}}(\mathbf{M}_i)\in [1,2]$  for all  $i\in [n],\kappa >1,\delta \in (0,1),\epsilon \in (0,1)$ ,  $\mathcal{A}_{\mathrm{pack}}$  which on input  $v\in \mathbb{R}_{\geq 0}^{n}$  returns  $w\in \mathbb{R}_{\geq 0}^{n}$  with  $\mathcal{M}(w)\preceq \mathbf{I}$  that  $\frac{\epsilon}{10}$ -multiplicatively approximates  $\max_{w:\mathcal{M}(w)\preceq \mathbf{I}}v^{\top}w$  with probability  $1 - \frac{\delta}{2T}$ ,  $T = O(\frac{\kappa\log d}{\epsilon^2})$ .
2 Output: With probability  $\geq 1 - \delta$ : "yes" or "no" is returned. If there exists  $w \in \mathbb{R}_{\geq 0}^{n}$  with
$$
\lambda_ {\max } \left(\sum_ {i \in [ n ]} w _ {i} \mathbf {M} _ {i}\right) \leq (1 - \epsilon) \kappa \lambda_ {\min } \left(\sum_ {i \in [ n ]} w _ {i} \mathbf {M} _ {i}\right), \tag {8}
$$
the algorithm must return "yes." If "yes" is returned, a vector  $w$  is given with
$$
\lambda_ {\max } \left(\sum_ {i \in [ n ]} w _ {i} \mathbf {M} _ {i}\right) \leq (1 + \epsilon) \kappa \lambda_ {\min } \left(\sum_ {i \in [ n ]} w _ {i} \mathbf {M} _ {i}\right). \tag {9}
$$
3  $\eta \leftarrow \frac{\epsilon}{10\kappa}, T \leftarrow \left\lceil \frac{10\log d}{\eta\epsilon} \right\rceil, \mathbf{Y}_0 \leftarrow \frac{1}{d}\mathbf{I}, \mathbf{S}_0 \leftarrow \mathbf{0}$
4 for  $0 < t < T$  do
5  $\mathbf{Y}_t\gets \frac{\exp(\mathbf{S}_t)}{\mathrm{Tr}\exp(\mathbf{S}_t)}$  6  $v_{t}\gets$  entrywise  $(\frac{\epsilon}{10},\frac{\epsilon}{10\kappa n})$  -approximations to  $\{\langle \mathbf{M}_i,\mathbf{Y}_t\rangle \}_{i\in [n]}$  , with probability  $\geq 1 - \frac{\delta}{4T}$  7  $x_{t}\gets \mathcal{A}_{\mathrm{pack}}(\kappa v_{t})$
8 if  $\kappa \langle x_t,v_t\rangle < 1 - \frac{\epsilon}{5}$  then return "no"
9  $\mathbf{S}_{t + 1}\gets \mathbf{S}_t - \eta \kappa \mathcal{M}(x_t)$
10  $\tau \leftarrow \frac{\log d}{\epsilon}$  -additive approximation to  $\lambda_{\mathrm{min}}(-\mathbf{S}_{t + 1})$  , with probability  $\geq 1 - \frac{\delta}{4T}$
11 if  $\tau \geq \frac{12\log d}{\epsilon}$  then return ("yes",  $\bar{x}$ ) for  $\bar{x} := \frac{1}{t + 1}\sum_{0\leq s\leq t}x_s$
12 end
13 return ("yes",  $\bar{x}$ ) for  $\bar{x} := \frac{1}{T} \sum_{0 \leq t < T} x_t$
Our applications in Section 3 are deferred to Section 4 of the supplement. Robust linear system solvers for perturbed variants of structured matrix families follow directly from Theorem 7, taking the matrix dictionary to be a suitable basis for the relevant non-perturbed structured family, which naturally satisfy (6). As an example, to prove Theorem 3, we use Theorem 7 with the dictionary of all  $b_{e}b_{e}^{\top}$  for edges  $e$  of a complete graph, where  $b_{e}$  is the associated incidence vector; the access (6) is then afforded by known Laplacian solvers. Finally, Theorems 4 and 5 follow by combining Theorems 6, 7 with homotopy techniques, alongside structural facts about M-matrices and Laplacians.
Further work. A natural open question is if, e.g., for outer scaling, the  $\kappa_{o}^{\star}(\mathbf{K})$  dependence in Theorem 1 can be reduced further, ideally to  $\sqrt{\kappa_o^\star(\mathbf{K})}$ . This would match the most efficient solvers in  $\mathbf{K}$  under diagonal rescaling, if the best known outer scaling was known in advance. Towards this goal, we prove in Appendix D of the supplement that if a width-independent variant of Theorem 6 is developed, it can achieve such improved runtimes for Theorem 1 (with an analogous improvement for Theorem 2). We also give generalizations of this improvement to finding rescalings which minimize natural average notions of conditioning, under existence of such a conjectured solver.
5 Additional related work
Matrix-dictionary recovery. Our algorithm for Theorem 6 is based on matrix multiplicative weights [WK06, AK07, AHK12], a popular meta-algorithm for approximately solving SDPs, with carefully chosen gain matrices formed by using packing SDP solvers as a black box. In this sense, it is an efficient reduction from structured SDP instances of the form (3), (4) to pure packing instances.