Title: The Structural Complexity of Matrix-Vector Multiplication
Venue: NeurIPS 2025
Accepted: yes
Keywords: data structures

VC-dimension

matrix-vector multiplication

matrix multiplication

dynamic high-accuracy algorithms
Abstract: We consider the problem of preprocessing an $n\times n$ matrix $\mathbf{M}$, and supporting queries that, for any vector $v$, returns the matrix-vector product $\mathbf{M} v$. This problem has been extensively studied in both theory and practice: on one side, practitioners have developed algorithms that are highly efficient in practice, whereas on the other side, theoreticians have proven that the problem cannot be solved faster than naive multiplication in the worst-case. This lower bound holds even in the average-case, implying that existing average-case analyses cannot explain this gap between theory and practice. Hence, we study the problem for \emph{structured} matrices. 
We show that for $n\times n$ Boolean matrices of VC-dimension $d$, the matrix-vector multiplication problem can be solved with $\smash{\tilde{O}(n^2)}$ preprocessing and $\smash{\tilde O(n^{2-1/d})}$ query time.  Given the low constant VC-dimensions observed in most real-world data, our results posit an explanation for why the problem can be solved so much faster in practice. Furthermore, we show how to extend this result to the non-Boolean setting with the Pollard pseudodimension. 

Our results yield the first non-trivial upper bounds for many applications.
In previous works, the online matrix-vector (OMv) hypothesis (conjecturing that quadratic time is needed per query, even over the boolean semi-ring) was used to prove many conditional lower bounds, showing that it is impossible to compute and maintain high-accuracy estimates for effective resistance, Laplacian solvers, shortest paths, and triangle detection in graphs subject to node insertions and deletions in subquadratic time.
Yet, via a reduction to our matrix-vector-multiplication result, we show we can maintain these problems efficiently if the input is structured, providing the first subquadratic upper bounds in the high-accuracy regime.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper shows that for n x n Boolean matrices with VC dimension d (where the function class is defined by considering each row of the matrix as the evaluation table of a function), there is an O(n^{2-1/d}) time-per-query algorithm for online matrix-vector multiplication with preprocessing. It also presents various applications and extensions of this result.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths.** The result gives a novel perspective on beyond-worst-case online matrix vector multiplication: VC dimension bounds are a plausibly natural assumption for real data, and this paper shows that they enable avoiding worst-case quadratic lower bounds. The basic result is an almost direct corollary of combining a (quite practical) algorithm due to (Bjorklund and Lingas, 2001) with an old (and to me unknown) lemma from the computational geometry literature showing that for any function class with bounded VC dimension, there are two points on which \"most\" of the functions agree (Welzl, 1988). Nevertheless making the connection between these two branches of literature is  a contribution in its own right, and the authors put substantial technical effort into extensions and applications of the basic result.\n\n**Weaknesses.** I think there is a (fairly minor) gap in the current proof of the basic result: Welzl's Lemma 4.1 hides some \"constants\" that, I think, actually depend on the VC dimension bound d. The statement of Lemma 4.1 says \"c is a constant depending on (X,R)\", which is completely vacuous since (X,R) is the entire problem instance. From inspecting the proof, my impression is that c ~ O(d) suffices, but this should be checked and perhaps a self-contained modern proof can be included in the paper for completeness. At any rate, a factor of d is not really important since the result already requires d << log n for it to be non-trivial.\n\nAlso, I think Definitions 3.8 and 3.10 are wrong: in 3.8, the symmetric difference between two sets is non-empty iff the sets are non-equal. The second formulation of the definition of crossing is I think the correct one, but it's not the same as the first formulation. In 3.10, I think the crossing number should be \"sum over edges of max over ranges of indicator that the range crosses the edge\", not \"max over ranges of sum over edges\". I believe the correct version is used for the proofs though, so this seems to just be a typo.\n\nI will increase my score if the authors can confirm that these are not serious issues.\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the problem of preprocessing a Boolean $m \\times n$ matrix $M$ to support fast matrix-vector product (MVP). The main result of this paper is that MVP can be supported in time $mn^{1 - 1/d}$ with $m n$ preprocessing time, where $d$ is the VC dimension of $M$.\nAs a bonus, this result extends to matrices that are \"close\" to the set $\\mathcal N_d$ of matrices with VC dimension $\\leq d$. Namely, $\\min_{N \\in \\mathcal N_d} \\max_i||M_i - N_i||_\\infty \\leq n^{1 - 1/d}$ suffice (denote with $A_i$ the $i$-th column of $A$).\n\nSince MVP is a fundamental problem, the complexity bound above propagates to several other problems.\n\nSTRENGTHS AND WEAKNESSES:\nThis work is, to the best of my knowledge, the first to parameterize the complexity of MVP with respect to the VC dimension of a the matrix. The techniques employed to design the data structure achieving the desired time bound are the following.\n\nFirst, the authors define a spanning tree for a matrix $M$ as a spanning tree of the weighted complete graph over $[n] \\times [n]$, where $w(i, j) = ||M_i - M_j||_1$. Then, they prove a structural lemma showing that any matrix $M$ with VC dimension $d$ admits a spanning tree of weight at most $m n^{1-1/d}$. Finally, they observe that a spanning tree $T$ for $M$ can be leveraged to compute MVPs with $M$ in time $w(T)$.\nTo turn these observations into an algorithm, they use a known algorithm from Indyk and Motwani that computes a $O(\\log n)$-approximate minimum spanning tree over hamming metric in  $\\tilde O(mn)$ time.\n\nThis paper introduces a novel parameterization for the MVP problem and derives novel bounds. I believe that the main result of this work is interesting and certainly worth observing. Nonetheless, I recommend rejection due to the lack of technical novelty. \n\nThe technical novelty of this work is minimal. This work's contribution is a series of simple observation that I believe could be expressed in a four-page note, without compromising clarity.\nLet me briefly evaluate the novelty (or lack thereof) of each of this work's main propositions.\n\nTheorem 1.1 follows from the simple observation that each $n \\times n$ matrix is the sub-matrix of any matrix with VC dimension at least $n$.\n\nLemma 3.4 can be proven through simple algebraic manipulations of the definition of MVP.\n\nLemma 3.12, which proves the main structural result of this work, follows by applying Lemma 4.1 of Welzl [1988] greedily.\n\nLemma 3.13 follows from the classic metric TSP analysis because crossing numbers are a metric.\n\nFinally, and perhaps less importantly, the quality of writing is low. This paper could be much shorter, and thus more readable, if the authors did not add a long list containing several of the obvious consequences of a faster MVP algorithm. Moreover, the density of typos made it hard to understand at first sight. A list of writing mistakes and typos follows.\n\nMISTAKES and TYPOS:\nIn the abstract, you do not say that your result holds for the class of Boolean matrices.\n\nLine 78, \"rows in S\" is unclear. I guess you mean the sub-matrix which columns are restricted to those in S.\n\nLine 79 - 84, you define VC dimension for matrices and then talk about hypothesis classes. A reader who's not familiare with learning theory would get lost here.\n\nLine 90 - 91. I do not understand what you mean here. \n\nDefinition 1.2. I imagine that we're taking the minimum over such d.\n\nLine 150 - 155. What is the threshold of a matrix?\n\nLine 258. Regime -> paradigm.\n\nDefinition 3.8. I imagine the definition that you wanted to give is that A crosses r if $A \\cap r, A \\Delta r \\neq \\emptyset$\n\nLine 368. \\max -> \\Sigma\n\nQUESTIONS:\nYou claim that your work bridges the gap between MVP heuristics used in practice and theoretical lower bounds. It would be interesting if you could further elaborate on why real-world MVP involve low-VC-dimensional matrices. Could you provide examples of successful heurstics which behavior is exampled by low VC dimension?\nAt line 83 you claim that \"real-world data has low VC dimension\": please elaborate more.\n\nAs written above, this work's results could be compressed into an extremely short note, and I believe that the community would benefit the most if this work was published (perhaps on Arxiv) as a coincise note.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper provides a theoretical explanation on the speed up observed in some recent practical algorithms for matrix-vector multiplication through preprocessing of matrices. In summary, the paper proves that for  an n × n matrix M of VC-dimension d, the matrix-vector product Mv can be computed in O(n^(2−1/d)) time for any vector v, after preprocessing in O(n^2) time. This is in contrast to the theoretical bounds showing that matrix-vector multiplication over sufficiently large fields require \\Omega(n^2/ log n) time, where the authors claim that the speed up in practice comes thanks to the structure present, measured through the low VC dimension.\n\nSTRENGTHS AND WEAKNESSES:\nOverall, this is an interesting theoretical paper that explains an important practical phenomenon. The paper is well written. It motivates the problem well with a very comprehensive background on the literature. This is far from my field of study, so I was not able to follow the proofs, but I was able to appreciate the solution steps and the logic behind the arguments. \n\nOn the other hand, the main weakness of the paper, in my opinion, is whether it is a good match for NeurIPS. I would have expected this result to appear in a more theoretically inclined venue. \n\nFrom a more technical point of view: the solution mainly provides a (partial)  theoretical explanation of the speed up observed in practice, but does not provide any insights or guidance to come up with more efficient algorithms based on the VC dimension of data.\n\nQUESTIONS:\n- The authors argue that most real data has “low constant VC-dimensions”. Can you please better qualify this claim? Can you provide examples? A better example would be to show (in practice) an increase in the speed up with a decrease in the VC dimension of the data. \n\nMinor issues: \n\n- Line 248: semiring -> semi-ring\n\n- Line 258: Can you better explain what you mean by “ algorithms with predictions regime” ? \n\n- Line 261: “ Our dynamic algorithms” -> which algorithms does this refer to?\n\nLIMITATIONS:\nYes. The work is purely theoretical.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 4, "DATE": "2025-07-02"}

comments: SUMMARY:
The paper considered the problem of preprocessing a $m\times n$ 0/1 matrix $M$, such that matrix vector queries on $M$ can be efficiently implemented. The main takeaway from this work is that if the VC-dimension of the rows of $M$ is at most $d$ then matrix vector queries can be implemented in time complexity $mn^{1-1/d}$. They have also considered both static and dynamic online variants of this problem. The paper as applications of the above result give efficient algorithms for the following problems 

1. Dynamic Laplacian solver

2. Dynamic effective resistance of graphs

3. Dynamic triangle detection

4. Dynamic approximate single-source shortest paths

The authors fundamentally use ideas from computational geometry. Specifically, they use trees of low crossing number for set systems with low VC dimension, a fundamental result in both computational and combinatorial geometry.

STRENGTHS AND WEAKNESSES:
Strengths

1. The paper is well written and well motivated. 

2. The overall presentation is good. 

3. It introduces ideas from computational geometry to address a matrix problem in a novel way. 

4. It gives multiple important applications of their central result.

There are no obvious weaknesses in this paper.

QUESTIONS:
None

LIMITATIONS:
None
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-08