In this paper we develop algorithms for approximating matrix multiplication with respect to the spectral norm. Let A ∈ R n×m and B ∈ R n×p be two matrices and ε > 0. We approximate the product A B using two sketches e A ∈ R t×m and e with high probability. We analyze two different sampling procedures for constructing e A and e B; one of them is done by i.i.d. non-uniform sampling rows from A and B and the other by taking random linear combinations of their rows. We prove bounds on t that depend only on the intrinsic dimensionality of A and B, that is their rank and their stable rank. For achieving bounds that depend on rank when taking random linear combinations we employ standard tools from high-dimensional geometry such as concentration of measure arguments combined with elaborate ε-net constructions. For bounds that depend on the smaller parameter of stable rank this technology itself seems weak. However, we show that in combination with a simple truncation argument it is amenable to provide such bounds. To handle similar bounds for row sampling, we develop a novel matrix-valued Chernoff bound inequality which we call low rank matrixvalued Chernoff bound. Thanks to this inequality, we are able to give bounds that depend only on the stable rank of the input matrices. We highlight the usefulness of our approximate matrix multiplication bounds by supplying two applications. First we give an approximation algorithm for the 2-regression problem that returns an approximate solution by randomly projecting the initial problem to dimensions linear on the rank of the constraint matrix. Second we give improved approximation algorithms for the low rank matrix approximation problem with respect to the spectral norm.
This dependence on rank is later improved to stable rank in REF , but at the cost of a weaker dependence on ǫ.
Low rank matrix-valued chernoff bounds and approximate matrix multiplication
SODA '11
Computer Science Mathematics