Title: Approximate Multi-Matrix Multiplication for Streaming Power Iteration Clustering
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Power Iteration, Clustering, Streaming, Random Projection

Abstract: In this paper,  we study the largest-cluster recovery by clustering the rows of a matrix obtained from approximate power iteration in the symmetric stochastic block model (SSBM). Specifically, we analyze the behavior of \$B^r\$, where \$B = A - q\mathbf{1}\mathbf{1}^\top\$ is a shifted version of the adjacency matrix \$A \in \mathbb{R}^{n\times n}\$ of a graph sampled from the SBM. We prove that, with high probability, interleaving powers of \$B\$ with oblivious sketches (e.g., Johnson–Lindenstrauss transforms) of dimension depending on the stable rank $\mathrm{sr}(B)=\frac{\|B\|_F^2}{\|B\|_2^2}$ preserves essentially the same row-distance threshold (“separation scale”) as the unsketched power method (Mukherjee et al., 2024). Notably, this algorithm admits cluster-recovery guarantees using a sublinear embedding obtainable in a single pass over a dynamic stream with edge insertions and deletions; in this streaming setting, the embedding dimension is $ m=\tilde{O}\big(\mathrm{sr}(B)\varepsilon^{-2}\big)$. We also derive a high-probability bound on \$\mathrm{sr}(B)\$ that holds for unbalanced communities. We further present an end-sketch variant: if one can store \$B\$ and iteratively compute \$B^r S\$ using a single sketch \$S\$ in \$r\$ passes over \$B\$, then the clustering threshold is preserved with embedding dimension $m=O(\varepsilon^{-2}\log n).$ In both cases, the embedding dimension is sublinear in \$n\$. Our main technical contribution is a streaming algorithm that propagates operator-norm approximate matrix multiplication (AMM) to products of multiple conforming matrices—an “approximate multi-matrix multiplication” (AMMM). Finally, experiments comparing the streaming and iterative variants show similar empirical recovery, with the streaming method typically requiring a somewhat larger embedding dimension.

TL;DR: Developed a streaming friendly version of Power Iteration Operator for the purpose of clustering