NeurIPS 2025

Low-Precision Streaming PCA
Sanjoy Dasgupta
University of California San Diego
sadasgupta@ucsd.edu
Syamantak Kumar
University of Texas at Austin
syamantak@utexas.edu
Shourya Pandey
University of Texas at Austin
shouryap@utexas.edu
Purnamrita Sarkar
University of Texas at Austin
purna.sarkar@utexas.edu
Abstract
Low-precision Streaming PCA estimates the top principal component in a stream-
ing setting under limited precision. We establish an information-theoretic lower
bound on the quantization resolution required to achieve a target accuracy for the
leading eigenvector. We study Oja’s algorithm for streaming PCA under linear and
nonlinear stochastic quantization. The quantized variants use unbiased stochas-
tic quantization of the weight vector and the updates. Under mild moment and
spectral-gap assumptions on the data distribution, we show that a batched version
achieves the lower bound up to logarithmic factors under both schemes. This
leads to a nearly dimension-free quantization error in the nonlinear quantization
setting. Empirical evaluations on synthetic streams validate our theoretical findings
and demonstrate that our low-precision methods closely track the performance of
standard Oja’s algorithm.
1
Introduction
Quantization (or discretization) is the mapping of a continuous set of values to a small, finite set
of outputs close to the original values; standard methods for quantization include rounding and
truncation. The current popularity of training large-scale Machine Learning models has brought a
renewed focus on quantization, though its origins go back to the 1800s. Some early examples include
least-squares methods applied to large-scale data analysis in the early nineteenth century [Sti86].
In 1867, discretization was introduced for the approximate calculation of integrals [Rie67], and the
effects of rounding errors in integration were examined in 1897 [She97]. For an excellent survey and
history of quantization, see [GKD`22].
In the context of efficie...