Singular Value Decomposition (SVD) plays a pivotal role in exploratory data analysis. However, in a Big Data setting computing the dominant singular vectors is often restrictive due to the main memory requirements imposed by the dataset . Recently introduced randomized projection schemes attempt to mitigate this memory load by constructing approximate projections of the true dataset in a streaming setting. However, these projection methods come at the cost of approximation errors in both top singular values and vectors . Furthermore, in order to bound the approximation error, an over-sampled projection is required, often much larger in dimension than the desired rank. This latter consideration can still be memory intensive when the data dimension is large or extraneous when the desired rank approximation is close to the full rank. We present a two stage neural optimization approach as an alternative to conventional and randomized SVD techniques , where the  memory requirement depends explicitly on the feature dimension and desired rank, independent of the sample size. The proposed scheme reads data samples in a streaming setting with the network minimization problem converging to a low rank approximationwith high precision. Our architecture is fully interpretable where all the network outputs and weights have a specific meaning. We evaluate our results on various performance metrics against state of the art streaming methods. We also present numerical experiments for Singular and Eigen value decomposition on real data at various scales to show the memory efficiency of our proposed approach .
In a Big Data setting computing the dominant SVD factors is restrictive due to the main memory requirements . Recently introduced streaming Randomized SVD schemes work under the restrictive assumption that the singular value spectrum of the data has exponential decay. This is seldom true for any practical data. Although these methods are claimed to be applicable to scientific computations due to associated tail-energy error bounds, the approximation errors in the singular vectors and values are high when the aforementioned assumption does not hold. Furthermore from a practical perspective, oversampling can still be memory intensive or worse can exceed the feature dimension of the data. To address these issues, we present Range-Net as an alternative to randomized SVD that satisfies the tail-energy lower bound given by Eckart-Young-Mirsky (EYM) theorem. Range-Net is a deterministic two-stage neural optimization approach with random initialization , where the main memory requirement depends explicitly on the feature dimension and desired rank, independent of the sample dimension. The data samples are read in a streaming setting with the network minimization problem converging to the desired rank-r approximation. Range-Net is fully interpretable where all the network outputs and weights have a specific meaning. We provide theoretical guarantees that Range-Net extracted SVD factors satisfy EYM tail-energy lower bound at machine precision. Our numerical experiments on real data at various scales confirms this bound. A comparison against the state of the art streaming Randomized SVD shows that Range-Net accuracy is better by six orders of magnitude while being memory efficient .
R
Singular Value Decomposition (SVD) plays a pivotal role in exploratory data analysis. However, in a
In a
R
singular vectors is often
SVD factors is
D
imposed by the dataset
R
randomized projection schemes attempt to mitigate this memory load by constructing approximate projections of the true dataset in a streaming setting. However, these projection methods come at the cost of
streaming Randomized SVD schemes work under the restrictive assumption that the singular value spectrum of the data has exponential decay. This is seldom true for any practical data. Although these methods are claimed to be applicable to scientific computations due to associated tail-energy error bounds, the
R
both top singular values and vectors . Furthermore, in order to bound the approximation error, an over-sampled projection is required, often much larger in dimension than the desired rank. This latter consideration
the singular vectors and values are high when the aforementioned assumption does not hold. Furthermore from a practical perspective, oversampling
R
when the data dimension is large or extraneous when the desired rank approximation is close to the full rank. We present a two stage neural optimization approach
or worse can exceed the feature dimension of the data. To address these issues, we present Range-Net
R
conventional and randomized SVD techniques
randomized SVD that satisfies the tail-energy lower bound given by Eckart-Young-Mirsky (EYM) theorem. Range-Net is a deterministic two-stage neural optimization approach with random initialization
A
main
R
size. The proposed scheme reads data samples
dimension. The data samples are read
R
a low rank approximationwith high precision. Our architecture
the desired rank-r approximation. Range-Net
R
evaluate our results on various performance metrics against state of the art streaming methods. We also present numerical experiments for Singular and Eigen value decomposition
provide theoretical guarantees that Range-Net extracted SVD factors satisfy EYM tail-energy lower bound at machine precision. Our numerical experiments
R
to show the memory efficiency of our proposed approach
confirms this bound. A comparison against the state of the art streaming Randomized SVD shows that Range-Net accuracy is better by six orders of magnitude while being memory efficient