Abstract. The CANDECOMP/PARAFAC (CP) decomposition is a leading method for the analysis of multiway data. The standard alternating least squares algorithm for the CP decomposition (CP-ALS) involves a series of highly overdetermined linear least squares problems. We extend randomized least squares methods to tensors and show the workload of CP-ALS can be drastically reduced without a sacrifice in quality. We introduce techniques for efficiently preprocessing, sampling, and computing randomized least squares on a dense tensor of arbitrary order, as well as an efficient sampling-based technique for checking the stopping condition. We also show more generally that the Khatri-Rao product (used within the CP-ALS iteration) produces conditions favorable for direct sampling. In numerical results, we see improvements in speed, reductions in memory requirements, and robustness with respect to initialization. Key words. canonical polyadic tensor decomposition, CANDECOMP/PARAFAC (CP), multilinear algebra, randomized algorithms, randomized least squares AMS subject classifications. 15A69, 68W20 , and many others. We consider the problem of accelerating the alternating least squares (CP-ALS) algorithm using randomization. Because randomized methods have been used successfully for solving linear least squares problems [16, 3, 42] , it is natural that they might prove beneficial to CP-ALS since its key kernel is the solution of a least squares problem. However, the CP-ALS least squares subproblem has a special structure that already greatly reduces its cost, so it is unclear whether or not sketching would be beneficial. Nevertheless, we find that our randomized algorithms significantly reduce the memory and computational overhead of the CP-ALS process for dense tensors and moreover positively impact algorithmic robustness. To the best of our knowledge, this is the first successful application of matrix sketching methods in the context of CP. The contributions of this paper are as follows: • The least squares coefficient matrix in the CP-ALS subproblem is a KhatriRao product of factor matrices. Our randomized algorithm prefers incoherent matrices. We prove that the coherence of the Khatri-Rao product is bounded above by the product of the coherence of its factors. • We introduce the CPRAND algorithm that uses a randomized least squares solver for the subproblems in CP-ALS and never explicitly forms the full Khatri-Rao matrices used in the subproblems. We also introduce the complementary CPRAND-MIX algorithm that employs efficient mixing to promote incoherence and thereby improves the robustness of the method.
A randomized least squares algorithm for CP decomposition is proposed by REF , which is significantly faster than traditional CP decomposition.
A Practical Randomized CP Tensor Decomposition
SIAM Journal on Matrix Analysis and Applications, Vol. 39, No. 2, pp. 876-901, 2018
Mathematics Computer Science