SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 6: Tested Sample Counts for hybrid CP-ARLS-LEV
TENSOR	VALUES OF J TESTED
UBER	{215x | x ∈ {1..13}}
ENRON	{215x | x ∈ {1..7} ∪ {10, 12, 14, 16, 18, 20, 22, 26, 30, 34, 38, 42, 46, 50, 54}}
NELL-2	{215x | x ∈ {1..7}}
AMAZON	{215x | x ∈ {1..7}}
REDDIT	{215x | x ∈ {1..12}}
For each configuration of tensor, target rank  $R$ , sampling algorithm, and sample count  $J$ , we ran 4 trials using the configuration and stopping criteria in Appendix A.11. The result of each trial was a set of (time, fit) pairs. For each configuration, we linearly interpolated the pairs for each trial and averaged the resulting continuous functions over all trials. The result for each configuration was a function  $f_{\mathcal{T},R,A,J} : \mathbb{R}^+ \to [0,1]$ . The value  $f_{\mathcal{T},R,A,J}(t)$  is the average fit at time  $t$  achieved by algorithm  $A$  to decompose tensor  $\mathcal{T}$  with target rank  $R$  using  $J$  samples per least squares solve. Finally, let
$$
\operatorname {S p e e d u p} _ {\mathcal {T}, R, M} := \frac {\min  _ {J} \operatorname {a r g m i n} _ {t \geq 0} \left[ f _ {\mathcal {T} , R , \mathrm {C P - A R L S - L E V - H} , J} (t) > P \right]}{\min  _ {J} \operatorname {a r g m i n} _ {t \geq 0} \left[ f _ {\mathcal {T} , R , \mathrm {S T S - C P}, J} (t) > P \right]}
$$
be the speedup of STS-CP to over CP-ARLS-LEV (hybrid) to achieve a threshold fit  $P$  on tensor  $\mathcal{T}$  with target rank  $R$ . We let the threshold  $P$  for each tensor  $\mathcal{T}$  be a fixed fraction of the fit achieved by non-randomized ALS (see Table 5).
Figure 9 reports the speedup of STS-CP over hybrid CP-ARLS-LEV for  $P = 0.95$  on all tensors except Enron. For large tensors with over one billion nonzeros, we report a significant speedup anywhere from  $1.4\mathrm{x}$  to  $2.0\mathrm{x}$  for all tested ranks. For smaller tensors with less than 100 million nonzeros, the lower cost of each least squares solve lessens the impact of the expensive, more accurate sample selection phase of STS-CP. Despite this, STS-CP performs comparably to CP-ARLS-LEV at most ranks, with significant slowdown only at rank 25 on the smallest tensor Uber.
(a) Large Tensors
(b) Small Tensors
Figure 9: Speedup of STS-CP over CP-ARLS-LEV hybrid (LK) to reach  $95\%$  of the fit achieved by non-randomized ALS. Large tensors have more than 1 billion nonzero entries.
On the Enron tensor, hybrid CP-ARLS-LEV could not achieve the  $95\%$  accuracy threshold for any rank above 25 for the sample counts tested in Table 6. STS-CP achieved the threshold accuracy for all ranks tested. Instead, Figure 10 reports the speedup to achieve  $85\%$  of the fit of non-randomized ALS on the Enron. Beyond rank 25, our method consistently exhibits more than 2x speedup to reach the threshold.
Guide to Sampler Selection. Based on the performance comparisons in this section, we offer the following guide to CP decomposition algorithm selection. Our experiments demonstrate that STS-CP offers the most benefit on sparse tensors with billions of nonzeros (Amazon and Reddit) at high target decomposition rank. Here, the runtime of our more expensive sampling procedure is offset by reductions in the least squares solve time. For smaller tensors, our sampler may still offer significant performance benefits (Enron). In other cases (Uber, NELL-2), CP-ARLS-LEV exhibits better performance, but by small margins for rank beyond 50.
Figure 10: Speedup of STS-CP over CP-ARLS-LEV hybrid (LK) to reach  $85\%$  of the fit achieved by non-randomized ALS on the Enron Tensor.
STS-CP reduces the cost of each least squares solve through a sample selection process that relies on dense linear algebra primitives (see Algorithms 3 and 4). Because these operations can be expressed as standard BLAS calls and can be carried out in parallel (see Appendix A.10, we hypothesize that STS-CP is favorable when GPUs or other dense linear algebra accelerators are available.
Because our target tensor is sparse, the least squares solve during each ALS iteration requires a sparse matricized-tensor times Khatri-Rao product (spMTTKRP) operation. After sampling, this primitive can reduced to sparse-matrix dense-matrix multiplication (SpMM). Development of accelerators for these primitives is an active area of research [28, 26]. When such accelerators are available, the lower cost of the spMTTKRP operation reduces the relative benefit provided by the STS-CP sample selection method. We hypothesize that CP-ARLS-LEV, with its faster sample selection process but lower sample efficiency, may retain its benefit in this case. We leave verification of these two hypotheses as future work.