SGD<inline-formula><tex-math notation="LaTeX">$\_$</tex-math><alternatives><mml:math><mml:mo>_</mml:mo></mml:math><inline-graphic xlink:href="li-ieq1-3047460.gif"/></alternatives></inline-formula>Tucker: A Novel Stochastic Optimization Strategy for Parallel Sparse Tucker Decomposition

Sparse Tucker Decomposition (STD) algorithms learn a core tensor and a group of factor matrices to obtain an optimal low-rank representation feature for the <underline>H</underline>igh-<underline>O</underline>rder, <underline>H</underline>igh-<underline>D</underline>imension, and <underline>S</underline>parse <underline>T</underline>ensor (HOHDST). However, existing STD algorithms face the problem of intermediate variables explosion which results from the fact that the formation of those variables, i.e., matrices Khatri-Rao product, Kronecker product, and matrix-matrix multiplication, follows the whole elements in sparse tensor. The above problems prevent deep fusion of efficient computation and big data platforms. To overcome the bottleneck, a novel stochastic optimization strategy (SGD<inline-formula><tex-math notation="LaTeX">$\_$</tex-math><alternatives><mml:math><mml:mo>_</mml:mo></mml:math><inline-graphic xlink:href="li-ieq2-3047460.gif"/></alternatives></inline-formula>Tucker) is proposed for STD which can automatically divide the high-dimension intermediate variables into small batches of intermediate matrices. Specifically, SGD<inline-formula><tex-math notation="LaTeX">$\_$</tex-math><alternatives><mml:math><mml:mo>_</mml:mo></mml:math><inline-graphic xlink:href="li-ieq3-3047460.gif"/></alternatives></inline-formula>Tucker only follows the randomly selected small samples rather than the whole elements, while maintaining the overall accuracy and convergence rate. In practice, SGD<inline-formula><tex-math notation="LaTeX">$\_$</tex-math><alternatives><mml:math><mml:mo>_</mml:mo></mml:math><inline-graphic xlink:href="li-ieq4-3047460.gif"/></alternatives></inline-formula>Tucker features the two distinct advancements over the state of the art. First, SGD<inline-formula><tex-math notation="LaTeX">$\_$</tex-math><alternatives><mml:math><mml:mo>_</mml:mo></mml:math><inline-graphic xlink:href="li-ieq5-3047460.gif"/></alternatives></inline-formula>Tucker can prune the communication overhead for the core tensor in distributed settings. Second, the low data-dependence of SGD<inline-formula><tex-math notation="LaTeX">$\_$</tex-math><alternatives><mml:math><mml:mo>_</mml:mo></mml:math><inline-graphic xlink:href="li-ieq6-3047460.gif"/></alternatives></inline-formula>Tucker enables fine-grained parallelization, which makes SGD<inline-formula><tex-math notation="LaTeX">$\_$</tex-math><alternatives><mml:math><mml:mo>_</mml:mo></mml:math><inline-graphic xlink:href="li-ieq7-3047460.gif"/></alternatives></inline-formula>Tucker obtaining lower computational overheads with the same accuracy. Experimental results show that SGD<inline-formula><tex-math notation="LaTeX">$\_$</tex-math><alternatives><mml:math><mml:mo>_</mml:mo></mml:math><inline-graphic xlink:href="li-ieq8-3047460.gif"/></alternatives></inline-formula>Tucker runs at least 2<inline-formula><tex-math notation="LaTeX">$X$</tex-math><alternatives><mml:math><mml:mi>X</mml:mi></mml:math><inline-graphic xlink:href="li-ieq9-3047460.gif"/></alternatives></inline-formula> faster than the state of the art.