SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\{\mathcal {R} _ {t} \mid \log \mathcal {R} _ {t} = (1 - t) \log \mathcal {P} _ {1} + t \log \mathcal {P} _ {2} - \phi (t) \}, \quad \{\mathcal {R} _ {t} \mid \mathcal {R} _ {t} = (1 - t) \mathcal {P} _ {1} + t \mathcal {P} _ {2} \},
$$
respectively, where  $0 \leq t \leq 1$  and  $\phi(t)$  is a normalization factor to keep  $\mathcal{R}_t$  to be a distribution.
We can parameterize distributions  $\mathcal{P} \in \mathcal{U}$  by parameters known as natural parameters. In Equation (1), we have described the relationship between a distribution  $\mathcal{P}$  and a natural parameter vector  $\theta =$
$(\theta_{2,\dots,1}, \ldots, \theta_{I_1,\dots,I_D})$ . The natural parameter  $\pmb{\theta}$  serves as a coordinate system of  $\mathcal{U}$ , since any distribution in  $\mathcal{U}$  is specified by determining  $\pmb{\theta}$ . Furthermore, we can also specify a distribution  $\mathcal{P}$  by its expectation parameter vector  $\pmb{\eta} = (\eta_{2,\dots,1}, \ldots, \eta_{I_1,\dots,I_D})$ , which corresponds to expected values of the distribution and an alternative coordinate system of  $\mathcal{U}$ . The definition of the expectation parameter  $\pmb{\eta}$  is described in Equation (3).  $\theta$ -coordinates and  $\eta$ -coordinates are orthogonal with each other, which means that the Fisher information matrix  $\mathbf{G}$  has the following property,  $\mathbf{G}_{u,v} = \partial \eta_u / \partial \theta_v$  and  $(\mathbf{G}^{-1})_{u,v} = \partial \theta_u / \partial \eta_v$ .  $e$ - and  $m$ -geodesics can also be described using these parameters as follows.
$$
\left\{\boldsymbol {\theta} ^ {t} \mid \boldsymbol {\theta} ^ {t} = (1 - t) \boldsymbol {\theta} ^ {\mathcal {P} _ {1}} + t \boldsymbol {\theta} ^ {\mathcal {P} _ {2}} \right\}, \quad \left\{\boldsymbol {\eta} ^ {t} \mid \boldsymbol {\eta} ^ {t} = (1 - t) \boldsymbol {\eta} ^ {\mathcal {P} _ {1}} + t \boldsymbol {\eta} ^ {\mathcal {P} _ {2}} \right\},
$$
where  $\pmb{\theta}^{\mathcal{P}}$  and  $\pmb{\eta}^{\mathcal{P}}$  are  $\theta$ - and  $\eta$ -coordinate of a distribution  $\mathcal{P} \in \mathcal{U}$ .
Flatness and projections A subspace is called  $e$ -flat when any  $e$ -geodesic connecting two points in a subspace is included in the subspace. The vertical descent of an  $m$ -geodesic from a point  $\mathcal{P} \in \mathcal{U}$  onto  $e$ -flat subspace  $\mathcal{B}_e$  is called  $m$ -projection. Similarly,  $e$ -projection is obtained when we replace all  $e$  with  $m$  and  $m$  with  $e$ . The flatness of subspaces guarantees the uniqueness of the projection destination. The projection destination  $\overline{\mathcal{P}}$  or  $\tilde{\mathcal{P}}$  obtained by  $m$ - or  $e$ -projection onto  $\mathcal{B}_e$  or  $\mathcal{B}_m$  minimizes the following KL divergence,
$$
\overline {{\mathcal {P}}} = \underset {\mathcal {Q} \in \boldsymbol {\mathcal {B}} _ {e}} {\arg \min } D _ {K L} (\mathcal {P}, \mathcal {Q}), \quad \tilde {\mathcal {P}} = \underset {\mathcal {Q} \in \boldsymbol {\mathcal {B}} _ {m}} {\arg \min } D _ {K L} (\mathcal {Q}, \mathcal {P}).
$$
The KL divergence from discrete distributions  $\mathcal{P} \in \mathcal{U}$  to  $\mathcal{Q} \in \mathcal{U}$  is given as
$$
D _ {K L} (\mathcal {P}, \mathcal {Q}) = \sum_ {i _ {1} = 1} ^ {I _ {1}} \dots \sum_ {i _ {D} = 1} ^ {I _ {D}} \mathcal {P} _ {i _ {1}, \dots , i _ {D}} \log \frac {\mathcal {P} _ {i _ {1} , \dots , i _ {D}}}{\mathcal {Q} _ {i _ {1} , \dots , i _ {D}}}. \tag {13}
$$
A subspace with some of its natural parameters fixed at 0 is  $e$ -flat [1, Chapter 2], which is obvious from the definition of  $e$ -flatness. The proposed many-body approximation performs  $m$ -projection onto the subspace  $\mathcal{B} \subset \mathcal{U}$  with some natural parameters fixed to be 0. From this linear constraint, we know that  $\mathcal{B}$  is  $e$ -flat. Therefore, the optimal solution of the many-body approximation is always unique. When a space is  $e$ -flat and  $m$ -flat at the same time, we say that the space is dually-flat. The set of discrete probability distributions  $\mathcal{U}$  is dually-flat.
Natural gradient method  $e(m)$ -flatness guarantees that cost functions to be optimized in Equation (13) are convex. Therefore,  $m(e)$ -projection onto an  $e(m)$ -flat subspace can be implemented by a gradient method using a second-order gradient. This second-order gradient method is known as the natural gradient method [42]. The Fisher information matrix  $\mathbf{G}$  appears by second-order differentiation of the KL divergence (see Equation (5)). We can perform fast optimization using the update formula in Equation (6), using the inverse of the Fisher information matrix.
Examples for Möbius function In the proposed method, we need to transform the distribution  $\mathcal{P} \in \mathbb{R}^{I_1 \times \dots \times I_D}$  with  $\theta$  and  $\eta$  using the Möbius function, defined in Section 2.1. We provide examples here. In Equation (2), The Möbius function is used to find the natural parameter  $\theta$  from a distribution  $\mathcal{P}$ . For example, if  $D = 2, 3$ , it holds that
$$
\theta_ {i _ {1}, i _ {2}} = \log \mathcal {P} _ {i _ {1}, i _ {2}} - \log \mathcal {P} _ {i _ {1} - 1, i _ {2}} - \log \mathcal {P} _ {i _ {1}, i _ {2} - 1} + \log \mathcal {P} _ {i _ {1} - 1, i _ {2} - 1},
$$
$$
\begin{array}{l} \theta_ {i _ {1}, i _ {2}, i _ {3}} = \log \mathcal {P} _ {i _ {1}, i _ {2}, i _ {3}} - \log \mathcal {P} _ {i _ {1} - 1, i _ {2}, i _ {3}} - \log \mathcal {P} _ {i _ {1}, i _ {2} - 1, i _ {3}} - \log \mathcal {P} _ {i _ {1}, i _ {2}, i _ {3} - 1} \\ + \log \mathcal {P} _ {i _ {1} - 1, i _ {2} - 1, i _ {3}} + \log \mathcal {P} _ {i _ {1}, i _ {2} - 1, i _ {3} - 1} + \log \mathcal {P} _ {i _ {1}, i _ {2} - 1, i _ {3} - 1} - \log \mathcal {P} _ {i _ {1} - 1, i _ {2} - 1, i _ {3} - 1}, \\ \end{array}
$$
where we assume  $\mathcal{P}_{0,i_2} = \mathcal{P}_{i_1,0} = 1$  and  $\mathcal{P}_{i_1,i_2,0} = \mathcal{P}_{i_1,0,i_3} = \mathcal{P}_{0,i_2,i_3} = 1$ . To identify the value of  $\theta_{i_1,\dots ,i_d}$ , we need only  $\mathcal{P}_{i_1',\ldots ,i_d'}$  with  $(i_1',\ldots ,i_d')\in \{i_1 - 1,i_1\} \times \{i_2 - 1,i_2\} \times \dots \times \{i_d - 1,i_d\}$ . In the same way, using Equation (3), we can find the distribution  $\mathcal{P}$  by the expectation parameter  $\pmb{\eta}$ . For example, if  $D = 2,3$ , it holds that
$$
\begin{array}{l} \mathcal {P} _ {i _ {1}, i _ {2}} = \eta_ {i _ {1}, i _ {2}} - \eta_ {i _ {1} + 1, i _ {2}} - \eta_ {i _ {1}, i _ {2} + 1} + \eta_ {i _ {1} + 1, i _ {2} + 1}, \\ \mathcal {P} _ {i _ {1}, i _ {2}, i _ {3}} = \eta_ {i _ {1}, i _ {2}, i _ {3}} - \eta_ {i _ {1} + 1, i _ {2}, i _ {3}} - \eta_ {i _ {1}, i _ {2} + 1, i _ {3}} - \eta_ {i _ {1}, i _ {2}, i _ {3} + 1} \\ + \eta_ {i _ {1} + 1, i _ {2} + 1, i _ {3}} + \eta_ {i _ {1} + 1, i _ {2}, i _ {3} + 1} + \eta_ {i _ {1}, i _ {2} + 1, i _ {3} + 1} - \eta_ {i _ {1} + 1, i _ {2} + 1, i _ {3} + 1}, \\ \end{array}
$$
where we assume  $\eta_{I_1 + 1,i_2} = \eta_{i_1,I_2 + 1} = 0$  and  $\eta_{I_1 + 1,i_2,i_3} = \eta_{i_1,I_2 + 1,i_3} = \eta_{i_1,i_2,I_3 + 1} = 0$ .
Figure 5: (a) Interaction representation corresponding to Equation (14) and its transformed tensor network for  $D = 9$ . (b) Tensor network of a variant of tensor tree decomposition.
B Formal definitions
We provide give formal definitions of Legendre decomposition [35] as follows:
Definition 1. For a normalized non-negative tensor  $\mathcal{P}$  and a set of indices  $B$ , Legendre decomposition of  $\mathcal{P}$  is  $\mathcal{Q}$  defined as
$$
\overline{\mathcal{Q}} = \operatorname *{arg  min}_{\mathcal{Q};\mathcal{Q}\in \boldsymbol {\mathcal{B}}}D_{KL}(\mathcal{P},\mathcal{Q})
$$
where  $\mathcal{B} = \{\mathcal{Q}\mid \theta_{i_1,\dots ,i_D} = 0$  if  $(i_{1},\ldots ,i_{D})\notin B$  for  $\theta$  -parameters of  $\mathcal{Q}\}$
As in the following definition, a particular case of the Legendre decomposition is the many-body decomposition.
Definition 2. For a normalized non-negative tensor  $\mathcal{P}$  and a natural number  $m$ , we define its  $m$ -body approximation  $\mathcal{P}^{\leq m}$  as
$$
\overline {{\mathcal {Q}}} = \underset {\mathcal {Q}; \mathcal {Q} \in \boldsymbol {\mathcal {B}}} {\arg \min } D _ {K L} (\mathcal {P}, \mathcal {Q})
$$
where  $\mathbf{B}$  used in  $B$  contains all the indices of  $n(\leq m)$ -body parameters.
C Theoretical remarks
C.1 Convexity and uniqueness of many-body approximation
It is widely known that the maximum likelihood estimation of ordinary Boltzmann machines without hidden variables is a convex problem. Since we optimize the KL divergence, the proposed many-body approximation is also a maximum likelihood estimation that approximates a non-negative tensor, which is regarded as an empirical distribution, by an extended Boltzmann machine without hidden variables, as described in Section 2.2. As with ordinary Boltzmann machines, the maximum likelihood estimation of extended Boltzmann machines can be globally optimized. Theorem 1 is a general proof of this using information geometry.
Theorem 1. The solution of many-body approximation is always unique, and the objective function of many-body approximation is convex.
Proof. As Definition 2 shows, the objective function of the proposed many-body approximation is the KL divergence from an empirical distribution (a given normalized tensor) to a subspace  $\mathcal{B}$ . We can immediately prove that a subspace  $\mathcal{B}$  is  $e$ -flat from the definition of the flatness [1, Chapter 2] for any  $B$ . The KL divergence minimization from a given distribution to  $e$ -flat space is called  $m$ -projection (see the second paragraph in Section A). The  $m$ -projection onto  $e$ -flat subspace is known to be convex and the destination is unique (see the third paragraph in Section A). Thus, the optimal solution of the many-body approximation is always unique, and the objective function is convex.
Figure 6: The em-algorithm. Dashed(Solid) arrows are  $e(m)$ -projection.
C.2 Another example: Tensor tree decomposition
As seen in Section 2.4, cyclic two-body approximation coincides with tensor ring decomposition with a constraint. In this section, we additionally provide another example of correspondence between many-body approximation and the existing low-rank approximation. For  $D = 9$ , let us consider three-body and two-body interactions among  $(i_1, i_2, i_3)$ ,  $(i_4, i_5, i_6)$ , and  $(i_7, i_8, i_9)$  and three-body approximation among  $(i_3, i_6, i_9)$ . We provide the interaction representation of the target energy function in Figure 5(a). In this approximation, the decomposed tensor can be described as
$$
\mathcal {P} _ {i _ {1}, \dots , i _ {9}} = \mathcal {A} _ {i _ {1}, i _ {2}, i _ {3}} \mathcal {B} _ {i _ {4}, i _ {5}, i _ {6}} \mathcal {C} _ {i _ {7}, i _ {8}, i _ {9}} \mathcal {G} _ {i _ {3}, i _ {6}, i _ {9}}. \tag {14}
$$
Its converted tensor network is described in Figure 5(a). It becomes a tensor network of tensor tree decomposition in Figure 5(b) when the region enclosed by the dotted line is replaced with a new tensor (shown with tilde) by coarse-graining. Such tensor tree decomposition is used in generative modeling [43], computational chemistry [48] and quantum many-body physics [50].
C.3 Reducing computational complexity by reshaping tensor
As described in Section 2.1.1, the computational complexity of many-body approximation is  $\mathcal{O}(\gamma |B|^3)$ , where  $\gamma$  is the number of iterations, because the overall complexity is dominated by the update of  $\theta$ , which includes matrix inversion of  $\mathbf{G}$  and the complexity of computing the inverse of an  $n\times n$  matrix is  $\mathcal{O}(n^3)$ . This complexity can be reduced if we reshape tensors so that the size of each mode becomes small. For example, let us consider a third-order tensor whose size is  $(J^{2},J^{2},J^{2})$  and its cyclic two-body approximation. In this case, the time complexity is  $\mathcal{O}(\gamma J^{12})$  since it holds that  $|B| = \mathcal{O}(J^4)$  (see Equation (12)). By contrast, if we reshape the input tensor to a sixth-order tensor whose size is  $(J,J,J,J,J,J)$ , the time complexity becomes  $\mathcal{O}(\gamma J^6)$  since it holds that  $|B| = \mathcal{O}(J^2)$ .
This technique of reshaping a tensor into a larger-order tensor is widely used practically in various methods based on tensor networks, such as tensor ring decomposition [47]. We use this technique in the experiment in Section 3.3 to handle large tensors in the both of proposal and baselines.
C.4 Information geometric view of LBTC
Traditional methods for tensor completion are based on the following iteration, called em-algorithm [51].
$$
e - \text {s t e p}: \mathcal {P} _ {\Omega} \leftarrow \mathcal {T} _ {\Omega}, \quad m - \text {s t e p}: \mathcal {P} \leftarrow \text {l o w - r a n k - a p p r o x i m a t i o n} (\mathcal {P}),
$$
with an initialized tensor  $\mathcal{P}$ , where  $\Omega$  is the set of observed indices of a given tensor  $\mathcal{T}$  with missing values. From the information geometric view,  $m$ -step does  $m$ -projection onto the model manifold  $\mathcal{B}$  consisting of low-rank tensors, and  $e$ -step does  $e$ -projection onto the data manifold  $\mathcal{D} \equiv \{\mathcal{P} \mid \mathcal{P}_{\Omega} = \mathcal{T}_{\Omega}\}$ . It is straightforward to prove that  $e$ -step finds a point  $\overline{\mathcal{P}} = \arg \min_{\overline{\mathcal{P}} \in \mathcal{D}} D_{KL}(\overline{\mathcal{P}}, \mathcal{P})$  for the obtained tensor  $\mathcal{P}$  by the previous  $m$ -step [52]. The iteration of  $e$ - and  $m$ -step finds the closest point between two manifolds  $\mathcal{B}$  and  $\mathcal{D}$ . Since we can immediately prove that  $tQ_1 + (1 - t)Q_2$  belongs to  $\mathcal{D}$  for any  $Q_1, Q_2 \in \mathcal{D}$  and  $0 \leq t \leq 1$ , the data manifold  $\mathcal{D}$
Figure 7: (a) Many-body approximation with three-body interaction among width, height, and index image can reconstruct the shapes of each image. (b) Without the interaction, many-body approximation cannot reconstruct the shapes of each image.
is  $m$ -flat, which guarantees the uniqueness of the  $e$ -step. See the definition of  $m$ -flat in Section A. However, since low-rank subspace is not  $e$ -flat, that  $m$ -projection is not unique. LBTC conducts tensor completion by replacing low-rank approximation in  $m$ -step with many-body approximation. Since the model manifold of many-body approximation is always  $e$ -flat and data manifold is  $m$ -flat, each step in the algorithm is unique.
D Related work
As discussed in Section 1, we assume low-rankness in the structure of the tensor in conventional decompositions. However, since tensors have many modes, there are too many kinds of low-rank structures. Examples include, the CP decomposition [15], which decomposes a tensor by the sum of Kronecker products of vectors; the Tucker decomposition [37], which decomposes a tensor by a core tensor and multiple matrices; the tensor train decomposition [29], which decomposes a tensor by multiple core tensors; and tensor-ring decomposition [41], which assume periodic structure on tensor train decomposition are well known and popularly used in machine learning.
However, in general, it is difficult to find out the low-rank structure of a given data set. Therefore, it is difficult to say how to choose a low-rank decomposition model from those options. To alleviate this difficulty in model selection and provide intuitive modeling, we have introduced the concept of body and formulated a decomposition that does not rely on ranks.
E Additional numerical results
E.1 Additional results for COIL dataset
In Section 3.1, we see that interactions related to color control color-richness in the reconstruction. We provide additional experimental results in Figure 7 to see the role of the interaction among width, height, and image index. The figure shows the influence of the interaction for reconstruction. While Figure 7(b), without the interaction, could not capture the shapes of images, Figure 7(a) reconstruct each shape properly. Based on these results, it is reasonable that the interaction is responsible for these shapes.
E.2 Additional results for LBTC
We provide the reconstructions by the experiments in Section 3.2 in Figure 8 with ground truth.
In traditional low-rank approximation, tuning hyper-parameters based on prior knowledge is difficult because its semantics are unclear. By contrast, the interactions in many-body approximation can be tuned easily by considering the expected correlation among modes of a given tensor. To support this claim, we also confirm that LBTC scores get better when we activate interactions between modes that are expected to be correlated based on prior knowledge. In this experiment, we use the same datasets in Section 3.2. We prepare two set of interactions, Interactions A and B, as shown on the left in Figure 9. In Interaction A, we activate interaction between modes such as (date, hour) and (hour, minute, lane), which we expect to be correlated. In Interaction B, we activate interaction between modes such as (date, minute) and (minute, lane), which we do not expect to be correlated. For a fair comparison, we chose these interactions so that the number of parameters used in Interactions A and B are as close as possible, where the number of parameters for Interaction A was 1847, and the number of parameters for Interaction B was 1619. As seen in Table 2, LBTC with Interaction A always has a better score than Interaction B, which implies that many-body approximation has better
performance when we choose interactions among correlated modes. We conducted the experiment 10 times and reported mean values with standard error. We also visualize the reconstruction in Figure 9 to show that LBTC with Interaction A can complete tensors more accurately than that with Interaction B.
As seen in Figure 14 and Section G.2, existing methods require hyper-parameters that are difficult to tune since we usually cannot find any correspondence between these values and prior knowledge about the given tensor. In contrast, as demonstrated in this section, our energy-based model is intuitively tunable that can skip time-consuming hyper-parameter tuning.
Table 2: Dependency of recovery fit score of LBTC on a choice of interaction.
Scenario 1	Scenario 2	Scenario 3
Interaction A	0.900 ± 0.00125	0.889 ± 0.000632	0.864 ± 0.001264
Interaction B	0.820 ± 0.00143	0.819 ± 0.000699	0.805 ± 0.000527
Figure 8: Ground truth and reconstruction by LBTC in the experiment in Section 3.2. Filled areas are missing parts, and blue lines are estimated values by the proposal. Green lines are ground truth.
Scenario 1
Interaction A	d	h	m	l	Speed (max)	90°	60°	30°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	GT Interaction A	Lane 1	GT Interaction A	Lane 1	GT Interaction A	Lane 1	GT Interaction A	Lane 1	GT Interaction A	Lane 1	GT Interaction A	Lane 1	GT Interaction A	Lane 1	GT Interaction A	Lane 1	GT Interaction A	Lane 1	GT Interaction A	Lane 1	GT Interaction A	Lane 1	GT Interaction A	Lane 1
Interaction B	d	h	m	l	Speed (max)	90°	60°	30°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°	0°C	0°C	0°C	0°C	0°C	0°C	0°C	0°C	0°C	0°C	0°C	GT Interaction B	Lane 1	GT Interaction B	Lane 1	GT Interaction B	Lane 1	GT Interaction B	Lane 1	GT Interaction B	Lane 1	GT Interaction B	Lane 1
E.3 Additional results for comparison with ring decomposition
We additionally provide Figure 10, which shows running time compassion in the experiment in Section 3.3. As described in the main text, we repeat five times each baseline method with random initialization while the proposal runs only once. We compare the total running time of them. As seen in Figure 10, for both synthetic and real datasets, the proposed method is always faster than baselines, keeping the competitive relative errors. The flatness of solution space enables many-body approximation to be solved with the Newton method. In contrast, NTR is solved by a first-order derivative. The quadratic convergence of the Newton method is well known, and it requires only a few iterations to find the best solution in the flat solution space  $\mathcal{B}_{\mathrm{cyc}}$ . That is the reason why the proposed method is fast and stable.
E.4 Comparison with traditional low-rank approximation
Figure 9: Interaction dependency of LBTC. Filled areas are missing parts, and blue lines are estimated values by the proposal with Interactions A and B, respectively. Green lines are ground truth. Due to space limitations, we show the reconstruction of Lane 1 from day 9 to 20, where many missing parts exist.
Figure 10: Comparison of reconstruction capabilities (top) and computational speed (bottom) of cyclic two-body approximation and tensor ring decomposition for synthetic low-tensor ring datasets (a) and real datasets (b). The vertical red dotted line is  $|\mathcal{B}|$  (See Equation (12)).
In low-rank approximation, we often gradually increase the rank to enlarge model capability. In the same way, if we have no prior or domain knowledge about a tensor, we can gradually increase  $m$  of  $m$ -body approximation and get more accurate reconstruction. To demonstrate that, we conducted additional experiments with the same dataset in Section 3.2. We compared the reconstruction ability
Figure 11: We compared the fit score of proposal  $\mathcal{P}^{\leq 1}$ ,  $\mathcal{P}^{\mathrm{cyc}}$ ,  $\mathcal{P}^{\leq 2}$ ,  $\mathcal{P}^{\leq 3}$  with baselines. (a) The ranks for baselines assumed as  $(r, \dots, r)$  with changing the value of  $r$ . (b) The ranks for baselines are tuned taking the tensor into size account.
Figure 12: Heatmaps for reconstructed tensor  $\mathcal{P}^{\leq 1}$ ,  $\mathcal{P}^{\mathrm{cyc}}$ ,  $\mathcal{P}^{\leq 2}$ ,  $\mathcal{P}^{\leq 3}$  and input tensor  $\mathcal{P}$ . We visualize the traffic on only the first lane at 00 minutes every hour, which corresponds to  $\mathcal{P}[:, :, 1, 1]$ .
of the proposed  $m$ -body approximation with non-negative tucker decomposition (NTD) [44] and non-negative tensor train decomposition (NTTF) [34].
In many applications based on matrix and tensor factorization, their objective functions are often the fit score (RMSE) and they try to find good representations to solve the task by optimizing the objective. Therefore, for fair evaluation without depending on specific applications, we examine the performance by fit score.
For  $m$ -body approximation, we gradually increased the order of the interaction by changing  $m$  to 1, 2, 3. We ran the proposal only once since the proposal does not depend on initial values. For baselines, we gradually enlarged model capacity by increasing the target rank. We repeated each baseline 20 times with random initialization. We defined the Tucker and train ranks for the baseline method in two ways. First, we assumed that all values in ranks are the same – that is,  $(r, \dots, r)$  – and increase the value of  $r$ . However, since the size of an input tensor is (28, 24, 12, 4) and each mode size is not unique, it can be unnatural rank setting, which may cause worse RMSE. To avoid that, we also adjusted target rank by taking the tensor size into account. In particular, the rank is set to be (1,1,1,1), (6,2,2,1), (6,5,2,1), (8,5,4,2), (10,7,4,2), (12,9,4,2), (16,12,6,2), (18,15,8,2), (20,18,11,3) for NTD and (1,1,1), (1,2,1), (1,2,2), (2,2,2), (4,2,2), (6,2,2), (6,3,2), (8,3,2), (10,3,2), (10,4,2), (15,4,2), (20,5,2), (20,8,2), (30,10,2), (30,15,8) for NTTF.
We evaluated the approximation performance with the fit score  $1 - \|\mathcal{P} - \overline{\mathcal{P}}\|_F / \|\mathcal{P}\|_F$  for the input  $\mathcal{P}$  and the reconstructed tensor  $\overline{\mathcal{P}}$ . We show these results in Figure 11. While baselines are often trapped in a worse local minimum, the proposal reconstructs the original tensor better. We also show the reconstruction by the proposed method as heatmaps in Figure 12. Even one-body approximation is able to reconstruct the rush hour around 17:00. When the number of interactions increases, we can observe the trend that the time of the busy hours on weekends are different from those on weekdays more clearly.