SlowGuess/NeurIPS_2023_OCR huggingface dataset:
There have been many works on tensor decomposition, such as (Yang and Dunson, 2013; Rai et al., 2014; Zhe et al., 2015, 2016b,a; Tillinghast et al., 2020; Pan et al., 2020b; Fang et al., 2021a,b; Tillinghast and Zhe, 2021; Tillinghast et al., 2022; Fang et al., 2022). To integrate temporal information, most approaches augment the tensor with a time mode (Xiong et al., 2010; Rogers et al., 2013; Zhe et al., 2016b; Ahn et al., 2021; Zhe et al., 2015; Du et al., 2018), which includes a list of time steps. To estimate the temporal dependencies, existing methods often employ a dynamic model over the time steps, such as a conditional Gaussian prior (Xiong et al., 2010), recurrent neural networks(Wu et al., 2019), and kernel smoothing/regularization (Ahn et al., 2021). To leverage continuous timestamps, Zhang et al. (2021) modeled the CP coefficients as a time function, and Fang et al. (2022) modeled the tensor-core as a time function in the Tucker decomposition. The most recent work (Wang et al., 2022) places a GP prior in the frequency domain, and construct a bi-level GP model to learn factor or embedding trajectories as a combination of Fourier bases.
Another set of works factorize the interaction events (Schein et al., 2015, 2016; Zhe and Du, 2018; Pan et al., 2020a; Wang et al., 2020; Pan et al., 2021; Wang et al., 2022), and they cannot predict the interaction results (e.g., tensor entry values like purchase amount and product ratings). These works mainly leverage Poisson processes, Hawkes processes, or more general point processes to estimate the event rate. Like the standard tensor factorization, these methods also estimate static embeddings for the event participants.
Our model can be viewed as an extension of the neural ODE model (Chen et al., 2018). If we only employ the reaction process for each entity, our model is a latent neural ODE (we have an additional NN that combines the latent trajectories to predict the tensor entry values). However, we further leverage the structural knowledge in data to construct a multi-partite graph so as to encode the correlations of the entities. Based on the multi-partite graph, we construct a diffusion process to co-evolve the embeddings of the entities. In doing so, we can better overcome the data sparsity issue. The most recent work (Li et al., 2022) uses a neural ODE to model the tensor entry value as a function of the involved embeddings and time. It differs from our method in that (1) it still learns a static (time-invariant) embedding for each entity, and (2) its modeling does not use the structure knowledge within the tensor data like our method. Many other works have developed graph diffusion processes based on graph data. For example, Chamberlain et al. (2021) proposed a graph neural network (GNN) by using multi-head attention to construct the adjacent matrix for a graph diffusion equation over the graph nodes. Atwood and Towsley (2016) introduced a diffusion operator to develop diffusion-convolutional neural networks. Huang et al. (2021) developed a GNN-based ODE to model both the nodes and edges in dynamic graphs.
6.1 Simulation Study
We first examined DEMOTE on a synthetic task. Specifically, we considered a two-mode tensor, where each mode includes 20 entities. Each entity has one underlying embedding trajectory. In the first mode, the trajectory of each entity is an exponential function,  $u_{j}^{1}(t) = c_{j}^{1}\exp (0.5c_{j}^{1}t)$  ( $1\leq j\leq 20$ ), while in the second mode is a linear function,  $u_{j}^{2}(t) = c_{j}^{2} + 2\pi c_{j}^{2}t$ . We generated two clusters of trajectories for each mode, where those of the first ten entities form the first cluster and the remaining the second cluster. To this end, for mode 1, we sampled the coefficients of the first ten entities' trajectories, namely,  $[c_1^1,\dots ,c_{10}^1 ]^\top$ , from  $\mathcal{N}\left([-5,\ldots , - 5]^\top ,0.1\mathbf{I}\right)$ , and the remaining ten coefficients  $[c_{11}^{1},\ldots ,c_{20}^{1}]^{\top}$  from  $\mathcal{N}\left([0.5,\dots ,0.5]^{\top},0.1\mathbf{I}\right)$ . Then for mode 2, we sampled each coefficient  $c_{j}^{2}$  conditioned on its counter-part for mode 1,  $c_{j}^{2}|c_{j}^{1}\sim \mathcal{N}(c_{j}^{2}|c_{j}^{1},0.1)$ . That means, the coefficients of cluster-1 trajectories across the two modes are close, and so are those of cluster-2 trajectories. The value for a particular entry  $\ell = (l_{1},l_{2})$  is generated by
$$
m _ {\ell} (t) = \left(u _ {l _ {1}} ^ {1} (t)\right) ^ {\mathbb {1} \left(l _ {1} + l _ {2} \mod 2 = 0\right)} \cdot \left(u _ {l _ {2}} ^ {2} (t)\right) ^ {\mathbb {1} \left(l _ {1} + l _ {2} \mod 2 = 1\right)} \tag {5}
$$
(a) Ground-truth: Mode 1
(b) NONFAT: Mode 1
(c) DEMOTE: Mode 1
(d) Ground-truth: Mode 2
(e) NONFAT: Mode 2
(f) DEMOTE: Mode 2
Figure 2: The estimated embedding trajectories for each mode. The color indicates the ground-truth cluster membership.
where  $\mathbb{1}(\cdot)$  is the indicator function. When  $l_{1} + l_{2}$  is even, the entry value is the trajectory value of the first entity; otherwise, it is the trajectory value of the second entity. To generate the training data, we randomly sampled entries from  $\{(l_1,l_2)|1\leq l_1,l_2\leq 10\} \cup \{(l_1,l_2)|11\leq l_1,l_2\leq 20\}$  (namely, interactions between cluster-1 entities of the two modes, and between cluster-2 entities). We then sampled  $t\sim$  Unifrom[0,5], to obtain the corresponding entry values. We randomly generated 6,400 entry values and the timestamps for training, and another 1,600 data points for testing.
We implemented our method with Pytorch (Paszke et al., 2019). We used torchdiffreq library (https://github.com/rtqichen/torchdiffreq) to solve ODEs and to compute the gradient w.r.t ODE parameters and initial states via automatic differentiation. For the NN of the reaction process, we used one hidden layer, with 10 neurons and tanh activation, and for the NN to predict the interaction result, we used two hidden layers, 50 neurons per layer and tanh activation.
We compared with NONFAT (NONparametric Factor Trajectory learning) (Wang et al., 2022), a bi-level latent GP model that uses Fourier bases to estimate factor trajectories for dynamic tensor decomposition. To our knowledge, this work is the only method (and also the most recent) that estimates trajectories. We used the original implementation (https://github.com/wzhut/NONFAT) and the default settings. We set the mini-batch size to 50, and used ADAM (Kingma and Ba, 2014) algorithm for stochastic optimization. The learning rate was automatically adjusted in  $[10^{-4}, 10^{-1}]$  by the ReduceLROnPlateau scheduler (Al-Kababji et al., 2022). The maximum number of epochs is 2K, which is enough for convergence. The estimated trajectories are shown in Fig. 2a-f. As we can see, our estimation (Fig. 2c and 2f) well matches the ground-truth and accurately recovers the cluster structure of the trajectories. The root-mean-square error (RMSE) on the test set is 0.032. By contrast, although the test error of NONFAT is close to DEMOTE (0.034), its learned trajectories (Fig. 2b and 2e) are far from the ground-truth, and fail to reflect the cluster structure. These have shown the advantage of DEMOTE in capturing complex relationships within data to recover the underlying trajectories and their structure.
6.2 Prediction Accuracy
Datasets. We next evaluated the predictive performance of DEMOTE in three real-world applications. (1) CA Weather (Moosavi et al., 2019) (https://smoosavi.org/datasets/lstw), weather conditions in California from August 2016 to December 2020. We extracted a four-mode tensor for 7 different weather types, 6 severity levels, 30 latitudes and 30 longitudes in GPS coordinates. The entry value is the count of the particular weather condition. We collected 15K observed tensor entry values and the timestamps. (2) CA Traffic (Moosavi et al., 2019) (https://smoosavi.org/datasets/lstw), traffic accidents in California from January 2018 to December 2020. We extracted a four
(a) CA Weather
(b) CA Traffic
(c) Server Room
Figure 3: Predictive performance of the diffusion and reaction processes.
mode tensor (traffic type, severity level, latitudes, longitude). There are 7 traffic types, 6 severity levels, 20 latitudes and 20 longitudes. We collected 30K entry values (accident counts) at different time points. (3) Server Room (https://zenodo.org/record/3610078#.XlNpAigzaM8), temporal temperature records of Poznan Supercomputing and Networking Center. The temperatures were measured at 34 locations, under different air-condition modes  $(24^{\circ}, 27^{\circ}$ , and  $30^{\circ})$  and power usage settings  $(50\%, 75\%$  and  $100\%)$ . Hence, we extracted a three-mode tensor (location, air-condition mode, power level). In total, 10K observed entry values and their timestamps were collected.
Competing Methods. The following popular and/or state-of-the-art temporal decomposition approaches were compared. (1) CP-DTLD, discrete-time CP decomposition with linear dynamics, where a conditional prior is placed over successive time steps,  $p(\mathbf{t}_{j+1}|\mathbf{t}_j) = \mathcal{N}(\mathbf{t}_{j+1}|\mathbf{A}\mathbf{t}_j + \mathbf{b}, v\mathbf{I})$ ; A, b and  $v$  were jointly estimated during the CP decomposition. Note that (Xiong et al., 2010) is an instance of this model where  $\mathbf{A} = \mathbf{I}$  and  $\mathbf{b} = \mathbf{0}$ . (2) GP-DTLD and (3) NN-DTLD, similar to CP-DTLD, except using GP (Zhe et al., 2016b) and NN decomposition models (similar to (Liu et al., 2019)), respectively. (4) CP-DTND, (5) GP-DTND and (6) NN-DTND — CP, GP and NN decomposition with nonlinear dynamics, where the conditional prior is  $p(\mathbf{t}_{j+1}|\mathbf{t}_j) = \mathcal{N}(\mathbf{t}_{j+1}|\sigma(\mathbf{A}\mathbf{t}_j) + \mathbf{b}, v\mathbf{I})$  where  $\sigma(\cdot)$  is a nonlinear activation. The dynamics can therefore be viewed as an RNN transition. (7) CP-CT (Zhang et al., 2021), continuous-time CP factorization, which models the CP coefficients as a time-varying function, with polynomial splines. (8) GP-CT, continuous-time GP decomposition that extends (Xu et al., 2012; Zhe et al., 2016b) by plugging the time in the GP kernel so as to estimate the entry value as a function of the embeddings and time,  $m_\ell = g(\mathbf{u}_{\ell_1}^1, \ldots, \mathbf{u}_{\ell_K}^K, t)$ . (9) NN-CT, continuous-time NN decomposition, where the input consists of both the embeddings and time  $t$ . (10) THIS-ODE (Li et al., 2022), a continuous-time decomposition, where a neural ODE is used to estimate the tensor entry values given the static embeddings and time. (11) NONFAT (Wang et al., 2022), a bi-level latent GP model that uses Fourier bases to estimate factor trajectories for dynamic tensor decomposition.
Settings and Results. All the approaches were implemented with PyTorch. The Square Exponential kernel was used for all the GP-related methods, including GP-  $\{\mathrm{DTLD}$ , DTND, CT\}. We used the same variational sparse approximation (Hensman et al., 2013) to fulfill scalable posterior inference. Following (Zhe et al., 2016b), the number of inducing point was set to 100. For the NN decomposition methods, we employed a three-layer network with tanh activation, and for THIS-ODE, we used a one-layer network. The layer width was chosen from  $\{10, 25, 50, 75, 100\}$ . We used tanh as the activation function in the nonlinear dynamic baselines, including  $\{\mathrm{CP}, \mathrm{GP}, \mathrm{NN}\}$ -DTND. For our method, we used the same NN architecture for both the reaction process and entry value prediction, which includes two hidden layers with 50 neurons per layer. For CP-CT, we employed 100 knots to fulfill the polynomial splines. For each discrete-time method, the number of time steps was chosen from  $\{25, 50, 75, 100\}$  via the cross-validation on the training set. We trained all the models with stochastic mini-batch optimization. We used the ADAM algorithm, and the mini-batch size was set to 100. We ran every method with 10K epochs to ensure convergence. The learning rate was automatically adjusted in  $[10^{-4}, 10^{-1}]$  by the ReduceLROnPlateau scheduler. We varied the dimension of the embeddings  $R$  from  $\{2, 3, 5, 7\}$ . For DEMOTE,  $R$  is the number of embedding trajectories; we used computational graphs to obtain the gradient. We followed (Kang et al., 2012; Zhe et al., 2016b) to randomly draw  $80\%$  observed entries and their time stamps for training, with the remaining for test. We computed the normalized root-mean-square error (nRMSE). We repeated the evaluation five times and computed the average nRMSE and standard deviation.
CA Weather	R=2	R=3	R=5	R=7
CP-DTLD	0.7440±0.0035	0.7372±0.0040	0.7290±0.0042	0.7270±0.0044
GP-DTLD	0.7417±0.0031	0.7414±0.0036	0.7444±0.0036	0.7449±0.0039
NN-DTLD	0.7228±0.0054	0.7116±0.0033	0.7070±0.0041	0.7065±0.0038
CP-DTND	0.7448±0.0031	0.7360±0.0035	0.7273±0.0037	0.7280±0.0044
GP-DTND	0.7399±0.0034	0.7346±0.0032	0.7448±0.0037	0.7467±0.0031
NN-DTND	0.7113±0.0045	0.6979±0.0126	0.6659±0.0122	0.6543±0.0155
CP-CT	1.0000±0.0096	0.9959±0.0067	1.0010±0.0017	1.0060±0.0034
GP-CT	0.7433±0.0038	0.7354±0.0027	0.7359±0.0034	0.7377±0.0033
NN-CT	0.8697±0.0014	0.8679±0.0022	0.8676±0.0018	0.8695±0.0016
NONFAT	0.7444±0.0042	0.7460±0.0032	0.7645±0.0061	0.7553±0.0029
THIS-ODE	0.7511±0.0052	0.7539±0.0041	0.7614±0.0024	0.7620±0.0032
DEMOTE	0.6327±0.0119	0.6109±0.0056	0.6172±0.0075	0.6354±0.0085
CA Traffic
CP-DTLD	0.6498±0.0257	0.6424±0.0266	0.6436±0.0268	0.6405±0.0262
GP-DTLD	0.6309±0.0167	0.6290±0.0185	0.6383±0.0204	0.6496±0.0193
NN-DTLD	0.6528±0.0230	0.6545±0.0244	0.6401±0.0282	0.6136±0.0338
CP-DTND	0.6497±0.0245	0.6456±0.0265	0.6431±0.0263	0.6419±0.0259
GP-DTND	0.6544±0.0213	0.6559±0.0224	0.6604±0.0243	0.6674±0.0214
NN-DTND	0.6578±0.0248	0.6528±0.0256	0.6519±0.0249	0.6482±0.0261
CP-CT	0.9858±0.0120	0.9972±0.0056	0.9816±0.0136	0.9991±0.0120
GP-CT	0.6610±0.0207	0.6668±0.0191	0.6756±0.0190	0.6768±0.0196
NN-CT	0.9804±0.0017	0.9815±0.0015	0.9791±0.0012	0.9802±0.0017
NONFAT	0.4461±0.0247	0.4610±0.0231	0.5031±0.0155	0.6307±0.0847
THIS-ODE	0.6603±0.0230	0.6536±0.0212	0.6838±0.0193	0.6378±0.0142
DEMOTE	0.3601±0.0334	0.2972±0.0099	0.3174±0.0118	0.3269±0.0162
Server Room
CP-DTLD	0.4211±0.0029	0.4209±0.0031	0.4208±0.0028	0.4208±0.0028
GP-DTLD	0.0914±0.0020	0.0791±0.0010	0.0739±0.0014	0.0753±0.0013
NN-DTLD	0.4213±0.0032	0.4213±0.0032	0.4212±0.0034	0.4205±0.0030
CP-DTND	0.2835±0.0160	0.1751±0.0020	0.1174±0.0011	0.0829±0.0044
GP-DTND	0.0925±0.0013	0.0784±0.0011	0.0739±0.0009	0.0774±0.0009
NN-DTND	0.4213±0.0032	0.4212±0.0030	0.4211±0.0032	0.4205±0.0030
CP-CT	0.9919±0.0096	0.9951±0.0050	0.9862±0.0109	1.0121±0.0070
GP-CT	0.1385±0.0020	0.1223±0.0016	0.1275±0.0014	0.1365±0.0014
NN-CT	0.1193±0.0030	0.1140±0.0015	0.1113±0.0027	0.1149±0.0028
NONFAT	0.1468±0.0026	0.1407±0.0023	0.1396±0.0022	0.1409±0.0030
THIS-ODE	0.1412±0.0024	0.1312±0.0013	0.1304±0.0016	0.1350±0.0019
DEMOTE	0.0536±0.0031	0.0403±0.0014	0.0393±0.0018	0.0403±0.0027
Table 1: Normalized Root Mean-Square Error (nRMSE). The results were averaged from five runs.
As shown in Table 1, DEMOTE consistently achieves the best prediction accuracy and in many cases outperforms the competing methods by a large margin. Although learning an embedding trajectory is much more challenging than learning a fixed-value embedding, the experimental results have demonstrated the advantage of our method in predictive performance. To investigate the effect of the two processes in our model, we also examined our method with the diffusion process only and with the reaction process only on all the datasets. Their performance, as compared with DEMOTE, is shown in Fig. 3. We can see that each individual component can lead to good prediction accuracy. However, each component is worse than DEMOTE that synergizes the two components together. Therefore, the results show that each process is effective, and more important, the two processes can bolster each other to further improve the performance when they are combined.
6.3 Learning Result Investigation
Next, we looked into the learned embedding trajectories and checked if they exhibit patterns. To do so, we set  $R = 3$  and ran DEMOTE on Server Room dataset. In Fig. 4, we show the learned embedding trajectories for the first location (a-c), the first air condition mode (d-f) and the first power usage level (g-i). As we can see, even for the same object, e.g., a particular location, the corresponding embedding trajectories vary quite differently, implying the evolution of different underlying properties, such as the workload, memory usage, and network latency. We further found there are underlying structures within the embeddings during their evolution. We listed the results in Appendix (Sec. A).
(a)  $u_{1,1}^{1}(t)$
(b)  $u_{1,2}^{1}(t)$
(c)  $u_{1,3}^{1}(t)$
(d)  $u_{1,1}^{2}(t)$
(g)  $u_{1,1}^{3}(t)$
(e)  $u_{1,2}^2 (t)$
(h)  $u_{1,2}^{3}(t)$
(f)  $u_{1,3}^{2}(t)$
(i)  $u_{1,3}^{3}(t)$
Figure 4: The learned embedding trajectories for location 1 (a-c), air conditional mode 1 (d-f), and power usage level 1 (g-i) in Server Room dataset.
We showcase the temporal predictions for two tensor entries. As we can see from Fig. 5, given only a few training points (blue), our method can predict the test points (green) much more accurately, as compared with GPCT, and the predictive uncertainty (reflected by the noise variance  $\sigma^2$ ) is much smaller. This might be due to that via the diffusion-reaction process, and our method can more effectively extract the temporal knowledge from sparse data. For example, DEMOTE successfully captured the periodic nature in the first entry (Fig. 5b) while GPCT treated the fluctuation as noises and ended up with much inaccurate predictions and larger predictive variances.
Computational Efficiency. We compared the per-epoch/iteration running time DEMOTE with the other methods. We tested all the methods in a workstation with one NVIDIA GeForce RTX 3090 Graphics Card, 10th Generation Intel Core i9-10850K Processor, 32 GB RAM, and 1 TB SSD. The results are shown in Table 2 in Appendix. We can see that the running speed of DEMOTE is comparable to NONFAT and other NN decomposition methods. We also compared with running DEMOTE with naive sampling (DEMOTE-NS). The stratified sampling led to  $4\mathrm{x}$  to  $22\mathrm{x}$  speed-up.
(a)  $(2,1,1)$
Figure 5: Entry value prediction on Server Room.
(b)  $(3,1,1)$
We have presented DEMOTE, a neural diffusion-reaction process model to learn dynamic embeddings for dynamic tensor decomposition. The predictive performance is encouraging, and the learned embedding trajectories exhibit interesting patterns. Currently, our method is limited to a small number of entities since it has to integrate the entire multi-partite graph to construct the diffusion process. In the future work, we plan to develop graph cut algorithms to partition the graph into a set of small sub-graphs so that we can construct multiple diffusion processes in parallel so as to scale up our model to big graphs and to large tensors.
This work has been supported by NSF CAREER Award IIS-2046295.
Wang, Z., Chu, X., and Zhe, S. (2020). Self-modulating nonparametric event-tensor factorization. In International Conference on Machine Learning, pages 9857-9867. PMLR.
Wang, Z., Xu, Y., Tillinghast, C., Li, S., Narayan, A., and Zhe, S. (2022). Nonparametric embeddings of sparse high-order interaction events. In International Conference on Machine Learning, pages 23237-23253. PMLR.
Wu, X., Shi, B., Dong, Y., Huang, C., and Chawla, N. V. (2019). Neural tensor factorization for temporal interaction learning. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pages 537-545.
Xiong, L., Chen, X., Huang, T.-K., Schneider, J., and Carbonell, J. G. (2010). Temporal collaborative filtering with bayesian probabilistic tensor factorization. In Proceedings of the 2010 SIAM International Conference on Data Mining, pages 211-222. SIAM.
Xu, Z., Yan, F., and Qi, Y. A. (2012). Infinite tucker decomposition: Nonparametric bayesian models for multiway data analysis. In ICML.
Yang, Y. and Dunson, D. (2013). Bayesian conditional tensor factorizations for high-dimensional classification. Journal of the Royal Statistical Society B, revision submitted.
Zhang, Y., Bi, X., Tang, N., and Qu, A. (2021). Dynamic tensor recommender systems. Journal of Machine Learning Research, 22(65):1-35.
Zhe, S. and Du, Y. (2018). Stochastic nonparametric event-tensor decomposition. In Advances in Neural Information Processing Systems, pages 6856-6866.
Zhe, S., Qi, Y., Park, Y., Xu, Z., Molloy, I., and Chari, S. (2016a). Dintucker: Scaling up Gaussian process models on large multidimensional arrays. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30.
Zhe, S., Xu, Z., Chu, X., Qi, Y., and Park, Y. (2015). Scalable nonparametric multiway data analysis. In Artificial Intelligence and Statistics, pages 1125-1134. PMLR.
Zhe, S., Zhang, K., Wang, P., Lee, K.-c., Xu, Z., Qi, Y., and Ghahramani, Z. (2016b). Distributed flexible nonlinear tensor factorization. Advances in neural information processing systems, 29.
A Investigation of Embedding Dynamics
We investigated if there are underlying structures within the embeddings during their evolution. To this end, we looked into the embeddings of the 34 locations on Server Room dataset at five time points  $(t = 1, 20, 50, 80, 100)$ . At each time point, we ran the k-means algorithm over the embeddings to extract the clustering structures. We used the elbow method (Ketchen and Shook, 1996) to select the number of clusters. We can see that at earlier time  $(t \leq 50)$ , the clusters are more compact, while at the later stages, the clusters become more scattered. This reflects how the structure of those entities (i.e., locations) evolves along with time. It is interesting to see that some locations are in the same cluster all the time, like location  $\{5,7\}$  and location  $\{16,32\}$ . It implies that their underlying properties might have quite similar (or correlated) evolution patterns. Some locations are grouped in the cluster at the beginning, e.g., location  $\{32,34\}$  (at  $t = 1$ ), but later moves to different clusters  $(t > 1)$ . It implies their evolution patterns can vary significantly, leading to the change of the cluster memberships.
CA Weather	CA Traffic	Server Room
CP-DTLD	0.037	0.086	0.023
GP-DTLD	0.246	0.247	0.248
NN-DTLD	2.400	4.730	1.080
CP-DTND	0.038	0.087	0.025
GP-DTND	0.119	0.242	0.080
NN-DTND	2.360	4.701	1.060
CP-CT	0.025	0.052	0.018
GP-CT	0.068	0.216	0.105
NN-CT	2.310	3.885	1.030
NONFAT	0.952	1.925	0.571
THIS-ODE	58.710	136.100	7.190
DEMOTE	1.390	1.895	0.309
DEMOTE-NS	6.12	10.42	7.06
Table 2: Per-epoch/iteration running time (in seconds). DEMOTE-NS means running DEMOTE with naive sampling of min-batches rather than the stratified sampling.
(a)  $t = 1$
(b)  $t = 20$
(c)  $t = 50$
(d)  $t = 80$
(e)  $t = 100$
Figure 6: Evolution of the clustering structure within the 34 locations on Server Room dataset.