# Advantages of Density in Tensor Network Geometries for Gradient-Based Training
## 5 Conclusions and Outlook  
The experiments and training pipeline presented in this work contribute to current research on TN architectures, starting with a surrogate tensor contraction that enables further research in heterogeneous geometries. Our main findings relate to how different structures reach increasing levels of quality when training them to learn random states. Specifically, more dense geometries reach smaller errors faster both in terms of time and iterations, despite an increased contraction cost. Our interpretation is that the way the TN structure connects and distributes the information that it stores is much more important than the total amount of information it can store. Evidence for this includes the worse performance of the MPS and some tree TNs with maximal bond dimensions, despite being as complex as their denser counterparts, as well as the lack of improvement when allowing artificially large bond dimensions. The emergence of barren plateaus is relevant in any gradient-based training, and it is expected to appear when using a global loss function based on infidelity, as happens in our experiment. It has previously been linked to MPSs, with tree TNs and other more complex structures being less prone to exhibiting them. We have expanded on these findings by showing that their behavior is tied to the density of the structure, with barren plateaus being absent for very dense structures and becoming slightly more prominent as sparsity increases. For the MPS, specifically, the training even fails to converge for \(n=16\) sites unless using hundreds of trials, but performs well for denser structures. Memory limits in the training of tensor networks are dependent on the largest tensor contraction at any point as they rely on the transfer of large tensors from fast to slow memory. Regardless, decreasing the total memory required contributes to reducing the overall duration of the computation by minimizing the movements of large tensors between memories. We have also introduced a compactification of tree TN structures, using our training examples to prove that it not only reduces the total memory needed but also improves the quality and speed of the training. The latter is due to a reduced number of iterations, as the cost of the contractions themselves increases slightly with the compactification. These results are consistent with the density effect observed. Finally, we have showcased an application of existing tensor network libraries with HPC resources, and also characterized performance improvements with the use of acceler  
Figure 8: The time cost (left axis, points) and energy cost (right axis, bars) of the training in this work for different sizes of the system, adding together all reviewed geometries. We use 20 CPU cores and 20 CPU cores + GPU, both with high and reduced precision, labeled as “CPU”, “GPU”, and “GPU low”, respectively. In the inset, the average power needed for each training is presented. ated hardware in this same setting, which is very impactful for the large-scale setting that these types of experiments are usually executed on. Due to the nature of the simulations, these benchmarks are also relevant for similar machine learning techniques in HPC. Overall, this work contributes to better understanding the training of TNs, and will also have an impact on dynamical approaches to finding optimal structures. We think it is of great interest to extend the study of these geometries beyond gradient training towards powerful methods like DMRG or TEBD in the future. Similarly, with the proposed surrogate procedure, one can test if entanglement structures hidden in a single tensor can be learned with specific TN ansatzes, either using gradient methods or DMRG.