# Advantages of Density in Tensor Network Geometries for Gradient-Based Training
## 1 Introduction  
Tensor networks (TNs) provide a characterization of quantum states with a connection to physical states of local Hamiltonians [1] that may be efficiently represented classically. They have become an indispensable tool in the field of simulating quantum systems, as they offer highly developed methods that are parallelizable for large-scale computations. For example, they are used for circuit simulation [2] with large network contraction heuristics [3] or using a state vector with bounded bond dimension [4], and in condensed matter computations with 1D [5] and higher-dimensional methods [6, 7]. These computations with classical tools push the frontier of experiments that require simulation with quantum resources [8]. In fact, they have recently played a role disproving quantum advantage claims [9] in cutting-edge Noisy Intermediate-Scale Quantum devices [10]. A particular type of 1D TN, namely Matrix Product States (MPSs) [11], has been identified as the variational structure operated by the celebrated Density Matrix Renormalization Group algorithm (DMRG) [12]. Moreover, their flexibility in storing data and the usefulness of the bond dimension \(\chi\), a parameter that controls how much correlation is allowed, has enabled this tool to see success in machine learning as well [13, 14] using gradient-based tools (either first- or second-order), similarly to classical machine learning. Other regular structures, such as 2D projected entangled pair states (PEPSs) [7] or the multi-scale entanglement renormalization ansatz (MERA) [6], have also been extensively used for successfully simulating many-body systems. The choice of the TN geometry is related to how physical systems present different entanglement structures: networks with high complexity might be able to represent interesting states better. On the other hand, the use of highly intricate TN structures is limited by the computational cost of contractions and increased memory requirements. The interplay between these behaviors is crucial when choosing the strategy to solve a problem, and a common and reasonable approach is to settle for one of these regular structures that is just complex enough for the target system, but not more [15]. Another option is to find heterogeneous geometries that are tailored to the target system, which has seen growing interest recently, with proposals to characterize them mathematically [16; 17], as well as algorithms to find the optimal choice [18; 19; 20]. The geometric traits used in these descriptions and algorithms have an impact on the training performance in real applications, but have not been characterized, despite their relevance in contraction strategies [3], numerical implementations [21], large-scale computations [22], and the appearance of barren plateaus [23; 24]. In this article, we present different low-dimensional TN structures and characterize them in terms of contraction costs and memory usage. We train them to copy random quantum states, which do not have any implicit correlation structure, using a set-up that relies on gradient-based tools with automatic differentiation. To do so, we propose a tensor contraction with a surrogate tensor, which gives us complete control over the complexity of the target system and avoids some statistical subtleties of training with samples. This enables the establishment of a training method much more suitable for testing different properties of heterogeneous TNs with systems of specific geometries. In addition, we introduce the _compact_ version of any tree TN structure and discuss when it can present an advantage. These simulations reveal links between the quality of the encoding and the density of these heterogeneous TN structures, as MPSs and some large-diameter tree TNs perform worse than their more dense counterparts, even though the bond dimension is high enough to store the same amount of information in principle.