# Advantages of Density in Tensor Network Geometries for Gradient-Based Training
## 2 Tensor Network Structure
### Choice of Structures  
We focus our study of gradient-based tensor network training mostly on different tree tensor networks, also including the 2D PEPS [7] geometry comparison when the size of the system allows it. In Figure 1, we show each different geometry used in the training. Beyond the known ansatzes of MPSs and PEPSs, we have an \"Antenna\" structure that reduces the distance between sites with respect to an MPS, without allowing tensors with more than three virtual indices, whereas \"Balanced\" achieves an even smaller distance at the cost of having bigger tensors with four virtual indices. We also propose a _highly connected_ tree with \"Star\({}_{n}\)\", where a single site is connected to many sites. For \(n=1\), the site distance is minimal without introducing loops, and the central tensor is as complex as the _dense_ tensor, but the contractions of tensors on a single site are minimally expensive (comparable to the edge of an MPS). As \(n\) grows, the beams of the star have more nodes and the distance between nodes increases. In the trainings, we use \(n=1,2,3\) (despite \(n=3\) not being depicted in Figure 1), and \(n=1,2\) especially require much bigger tensors than the other tree structures.  
Tree tensor networks have traditionally been used with the binary tree structure [37], which was also a precursor to the success of MERA [6]. This structure can be made more flexible by dropping the binary requirement [38]. The structures that we study here are equivalent to this generalized tree TN but with some of the branches contracted. One can also allow for loops in the connectivity, which increases the representation power of the structure, but has to forgo most of the performance in the algorithms that make MPSs very efficient (for example, using canonical tensors becomes much more complicated [39; 40]). The connectivity of the structure, once loops are allowed, can be taken beyond 1D MPSs and 2D PEPSs into the extreme all-to-all case, in which an \(n\)-site system would be encoded into an \(n\)-dimensional tensor network, reaching maximal contraction complexity and representation power. On the other hand, restricting the ansatz to be loop-free makes it possible to establish an ordering inside the structure, which allows MPS tools to be used while still making use of the increased flexibility beyond a chain structure [37].  
Heterogeneous TNs such as the irregular trees we study here are not very commonly used. A structure that works well in general and has highly developed tools can present good performance for optimizing a systems that is not characterized, even if it is not optimal. However, an efficient heuristic that tailors a structure to a given problem could offer an additional advantage. Efforts in this direction have recently been made, both with intuitive heuristics [20] and with machine learning methods [18; 19], achieving good results in terms of identifying entanglement structures in a specific system [41]. These structures are thus a good fit for systems whose known properties picture a certain entanglement structure, from which one can propose a physically motivated ansatz. We contribute to this application by studying the benefits and drawbacks of the resulting structures in regard to training methods. In the future, these experimental findings can help bridge the field with research that aims to characterize TN geometries mathematically [16; 17].