# Advantages of Density in Tensor Network Geometries for Gradient-Based Training
## 2 Tensor Network Structure
### Tensor Budget  
In the design of our tensor network, it is important to budget the resources properly. Memory limitations are very relevant to the size of simulations that can be achieved [45; 46], and common strategies to improve performance involve a tradeoff between memory and computational cost, especially in the HPC setting [47]. This tradeoff has a direct equivalent in tensor networks. In an MPS with a fixed bond dimension \(\chi\), for example, we can contract all sites with a virtual index smaller than \(\chi\) without losing information or correlations. This is evident when using the maximum \(\chi=\chi_{S}\): for an odd \(n\)-qubit system, the central tensor with one physical bond and two virtual bonds has the same dimension (\(2^{(n-1)/2)}\times 2\times 2^{(n-1)/2)}\) as a dense representation of the same quantum system (\(2^{n}\)). On the other hand, each contraction on these _redundant_ sites becomes more expensive, which is relevant for Time-Evolution Block Decimation (TEBD) [48; 49] or circuit simulation [46], as depicted in Figure 3. This method can be trivially generalized to any TN structure without a loop. Such structures employ more tensors than strictly necessary for the amount of information that they store; from another point of view, the information is distributed over a larger number of variables that need to be trained. Figure 2: The usual training process of a TN model using samples of a target unknown physical system, to the left, compared to our method with a surrogate TN, and to the right, where we control the number of correlations in a TN with its bond dimension, and then we contract it into a single dense tensor. Both approaches output a model of the physical system in the form of another TN (in the example, an MPS). We can show explicitly when this contraction effect is relevant in the MPS. For a given \(\chi\), and assuming that the physical indices have a fixed dimension \(p\), let us consider two tensors that share a bond with dimension \(d<\chi\). The tensor that is further from the center of the MPS has a left virtual index of dimensions of at most \(d/p\), so in total, it needs a memory of at most \(d^{2}\), whereas the other tensor needs \(d\cdot p\cdot d_{r}\), where \(d_{r}\) is its other virtual bond. The contracted tensor then occupies a memory of \(d/p\cdot p\cdot p\cdot d_{r}\), which is the same as the second tensor. Thus, the sum of both uncontracted tensors will always be strictly larger. Starting from one end of the MPS, if we carry out this process \(m\) times before reaching the tensor with virtual bond dimension \(\chi\), we will have a tensor with dimensions \(p^{m}\cdot\chi\). Therefore, \(m\) is the largest integer smaller than \(\log_{p}\chi\). In particular, for the usual case where \(p=2\), we have \(m=\lfloor\log_{2}\chi\rfloor\). We can repeat this process from each end of the MPS. The closed formula for the MPS case cannot be generalized to arbitrary tree TNs, as it depends on the branching structure, but we can apply the same concept to decide which tensors to contract. Considering the leaves of the tree to be the tensors, with only one physical bond and one virtual bond, we start contracting the branches until we reach a tensor with a bond of size \(\chi\). For tree tensor networks with few branches (or where most branches connect to few tensors) the effect will be similar to that of an MPS, whereas for many branches, the memory reduction will be much more substantial. This effect of compactification is rarely the leading performance or memory factor in large MPS simulations, since that depends on the large tensors that are far from the endpoints (leaves) of the network. Specifically, it is negligible when \(n>>2\lfloor\log_{2}\chi\rfloor\). Regardless, it can be used to increase the quality of the results that are achievable with a fixed memory budget, especially with low bond dimensions or sites. On the other hand, tree structures can benefit much more, namely those with a small average distance between nodes, as they have a larger number of leaves.