# Advantages of Density in Tensor Network Geometries for Gradient-Based Training
## 2 Tensor Network Structure
### Training with a Surrogate System  
Training a model in machine learning usually requires a large number of samples [42] which contain the information that we want to learn. Thus, one can expect a training to consist of a set of \(k\) samples \(\phi_{i}\), which are extracted from a target physical system, and a model \(\psi\)[43], which is expected to generalize well following some known properties or theorems. In the setting of using tensor networks as the model \(\psi\), this translates to a contraction between samples and the TN ansatz, which is commonly a simple MPS [13] or a similar structure [14]. These contractions are accounted for as an ensemble in some appropriate cost function \(\mathcal{L}(\phi_{i},\psi)\). Such an approach must deal with issues like generalization of the model, sampling complexity, and so on, which are quite complex [44] and depend not only on the structure of the TN ansatz but also the training set and the properties of the model we are trying to learn.  
We propose training with a surrogate TN that represents the physical system directly in a single tensor, avoiding some of these learning issues. With this method, depicted in Figure 2, we can prepare physical systems that have specific entanglement structures or limited correlations and study how well they can be learned for a given geometry. This is achieved by initializing a random tensor network with the structure ansatz we are interested in, fixing \(\chi\), and then contracting it into a single tensor. In the final state vector-like form, the surrogate _hides_ its origin, as would happen for a target physical system with unknown properties. Because we have to prepare the tensor, it is not a substitute for learning unknown systems, but rather, a useful tool for controlled experiments. Instead of many contractions with each sample \(\phi_{i}\), we perform a single contraction between the surrogate and the model which avoids the statistical problems of sampling but, on the other hand, is more costly computationally. The final trained model in the ideal case of perfect training should be equivalent to the physical system itself, and since the model has a TN form, it is then equivalent to the surrogate itself. In this work, we focus on controlling the complexity of the target system (which would extend to that of the samples) via the bond dimension \(\chi\) of the surrogate for two different scenarios, as we will explain in Section 3, and leave for future work the study of encoding different structures into the surrogate to see if we can recover them.  
The structure of the TN model plays a crucial role in several ways. It directly constrains how hard it is, computationally, to contract the network. Consequently, it also characterizes how hard it is to find the optimal path for such contraction, with an MPS on one end (the optimal path is known beforehand) and the contraction of non-structured tensor networks, like in arbitrary quantum circuits, [3] on the other (known heuristics can find good paths, but not optimal). It also relates to how well they can represent the system that is being encoded in them. This is an obstacle that can be overcome with enough resources, as we have seen that without binding \(\chi\), even an MPS could represent the most complex quantum state. However, we will see that even with large \(\chi\), wide geometries that have enough correlations between sites can still fail to train properly. More generally, finding the appropriate geometry that reflects the correlations between different parts of the target quantum system can keep bond dimension costs and, thus, the computational cost lower, but such benefits must compete with how easy it is to find and train them.