SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Tree tensor network is a fully binary tree structure, of which the number of tensor nodes depends on the height. We reload the initialization function and write the build function as follows,
TensorNetwork:: TensorNetwork(int H) { // init the graph
this.H = H; this.V = pow(2,H) - 1; this.E = 0;
this(adj = new Queue[V];
for (i = 0; i < V; i++) {
    this.adj[i] = new Queue {};
}
void TensorNetwork::Build() { // Build for tensor ring
    this.AddEdge(0, 1); this.AddEdge(0, 2);
    for (h = 1; h < this.H; h++) {
        for (v = pow(2, h-2); v < pow(2, h-1) + 1; v++) {
            this.AddEdge(v, 2v); this.AddEdge(v, 2v+1);
        }
    }
}
The generation code for the tree tensor network is saved in https://github.com/XiaoYangLiu-FinRL/RL4QuantumCircuits/blob/main/datasets/tree/tree_generate.py.
PEPS Network: The PEPS (projected entangled pair state) tensor network [49, 58] generalizes MPS from a one-dimensional network to a network on an arbitrary graph, whose tensor diagram notation can be represented in Fig. 11.
Some work applies the PEPS network to quantum circuits [43] and quantum systems [40, 31].
1   
2   
3
1   
2   
3   
4   
5   
6   
7   
8   
9   
10   
11   
12   
13   
14   
15   
16
Figure 11: PEPS tensor network.
The generation code for the PEPS network is saved in https://github.com/XiaoYangLiu-FinRL/RL4QuantumCircuits/blob/main/datasets/peps/peps_generate.py.
MERA Network: The MERA (Multiscale Entanglement Renormalization Ansatz) [14] tensor network colleagues as a refinement of the MPS and PEPS. It has a hierarchical structure, with layers of tensors representing increasingly coarse-grained degrees of freedom, which can be represented in Fig. 12.
Figure 12: MERA tensor network.
Some work applies the MERA network to quantum circuits [1, 20, 1, 30].
The generation code for the MERA tensor network is saved in https://github.com/XiaoYangLiu-FinRL/RL4QuantumCircuits/blob/main/datasets/mera/mera_generate.py.
Google's Sycamore Circuits: Some work has put efforts into transforming the Sycamore circuits into tensor representations. The provided data is usually organized into the adjacent graph structure, while the indices of the  $i$ -th row are the indices of connected tensors to  $i$ -the tensor. The tensor network representation of the Sycamore circuits corresponds to a complicated net structure. To map the Sycamore circuits into the tensor network environment, we need to iteratively read each line of the given Sycamore file and fill it up with corresponding dimension information.
The generation code for the Sycamore circuit is saved in https://github.com/XiaoYangLiu-FinRL/RL4QuantumCircuits/tree/main/datasets/sycamore.
B.5 Environment
We provide the gym-environment for the tensor network contraction problem in our classical simulation of quantum circuits. The code is provided in https://github.com/XiaoYangLiu-FinRL/RL4QuantumCircuits/blob/main/rl/mps/env.py for tensor-train tensor network and https://github.com/XiaoYangLiu-FinRL/RL4QuantumCircuits/blob/main/rl/sycamore/env.py for Sycamore circuits.
Specifically, the gym-environment for the tensor network contraction problem in our classical simulation of quantum circuits is designed as follows,
- Init/Restart: initialize the states of environments, including the number of nodes, the number of edges, etc.
- Step: Given the tensor network contraction order, compute the number of involved multiplications as the reward. The rewards are represented in log scale.
The implementations can be found in init(), and get_log10_multiple(times() functions of our env.py.
B.6 High-Performance Reinforcement Learning for TNCO
Mapping onto the K-spin Ising Model: We use the K-spin Ising model to formulate the TNCO problem as follows,
$$
H (x) = \sum_ {i = 1} ^ {N - 1} \left[ \left(2 - \sum_ {u = 1} ^ {N - i} x _ {u, i}\right) ^ {2} + \sum_ {u = 1} ^ {N} \sum_ {v = 1} ^ {N} J _ {u _ {i}, v _ {i}} x _ {u, i} x _ {v, i} \right], \tag {12}
$$
$$
H (x ^ {1},..., x ^ {K}) = \sum_ {k = 1} ^ {K} H (x ^ {k}) + \sum_ {k = 1} ^ {K} \sum_ {i _ {1} \in V ^ {1}} \dots \sum_ {i _ {1} \in V ^ {k}} J _ {i _ {1} \dots i _ {k}} x _ {i _ {1}} ^ {1} \dots x _ {i _ {k}} ^ {k},
$$
where we denote the  $N(N - 1)$  spin as  $x_{u,j}$ ,  $u$  as the tensor and  $j$  denotes its order in the TNCO path.
We use the variational annealing methods to solve the K-spin Ising model problem of TNCO. Specifically, we minimize the KL divergence between the transition distribution of  $x$  to  $x'$  with the target Boltzmann distribution as follows,
$$
\begin{array}{l} \mathcal {D} _ {K L} (q _ {\theta} | | p) = \sum_ {x} q _ {\theta} (x \to x ^ {\prime}) \ln \left(\frac {q _ {\theta} (x \to x ^ {\prime})}{p (x \to x ^ {\prime})}\right) \\ = \sum_ {x} q _ {\theta} \left(x \rightarrow x ^ {\prime}\right) \ln q _ {\theta} \left(x \rightarrow x ^ {\prime}\right) \tag {13} \\ + \frac {q _ {\theta} \left(x \rightarrow x ^ {\prime}\right)\left(H (x) - H \left(x ^ {\prime}\right)\right)}{T} + \ln Z _ {x}, \\ \end{array}
$$
where  $T$  is the temperature,  $q_{\theta}$  is the distribution of TNCO path  $x \to x'$  parameterized by  $\theta$ ,  $p(x'|x)$  is the transition distribution of Boltzmann distribution  $p(x'|x) = \frac{\exp(-\frac{H(x')}{T})}{Z}$ ,  $Z_x = \sum_{x'}^{H(x') < H(x)} e^{-\frac{H(x')}{T}}$ . During the learning process, we gradually anneal the temperature to optimize (13).
Parallel data sampling: We use RNN parameterized by  $\theta$  as the policy network to model the transition probability  $q_{\theta}(x \to x')$ . We input the K-spin representation of tensor contraction order,  $(x_1, \ldots, x_K)$ , sequentially from left to right, and compute the transition probability as follows,
1. Randomly initialize the hidden variable  $h_1$ ;
2. Input  $x_{1}$  and  $h_{1}$  to the RNN, and output the next hidden variable  $h_{2}$  and a transition probability  $q_{\theta}(x_{1})$ ;
3. Input  $x_{2}$  and  $h_{2}$  to the RNN, and output the next hidden variable  $h_{3}$  and a transition probability  $q_{\theta}(x_1 \to x_2)$ ;
4. ......
5. Input  $x_{K}$  and  $h_K$  to the RNN, and output the next hidden variable  $h_{K + 1}$  and a transition probability  $q_{\theta}(x_{K - 1}\to x_K)$ .
Then, by sampling the tensor contraction ordering trajectories,  $(x_{1},x_{2},\dots,x_{K})$ , we can optimize (13) to train the RNN. The sampling process can be batched onto multiprocess to achieve high-performance data sampling, thus alleviating the performance bottleneck of reinforcement learning training.
Parallel training: We initialize multiple optimizers in parallel to learn to optimal tensor contraction order. Specifically, at the  $t$ -th iteration
1. We parallelly sample  $N$  tensor contraction ordering from the TNCO environment and store them in the reply buffer.
2. For the optimizer  $i$ -th, we first freeze  $\rho_{t,i}$  parameters,  $0 < \rho_{i} < 1$ , then sample data from replay buffer, and compute the loss function (13) with the temperature  $T_{t,i}$ , independently.
3. For  $i$ -th optimizer, we compute the gradient and use LSTM to learn to optimize the parameter.
We vary the parameter masked ratio  $\rho$  and temperature  $T$  by increasing the number of iterations and initialized all the parameters differently to achieve swarm intelligence integrated with the curriculum learning.
Appendix C Accessibility, Usage, License, and Maintenance
Accessibility: All the code, dataset, and tensor network contraction orderings, including the Sycamore circuits, can be found in our open source project https://github.com/XiaoYangLiu-FinRL/RL4QuantumCircuits without personal request.
Dataset generation: We generate the classical simulation of quantum circuits synthetically using https://github.com/XiaoYangLiu-FinRL/RL4QuantumCircuits/tree/main/datasets.
For example, we run https://github.com/XiaoYangLiu-FinRL/RL4QuantumCircuits/blob/main/datasets/mps/mps_generate.py to generate quantum circuits based on the tensor train, where  $V$  in code controls the number of nodes in generated data.
Code organization: We implement the baseline methods using Opt-einsum and Cotengra to search for the tensor network contraction orderings. The codes for calling these solvers are provided in https://github.com/XiaoYangLiu-FinRL/RL4QuantumCircuits/tree/main/baseline. We provide two methods to solve each type of quantum circuit, like tensor train-based quantum circuits.
Usage: To run the baseline methods, execute "python cotengra.py" or "python opt_einsum.py", where the variable  $n$  is the number of nodes. To generate the dataset, please execute "python generate.py", where the variable  $V$  in the central part is the number of tensor nodes.
License: MIT License.
Maintenance: On GitHub, we keep updating our codes, merging pull requests, and fixing bugs and issues. We welcome contributions from community members and researchers.