Lecture 9: Tensor-Network Renormalization Group(TNRG)
Naoki KAWASHIMAISSP, U. Tokyo

June 10, 2024

Statistical Mechanics I: Lecture 9

June 10, 2024

1 / 24In this lecture, we see ...
The Migdal-Kadanoff RG was easy to compute, but, we do not know
when we can expect this approximation to be good or how we canimprove it systematically.
Real-space renormalization group method based on tensor-network
representation (TNRG) provides us with a method for computing thepartition function. While TNRG is also an approximation, it comeswith a method for systematic improvements, and may produce theexact critical exponents in the limit.

Statistical Mechanics I: Lecture 9

June 10, 2024

2 / 24

Tensor-network renormalization group (TNRG)
Most of statistical-mechanical models on lattices are tensor networks.
Quantum many-body states on lattices are also described by tensor
networks.
As we have seen, after renormalization transformation, we needinfinitely many parameters to describe the resulting system.
By working with the TN representation, and introducing “datacompression” at all length scales, we can overcome the disadvantageof the previous approximates for the real-space RG.

Statistical Mechanics I: Lecture 9

June 10, 2024

3 / 24

What is a tensor network?
Consider a graph consisting of circles and crosses connected by lines
so that all lines are terminated by circles or crosses. A cross canterminate only one line.
To each line, we assign an integer variable. To each circle we assign a
degree-k tensor Tσ1 σ2 ···σk where k is the number of lines connected tothe circle, and σk is the variable assigned to each line. We call thevariable virtual if the line is terminated by two circles, whereas we callit physical if an end of the line is terminated by a cross.
Consider the product of all the tensors, and apply the Einsteinconvention. The result is some function of physical variables. This is
the tensor network representation of the function.

Statistical Mechanics I: Lecture 9

June 10, 2024

4 / 24

Statistical-mechanical models are tensor networks
The partition function of lattice models can be regarded as a TN with nophysical variables. In the case of the Ising model, for example,
X YK(Si Sj +Sj Sk +Sk Sl +Sl Si )Z=W Si Sj Sk Sl W Si Sj Sk Sl ≡ e
S p=(i,j,k,l)

We can regard WSi Sj Sk Sl as a 2 × 2 × 2 × 2 degree-4 tensor. Then, theabove equation is a TN representation of the partition function.

Statistical Mechanics I: Lecture 9

June 10, 2024

5 / 24

Wave function can be represented as TNConsider a quantum many-body system defined on N sites.
The local Hilbert space Hi (i = 1, 2, · · · , N ) associated with the site
i is spanned by the local state vectors, |Si ⟩i , with Si taking one of dvalues.
NThe whole Hilbert space is the product of them H ≡ i Hi .
Any global wave function |Ψ⟩ can be expanded asXX|Ψ⟩ ≡CS1 S2 ···SN |S1 , S2 , · · · , SN ⟩ ≡CS |S⟩
S{Si }

where S ≡ (S1 , S2 , · · · , SN ), and |S⟩ ≡ |S1 ⟩1 ⊗ |S2 ⟩2 ⊗ · · · ⊗ |SN ⟩N .
In the case of the S = 1/2 Heisenberg model, for example, |S⟩ may
be the simultaneous eigenstates of the z-components of all the spin
operators, i.e., Ŝiz |S⟩ = Si |S⟩ where Ŝiz is the spin operator at i.
Now, CS1 S2 ···SN can be viewed as a degree-N tensor.
be approximated by some tensor network, i.e, CS ≈It mayQ
Cont ( k Tk ) .
Statistical Mechanics I: Lecture 9June 10, 2024

6 / 24

TN is a compressed representation of a wave function!
YCS ≈ Cont

Tkk=Note that CS has dN parameters (d = 2 for S = 1/2 spin systems),
whereas the tensor network can be specified by only O(N ) number ofparameters. By the tensor network representation, we may be able toreduce the computation for large N down to a manageable level.

Statistical Mechanics I: Lecture 9June 10, 2024

7 / 24

Trivial Tensor-network RG
Let us condider classical systems, and ask how
we can use the tensor network for RG.
How can we replace the original tensor lattice
into something similar but with the unit cellbigger than the original?
Let us solve this problem starting from the trivial
TNRG:
X′TS ′ ,S ′ ,S ′ ,S ′ ≡(T1 )S1 ,S9 ,S12 ,S8123⇓

4

S9 ,S10 ,S11 ,S12× (T2 )S2 ,S3 ,S10 ,S9 (T3 )S10 ,S4 ,S5 ,S11 (T4 )S12 ,S11 ,S6 ,S7
where Si′ is simply “bundled spins”, i.e.,S1′ ≡ (S1 , S2 ), S2′ ≡ (S3 , S4 ), · · ·Statistical Mechanics I: Lecture 9June 10, 2024

8 / 24

What’s good and bad with trivial TNRG?
Using T ′ , we can exactly express the original
partition function with lattice constant twice
larger than the original, which is good.
However, the dimension of each index of T ′ is χ2
where χ is the index dimension (called “bonddimension” in many papers) of Ti .
To renormalize a b × b cluster into a single
tensor, its index dimension is χb . If we repeat
this, it will cause exponential divergence.
To make the whole procedure practically usefulfor large systems, we need to keep the index
dimension below some constant.

Data compression is necessary!

Statistical Mechanics I: Lecture 9

June 10, 2024

9 / 24

June 10, 2024

10 / 24

Rank-reducer
What we need is a ‘rank-reducer’.
A rank-reducer is a matrix with rank χ, insteadof χb , that approximates the identity matrix.
If a good rank-reducer exists, we cansingular-value-decompose (SVD) it and insert
the SVDed rank-reducer as illustrated.
Then, by cutting the network at the reduced
indices and tracing out all the internal degrees of
freedom within each cluster, we can define therenormalized tensor with index dimension χ, as
we desired.
Now, we must ask whether such a magicalrank-reducer exists or not, and if it does, how wecan compute it.

Statistical Mechanics I: Lecture 9

A practical remark
In the previous slide, we compressed 2x2 tensors into one coarse-grainedtensor. This is conceptually simpler but technically more expensive thanhandling each direction one by one. In practice, the single-direction
coarse-graining is applied to each direction sequentially. (The following
illustrates the “y-direction” coarse-graining.)

After applying the coarse-graining to y and then x direction, we obtain

Statistical Mechanics I: Lecture 9

June 10, 2024

11 / 24

The optimal projector
We consider a projector P that satisfies
P 2 = P and minimizes the “error” causedby its insertion. Our problem is∗ to minimize
C ≡ ||AP − A||2 with a rank-χ matrix P .
With the singular value decomposition
The matrix A is a (χ4 ) × (χ2 ) matrix
A = V T ΛU with Λ = diag(λ1 , λ2 , · · · )whereas P is (χ2 ) × (χ2 ) with rank-χ.
(λ1 ≥ λ2 ≥ · · · ≥ 0), we have
C = Tr((AP − A)(AP − A)T ) = Tr(Λ2 (1 − U T P U )).
This suggests∗∗ that the condition for the optimal P , U T P U = 1χ , where1χ is the truncated identify matrix with rank χ. Thus we have obtained
the optimal projector P = U 1χ U T = Û Û T where Û is the truncated U .
∗This is just a compromise between the accuracy and the cost. (It would be better to minimize the error of the whole system,

not just A.) Also, preservation of the symmetries, such as reflection, may be also important, though we omit the issue here.
∗∗A more rigorous proof can be made by the EYM theorem.

Statistical Mechanics I: Lecture 9

June 10, 2024

12 / 24

Supplement: Theorem for Low-Rank Approximation (LRA)Λ≡Theorem 1 (Eckhart-Young-Mirsky)

For a given n × m matrix A, consider its
approimation by a rank-l (l ≤ m ≤ n) matrix Xand its error E 2 = |A − X|2 where|A|2 ≡ Tr AT A. Let A = U ΛV T be the singularvalue decomposition (SVD) of A with an n × m
diagonal matrix Λ and n and m dimensional
unitaries, U and V , respectively. Then,


λ1

0···
.

0.
.
.
00.
.
.
0λ2..

.

.

..

.
······.
0·········0.
.
.


0 .
λm 0 .
.
.
0λ1 ≥ λ2 ≥ · · · ≥ λm ≥ 0

E 2 ≥ λ2l+1 + λ2l+2 + · · · + λ2m
where λi is the i-th largest singular value. Thelower bound is attained by X ≡ Û Λ̂V̂ T where ‘ ˆ’
represents truncation at the l-th row/column.

Statistical Mechanics I: Lecture 9

June 10, 202413 / 24

TNRG provides accurate estimates
The free energy can be obtained
to the accuracy of nearly 8digits. (“TRG” in the figure.)(“TRG” is essentially the same, but technically different
way of realizing TNRG from the one discussed in thislecture. See Levin and Nave, Phys. Rev. Lett. 99,120601 (2007) for details.)

An improvement (“TNR”)
pushes it even up to 10 digits.

[Evenbly and Vidal, Physical Review Letters
115, 180405 (2015)]

Statistical Mechanics I: Lecture 9

June 10, 2024

14 / 24

How we can compute other quantities
From the method described so far, we can obtain F, E, S and C. Whatabout the magnetization, M , χ, and the Binder ratio?
Define “impurity tensors”,T (0) ≡ T, T (n) ≡ 0 (n > 1)
(1)
TS1 S2 S3 S4 ≡ TS1 S2 S3 S4 × m(S1 , S2 , S3 , S4 )where m = (S1 + S2 + S3 + S4 )/2 .
Define “renormalized impurity tensors”:
X(n)
T́
≡
Cont(T (n1 ) T (n2 ) T (n3 )n
1 n2 n3 n4P( k nk =n)(n4 )×T× (triangle tensors))

iterations,At the end
Xof allX (0)(n)n⟨M ⟩ =T S1 S2 S1 S2 /T S1 S 2 S1 S2S1 S 2

S1 S2Statistical Mechanics I: Lecture 9

June 10, 2024

15 / 24

Application of TNRG to q-state Potts model (1)q-state Potts model in 2D.
[S. Morita and N.K., Computational
Physics Communications, 236 65-71(2019).]n-th moments of magnetization arecomputed (e.g., magnetization(n = 1), susceptibility (n = 2),
Binder ratio (n = 4), etc)
The result of 20 RG iterations (i.e.,L = 220 ≈ 106 ) was obtaind for
q = 2, 3, · · · , 7 for the truncation
dimension (‘bond-dimension’)χ = 48.

Statistical Mechanics I: Lecture 9

June 10, 2024

16 / 24

Application of TNRG to q-state Potts model (2)According to the finite-size scaling(FSS), which we will discuss later, theBinder ratio is defined as U4 ≡⟨M 4 ⟩/⟨M 2 ⟩2 depends on T and L as
dU4
1= log L+a+bL−ω +· · ·dT T =Tc
ν
For first-order transitions, 1/ν = d isexpected.
The 1st order nature of the transition of5-state Potts model has beenconfirmed. (CF: ξ ≈ 2500 at Tc ).
[S. Morita and N.K., Comp. Phys.
Comm. 236, 65-71 (2019).]

Statistical Mechanics I: Lecture 9

June 10, 2024

17 / 24

Superoperator and its linearization
The linearized RGT can be expressed in terms of the T ∗ itself and theprojection operators as follows. The scaling dimensions can be obtained bydiagonalizing this operator as in the general case.
The diagram meansδA′ = R δA
By diagonalizing R,ϕ′µ = λµ ϕµλ µ = 2 xµ
The linearized TRG. The shaded half circles are the projection operators reducing the dimensionsof indices. (Not discussed in the lecture.) The half circles and the triangles are computed for,A∗ , the fixed point tensor (T ∗ in this lecture note). While there are four terms on the righthand side, only one is shown explicitly. For more details, see [Lyu et al, PRR 3 023048 (2021)].

Statistical Mechanics I: Lecture 9

June 10, 2024

18 / 24

Summary
Tensor-network RG (TNRG) is a scheme that realizes “datacompression” at every length scale (by the low-rank approximationof the identity matrix).
With TNRG, we can systematically improve the real-space RG by
adjusting the compression level, i.e., by increasing the cut-offdimension χ (often called “bond-dimension”).
TNRG provides us with rather accurate estimates of various quantities
and critical indices.
We have seen one particular implementation of TNRG, which is called
HOTRG. There are many other proposals for realizing TNRG, such as
MERA, TRG, TNR, loop-TNR, etc. (See Tao Xiang, “Density matrixand tensor network renormalization”, Cambridge 2023)

Statistical Mechanics I: Lecture 9

June 10, 2024

19 / 24

June 10, 2024

20 / 24

AppendicesStatistical Mechanics I: Lecture 9

Low-rank approximation (no reflection symmetry)
How can we optimize the rank-reducer X for the given rank χ?
For the cost function, we may take the amplitude of the local
disturbance caused by the insertion of X, i.e.,

Let us regard A and B as χ4 × χ2 matrices and the rank-reducer Xas a χ2 × χ2 matrix whose rank is χ (or less).

Low-rank approximation problem
Suppose 3 integers, l, m, n, that satisfy l < m < n. For two given n × mmatrices A and B, find a rank-l, m × m matrix X that minimizes
C(X) ≡ |AB T − AXB T |2 .

(1)

Statistical Mechanics I: Lecture 9

June 10, 2024

21 / 24

June 10, 2024

22 / 24

Solution to LRA problem I
We want the rank-l matrix Xminimize
C ≡ |AB T − AXB T |2 .

(2)

By the QR-decomposition,
A = QA RA , B = QB RB with QX
and RX the relevant parts oforthogonal and upper triangle
matrices, respectively,TT 2C ≡ |RA RB− RA XRB|(3)

T= U ΛV T .
Consider SVD: RA RB

If X satisfies
T= Û Λ̂V̂ T ,RA XRB(4)

it is optimal. (See supplement)

Statistical Mechanics I: Lecture 9

Solution to LRA problem II
Now, let us define“triangleoperators,” PA and PB , by1

TPA ≡ R BV̂ Λ̂− 2 ,1TPB ≡ R A
Û Λ̂− 2T = U ΛV T ,Then, because RA RB
1

RA PA = Û Λ̂ 2 ,1RB PB = V̂ Λ̂ 2 .
Therefore, X ≡ PA PBT , satisfies (4),TRA XRB= Û Λ̂V̂ T , so it yields theoptimal value of (3). This meansthat we can take PA and PB as“triangle operators”.

Statistical Mechanics I: Lecture 9

June 10, 2024

23 / 24

Summary of the TNRG procedure (no reflection symmetry)1

QR-decompose A and B matrices.
A = Q A RA , B = Q B RB
TRA RB
= U ΛV T

2SVD.

3Compute the “triangle operators”.
1TPA ≡ R B
V̂ Λ̂− 2 ,1TPB ≡ R AÛ Λ̂− 2456

Do the same for other d − 1 directions.∗
Contract the cluster of bd original operators
together with the 2d triangle operators andobtain the new element tensor T́ .
Repeat these till the desired system size hasbeen reached.

∗ Computationally, it is less expensive to bunch up one direction first before computing thetriangular operators in the other directions.

Statistical Mechanics I: Lecture 9

June 10, 2024

24 / 24