Lecture 5: Tensor-Network Renormalization Group(TNRG)
Naoki KAWASHIMAISSP, U. Tokyo

May 13, 2019

Naoki KAWASHIMA (ISSP)

Statistical Machanics I

May 13, 20191 / 22

In this lecture, we see ...
The MKRG was manageable, but is rather crude an approximation.
Even worse, we do not know when we can expect this approximation
to be good or how we can improve systematically.
Real-space renormalization group method based on tensor-network
representation (TNRG) provides us with a method for computing thepartition function. While TNRG is also an approximation, it comeswith a method for systematic improvements, and may produce theexact critical exponents in the limit.

Naoki KAWASHIMA (ISSP)

Statistical Machanics I

May 13, 20192 / 22

[5-1] Tensor-network renormalization group (TNRG)
Most of statistical-mechanical models on lattices are tensor networks.
Quantum many-body states on lattices are also described by tensor
networks.
As we have seen, after renormalization transformation, we needinfinitely many parameters to describe the resulting system.
By working with the TN representation, and introducing “datacompression” at all length scales, we can overcome both the faults in
the real-space RG.

Naoki KAWASHIMA (ISSP)

Statistical Machanics IMay 13, 20193 / 22

What is a tensor network?
When an object is expressed as the result of (full or partial)
contraction of tensor-product of multiple tensors, we call such an
expression a “tensor-network”. An expression such as!
YX YkContT≡
TSk k ,S k ,··· ,S k
(1)k(Si )i∈Ω ki
1i2inkis a tensor-network, where Ω is a subset of all indices, {iαk }, appearing
multiple times (typically twice) in the summand.
Example:TS1 ,S2 ,S3 ,S4 =X

TS11 ,S8 ,S5 TS22 ,S5 ,S6 TS33 ,S6 ,S7 TS44 ,S7 ,S8S6 ,S7 ,S8 ,S9Naoki KAWASHIMA (ISSP)

Statistical Machanics I

May 13, 20194 / 22

Statistical-mechanical models are tensor networks
The partition function of the Ising model on thesquare lattice can be expressed asXYZ=
W (Sip , Sjp , Skp , Slp )S p : shaded square

W (S1 , S2 , S3 , S4 ) ≡ e K (S1 S2 +S2 S3 +S3 S4 +S4 S1 )We can regard W (S1 , S2 , S3 , S4 ) as a degree-4 tensor.
Then, the above equation is a tensor network representation of thepartition function.

Naoki KAWASHIMA (ISSP)

Statistical Machanics I

May 13, 20195 / 22Graphical notation
In TN-related discussions, we use more diagrams than equations
because it is often much easier to grasp the idea.

For tensors, we often use bulkier symbols than just dots such as
circles, triangles, squares, etc, while we use simple lines for indices.
(This is more natural from the information-scientific point of view
because tensors are the carriers of most of the information.)

Naoki KAWASHIMA (ISSP)

Statistical Machanics I

May 13, 20196 / 22

Wave function can be represented as TN (1)Consider a quantum many-body system defined on a lattice.
A local quantum degree of freedom, say Si , is defined on each site.
Accordingly, we have a local Hilbert space Hi ≡ {|Si ii } associatedwith each site, e.g., Hi is 2-dimensional for S = 1/2 spin models.
NThe whole Hilbert space is the product of them H ≡ i Hi .
Any global wave function |Ψi can be expanded asX
X|Ψi ≡
CS1 ,S2 ,··· ,SN |S1 , S2 , · · · , SN i ≡CS |Si{Si }S

where |S1 , S2 , · · · , SN i ≡ |S1 i1 ⊗ |S2 i2 ⊗ · · · ⊗ |SN iN .
Now, CS1 ,S2 ,··· ,SN can be viewed as a degree-N tensor. It may beapproximated by some tensor network, i.e,!
YTkCS ≈ Cont
kNaoki KAWASHIMA (ISSP)

Statistical Machanics IMay 13, 20197 / 22

Wave function can be represented as TN (2)!
CS ≈ ContYTkk=Note that CS has d N parameters (d = 2 for S = 1/2 spin systems),
whereas the tensor network can be specified by only O(N) number ofparameters. By the tensor network representation, we may be able toreduce the computation for large N down to a manageable level.

Naoki KAWASHIMA (ISSP)

Statistical Machanics IMay 13, 20198 / 22

Trivial Tensor-network RG
Let us condider classical systems, and ask how
we can use the tensor network for RG.
How can we replace the original tensor lattice
into something similar but with the unit cellbigger than the original?
Let us solve this problem starting from the trivial
TNRG:
XT́Ś1 ,Ś2 ,Ś3 ,Ś4 ≡TS11 ,S9 ,S12 ,S8⇓

S9 ,S10 ,S11 ,S12
2× TS2 ,S3 ,S10 ,S9 TS310 ,S4 ,S5 ,S11 TS412 ,S11 ,S6 ,S7

where Ś1 ≡ (S1 , S2 ), Ś2 ≡ (S3 , S4 ), · · ·Naoki KAWASHIMA (ISSP)

Statistical Machanics I

May 13, 20199 / 22

What’s wrong with trivial TNRG?
Using T́ , we can exactly express the original
partition function with lattice constant twice
larger than the original, which is good.
However, the dimension of each index of the newtensor is χ2 where χ is the index dimension ofthe original tensor.
To be more specific, to handle an L × L system
we end up with a big tensor with χL -dimensional
indices. (We cannot go to so large L.)
To make the whole procedure practically usefulfor larger systems, we need to make the index
dimension back to χ in each iteration.
Data compression is necessary!

Naoki KAWASHIMA (ISSP)

Statistical Machanics IMay 13, 201910 / 22

Rank-reducer
What we need is a ‘rank-reducer’.
A rank-reducer is a tensor whose rank (whenviwed as a matrix) is χ in stead of χ2 , andwhose insertion keep things unchanged.
If such a thing exists, we can define triangle
operators as illustrated in the figure by SVD.
Then, by cutting the network at the reduced
indices, we can define the renormalized tensor
with index dimension χ, as we disired.
Now, we must ask whether such a magicalrank-reducer exists or not, and if it does, how wecan compute it.

Naoki KAWASHIMA (ISSP)

Statistical Machanics I

May 13, 201911 / 22

Low-rank approximation (LRA)
How can we optimize the rank-reducer X for the given rank χ?
For the cost function, we take the amplitude of the local disturbancecaused by the insertion of X , i.e.,

Let us regard A and B as χ4 × χ2 matrices and the rank-reducer X asa χ2 × χ2 matrix whose rank is χ (or less).

Low-rank approximation problem
Suppose 3 integers, l, m, n, that satisfy l < m < n. For two given n × mmatrices A and B, find a rank-l, m × m matrix X that minimizes
C (X ) ≡ |AB T − AXB T |2 .

Naoki KAWASHIMA (ISSP)(2)

Statistical Machanics I

May 13, 201912 / 22

Solution to LRA problem (1)We want the rank-l matrix X
that minimizes
C ≡ |AB T − AXB T |2 .
Consider the QR-decomposition,
A = QA RA ,B = QB RB .

Then, C ≡ |RA RBT − RA XRBT |2Consider SVD: RA RBT = UΛV T .
If X satisfies
RA XRBT = Û Λ̂V̂ T ,(3)it is optimal. (∗) (Here, Û, Λ̂,and V̂ are truncated matrices atthe l-th row and/or column.)
Naoki KAWASHIMA (ISSP)

Statistical Machanics IMay 13, 201913 / 22

Solution to LRA problem (2)Now, let us define “trianguleoperators,” PA and PB , by1

PA ≡ RBT V̂ Λ̂− 2 ,1PB ≡ RAT Û Λ̂− 2Then, because RA RBT = UΛV T ,1

RA PA = Û Λ̂ 2 ,1
RB PB = V̂ Λ̂ 2 .
Therefore, X ≡ PA PBT , satisfies
Eq.(3), RA XRBT = Û Λ̂V̂ T , andtherefore is the optimal
rank-reducer.

Naoki KAWASHIMA (ISSP)

Statistical Machanics I

May 13, 201914 / 22

Supplement: Theorem for Low-Rank Approximation (LRA)
Theorem 1 (Eckhart-Young-Mirsky)Λ≡For a given n × m matrix A, consider its
approimation by a rank-l (l ≤ m ≤ n) matrix Xand its error E 2 = |A − X |2 where |A|2 ≡ Tr AT A.
Let A = UΛV T be the singular value
decomposition (SVD) of A with an n × m diagonal
matrix Λ and n and m dimensional unitaries, Uand V , respectively. Then,


λ10···.

0.
.
.
00.
.
.
0λ2.

.

.
..
······.

..
0·········0.
.
.


0 
.
λm 
0 

. . . 0

λ1 ≥ λ2 ≥ · · · ≥ λm ≥ 0

E 2 ≥ λ2l+1 + λ2l+2 + · · · + λ2m
where λi is the i-th largest singular value. Thelower bound is attained by X ≡ Û Λ̂V̂ T where ’ ˆ’represents truncation at the l-th row/column.

Naoki KAWASHIMA (ISSP)Statistical Machanics IMay 13, 201915 / 22

Supplement: LRA used in the derivation
Theorem 2 (LRA)
Consider an m × m matrix Y expressed as Y = RA RBT with RA and RB ,and consider its SVD, Y = UΛV T . Then, Y ’s optimal LRA of the form
RA XRBT with rank l (l < m) matrix X is obtained when RA XRBT = Û Λ̂V̂ T.
Proof: When the condition of the theorem is satisfied,|Y − RA XRBT |2 = |UΛV T − Û Λ̂V̂ T |2= |UΛV T − U Λ̃V T |2 = |Λ − Λ̃|2 =mXλ2k ,k=l+1

where Λ̃ is Λ with singular values λk (k > l) replaced by 0. Therefore,
RA XRBT saturates the inequality of the EYM theorem.
Naoki KAWASHIMA (ISSP)

Statistical Machanics I

May 13, 201916 / 22

Summary of the TNRG procedure
1

QR-decomposition of A and B matrices.
A = QA RA , B = QB RB
RA RBT = UΛV T

2SVD.

3

Compute the “triangle operators”.
1

PA ≡ RBT V̂ Λ̂− 2 ,1
PB ≡ RAT Û Λ̂− 24

Do the same for other directions.

5

Using the triangular operators, contract fouroriginal tensors to obtain the new elementtensor T́ .

6Repeat these till the desired system size has
been reached.

Naoki KAWASHIMA (ISSP)

Statistical Machanics I

May 13, 201917 / 22

TNRG provides accurate estimates
The free energy can be obtained
to the accuracy of nearly 8digits. (“TRG” in the figure.)(“TRG” is essentially the same, but technically different
way of realizing TNRG from the one discussed in thislecture. See Levin and Nave, Phys. Rev. Lett. 99,120601 (2007) for details.)

An improvement (“TNR”)
pushes it even up to 10 digits.

Naoki KAWASHIMA (ISSP)[Evenbly and Vidal, Physical Review Letters
115, 180405 (2015)]

Statistical Machanics IMay 13, 201918 / 22

How we can compute other quantities
From the method described so far, we can obtain F , E , S and C . Whatabout the magnetization, M, χ, and the Binder ratio?
Define “impurity tensors”,
T (0) ≡ T , T (n) ≡ 0 (n > 1)
(1)
TS1 S2 S3 S4 ≡ TS1 S2 S3 S4 × m(S1 , S2 , S3 , S4 )where m = (S1 + S2 + S3 + S4 )/2 .
Define “renormalized impurity tensors”:
XT́ (n) ≡Cont(T (n1 ) T (n2 ) T (n3 )(n1 n2 n3 n4Pk nk =n)(n4 )×T× (triangle tensors))

At the enditerations,Xof allX (0)(n)
hM n i =TS1 S2 S1 S2 /TS1 S2 S1 S2
S1 S2
Naoki KAWASHIMA (ISSP)S1 S2Statistical Machanics I

May 13, 201919 / 22

Application of TNRG to q-state Potts model (1)q-state Potts model in 2D.
[S. Morita and N.K., ComputationalPhysics Communications, 236 65-71(2019).]
n-th moments of magnetization arecomputed (e.g., magnetization(n = 1), susceptibility (n = 2),Binder ratio (n = 4), etc)
The result of 20 RG iterations (i.e.,L = 220 ≈ 106 ) was obtaind for
q = 2, 3, · · · , 7 for the truncation
dimension (‘bond-dimension’)χ = 48.

Naoki KAWASHIMA (ISSP)

Statistical Machanics I

May 13, 201920 / 22

Application of TNRG to q-state Potts model (2)According to the finite-size scaling(FSS), which we will discuss later, theBinder ratio is defined as U4 ≡hM 4 i/hM 2 i2 depends on T and L as
1dU4= log L+a+bL−ω +· · ·dT T =Tc
ν
For first-order transitions, 1/ν = d isexpected.
The 1st order nature of the transition of5-state Potts model has been
confirmed. (CF: ξ ≈ 2500 at Tc ).
[S. Morita and N.K., Comp. Phys.
Comm. 236, 65-71 (2019).]

Naoki KAWASHIMA (ISSP)

Statistical Machanics I

May 13, 201921 / 22

Summary
Tensor-network RG (TNRG) is a scheme that realizes “datacompression” at every length scale.
With TNRG, we can systematically improve the real-space RG byadjusting the compression level, i.e., by increasing the cut-offdimension χ (often called “bond-dimension”).
TNRG provides us with rather accurate estimates of various quantities
and critical indices.
While we have seen just one way of implementing the idea, there aremany proposals for realizing TNRG. (MERA, TRG, TNR, loop-TNR,etc)

Naoki KAWASHIMA (ISSP)

Statistical Machanics I

May 13, 201922 / 22