Machine Learning
synthetic data, scanning probe data, andreciprocal space data on quantum materials.
Eun-Ah Kim (Cornell)
SLAC Oct 19, 2018

APS March Meeting 2019 Focus Topic• 16.01.05 Machine Learning in Condensed Matter Physics (DCOMP,
DCMP, DAMOP)• Submission Deadline for Contributed talks Oct 26.
• Under DCOMP

Data Revolution in R-space

Tunneling Density of States,in 1962Tunneling Density of States,in 1962

Differential conductance dI/dV @ V
proportional to N(E=eV)

Giaever et al, Phys.
Rev. 126, 941 (1962)

Tunneling Density of States,in 2000’s

Imaging N(r,E):
Scanning Tunneling SpectroscopyTunneling Density of States,in 2000’s

Imaging N(r,E):
Scanning Tunneling Spectroscopy

Data Revolution in Q-space

Sparse Data with Point Detectors~~r) = Aeikin ·~r0 S ⇢(~r0 )d~r0 eikout ·(~r ~r0 )out (~
Intensity at ~r(~kin , Ein )d~r0(~kout , Eout ):I(~q , !)
~kout

~
q
~kin

r0 ) = Aein (~i~kin ·~r0! ⌘ (Eout

Ein )Comprehensive Data with Modern Detectors

https://neutrons.ornl.gov/sequoia

High energy X-ray Data at CHESS> Possibly 5 TB per day

Neutron data from Spallation
Neutron Source> Possibly 200 GB per hourComprehensive, too comprehensive!

Learning Emergence
SimplePrinciples

Regression,
Classification
Hypothesis Test

Complex Realityof Data

Generation
Entropy
Phonons
Topological Invariant
Fermi statistics

Quantum
Complexity

Phase Diagrams
Many-body WF

What can we do with ANN for
Quantum Matter?

Use Neural Networks to
Represent
Many-Body Wave Functions

Classify
Numerical & Experimental
DataML in Quantum Matter Physics••

Representing Wave functiono Variational Wave Function represented through neural networks
https://arxiv.org/abs/1606.02318, Carleo & Troyer, Science (2017)
o Mapping Tensor Network to Neural networkhttps://arxiv.org/pdf/1701.04831.pdf Tao Xiango Neural Network Representation of Ground State WF of solvable models
Dong-Ling Deng, Xiaopeng Li, Das Sarma
https://arxiv.org/abs/1609.09060
https://arxiv.org/abs/1701.04844, PRX (2017)
Detecting Phaseso Supervised§ 2D Ising model & 2D Ising lattice gauge theory arXiv:1605.01735 Carrasquilla and Melko, Nature Physics (2017)§ Finite-T repulsive U 3D Hubbard arXiv:1609.02552 Melko, Khatami et al§ Zero-T repulsive U honeycomb Hubbard arXiv:1608.07848 Melko, Trebst et al§ Fractional Chern Insulator, arXiv:1611.01518, Yi Zhang & E-AK, PRL, Physics Viewpoint (2017)§ Z2 QSL with mutual statistics, arXiv:1705.01947, Yi Zhang, Melko, E-AK
§ MBL, arXiv:1704.01578 Neupert et al§ Hard-core bosons: superfluids, KT, Semi-unsupervised, arXiv:1707.00663, Broecker, Assaad, Trebst
Unsupervised (PCA and Autoencoders): so far, all classical.
arXiv:1606.00318 Lei Wang: 2D Ising
https://arxiv.org/abs/1703.02435 S. Wetzel: 2D Ising, 3D XY
https://arxiv.org/pdf/1704.00080.pdf Hu, Singh, Scalatter, Various spin models including highly frustrated three component (S in{-1,0,1}) spin model).
https://arxiv.org/pdf/1706.07977.pdf Ce Wang &Hui Zhai, Classical frustrated spin model
Theoretical Physics of Deep Neural Networks:o Connection between RG and fully connected deep network, arXiv:1410.3831, Mehta and Schwabo•Used Neural Networks to
Represent
Many-Body Wave Functions
Carleo and Troyer, Science
355, 602 (Feb, 2017)2
M (S; W) =

X

{hi }

exp 4

X
j

aj jz +

Xib i hi +X
ij3

Wij hi jz 5

The network parameters W = {a, b, W }: A compact representation of the
many-body stateUsed Neural Networks to
Carrasquilla and Melko, Nat.
Phys. ,13, 431 (May, 2017)Classify
Numerical Data

• Supervised Learning on the (thermalized) raw
configurations• Speed-up from “seeing through” noisy data.

Bench-Marked against known results for• The 1D Transverse Field Ising Model• The Antiferromagnetic Heisenberg Model in 1D and 2D(square lattice)• The Ferromagnetic Ising ModelBut all Long-Range Ordered States are Classical!!

Beyond Long Range Order @ Cornell
1. Discerning Topological Phases,
Many-body Localized Phases inSynthetic Data.
2. Seeking Theoretical Insights inExperimental Data from STM.

Discerning Numerical Data
Yi Zhang• Chern Insulators• Z2 Quantum Spin Liquid
PHYSICAL REVIEW B 96, 245119 (2017)

Machine learning Z2 quantum spin liquids with quasiparticle statistics
Yi Zhang,1,* Roger G. Melko,2,3 and Eun-Ah Kim1,†
1

Department of Physics, Cornell University, Ithaca, New York 14853, USA
Perimeter Institute for Theoretical Physics, Waterloo, Ontario N2L 2Y5, Canada
3
Department of Physics and Astronomy, University of Waterloo, Ontario, N2L 3G1, Canada(Received 20 May 2017; revised manuscript received 30 October 2017; published 13 December 2017;
corrected 12 February 2018)
2

Interpretability:
What did Neural Network Learn?

After decades of progress and effort, obtaining a phase diagram for a strongly correlated topological systemremains a challenge. Although in principle one could turn to Wilson loops and long-range entanglement, evalua
these nonlocal observables at many points in phase space can be prohibitively costly. With growing excitem
over topological quantum computation comes the need for an efficient approach for obtaining topological phdiagrams. Here we turn to machine learning using quantum loop topography (QLT), a notion we have rece
introduced. Specifically, we propose a construction of QLT that is sensitive to quasiparticle statistics. We thenmutual statistics between the spinons and visons to detect a Z2 quantum spin liquid in a multiparameter phspace. We successfully obtain the quantum phase boundary between the topological and trivial phases usin
simple feed-forward neural network. Furthermore, we demonstrate advantages of our approach for the evalua

Chiral Topological Phase:
Chern insulator TQPT

ve a well-defined gauge-invariant quantity.
We will give a physical interpretation of in the next section as a one-dimensional
olarization by relating changes in to electrical currents. (A generalization of thiserry phase is remarkably useful for the theory of polarization in real, three-dimensional
aterials.) In the next section we will understand how this one-dimensional example
related to the two-dimensional integer quantum Hall e↵ect. Historically the underanding of Berry phases in the latter came first, in a paper by Thouless, Kohmoto,en Nijs, and Nightingale [?]. They found that, when a lattice is put in a commensute magnetic field (one with rational flux per unit cell, in units of the flux quantum
that Bloch’s theorem applies), each occupied band j contributes an integerZinj =dkx dky h@kx uj |@ky uj i h@ky uj |@kx uj i
(2.6)
2⇡

Chern Insulator• Chern Number= Integral of Berry Flux
associated with Bloch Wave Function• Hall Conductivity

the total Hall conductance:

e2 X
nj .
xy =
h j

(2.7)

ow we derive this topological quantity (the “Chern number”, expressed as an integral

opography is not restricted to any particometry as di↵erent lattice geomtery willough di↵erent dimension D(dc ) for given
the entire procedure takes place in real
any need for diagonalization or flux inserrocedure does not depend on translational
ence TQL topography should be able tommodate heterogeneity, disorder and innstruction.[36] In the rest of this paper wehout loss of generality, to build the TQL
sampling the many-body ground state of
domly selected Monte Carlo steps (see Ap-

Topological quantum phase transition in a free fermion
model– We first apply the TQL topography-based machine learning to the topological quantum phase transition between a trivial insulator and a Chern insulator.
Consider the following tight-binding model on a squarelattice:X
H() =( 1)y c~r†+x̂ c~r + [1 + ( 1)y (1 )]c~r†+ŷ c~r

Model part I: Free Fermion

QL topography is obtained for a given
the image x into a neural network(Fig. 1).
signed a feed-forward fully-connected neuith only one hidden layer consisting ofd neurons. The network takes the TQL

~
r

h
i
i
†
†
c
c~r + c~r+x̂ ŷ c~r + h.c.
+ ( 1)y2 ~r+x̂+ŷ(2)where ~r = (x, y) (see Fig. 2) and  is a tuning parameterwith 0    1. The  = 1 limit is the ⇡-flux square• Topological
QuantumPhaselatticemodel for a Chern
insulator witha Chern number
C = 1 Transition[5], while the at =κ=0.5
0 limit amounts to decoupled
two-leg ladders. H() interpolates between a Chern insulator•anda trivial
insulator
with a topological quantumκ<0.5
trivialinsulator
phase transition at  = 0.5. To observe the quantum• κ>0.5 Chern insulator

Each entry of this vector is a chained
productMonteof twoindependent
Car
point correlators on each side of a triangle
that acts
Markov chain.
For oninst
Quantum
Loop
Topography
independent
Monte
Carlo
steps without
averagingovermasite j forthe triangle
Markov chain. For instance TQL data associated withP̃• QLTmadedata entry
forkinput
site j for the triangleof sitesand lxwould beDE†
cj ck (1)fo↵

where P̃jk ⌘Monte CarloD
E for a particularparticularMC
config.
†phy evaluatedcan be systematicall
where P̃jk ⌘ cj ck
for free fermions
with a↵ entry of x is complex
more non-local
correlatio• Eachvaluedparticular Monte Carlo sample ↵. Hencecut-o↵ TQL
lengthtopograscale dc .
phy can be systematically
expandedconverges
to include
more
and
for
small
dc ,
• Length(x)=2xLxLxD(d)
cmore non-local correlations involvingthreesite dimensional
j by increasing
image
cut-o↵ length scale dc . Conversely,Clearly
when TQL
the outcome
topographyular lattice geometry
converges for small dc , the TQL topography
will be a as
FIG. 1. Schematic illustration of our machine learning algoonlytwo-dimensional.
enter through di↵edimensional
imagethat is quasi
rithm consisting of three
TQL topography
and a neuralnetworkarchitecture. The TQL topography for each site j consists
Moreover,the
entir
FIG. d2. .Model
illustrationof Eq.
2.

P̃jk P̃kl P̃lj

high
can thebe gleaned
by observing
ned by observing
connection
between the
Hallconnection
syste
conductivity
andontwo-pointoInsightfromResponse
Functionty and two-point
correlators
trianglescorrelators
aspointed out by Kitaev [39]:dlest by Kitaev [39]:
More
2
X
e
1
2
X
X
e
1
e
1
=
4⇡iP
P
P
S
to
V
=
· xy 4⇡iP
P P· S(4)h P
N kl Plj S·
4⇡iPjk
(4)
xy =
h4jkl
N
h N where P ⌘ hc c i is the equal-time two-point correlaCarl
for free fermion
tor between site i and site j, †S
is the signed area
⌘N hc
i is
theof sites.
equal-timebody
two
ij and†ofwherethe triangleP
jkl,
is thenumberi cjtotal⌘ hci cj i is theequal-timecorrelaOur alternative
proof of Eq. two-point(4) (see Appendix) buildsontorbetween
site
i
and
site
j,
S
is
th
PEP
4jkl
adiabatic continuity protected by the gap in the spectra
Acknowledgemen:
signedareaofthetrianglejkl
n site i and site
j,
S
is
the
signedarea
4jkl
without
requiringthesystem
be
made
of
free
fermions.
T.
Neupertand S.
Tof the triangle jkl, and N is the totalnu
plica
ngle jkl, and Our
N isalternative
the total proof
numberof sites.
of Eq.
(4) (see Appen
into
Kitaev, Annals ofbuilds
Physics 321,2(2006)
ative proof of Eq.
(4)(see
Appendix)onadiabatic continuity protected by the gap
ontinuity protected by the gap in the spectraAckphases by introducing on TQL topography as the relevant input data interfacing the quantum many-bodyproblem with a simple neural network. A valuable theoretical insight into the success of the TQL topography
can be gleaned by observing the connection between Hall
conductivity and two-point correlators on triangles as• Triangles arein the Chern Number!
pointed out by Kitaev [39]:
2

xy

ij

jk

kl lj 4jkl

†
i j

4jkl

real space formulatio
metry and it does noinsertion it is extrem
proach can be appli
higher Chern numbsystems. The fact thdles degenerate gro
Moreover there is n
to VMC data. It ca
jk kl lj
Carlo samples as wel
body wave function
PEPS. Hence our coplication of machine
in topological phase

Hall conductivity

v.s.

QLT
with a neural network:

expectation value

Inputs from all triangles:

quantum fluctuations

Quantum loops have cut off d≤ dc

QLT as the input vector‘nontrivial’. This indicates that the neural network unfortunately does not pick up the universal features about
the topological phase, but rather memorizes the more detailed information of the specific model at  = 1.0 itself.
The contrast in the results based on TQL topography
input is striking. Now we use TQL topography from 2000
VMC samples for each  2 [0.1, 1.0] as our input, andc Carlo instance, which
compare the output of each Monteis topological if y > 0.5 and trivial otherwise. In order tolook for the convergence upon change of the TQL cut-o↵length scale, we test di↵erent cut-o↵s dc = 1, 2, 3. Now
the network returns results with a > 99.9% accuracy deepin either the topological phase or trivial phase. Moreover,it estimates the phase diagram and the location of thephase transitions rather accurately, i.e., right about expected critical point the proposed probability p ⇡ 0.5 indicating that the neural network is maximally confused.
Note the highly symmetric departure from p ⇡ 0.5 on
both sides of  = 0.5 is reflects the symmetry in gap• Train with two known points:
κ=0.1 (trivial), κ=1 (topo)• Smallest triangles (d =1) aresufficient in the gapped phases• Once trained, get PD in 10min on
a laptop.
• 99.9% accuracy in the phaseverified with 2k test samples.
FIG. 3. The ratio p of ‘topological’ response from the neuralnetwork on the model in Eq. 2 over the parameter region
 2 [0.1, 1, 0]. The neural network is trained with  = 0.1

3

How to ”image” Quantum Loops• Organize loops by lateraldimension d=1,2,3…
• Associate each site with all thetriangles that involves the site asa vertex.
• Gap & quantization allow dc<<L• Quasi-2D
FIG. 1. Schematic illustration of our machine learningalgo- ”image” input vector xrithm consisting of TQL topography and a neural networkarchitecture. The TQL topography for each site j consists
FIG. 2. Model illustration of Eq. 2.

non-Chiral Topological Phase:Z2 quantum spin liquid

Yi Zhang , R. Melko &E-AK, PRB, 96, 245119 (2017)aracterized bys!T xIup an overall phase factor e with statistical angle ✓ = ⇡ upon
braiding a spinon around a vision, or vice versa.
Microscopically, we consider the following model on a
two-dimensional square lattice[13]:
X
X
X
X
x
z
H2D = J x
A s Jz
Bp hx
h
(3)
z
b
b

Kitaev Model under field

= e

i✓ Ix

=

y around dis-which are lahe action(2)

so expected toing! Eq. 2 with

s

p

b

b

where the spin-1/2 lives on the square-lattice bonds, A s =
Q
Q
z
x
and
B
=
are the productsof spinoperatorsp
j2s j
j2p •j Finite
region ofZ2 spinliquidaround a site s and on the boundaries
of a correlation
plaquette p, respecwith finite
lengthtivelyFigure illustration of toric code necessary or not?. The
hz and h x terms are external magnetic fields in the x̂ and ẑ di• Spinons and Visonsrections, respectively. For the rest of the paper, we consider
system size 12 ⇥ 12 unless noted otherwise.
statistics
At the special point h x• Mutual= hz = 0,
H2D amounts to Kitaev’s toric code[1]. The exact solvability of the toric code
has allowed for much progress in explicit understanding ofZ2 quantum spin liquid. Since all A s and B p commute in theup an overall phase factor e with statistical angle ✓ = ⇡ upon
braiding a spinon around a vision, or vice versa.
Microscopically, we consider the following model on a
two-dimensional square lattice[13]:
X
X
X
X
I
i✓ Ix
x
z
s!T x = e =
H
=
J
A
J
B
h
h
(3)
2D
xof thes latticez
p spinx modelz in Eq.
IG. 2. (a) Illustration
3. The spinb
b
s
p
b
b
y/2’s
arounddisreside on the bonds of the two-dimensional square lattice. A
where
the spin-1/2 lives on the square-lattice bonds, A s =
z
x
which
are
laQ
Q
ring of( ) operators
creates
at
its
two
ends
a
pair
of
spinonsz
x•
QLT
designedtoprobe
mutualj2s j and B p =j2p j are the products of spin operators
he action
visions) denotedasa ‘e’orstatistics
equivalently,
moves
one spinon (viaroundsite s(‘m’),and on theboundaries
of a plaquettep, respecillustrationnecessary orofnot?.
Theon) from onetivelyFigure
end to theother.of toric(b) code
Illustrationa nontrivial
knot
hz and h x terms are external magnetic fields in the x̂ and ẑ dietween the world-lines
of aD spinon
anda vision.
operator
rections, respectively.
For the restof Ethepaper,
considerh we Thei corzx notedx otherwise.
z
x z z
x
systemsize
12⇥
12
unlessesponding to the process is r r0 r0 r = tr ⇢ r r r0 r0 , where
At the special point h x = hz = 0, H2D amounts to Kiz
z
x
x
(2)
and
(
and
spinon(vision)along
The aexact
solvabilityof the toric
codethe x̂ and ŷ
0 ) movesr
rtaev’s toricrcode[1].
r0has allowed forThemuchvision
progressandin explicit
understandingof
so expected torespectively.
irections,spinontrajectories
intersect
Z20 quantum spin liquid. Since all A s and B p commute in the
ing
Eq.
2
with
!

aracterized by

Quantum Loop Topography for Z2 QSL

informative candidate for machine learancy, the J x , hz , h x and Jz terms in Eq. 3 are interpreted asnally, string tension and the rapidly dec
cal bond, horizontal bond, vertical plaquette and horizonand vision-vision correlations allow us tlaquette terms in the anisotropic Z2 gauge Higgs model.
longer than a cut o↵, which is set to dc =implicity, we consider the isotropic case whereX
XY
erwise.
•2+1D
Kitaev
Model under field
H3D = b
Sb
Sj
(4)
p

Kitaev Model under field

6~Classical Z2 gauge Higgs
5
MACHINE LEARNING TOPOLOGICAL
and half topological)
20% ofthe
the sizeof the training
set model
nit temperature= 1,of withfollowing
relationsbe- in 3D
b

p

j2p

for validation purposes including learning speed control andn the relatedparameters:
termination[23].

Quantum loop topography data is obt
Once the neural network is trained successfully, it can
1 topography obtained from difrapidly process quantum loop
space of Eq. (4), and supervised mach
=
h
⌧
=
ln
tanh
J
⌧
b
z
x
ferent parts of the phase space
2 to yield a phase diagram. In
formed. As illustrated in Fig. 1, once
order to establish level of confidence on the trained network’s
1 is topological or not, we
assessment of whether the system
set serves as an “image” for a fully-con
=
J
⌧
=
ln
tanh
h
⌧
(5)
p
z
x
process 2000 inputs at each point and take the ratio p of ‘topo2
neural network with one hidden layer clogical’ output, i.e., y > 0.5. Our results are compared withdiagram
Ref. note[13] as thatwell asthe
our ownbench- we will
e limit theof phasesmall⌧. in We
methodsigmoid neurons. Each neuron processes
mark on the positions of the topological quantum phase tranent also sitions.
o↵ersPrecise
an alternative
convenientsignatureofdependent weights and biases w · x + b. A
benchmarks areandindependently
obtainedusing a specificmethod.lattice
A conventional
onfinement[42]inheatan collapse
equivalentgaugeMontetheory, eventhe outcome is fed forward to be process
Carlo method is used to obtain specific heat data for a range
gh our focus
ofsizes,
discussion
onthetheZ2 quantumron. The final output 0  y  1 correof system
and a collapseisis mainly
performed tofinite-size
scalingfunctionusing criticalthe expected
FIG.
3.
ratio
ofcritical‘topological’as a function
ofwhetherp . Forliquids.
For
theinrest
theexponents
paper,of we
use threeasThe
network’s
judgementofFIG. 4. Phase
diagramboth ofFIG.
5.
locatethevalue responseofthe topologicalquantum
phase the qu
b and
p To
b and p obtained with a single neudimensional Ising universality class. On simulations with up
each
,
we
train
the
neuralnetwork
with
=
1.25
for
the
topologZhangb ,weR.look
Melko
PRB,
(2017)
p
ral network with supervisedmachine learning and quantum loop to- Yi transition,
for the&E-AK,
non-analyticity
of96,
ratio245119
p over 20000
4

Shallow Network, Deep Insight?

𝑦 𝑥⃗ = 𝜎 𝕨(' ⋅ 𝜎 𝕨* 𝑥⃗ + 𝑏* + 𝑏'

Hidden layer neurons actively involved
in decision making for topological phases

What did the AI learn for CI?
• Largest w1 weights associated with the imaginaryparts of the dc=1 loops• All sites contribute evenly.

−4.84×𝑚𝑎𝑥 0.208 6 iPjkPklPlj + 3.73,0 + 9.03 > 089:*

1
6 2𝜋iPjkPklPlj > 0.392
𝑁89:*

What did the AI learn for CI?

A topological invariant, the Chern Number:

What did the AI learn for Z2 QSL?

What did the AI learn for Z2 QSL?
1. Full Non-linearity at play!
2. Non-linear products of QLT?

Non-linearity = Large loops with local info !!

Theoretical Insight from
Experimental Data

Machine Learning Intertwined orderYi Zhang Andrej Mesaros

Y. Zhang, A. Mesaros, K. Fujita, J.C. Davis,
E. Khatami, EAK et al arXiv:1808.00479

Hypothesis Test:

96mV

D

y

96mV

N

126mV

O

I

qy

x

qx

2nm
126mV

E

y

J

qy

x

qx

2nm
Low

High

Low

High

Questions
1. Origin: r-space or k-space?
2. Checkerboard or Stripe
Robertson et al (2006), Del Maestro et al (2006)
declared defeat within manual schemes!

idealized

3-stage protocol: I

3-stage protocol: II

y

x
Low

3-stage protocol: IIICCategorized
DiverseTraining Set

noise,
ﬂuctuations,disorder,and TopologicalFigure3
Defects…

…

Categories (siTargetting Hypothesis
Charge Order
Testingin underdoped
N
Cuprates with AI
Test Data

Generated Training Data

C

y

y

x

qx

x

Low

Low

Low

2nm

1nm

D

Y

D

High High

High

qy

B

1

E
Cost

1

qx

Architecture

AC

B

qy

2

Low

Doping Dependence

ImageNet Classification with Deep
Convolutional Neural Networks

Energy DependenceHypothesis test

Unidirectional, lattice commensurate, period 4 charge modulations.

Learning Quantum Emergence with AI
The journey has just begun….
• Synthetic Data: topological order, non-equilibrium
states, non-Fermi liquid transport, SC fluctuation….
• Experimental Data: STM, X-ray, Neutron, Resonant
Spectroscopy, …
• Quantum Information: Error Correction, Quantum
State Tomography,…

Acknowledgements

Yi Zhang(Cornell)Andrej Mesaros(CNRS)

K. Fujita(BNL)

S. Trebst(Cologne)

R. Melko(Waterloo)

P. Ginsparg(Cornell)E. Khatami(SJSU)

K. Winberger(Cornell, CS)

J.C. Davis(Cornell/BNL)

A. Wilson(Cornell, ORIE)