Machine learning for lattice theories1
Michael S. Albergo, Gurtej Kanwar, Phiala E. Shanahan
Center for Theoretical Physics, MIT1

Albergo, GK, Shanahan [PRD 100 (2019) 034515]

Deep Learning and Physics
Kyoto, Japan (November 1, 2019)

Machine learning for lattice theories

Real-world
lattices2

Machine learning for lattice theories

Real-world
lattices

Quantum fieldtheories

3

Lattices in the real world●

Many materials have degrees of freedom pinned to a lattice structure[Ella Maru Studio][Mazurenko et al. 1612.08436]
4

Lattices in the real world●Thermodynamics describes collective behavior of many degrees of freedom●At some temperature T, Boltzmann distribution over microstates

with

5

Lattices in the real world●Thermodynamics describes collective behavior of many degrees of freedom●

At some temperature T, Boltzmann distribution over microstates

with

Ising model has spin s = {↑,↓} per site,with energy penalty for neighboring
spins differing.
Typical microstates have patches of thesame spin at some scale.
[ "Ising Model and Metropolis Algorithm",
MathWorks Physics Team ]6

Lattices in the real world●Derive thermodynamic observables by averaging microstates

Boltzmann
distribution

Partition function

total energy

total energy

Helmholtz freeenergy

correlation
function

.
.
.

.
.
.
7

Lattices for quantum field theories●

Quantum-mechanical properties also computed as statistical expectation
values via Path Integral

similar topartition function8

Lattice Quantum Chromodynamics●

Predictions relevant to interpret upcoming
high-energy expts○○

Electron-Ion Collider will investigate detailed nuclear structure
Deep Underground Neutrino Expt requires nuclear crosssections with neutrinos

bnl.gov/eic

So far! Hong-Ye's talkfor holography ideas
dunescience.org●

Pen-and-paper methods fail, numerical evaluation of path integral req'd[D. Leinweber, Visual QCD Archive]

9

Computational approach to lattice theories●

Partition functions and path integrals are typically intractable analytically●

Numerical approximation by Monte Carlo sampling
sample integral according toestimate observables●

Markov Chain Monte Carlo converges to samples from p(𝜙)...
approximately 10
distributed ~ p(𝜙)Machine learning for lattice theories

Real-world
lattices

Quantum field
theories

lattice theories

11Machine learning for lattice theories

Real-world
lattices

Quantum field
theories

lattice theories

numerical methods––––

Thermodynamics
Collective phenomena
Spectrum...
12

Machine learning for lattice theories

Real-world
lattices

Quantum field
theories

lattice theories

numerical methods

✘
–
––
–

hard to reach continuum limit /critical point in some theories

Thermodynamics
Collective phenomena
Spectrum...
13

Machine learning for lattice theories

Real-world
lattices

Quantum field
theories

lattice theories

numerical methods

+ ML✘
–––
–

hard to reach continuum limit /critical point in some theories

Thermodynamics
Collective phenomena
Spectrum...
14

Machine learning for lattice theories

Real-worldlattices

Quantum fieldtheories

lattice theories
Sampling using ML

2numerical methods

+ ML✘
–––
–

3

Toy model results

1

Critical slowing down

Thermodynamics
Collective phenomena
Spectrum...
15

Difficulties with Markov Chain Monte Carlo●

Need to wait for "burn-in period"●Configurations close to each other on the chain will be correlated, so musttake many steps before drawing independent samples
correlated...
~ p(𝜙)~ p(𝜙)

burn-in●

Burn-in and correlations both related to Markov chain "autocorrelation time"→ smaller autocorrelation time means less computational cost!
typically quantify withintegrated autocorrelation time:16

Critical slowing down●

As params defining distribution approachcriticality, for Markov chains using local
updates, autocorrelation time diverges

continuum limit●

Fitting 𝜏int to power law behavior gives
dynamical critical exponents●

Smaller dynamical critical exponent =cheaper, closer approach to criticality17

CSD in scalar theory used in this work:

Critical slowing down
●As params defining distribution approachcriticality, for Markov chains using local
updates, autocorrelation time diverges

continuum limit●●Fitting 𝜏int to power law behavior gives
dynamical critical exponents

Smaller dynamical critical exponent =cheaper, closer approach to criticality

CSD also affects more realistic,
complex models:○ CPN-1 [Flynn, et al. 1504.06292]○ O(N) [Frick, et al. PRL 63, 2613]○ QCD [ALPHA collaboration 1009.5228]
○ ...
18

Machine learning for lattice theories

Real-world
lattices

Quantum fieldtheories

lattice theories
Sampling using ML

2numerical methods

+ ML✘
––––

3

Toy model results

1

Critical slowing down

Thermodynamics
Collective phenomena
Spectrum...
19

Sampling lattice configs
likely(log prob = 22)

likely(log prob = 25)

likely(log prob = 5)

unlikely(log prob = -6107)
20

Sampling lattice configs ≅ generating images
likelylikely[Karras, Lane, Aila / NVIDIA 1812.04948]

likelyunlikely

likelyunlikely21Unique features of the lattice sampling problem✓

Probability density computable (up to normalization)✓

Many symmetries in physics○○

Lattice symmetries like translation, rotation, and reflection
Per-site symmetries like negation✘

High-dimensional (109 to 1012) samples✘Few (~1000) samples available ahead of time (fewer than # vars!)○

Hard to use training paradigms that rely on existing samples from distribution

22

Image generation via ML
1.

Likelihood free methods:[Goodfellow et al. 1406.2661]

E.g. Generative Adversarial Networks (GANs)✘✘

2.

Needs many real samples
No associated likelihood for each produced sample

Autoencoding:[Kingma & Welling 1312.6114]

E.g. Variational Auto-Encoders (VAEs)✔
✘

3.

[Shen & Liu 1612.05363]

Good for human interpretability
Same issues as GANs

Normalizing flows:[Rezende & Mohamed 1505.05770]

Flow-based models learn a change-of-variables thattransforms a known distribution to the desired distribution✔
✔

Exactly known likelihood for each sample
Can be trained with samples from itself

23

Image generation via ML
1.

Likelihood free methods:[Goodfellow et al. 1406.2661]

E.g. Generative Adversarial Networks (GANs)✘✘

2.

Needs many real samples
No associated likelihood for each produced sample

Autoencoding:[Kingma & Welling 1312.6114]

E.g. Variational Auto-Encoders (VAEs)✔
✘

3.

[Shen & Liu 1612.05363]

Good for human interpretability
Same issues as GANs

Normalizing flows:[Rezende & Mohamed 1505.05770]

Flow-based models learn a change-of-variables thattransforms a known distribution to the desired distribution✔
✔

Exactly known likelihood for each sample
Can be trained with samples from itself

24

Many related approaches●●Continuous flows

Normalizing flows for many-body systems[Noé, Olsson, Köhler, Wu
Science 365 (2019)
Iss. 6457, 982]
[Zhang, E, Wang1809.10188]●Self-Learning Monte Carlo●

Hamiltonian transforms[Li, Dong, Zhang, Wang 1910.00024]

See talks by Junwei Liu,Lei Wang and Hong-Ye Hu[Liu, Qi, Meng, Fu 1610.03137]

25

Flow-based generative models
Using a change-of-variables, produce a
distribution approximating what you want.
[Rezende & Mohamed 1505.05770]

26

Flow-based generative models
Using a change-of-variables, produce a
distribution approximating what you want.
[Rezende & Mohamed 1505.05770]

Invertible&
Tractable
Jacobian

Easily sampled

Approximatesdesired dist.
27

Flow-based generative models
We chose real non-volume preserving(real NVP) flows for our work.
[Dinh et al. 1605.08803]

Invertible&
Tractable
Jacobian

Many simple layers
composed to produce f
Easily sampled

Approximatesdesired dist.
28

Flow-based generative models
We chose real non-volume preserving(real NVP) flows for our work.
[Dinh et al. 1605.08803]

Invertible&
Tractable
JacobianEasily sampled

Approximatesdesired dist.
29

Real NVP coupling layer
Application of gi-1

1.

Freeze 1/2 of the inputs, za

2.

Feed frozen vars into neural networks s and t

3.

Scale exp(-s) and offset -t applied to unfrozen, zb●

Simple inverse and Jacobian30

Loss function●

Use known target probability density:●

For our application, train to minimize shifted KL divergence
shift removes
unknownnormalization Z●

Can apply self-training: sampling model distribution p̃f(𝜙) to estimate loss

31

Correcting for model error●Known model and target densities, many options to correct for error●We use MCMC with proposals from ML model (interoperable with standardMC updates)●Metropolis-Hastings step:model proposal, independentof previous sample

MarkovChain✘
ML modelproposals
32Overview of algorithm
Parameterize flow using RealNVP coupling layers

Each layer contains
arbitrary neural netss and t33

Overview of algorithm
Parameterize flow using RealNVP coupling layers

Each layer contains
arbitrary neural netss and tTraining stepDesired accuracy?

Draw samples from model
Compute loss functionGradient descent

Save trainedmodel34

Overview of algorithm
Parameterize flow using RealNVP coupling layers

Each layer contains
arbitrary neural netss and tTraining stepDesired accuracy?

Draw samples from model
Compute loss function
Gradient descent

Save trainedmodel

Markov chain using
samples from model
generating samples is"embarrassingly parallel"

35

Machine learning for lattice theories

Real-world
lattices

Quantum field
theories

lattice theories
Sampling using ML

2numerical methods

+ ML

✘
––––

3

Toy model results

1

Critical slowing down

Thermodynamics
Collective phenomenaSpectrum...
36

Toy model: scalar 𝜙4 lattice field theory●

One real number 𝜙(x) ∊ (-∞,∞) per lattice site x (2D lattice)●

Action: relativistic scalar with quartic coupling37

Toy model: scalar 𝜙4 lattice field theory●

One real number 𝜙(x) ∊ (-∞,∞) per lattice site x (2D lattice)●

Action: relativistic scalar with quartic coupling●

5 lattice sizes L2 = {62, 82, 102, 122, 142} with bare parameters tuned to follow
a line of constant physics (symmetric phase)●

HMC and local Metropolis compared against our ML method38

Samples from ML model vs standard algorithms

By eye, ML model produces varied samples and correlations at the right scale
39

Toy model: scalar 𝜙4 lattice field theory●Measured observables:○

Correlation functions, relating to masses inthe (discretized) quantum field theory○

Response of the vacuum to an impulse(two-point susceptibility)○

Energy measurement relating to Ising model
microstate energy in a particular limit of 𝜙4 theory

40

Comparing observables (1)Ising energy and two-pointsusceptibility agree.

41

Comparing observables (2)Correlation functions
and pole masses agree.

correlation falls off
with separation inboth directions on
periodic lattice42

No critical slowing down in ML approach

HMCLocal

ML modelsby spending time training up-front,
autocorrelations are fixed during sampling43Moving towards QCD●

Developing the method to apply to lattice QCD, tackling difficulties withscaling volume, # dimensions, and to gauge theories

David Murphy

Dan Hackett

Denis Boyda

44

Convolutional architectures●Work in progress using convolutional networks for s and t, natural due tolocality of physical distributions●Conv + checkerboard mask = translational invariance (up to parity)●Convolutions also make scaling physical volume easy
14 x 14

Transfer trained net +10 mins retraining

20 x 20

3.5 days training

45

Hierarchical models[Dinh, Sohl-Dickstein, Bengio 1605.08803]●

Multiple flows, refining lattice between each

[Li & Wang PRL 121 (2018) 260601]

flow

flow

flow

flow

hier flow for 32x32 𝜙4●Intuition: build correlations at multiple scales efficiently●

Computational gains allow scaling ML method
46

Towards 3D and 4D lattices●No theoretical obstacle to moving from two dimensions to 3D and 4D●

Work in progress to implement 4D convolutions efficiently in PyTorch

8x8x8 model easily trained to 30% acceptance47

Towards gauge theories●Real NVP only directly works on fields taking real values 𝜙(x) ∊ (-∞,∞)●

Gauge theories (and O(N), CPN-1, ...) have compact domains●

Early success applying stereographic projection + Real NVP in collaboration
with DeepMind team[Gemici, Rezende, Mohammed 1611.02304]●

Applications to discrete models (Ising, Potts, etc.)?
○

Some recent ideas emerging [Ziegler & Rush 1901.10548]

48

Towards gauge theories●Real NVP only directly works on fields taking real values 𝜙(x) ∊ (-∞,∞)●

Gauge theories (and O(N), CPN-1, ...) have compact domains●

Early success applying stereographic projection + Real NVP in collaboration
with DeepMind teamThanks!
Questions?

[Gemici, Rezende, Mohammed 1611.02304]●

Applications to discrete models (Ising, Potts, etc.)?
○

Some recent ideas emerging [Ziegler & Rush 1901.10548]

49

Backup slides50

ML method for scalar lattice field theory●Prior distribution chosen to be uncorrelated Gaussian,i.e. for each site x,●Real NVP model:○○○●

8-12 Real NVP coupling layers
Alternating checkerboard pattern for variable split
2-6 fully connected layers with 100-1024 hidden units

Trained using shifted KL loss with Adam optimizer○.
.
.

Models trained until 50% and 70% acceptance rate in ML MCMC51

Autocorrelation time●

Well-behaved Markov chains have a "mixing time" determining how many
updates required to burn in / decorrelate samples○○●

Hard to compute directly except for very special chains
Dominated by the slowest mixing mode

Practically useful alternative: integrated autocorrelation time for an observable

two-point correlation
separated by tau
Markov chain steps
52