Generative Models
and Statistical Predictions inString Theory
Jim Halverson
Northeastern University
Deep Learning and Physics 2019
@ Yukawa Institute for Theoretical Physics
based on: 1911.xxxxx with C. Long [arXiv + GitHub]
see also: 1809.08279 with F. Rühle

String Theory● quantum theory of gravity, candidate TOE.
can give rise to semi-realistic particle-cosmo.
● extra dimensions → compactify.
geometry and topology determine 4d physics.
● landscape: many solutions / vacua of theory.
e.g. Kreuzer-Skarke CY3s, or 10755 F-theory geometries.
● Upshot: if string theory is true, “fundamental physics”
is a complex system, many metastable states.
(as in, for example, spin glasses, protein folding)2Strings andMachine Learning
●
●
●
●
●

[Candelas, de la Ossa, Green, Parkes]

unexpected patterns and correlations.
deep searches.
generating conjectures.
improve + accel. predictions.
faking actual (theoretical) data.
[Candelas, Constantin, Skarke]. . . and much more. Learning rapidly, wide open O(1) questions.
Not everything works, but probably still much to discover.
Summer 2017: [He] [Ruehle] [Cariﬁo, J.H., Nelson, Krioukov] [Kreﬂ, Song][Hashimoto, Sugishita, Tanaka, Tomiya] [Cole, Shiu] [Cariﬁo, Cunningham, J.H., Krioukov, Long, Nelson] [Mutter,Parr, Vaudrevange] [Erbin, Krippendorf] [Klaewer, Schlechter] [Liu] [Wang, Zhang] [Jinno] [Bull, He, Jejjala, Mishra][He, Jejjala, Nelson] [Rudelius] [Altman, Cariﬁo, J.H., Nelson] [Jejjala, Kar, Parrikar] [Dias, Frazer, Westphal] [Bull,He, Jejjala, Mishra] [Hashimoto] [J.H., Nelson, Ruehle] [He, Lee] [He, Kim] [Brodie, Constantin, Deen, Lukas]+ . . . (sorry if I forgot! let me know and I’ll add you next time around)
Review: [Ruehle, 2019]3[ Bousso, Polchinski]Landscape Statistics● Recall: if string theory is true, physics is complex.
i.e. many metastable states, must ask: “why one like ours?”● Predictions must be statistical. e.g., [Douglas]
sum over vacua, labelled by iprobability distribution U(i) A(i),U(i) cosmological factor
A(i) anthropic factor● Ensembles of vacua.
Compute expected value of observables in ensemble.
● Problem: don’t know full U, A, or {i}, and even if we did intheory, in practice we’d be complexity-limited.

4What we’reup against[J.H., Long, Nelson, Salinas]● 10755 F-theory geometries:~ 1800 axions,~ 700 gauge sectors.
[J.H., Long, Sung]● Reheating critical due toDM concerns. [J.H., Nelson, Ruehle]● Result: at ~ 188 axions,asymmetric reheating.
Couldn’t do large N calculation!

5

Where we’re going● Complexity.
affects practical matters and dynamics.
● Learning random matrix approximations.
simulate string theory to get statistical predictions.
● Examples: Kähler metrics.
the physics case.
simulating with conditional Wasserstein GANs.

6

Complexity
large N is hard.
not just b/c # vacua, but also complexity of physics tasks.
7

Complexity 101
The Good:●

polytime to solve = P●

polytime to check = NP●

NP-hard if at least as hard as any
problem in NP.

●The Bad:●

intuitively, ﬁnding solutions isharder than checking solutions.
i.e. expect P ≠ NP.

●

in that case,NP-hard problems are exp-time.

polytime for NP-hard? P=NP.
Get rich.
8Complexityin String Theory
The ugly:●

Bousso-Polchinski NP-hard.

Ways out?
●[Denef, Douglas]●

Instantons undecidable.

e.g. protein folding[Cvetič, J.H., García-Etxebarria]●

Finding string scalar potentials
and their minima both (co) NP-hard.
[JH, Rühle]●Dynamics: if a process is exp-time,
how did nature achieve it?

avoid worst-case instances,entirely or by selection.

●

allow error: approximate.
e.g. FPTAS for NP-hard.
polytime in N and 1/ε.

[Denef,Douglas] [Denef, Douglas, Greene, Zukowski]

9String vacua encode physical observables inensembles of matrices and tensors.
Can we machine learn random matrix
approximations of them?
- Cody Long
10

Learning
Random Matrix
Approximations
Goals: 1) interpolate rel. training set. 2) extrapolate to large N11

Random Matrices

see [Denef, Douglas] [Marsh, McAllister, Wrase][Chen, Shiu, Sumitomo, Tye] [Achucarro, Ortiz, Sousa][Pedro, Westphal] for applications of RMT in string theory.

x1
x2
x3..
..
..

FxNe.g. Wishart ensemble: F = ATA
entries of A drawn from 𝛀(0, 1/N1/2), are random vars.
Question: learn accurate F from sophisticated ansätz?

Learning Random Matrix
Approximations of String Theory[J.H., Long]

Learn the ansatz:

Search ansatz space:●

Restrict F by human cleverness,
likely has params, optimize them.

●e.g. Bergman approx to Kähler.

●Use very expressive form for F,
i.e. can hit a lot of function space.
F = a deep neural network.

[Donaldson, Tian, Yau]●
●Optimization possibilities:1) random walker
2) reinforcement learning3) gradient descent

Training possibilities:
1) simple network:
decrease distance of eigenspectra

talk of [Nelson]2) generative adversarial network:
competitive game until Nash eq.

13Generative
Adversarial Networks[Goodfellow +] 9740 cites
1406.2661

GRealvs.
Fake

D[Karras +] 2018●

G generates fake data from noise.

●

D discriminates real vs. fake data.

●zero sum game until Nash equil.

D gets v(𝜃g,𝜃d),●

G gets -v(𝜃g,𝜃d).

learns data distribution.
see [Erbin, Krippendorf] for a string
application of GANs, as well as the present
work and [J.H., Long, Ruehle] to appear.

14

Architecture:DCGAN[Radford +] 3755 cites
1511.06434

Wasserstein
GAN (Loss)[Arjovsky +] 1865 cites
1701.07875●

improves training stability due
to better behaved gradients.

●

idea: use Wasserstein loss,a.k.a. earth-mover distance.

●

Estimate with
Kantorovich-Rubinstein duality.

●

open source: hacked for current
application in a morning.
15

Interpolating andExtrapolating: CGANCGAN● GAN: fakes data, given noise.
● What if there are multiple classes
that appear in the data?
CGAN: pass condition, generate
certain types of data.
● e.g. MNIST: gen. a particular #.
● Q: extrapolate out of sample?

16[Mirza, Osindero] 2014

Kähler
Metrics

an example of demonstrating andlearning structure using RMAs.
17

Concrete physics:Axion kinetic terms● kinetic terms in QFT crucial for propagation of particles.
● left: axion kinetic terms with mixing matrix Kij.
● right: ﬁeld redeﬁnition with “canonical kinetic terms”,
eigenvalues of K in couplings of canonical ALPs 𝜙18

where does Kijcome from instring theory?

derivatives to get Kij

Concrete physics:
ALP cosmology
Physics:e.g. CAST (existing), IAXO (future)1) moduli stabilization.
2) axion couplings, have many purposes / uses:- cosmological inﬂation- dark matter- axion-like particles (ALPs), search for in sun orgalaxy clusters via ALP-photon coupling.

Abell 1689, credit:
Chandra + Hubble

20

ALP-photon couplings& Kahler metrics● result: ALP-photon
couplings depend cruciallyon metric eigenspectra.

21

so what is this data andhow do we compute it?
Kahler metrics on the Kahler moduli space of
Kreuzer-Skarke Calabi-Yau threefold hypersurfaces.

(the most popular ensemble of six-dimensional manifolds for extra dimensions in string theory).

Kähler Metric Data
A data point is:1) particular CY3 type. Kreuzer-Skarke pushing CY3.
(speciﬁed by ﬁne regular star triangulation of reﬂexive 4d polytope).

2) Evaluate at special point. stretched Kähler cone apex:
min vol(Σij Cij)s.t. vol(Cij) > 0were Cij = Di · Dj.
3) Compute Kij at A from K = -2 log(V).

23

StructureReal Data●

batches of 64, 10x10 K in upper
left of 0-padded 16x16.

●

hot spots and voids.

●

presumably due to topology
and / or narrowing Kähler cones?

●

see from eyes + string theory.

●

how to discover structure?

●

Is it human-interpretable?

Training●

Few thousand K-S h11=10 CY3s.

●

Choose subset to train on,generate Kahler potential.

●

Test ﬁtness of generated model.

Question: can GAN learn bothfeatures and the pad?

Real Data

Real DataReal Data

Real Data

Real Data

Real Data

Blue: Real Data
Green: GAN @ Epoch
Orange: Naive Wishart

Real Data
Note: not training eigenvalue loss. instead,learned matrix distribution, eigenvaluelearning implicit. Distance = WD.

Real Data

Real Data

Real Data

Real Data

Real Data
paper version: can get to ~.04

nz and Implicit correlations

Sculpting material = nz,
number of rand. vars in.
note:1) inversion at late times.
2) initial values ~ 10.
small nz good → correlations.

38

This was all at ﬁxed N.
Can we interpolate
and extrapolate in N?
often out of sample extrapolation too hard,
but maybe actually doable, due to Reid’s fantasy?

(the fantasy that people believe: one big network of CY’s, related by topology changing transitions)Interpolating &Extrapolating● setup: take Wasserstein DCGAN, make conditional on h11● train on data across different h11 values● Q: do samples from train h11 match eigenspectrum?
Q: generate samples with h11 not used in training,
do they match e-spectrum?
(h11 between h11 trains = interpolation)(h11 above h11 trains = extrapolation)

40

Interpolation
Results● trained on h11 = 20, 30,2500 geometries each● test h11 = 20, 25, 30 on unseen
e-spectrum, O(thousands) geoms● fairly good interpolation (h11=25)● mediocre @ smaller h11.
correlated w/ 0-padding!?

41

Extrapolation
Results● trained on h11 = 20, 25,2500 geometries each● test h11 = 20, 25, 30 on unseen
e-spectrum, O(thousands) geoms● excellent extrapolation (h11=30)● again, mediocre smaller h11.
padding issue clearer!

42

UpshotsPredictions in string theory are statistical.
Necessitated by complexity at large N, can we learn an approximation to it?
Concrete: arising from ensemble of matrices, can we learn random matrix approximation?
Complexity

Learning RMTCorrelations

Instances of NP-hard andundecidable problems arise
naturally in string theory.

Frameworks:
1) optimize a clever ansatz.
2) learn a new one.

High accuracy with few
random vars? Structure.
Have it in these examples.

e.g. determining andExtrapolate out @ ﬁxed N.
Interpolate and extrapolate toother N, ideally large!

What is controlling thisin Kahler metrics?

minimizing scalar potentials.

Here: use Wasserstein (c)GANwith DCGAN architecture.

Physics: directly related
to ALP sun-searches.

43Thanks!

Real

Any questions?
Ask away, or at a coffee break,or feel free to write:▪ @jhhalverson▪ jhh@neu.edu

special thanks to A. Achúcarro, D. Robbins,
L. McAllister, F. Rühle, G. Shiu, and A. Westphal for
discussions on random matrices and string theory.

44

Fake