Towards reduction of autocorrelationin HMC by machine learning

CCNU→ 理研(和光)→ 理研(ブルックヘブン)
富谷 昭夫
共同研究者: 田中章詞 (理化学研究所 革新知能統合研究センター)
Based on arXiv:1712.03893

2/June/2018 Deep Learning and Physics 2018

akio.tomiya@mail.ccnu.edu.cn

自己紹介

A. Tomiya

前回は機械学習を用いた測定、今回は機械学習を用いた配位生成

CCNU(中国の真ん中らへん、武漢) のポスドク
2015年 大阪大学 素粒子論研究室でPhD(田中くんと同期)。
専門: 格子QCD(数値計算で原子核などの性質を調べる)
相転移が好きで超伝導→テクニカラー→有限温度QCD、エンタングルメントエントロピー等…
イジング模型の相転移の推定※
JPSJ86, 063001 (2017)



※※

配位生成の高速化(このトーク)
arXiv:1712.03893

AdS/DL (橋本さんのトーク)




!

  

arXiv:1802.08313

2/June/2018 Deep Learning and Physics 2018

関連するトーク※柴さん、青木さん、大槻さんのトーク※※永井さんのトーク2まとめ：問題設定と結果

A. Tomiya

提案したアルゴリズムで自己相関は短縮。相転移点で不具合

問題: 格子QCDでの配位生成を効率化したい！
効率化できれば、モンテカルロ法(HMC)からくる統計誤差を
短時間で減らすことが出来る。
手法: 機械学習をHMCに組み込む
機械学習は、それっぽい画像を作れる→ 応用できないか？
性質が良く分かってる3次元φ4理論で試してみた

Radford et al. (2015)

結果: 対称相、破れ相で、1点関数, 2点関数は無矛盾臨界点で、不具合がある
2/June/2018 Deep Learning and Physics 20183

背景・動機背景: クォークとグルーオンの統計系の相構造を調べたい

A. Tomiya

原子核等は、一兆度でとけてクォークとグルーオンへ(〜相転移)予想されるQCD相図

温度

QCD相図
20

学校

1兆度

密度
※福嶋さんトーク

Fukushima-Hatsuda (2010)

2/June/2018 Deep Learning and Physics 2018 2011年8月18.19日＠夏の学校

25

5

背景: クォークとグルーオンの統計系の相構造を調べたい

A. Tomiya

クォークとグルーオンの量子系は、摂動できない& 非摂動効果が重要

クォーク場
q̄(x) 反クォーク場
Aµ (x) グルーオン場q(x)量子色力学(クォークとグルーオンの力学)の経路積分量子化
Z=

Z Y
x

dq̄(x)dq(x)dA(x) exp⇥1~

Z 1/(kT )
0

d⌧

Z

3

d x Lgluons (A) + Lquarks (q, q̄) + Linteractions (A, q, q̄)- 結合定数が大きいので基本的に摂動できない- 非摂動効果が重要であることが知られている↑を見てみると、統計力学の分配関数と似てる…
2/June/2018 Deep Learning and Physics 20186⇤

背景: クォークとグルーオンの統計系の相構造を調べたい

A. Tomiya

離散時空上のクォークとグルーオンの統計力学は、モンテカルロできる

クォーク場
q̄(x) 反クォーク場
Aµ (x) グルーオン場q(x)量子色力学の経路積分量子化(クォークとグルーオンの量子統計力学)
Z=

Z Y

dq̄(x)dq(x)dA(x) exp

x

⇥

1
~

Z 1/(kT )
0

d⌧

Z

3

d x Lgluons (A) + Lquarks (q, q̄) + Linteractions (A, q, q̄)

形式的に古典統計力学と似てるがxは、連続なので微妙に違う※→ xを離散化すれば 原子ラベルと同じ→基本的に統計力学の手法が使える→ 離散化した時空上の量子色力学(格子QCD)は、モンテカルロで計算できる※柏さんのトーク

2/June/2018 Deep Learning and Physics 2018

K. Wilson(1974)
M. Creutz(1980)

7⇤

These predictions build on successful calculations of the bb̄ and cc̄ spectra (37,
A. Tomiya
42, 43, 44, 45), which reproduce the experimental results well.
格子QCD=離散時空上のクォーク・グルーオンの量子多体系
The most striking aspect of the spectrum is how well it agrees with nature.
今まで成功してきた。
The
nucleons provide almost all the mass in everyday objects, and their masses
have been verified within 3.5%. Their mass mostly comes, via m = E/c2 , from
線が実験で決定したハドロンの質量
the kinetic energyof the quarks and the energy stored in the sausage-like flux
シンボルが格子QCDで決定したハドロンの質量
tube(s)holding the quarks together.

格子QCDへ3,
4つのインプット->あう。
H
H
B
B

ハドロン 2200
質量 2000
2400

H

*

H

*
s

s

c

*
c

© 2012 Andreas Kronfeld/Fermi Natl Accelerator Lab.

1800

(MeV)

1600
1400
1200
1000
800
600
400
200
0

π

ρ

K

∗

K

η

η′

Figure Deep
2: Learning
Hadronandspectrum
from
2/June/2018
Physics 2018

ω

φ

N

Λ

Σ

Ξ

∆

∗

Σ

Ξ

∗

Ω

ハドロン
の種類

arXiv:1203.1204

lattice QCD. Comprehensive results for8ハイブリッドモンテカルロ法(HMC)

A. Tomiya

グルーオン場配位を連鎖的に作り出し、経路積分の結果を推定

経路積分(状態和) = すべての可能な状態の重み付け和(積分)重み = ボルツマンweight ~ exp(- L)=exp(-S)→作用S の値が大きいとき、結果への寄与は小さい
作用が小さい所の周りを重点的に計算= ハイブリッドモンテカルロ法(HMC)
HMCの気持ち:量子系 〜 運動方程式 + ランダム運動量
Sが小さい所量子性
2/June/2018 Deep Learning and Physics 20189

HMC(ハイブリッド(ハミルトン)モンテカルロ)とは

A. Tomiya

ランダム運動量を入れた運動方程式で場を更新し、経路積分をミミックするHMC(スカラー場版):

0. 場の配位φold=φ(x)を一つ選ぶ
1. ランダム運動量(場) p(x)を確率~exp(-p2/2)で生成
2. H=p2/2 + S[φ] に対するハミルトン方程式を使い
場φと運動量pを発展 = 数値的に運動方程式を解く
3. メトロポリス法で数値誤差を消去、φnewを得る
1〜３を繰り返す(MD)

Simon Duane, A.D. Kennedy, Brian J. Pendleton, andDuncan Roweth. Hybrid monte carlo. Physics Letters B,
195(2):216 – 222, 1987.

場の配位の列φ1, φ2, … の平均 = 期待値
無限回やれば厳密に経路積分に一致することが知られている
2/June/2018 Deep Learning and Physics 201810

自己相関時間

A. Tomiya

ランダム運動量を入れた運動方程式で場を更新し、経路積分をミミックするHMC(ハイブリッドモンテカルロ)を使うと、場の配位の列φ1, φ2, … が得られる。
…
場のラベル φ1

φ2

φ3

…

φN

4

5

4

…

7

黒の数

黒のセルの数が近いので、独立な配位とみなせない→ 似ている範囲を自己相関時間と呼ぶ(短い方が良い)
2/June/2018 Deep Learning and Physics 201811

自己相関を短くして、高効率なサンプリングを

A. Tomiya

似たような場の配位のサンプル → 有効的な統計数の減少

HMCの気持ち:量子系 〜 運動方程式 + ランダム運動量
Sが小さい所量子性→ (仮想)時間での発展が有効的に遅くなるときを考える= 同じ計算時間でも似た場の配位のみが出てくる→ ランダムサンプリングとして効率の低下
なんとかしたい！
2/June/2018 Deep Learning and Physics 2018

12

目次

A. Tomiya

1.動機と導入
2.ボルツマンマシン
3.先行研究4.やったこと
5.結果
2/June/2018 Deep Learning and Physics 201813

ボルツマンマシン

生成模型、ボルツマンマシン

A. Tomiya

生成モデルは、経験確率に基づき、模倣したデータを生成できる生成模型とは…

？

! ! " ! # !

P
" ! #

ブラックボックス(情報源)から情報が出て来るとして、情報源をモデル化、模倣する機械学習の枠組みの1つ.
情報は、適当な確率分布に従って生成されてると考える
ボルツマンマシンは、生成模型の1つ情報源から出てくる確率と模型の確率を似せる(最尤法)
2/June/2018 Deep Learning and Physics 201815

制限ボルツマンマシン

A. Tomiya

生成モデルの一種で、経験に基づき、模倣データを生成できる

ボルツマンマシン

イジング模型、2種のスピン自由度v, h(両方2値)：※大関さん、柴さんのトーク

制限ボルツマンマシン: v同士、h同士には結合のない生成模型
E[v, h] =X

bn v n +nXi

c i hi +X

vn wnj hj

nj

1 E[v,h]P(v, h) = e
Z
学習パラメータ: b, c, w

v, h は、スピン自由度(に対応)
v に値を入力→hに関する確率分布
h に値を入力→vに関する確率分布
通常 v(可視層)には、画像データを入力し、画像の分布を模倣する
学習パラメータは、勾配法でPを最大化するように学ばれる
2/June/2018 Deep Learning and Physics 2018

16

(b

A. Tomiyasymmetry and psudofemion formalism in the case of latガウシアン制限ボルツマンマシンGRBM
tice QCD and this leads long autocorrelation. Towardvのスピン自由度を連続変数にしたもの.
solving this problem, we hは2値
introduce a Gaussian-BernoullibeRestricted Boltzmann Machine (GRBM) into the algobl
rithm (Bottom half of Figure. 2).
ics

ガウシアン制限ボルツマンマシン = 可視層の変数を連続なモノにしたversion

GRBM canXbe regarded
asX
a physical system composedX
E[v, h] = fieldbn v n φ+ andci hauxiliary
vn wnjhji+by dynamical
binaryfield h with theninjfollowing Hamiltonian:
HθGRBM [φ, h] =! (φn − an )2
n

2σn2

−

!

a, σ, b, w が学習パラメータjbj h j −

! φn
n,j

σn

su
H

wnj hj ,(6)

Thwhere θ = (an , σn , bj , wnj ). In contrast to usual statistical physics problem which discuss property of the system
通常 v(可視層)には、画像データを入力し、画像の分布を模倣する
with fixed θ, we will determine appropriate θ by the following steps.
学習パラメータは、勾配法でPを最大化するように学ばれる• Preparing a teacher data D ∼ e−S[φ] /Zlat which wewould like to mimic (top half of Figure. 2).
2/June/2018 Deep Learning and Physics 2018

• Updating parameters θ in (6) via contrastive diver-

10
la
17

先行研究

Self-Learning Monte-Carlo(SLMC)

A. Tomiya

有効模型の係数を実際のシミュレーション結果から決める

永井さんのトーク：
アイデア: 自己相関が長い系の厳密なハミルトニアンが分かってるとする。
係数をパラメータにした有効ハミルトニアンを用意し、シミュレーションと回帰で係数を決める。有効系でシミュレーションし、自己相関を切る。
メトロポリステストで、厳密なハミルトニアンに対する分布に補正する。

上手いこといってる

2/June/2018 Deep Learning and Physics 2018

J.Liu et al arXiv: 1610.03137 など
19

−F (h)=bjrandomly
h"
ln(1
+!
e configurations).
(9) developed"j +nte Carlo
methods
sampleandproposes
updateswithcorrespondingly
high acceptanc
forstatisticalmechanics
problems
To makethe presentation
cleaner,we [11–16,44–47].
divided the compuai + M
Wij hjj=1−F (h) =
bjj=1
hj +
ln(1
+
e
).
(9)
i=1
A. TomiyaTomake thepresentation
cleaner, collecting
we divided
thetraining
compu-data, makes
the answer
asastatistical
average.
However,
rates
and
low
autocorrelations.
This
approachbettetational
tasks
into
three
phases:
the
j =1
i=1
tasks
three
phases:collecting
theCarlo
training
data,
fittingtheintoRBM,
andthe actualMonte
Noticethat the visible
variables
have been traced
out forin tational
configuration
spacesare exceptionally
large
use
of
the
sampledMonteCarlodatasimulations.
because,in additio
fitting
the
RBM,
and
the
actualMonteCarlosimulations.
In
future, one can use online learning to optimize thecethattheitsampling
visiblevariables
have
beencapturesout genericin
Eq.
(9), andaccordingto p(h)thesystems,
is typically
impossible
totracedperform
directtoestimatingthephysicalobservables,theRBM builds aInthefuture,one
canuse
onlinelearning
tooptimizethe
RBM
progressively
with
newly
collected
configurations.
After
RBMのみでイジングの配位を生成する
9),
andsampling
accordingtop(h)capturesthegeneric
distribution of the hidden features and is unaffected by theherefore
resorts
to theandMarkov
chain
randomadaptive
model
forphysicalprobability
distribution an
RBM
progressively
withinnewly
collected
configurations.
After
training
the RBMthe theequilibration
phaseof theMontebutionof onethe variables.
hiddenfeaturesis improves
unaffectedtheurrent visible
This further
thebysamplingtheRBM
in onethe canequilibration
phase ofMontewithach
to byexplore
configuration
space.
In
thistraining
guidesbetter
explorations.
Carlosimulation,use it to generatenewthesamplesntRBM
visiblevariables.
Thisfurtherand
improves
samplingof
avoidingthe
the visible
hidden thevariables
to lock
simulation,
one canOne
use italso
togeneral
generatenewsampleswithimproved
efficiency.
needs toideas
checkthe scalabilityBMavoiding
the visible
hidden ratio
variables
to lock theCarloach byothers’feature.
nly
needs
to knowtheandrelative
between
We illustrate
these
using
the Falicov-Kimba
improved
efficiency.
One
also
needs
to
check
the
scalability
ofmodelthe proposed
approachfor larger andmorecomplicatedfeature.
Wetwo
demonstrate
the improvement of using RBM in thesothers’ofconfigurations.
[32]
as
an
example.
The
model
describes mobiloftheproposed
approachforlargerandmorecomplicatedphysical
systems.
e demonstrate
the improvementof using
RBM ratioin theMonte
Carlo simulationin Fig. 5. The
acceptanceof thegefficientstrategies
toexplorethe
configuration
fermions
and extension
localized
interacting
中国科学院組physical
systems.future
Another
is tofermions
explore deep
Boltzmann with on-sit
te Carlo simulation in Fig. 5. The acceptance ratio of theThe
Falicov-Kimballfutureisbelief
to explore
deepforBoltzmann
ently is at the heart of Markov chain Monte Carlo Anotherinteractions.
The
Hamiltonianreads
machines[60] extension
anddeep
nets [51]Montemodel
Carlo[60] anddeep hierarchicalbelief nets structures[51] for"MonteCarlosimulations.
Deeper
may allow
evenThis is, however, a challenging endeavor. Notmachines
#"
#
N
!
!
Deeperhierarchicalstructures
allow
even
higher level
abstractions
physical may
degreeof 1
freedoms.
1† of theoning the fundamentally difficult case of glassysimulations.
n̂ideepxi −,(1ĉofi Kĉj + U degree− structure
higherlevelĤ
abstractions
theijindicating
physicalfreedoms.
FK =
There
were
observations
that aof
2 [61]. 2
scapes, naive Monte Carlo samplings are usuallyThere
were observations
indicating
thati=1
aCarlodeepsampling
structure
improves
the mixingof the Montei,j time
ow close to the phase transitions. These drawbacksimproves
the mixing
the Montesymmetry
Carlo sampling
To further
exploittime
the oftranslationalof the [61].
physicaltranslational
symmetryof thephysical
where
xone
{0,1}
is
classical
variable
representing th
problems,
mayconsiderusing
abinary
shift
invariantRBM
LIexploit
HUANG
AND
LEIa WANG
i ∈theoteworthy algorithmic developments in the pastTo furtherone
consider ofusing
alocalized
shift invariant
RBMat site i. ĉ is th[22,62]or a mayconvolutionalRBM
occupation
numberthe[63–65].
fermioni–16]. In essence, those algorithms exploit variousproblems,[22,62]Lastor a convolutional
but not least, RBM
in this[63–65].
paper we view the RBM
as
a
†
si
G
G+M
ĉ
≡
ĉ
is
the
occupatio
fermionannihilation
operatorandn̂
pects of the problem for efficient Monte Carlo Last
i
i
i
but
not
least,
in
this
paper
we
view
the
RBM
as
a
generative model and use it to accelerate the Monte Carlo(1
numberoperator
mobileis thethekinetic energs, however, difficult to devise a general strategygenerativemodel
and use of
it theto acceleratethesimulationofphysical
systems.
On thefermion.
otherMontehand,KCarloalongshsimulation
physical
systems.
On
the other
hand,
along
theas an we conside
lines
ofofRef.
is also interesting
to In
view
the
RBM
matrixof[18],
the itmobile
fermions.
thefollowing,
imal Monte Carlo algorithm design.
in
lines
of
Ref.
[18],
it
is
also
interesting
to
view
the
RBM
as
an
unsupervised
featuredetector
andexplorethepatterns
inthethe model on a periodic square lattice with N sites. Thus, Kij co=ess these difficulties in a general setting with
unsupervised
featuredetectorand explore
patternsofinphysicaltheweights andthe latentvariables
for thethe
discovery−tandforthenearest
neighborsandis zerootherwise. The −1/m machine learning. Recently, there has been aweights
invariables
foroftheproposing
discoveryof physicalknowledge.
FIG.latent
4. Twostrategies
Monte
Carlo updates usingT
offsets
inweEq.
(1)blocked
ensurethatboth Given
the mobile
localize
estFIG.
in 5.applying
machine learning approaches toRecently
noted
a related
work
[66].
the RBM.
(a)
The
Gibbs
sampling.
the visibleandvariables(a) The acceptance ratio and (b) the total energy autocor- knowledge.
thwe sample
noted
athe
relatedwork
fermions
are halffilledon[66].
average.
Previous
studiesshow tha
physics
problems.
This
includesclassifying
phasesx, wehiddenvariableh under theconditional
probability
G. 5. (a)Theratio and(b) theon
total
autocorlattice. Recently
elationtime
ofacceptancethe Falicov-Kimballmodel
a Nenergy= 82 squareFiis=supportedbysamplethe Ministryofvariables
ScienceandTechp(h|x),Eq.
(7), thenthe visible
under
the conditionallattice.
on timeof theFalicov-Kimball
onbit-flip
a N = updates.
82 squareBluesquaresdenotethe
results
ofmodellocal
Theyellow
atL.W.
U/t
4 andtemperature
T
/t
≈
0.15
the
systemundergoe
17–21],
using
neuralnetworks
as variational
有効ハミルトニアンを使わずに、配位を生成M′ the Ministry
isprobability
supportedbyofScienceTechnology
of Chinap(x
under
Grant
No.
and with ansquaresdenoteresultstheof RBM
local update
bit-flip schemes
updates. ofTheyellow|h), Eq.
(8). (b)The2016YFA0302400
blockedandGibbs
samplingnd
red dotsare usingFigs.
4(a) and L.W.
a phaseto thecheckerboard
density-wave(CDW
ons[22,23],
fitting
the density
functionals[24–26],th
nologyChinatransition
under
No.
2016YFA0302400
andthe ofstartup
grantof Grant
IOP-CAS.
L.H. is Itsupportedby
the variableed
are using The
the critical
RBM update
schemesFigs.
4(a) [33–35].
and(b),dots
respectively.
temperature
is atofT /t≈ 0.15
additional
Metropolis
stepin between.
updates thehiddenstate[33–35].
inverseproblemsinquantum
physics
startup
grantof Foundation
IOP-CAS.
L.H.
is supportedby
the variables,of
Science
China
Grant No.
11504340.
respectively.
The
criticalobservables
temperature
is atmany-bodyTwithin/t ≈ 0.15
The
estimated
physicalagreeerror[33–35].
bars
for all the Naturalto theoflog
probabilityof thehiddenh→h′ according
上手いこといってるらしい。
NaturalScience
China
Grant
No. Zi-Yang
11504340.
un
stimatedapproaches.
physical observables agree within error bars for all
We acknowledge
Jun-WeiLiu,
Ye-HuaLiu,
Meng,
ampling
Tracingout
the ofmobile
fermions,
the occupation
numbe
Eq. (9). Foundation
N Meng,Jun-Wei fermionsLiu, Ye-Hua
Zi-Yang
ing approaches.
ofof the localized
x Liu,∈ {0,1}
follows the probabilitper,
we propose a general way to accelerate MonteWe acknowledgeis035105-4−FFK (x) PRB, 95(3):035105, 2017.
Li
Huang
and
Lei
Wang.
distribution
p
(x)
=
e
/Z
,
where
Z
is
a
normalations of statistical physics problems. We present
FKFK
FK
physics model. Importantly, flipping a hidden variable with the035105-4
di2/June/2018
Deep
LearningandPhysics
2018
20 aization
The negative
energy”
reads(omittinghmic innovations: a simple supervised learning
local factor.
Gibbs sampling
may have“freenonlocaleffects on
the physical

RBMを用いたイジングの配位生成やったこと機械学習で場の理論の配位を生成した

A. Tomiya

初の試みなので、多面的にチェックが必要

ボルツマンマシン+HMCで配位生成をし、自己相関の短縮を試みた。
!

場の理論の配位生成への初の応用なので何が起こるかわからない…
!

格子上の3次元φ4でやってみた。
→ よく理解されている模型なので問題があればすぐにわかる。
2/June/2018 Deep Learning and Physics 201822

HMC(ハイブリッドモンテカルロ)とは

A. Tomiya

ランダム運動量を入れた運動方程式で場を更新し、経路積分をミミックするHMC(スカラー場版):

0. 場の配位φold=φ(x)を一つ選ぶ

再掲

1. ランダム運動量(場) p(x)を確率~exp(-p2/2)で生成
2. H=p2/2 + S[φ] に対するハミルトン方程式を使い
場φと運動量pを発展 = 数値的に運動方程式を解く
3. メトロポリス法で数値誤差を消去、φnewを得る
1〜３を繰り返す
場の配位の列φ1, φ2, … が得られるので、期待値を求める
2/June/2018 Deep Learning and Physics 201823

提案したアルゴリズム

A. Tomiya

配位の更新を運動方程式のみでなく、経験分布もつかう

Boltzmann machine + HMC(スカラー場版):
0. 場の配位φold=φ(x)を一つ選ぶ

Added

1. ランダム運動量(場) p(x)を確率~exp(-p2/2)で生成

1.5. 学習済のボルツマンマシンに配位φをいれ、新配位φを生成

2. H=p2/2 + S[φ] に対するハミルトン方程式を使い
場φと運動量pを発展 = 数値的に運動方程式を解く
3. メトロポリス法で数値誤差を消去、φnewを得る
1〜３を繰り返す

場の配位の列φ1, φ2, … が得られるので、期待値を求める
2/June/2018 Deep Learning and Physics 201824

実際の運用のながれ

A. Tomiya

最初は、HMCのみで配位を生成。学習が進めば、ボルツマンマシンを挿入。

Figure 3: Two p
the symmetric ph2/June/2018 Deep Learning and Physics 201825

any improvement. m and λ are real parameters anddA. Tomiya
2
one proposal’s
validity,
we
employinteracting
realscalarfield
m can be negative. In general, interacting scalar filed
φ4理論
s6
BM. theoryinthree
dimensional
discretizedspacetime.
The
theoriesinthreedimensionareallowedtocontainφ
term
3次元格子上のφ4理論の計算を行った
pmann theory
in theby thebut we just omit
is action
described
by renormalizability
the action,aneed
it for simplicity. φn is enjoined the periodic boundary
"
#
N
mann
2
condition for!
each direction.
1
m 2
λ 4
orks.
S[φ]
=
−value
φn ∆φ
φn +
The
expectationof anoperator
O is φdefined
by(1)
n+
n .
2
2
4!
and
the euclidean npath integral,rob$
1 are summarized
dels Our notation and setup−S[φ]in appendix A.
⟨O⟩ =Dφ O[φ] e,(8)validityof BHMC, we compare folonal In order to confirm Z
lat
fixed lowing quantities
calculated by configurations
generated
&
%N
p
−S[φ]
Dφthe
= original
Zlatand= BHMC.
Dφ e First. In isthetheard- by where
both ofHMCn dφn anddother words,
realization
probability
for aforcertainconfig古典的には、
r the expectation
valueof the action
density(1), ⟨S⟩/V ,2 < 0: Z φurationis given by,m2対称性が自発的にやぶれた相に。そうでなければ、対称相
tion
N
Here
conf
!
−S[φ]
1 1 e
(c)
blem
S/V = plat [φ] =S[φ],.
(9)(2)
対称相、破れ相そして臨界点で計算をおこなった
V Nconf Zc=1
latwhere V = Nx Ny Nt is the spacetime volume, Nconf is

2/June/2018 Deep Learning and Physics 2018

(c)

26

MONT
theory is described by the action,
details).
HMnalgorithm
A. Tomiya
Our notationand ofsetup
are summarized
appendix
expectation
value
the action
density forin(1),⟨S⟩ /V A.
,#
測定した物理量N " validity of BHMC,
2In order to confirm
wecomparefol!
1. order
Momen
1
m
λ
In
to
2
4
N
conf
!
=
− 1φn ∆φφ(c)φngenerated.
(1)
lowing S[φ]quantities
calculatedbyconfigurationsn+1
作用密度、1点関数、2点関数
n+every pprocedures
o
2
2
4!
S/V
=
S[φ
],
(2)
BOLTZMby both of the noriginal
and BHMC. First is the
V NHMC
conf c=1
details).
HM
MON
2. The m
expectation
value
of
the
actiondensity
for
(1),
⟨S⟩
/V
,
Our notation and setup are summarized in appendix A.
代表的な物理量である作用密度、1点関数、2点関数を
′
to
(φ
,
1.
Momen
where
V
=
N
N
N
is
the
spacetime
volume,
N
is
x
y
t
conf
In order to confirm validity of
BHMC,
we compare folN
conf
Infor
order
!
(c)
HHp
1
1
every
the
numberofconfigurations
and
φ
is
c-th
configuraしらべた。作用密度と1点関数はヒストグラムも計算。
(c)lowing quantities
by configurations
generated
S/Vcalculated=
S[φ ],(2)
procedures
tion.
Secondisthevacuumexpectation
value
of
oneV
N
conf c=1
by both of the original HMC
and BHMC. First is the3.
TheM
details).
HM
2.
The
m
1点関数の自己相関を2つのアルゴリズムで比較した
pointGreen’sfunction
(vev)
⟨φ⟩
/V
,
expectation value of the action density for (1), ⟨S⟩ /V ,′didatesto(φ,where V = Nx Ny Nt is the spacetime volume, Nconf is
1.for
Mom
N
N(c)
conf
HH
!
!
N
the number of configurations
and
φ
is
c-th
configuraconf
1
1
!
(c)
This algorith
1 1
every
φ/V
=
φ
,
(3)
(c)
n
tion. Second S/V
is the=N
vacuumexpectation
value
ofoneS[φ ],(2) Figure 2. In
conf c=1 V n
3. The M
Nconf⟨φ⟩c=1
point Green’s functionV(vev)/V ,ated2.
bysolvimThe
didates(c)
3 ensuresto gen(φ
whereis aNfield
value
point
n
for
c-th
configuration.
N
N
where φVn =thespacetime
volume,Nisconf
x Ny Nt is at
conf
! 1 !
1
(c) function withThis thealgorith
with
Bol
for
H
Third
quantityis =the two-point,(3)
the numberofφ/V
configurationsandGreen’s
φ(c)φisc-th
configuran
N
V
2. In
conf c=1
In total,w
zerontion.momentum
Second is projection,
the vacuum
expectation
value of one- Figure
3.byThe
M
ated
solvi
φ
which
obey
point Green’s(vev) ⟨φ⟩/V ,Nx ,Ny function
(c)
N
N
conf
didat
3 ensuresge!at 1point! n(c)
three
steps.
where φn !
is a field1 value
for c-th configuration.
(c)
G(t)
=
φ
φ
N
Nt ,dx+nfunction
with the don
Bol
n ,
conf
t+n
sentially
Third quantity N
isconfthe two-point
Green’s
x ,dy+ny with
!
!
V
1
1
This
algorinc=1dx,dy=1
In
total,
w
φ/Vprojection,
=
φ(c)
,
(3)
motionfor Hzero momentumnNconf c=1 V n
2. I(4)φFigurewhich
obey(破れ相では、con.
part)
for
the field
Nx ,Ny
N
N
ated
by
solv
conf
!
!
!
three
steps.
1
1
(c)
(c)
(c)
G(t)
=
φ
φ
3 ensuresgwhere φn is a field
value
at
point
n
for
c-th
configuration.
t+nt ,dx+nx ,dy+ny n ,sentially
don
N2018
V
2/June/2018 Deep Learning and Physics
27
conf
dx,dy=1

c=1

n

結果

統計と結果

A. Tomiya

数字だけ見ると良さそうに見える

Alg.
HMC

Phase

Symmetric
BrokenBHMC SymmetricBrokenNconf⟨φ⟩ /V⟨S⟩ /Vτint

104 0.00±0.05 0.48±0.03 4.4±0.3
104 -3.94±0.04 -2.70±0.03 2.8±0.2
104 0.00±0.04 0.50±0.04 2.0±0.1
104 -3.95±0.03 -2.73±0.04 2.5±0.2

Table I: BHMC in the Alg. column is our proposing algorithm (HMC+GRBM). V is the system volume 83 and Nconf
is the number of configurations. The integrated autocorrelation time一見良さそうに見えるが…？
τint is defined by the expectation value for the
spacetime averaged one-point Green’s function. Here the initial configuration both for HMC and BHMC is prepared froma configuration which is well thermalized configuration from
HMC.
2/June/2018 Deep
Learning and Physics 2018

In thisthe brok
Goldston
the two-in steadquantitialgorithm
rations o
Besidemated n29

tion diﬀers from the one from HMC. We discuss the

y
n
e
g
e
D
t
d

結果

A. Tomiya

対称相での結果

3- 自己相関は、短くなる- 1点関数、2点関数はコンシステント- 作用密度が少しズレている

2点関数

3

Figure 3: Two point function G(t) for generated samples inthe symmetric phase. Left: HMC, Right: BHMC

1点関数

Figure 1: Left: Autocorrelation function for the condensate
in the symmetric phase. Right: The same plot but in thebroken phase. Black lines represent autocorrelations for ourBoltzmann machine supported HMC.
Figure 2: Our proposal for update (BHMC). GaussianRBM(GRBM)
to thesamples
original in
HMC after
Figure 3: Bernoulli
Two pointfunctionG(t) isforadded
generatedtraining.
the symmetricphase. Left: HMC, Right: BHMC

作用密度e
Figure 4: Histograms for generated samples in the symmetricLeft: vev for HMC(red) and trained BHMC (black),nand determine the couplings throughphase.
linearregression
Right: the action
density for HMC(red) and trained BHMC
(black).
f 2/June/2018
symmetryandpsudofemion
formalism
in
the
case
of
latwiththeoriginal
Hamiltonian.
After
the
regression, they
Deep Learning and Physics 201830

MC. We discuss the結果

A. Tomiya

破れ相での結果

4- 自己相関は、(少し)短くなる- 1点関数、2点関数はコンシステント
We call the pair of procedure (i) and (ii) as epoch. We
train GRBM with 40 epochs.
Test details: After training, we run BHMC to gener-作用密度が少しズレている(ii) We repeat updating process for b. Each update is(b)performed by using mean values on Dmini of δθ.

ate 104 configurations. In addition, we perform the original HMC to generate 104 configurations for a reference.
We use ∆τ = 0.2 in both molecular dynamics. Both ofalgorithms start with the same initial thermalized con4
figuration. By using each configuration, we compute
vev
2点関数(3), the action density (2), two-point function (4), andexamine BHMC can generate sane configurations or not.
In addition we calculate autocorrelation function (25),integrated autocorrelation (26), and examine BHMC canprovide small autocorrelation time or not.
We perform experiments in two phases of the φ4 theory, namely the symmetric phase and broken phaseas follows.

Figure 5: Two point function Gcon (t) for generated samples
in the broken phase. Left: HMC, Right: BHMC.

1点関数

作用密度

tion function for the condensate
ight: The same plot but in theepresent autocorrelations for our
ed HMC.

Figure 5: Two point function Gcon (t) for generated samples
The symmetric phasein the broken phase. Left: HMC, Right: BHMC.

We take m2 = 0.8, λ = 0. Acceptance rate for each
2/June/2018
Learning and Physics 2018
MCMCDeepis,ngs through linear regression

Figure 6: Histograms for generated samples in the broken
phase. Left: vev for HMC (red) and trained BHMC (black),
Right: the action density for HMC (red) and trained BHMC
(black).

31

結果

A. Tomiya

臨界点での結果

If we want to u
sampler, we must
probability distrib- 1点関数すら合わない
configurations whi!
obey a conclusion
ボルツマンマシンの限界？
機械学習のモデルを変えればうまくいく?
pectation values ozero. For example,
mechanism. In this
ative models for ga
generate gauge fie
Histogram for vev near the criticality. Red: HMC,
course, if we fix thMC.
els might work. Ho
we believe that32suc
2/June/2018 Deep Learning and Physics 2018

rithmappendix,(Bottom half
Figure. 2).Gaussian-Bernoulli ReIn thisweof explain
A. icsTomiyaandn- 結果を省みる
stricted Boltzmann Machines (GRBM) and their update.summame
GRBM can be regarded as a physical system composed
モデルの表現力？
In contrast to binary Boltzmann machines, GRBM canHMC
by dynamical field φ and auxiliary binary field h with theatreat real-valued
inputs. It consists of visible continu- 2. G
following Hamiltonian:
機械学習のモデルを変えればうまくいく?
ous variables, φ =!
{φn }, and2 hidden
binaryvariables,b!
!
(φn − an )
φn
GRBM
(
Hiθ}. Basically,[φ, h] =
−
b
h
−
w
h
,
h = {hGRBM
isdefinedasastatistical
j
j
nj
j
2
2σ
σn
n
3)
b
n
j
n,j
physics system controlled by joint probability,

4)

(6)

−HθGRBM [φ,h]This iswhere θ = (an , σn , bj , wnje). In contrast to usual statistipθ [φ, h] =,(29)
cal physics problem which discussZθproperty of the systemwith fixed
will determine appropriate θ by the fol& θ, we'
GRBM
−H
[φ,h]
θ
lowingsteps.
where Zθ = Dφewith the “Hamiltoh
ボルツマンマシンは、本質的にガウス分布。

nian”
for •thissystem(6), data D ∼ e−S[φ] /Zlat which we
Preparinga teacher
混合ガウス分布にする？

Prep
4
would
like
to
mimic
(top
half
of
Figure.
2).
10te
GAN(Generative Adversarial
Networks)などより表現力の高い模型？
2
#
#
# φn
(φ
−
a
)
is
n
n
lattice
GRBM
理論的に経路積分に収束する模型？(あるのか？)
Hθ
[φ,
h] =
−
b
h
−
w
h
,
• Updating
parameters
θ
in
(6)
via
contrastive
diverj
j
nj
j
2
∆τ
=
),
2σ
σ
n
gence method[26,n27] (See appendix
C for
n
j
n,ja brief
dynammreview.).
and afwhere Throughout
θ = (an , σthis) represents
parameterstobenn , bpaper,j , wnjwe
Traiprepare teacher data D by the
Deep Learning
and Physics
2018
updatedin thelearning/training
process. isOnce
we fixas33ϵ =ed2/June/2018
original
HMC.
In
this sense our algorithm
not com-まとめ：問題設定と結果

A. Tomiya

提案したアルゴリズムで自己相関は短縮。相転移点で不具合

問題: 格子QCDでの配位生成を効率化したい！
効率化できれば、モンテカルロ法(HMC)からくる統計誤差を
短時間で減らすことが出来る。
手法: 機械学習をHMCに組み込む
機械学習は、それっぽい画像を作れる→ 応用できないか？
性質が良く分かってる3次元φ4理論で試してみた

Radford et al. (2015)

結果: 対称相、破れ相で、1点関数, 2点関数は無矛盾臨界点で、不具合がある
2/June/2018 Deep Learning and Physics 201834

バックアップto HHMC + O( ⌧ ) during the evolution (see for example [33]). This is due to violation of the low of energy
自己相関
conservation of a numerical solution. Thus we need to
MCMCでつくられる配位間の類似性
perform the Metropolis test in step 3 to obtain correct
distribution of e HHMC [ ,⇡] . By repeating step 1 to 3,
we obtain a sequence of configurations
which obeys
e HHMC [ ,⇡] ⇠ e S[ ] .
As we have mentioned, a sequence of configurations area↵ected by the autocorrelation, which is evaluated by the
autocorrelation function. The approximated autocorrelation function [24, 25] is defined by,(⌧ ) =

1
Nconf

⌧

N
conf
X

(Oc

O)(Oc+⌧O),

A. Tomiya

p( |h

p(hj

p(hj

and p(h|tion andprobabilit
cedure cal

(25)

c

For a give

where Oc = O[ (c) ] is the value of operator O for c-th
configuration (c) and ⌧ is the fictitious time of HMC.
Nconf is the number of configurations. We use normalized
one, ⇢(⌧ ) = (⌧ )/ (0).
The integrated autocorrelation time ⌧int quantifies effects of autocorrelation, which is given by,

2/June/2018 Deep Learning and Physics 2018

abilities aWX1⌧int = +⇢(⌧ ).
2 ⌧ =1

an

an

bj

bj +

wnj

w

+
n

(1

(26)
36

ニューラルネットワークとボルツマンマシン

A. Tomiya

ニューラルネットワーク=一方向, ボルツマンマシン=双方向

ニューラルネット

ボルツマンマシン

決定論的

確率論的

正解と出力を合わせる

分布を近づける

2/June/2018 Deep Learning and Physics 201837