ガウス過程と機械学習

持橋大地
統計数理研究所
daichi@ism.ac.jp
Summer School 数理物理
2020-8-28 (土)持橋担当分のスケジュール
l 1日目：概要と目次、ノンパラメトリックベイズ法(離散的な場合; 無限モデル)
l 2日目：ガウス過程とその適用(連続的な場合; ベイズ的関数回帰)
l 3日目：ノンパラメトリックベイズ法と自然言語処理への応用(研究紹介)

1/124

Summer School 数理物理 2021 Lecture2

今日の概要l はじめに: ガウス過程回帰とは何か
l 線形回帰モデル
l ガウス過程とガウス過程回帰
l カーネル関数の学習
l 深層学習との関係
l ガウス過程潜在変数モデル
l 様々な分野での応用

2/124

Summer School 数理物理 2021 Lecture2

教科書「ガウス過程と機械学習」
l 講談社機械学習プロフェッショナルシリーズ(MLP),
2019– 持橋大地・大羽成征著– 現在、レビュー49件
l 線形回帰モデルの非常にやさしい導入から入っています
l 確率過程としての話ではなく、統計の道具としての意味と使い方の話
3/124

Summer School 数理物理 2021 Lecture2

教科書 (GPML)
l “Gaussian Processes

for Machine Learning”
by Carl Rasmussen &Chris Williams (2006)
l 中級者以上向け
l 数学的な詳細やカーネル設計などについて
知りたい場合はこちら
l PDFがフリーでダウンロード可能
http://www.gaussianpr
ocess.org/gpml/
4/124

Summer School 数理物理 2021 Lecture2

注意：二つの見方について
l ガウス過程には、回帰としての見方と関数解析としての見方の2つがある– GPMLでは “weight-space view” と“function-space view” と呼ばれている
l 本講演と教科書では、前者の見方を主に紹介する
l 後者の見方も時に重要だが (3日目の内容も参照) 、
抽象的構成がわかれば、ガウス過程の全てを理解したということにはならない– カーネルの意味、ハイパーパラメータの学習– ガウス過程の様々な応用

5/124

Summer School 数理物理 2021 Lecture2

ガウス過程(正規過程)の歴史
l 数学や信号処理では、古くから知られていた概念
l ケンブリッジのCavendish Labの
MacKayが、1998年ごろ機械学習・
データ解析への適用可能性を示した
l 2006年にGPML発売 (Rasmussen&
Williams)l 最近になり、特定の研究室以外でも様々な研究が広まりつつある

6/124

Summer School 数理物理 2021 Lecture2(飛田・櫃田,1976)なぜガウス過程?
l 通常の線形回帰や一般化線形モデル (例: 緑本) にとどまらず、もっと柔軟な非線形な回帰を行いたい
l 空間的・時間的なスムージングを数学的に見通しよく行いたい– 空間統計学では、ガウス過程はKrigingとして知られている (ただし物理的な次元に限定)
l ニューラルネットの数学的代替：ニューラルネットは素子数→∞の極限でガウス過程に漸近 (Neal 1996)– ニューラルネットの重みを計算する必要がない！

7/124

Summer School 数理物理 2021 Lecture2

なぜガウス過程？ (2)l 自然言語処理でも、連続値を扱う機会が増えている／カテゴリに分類すれば終わりではない– 単語ベクトルの座標– 時間データ– 地理データ– 物理データ (ロボットや車の動作など)– 価格、得点、評価値など
l ニューラルネットにただ突っ込むだけでは、あまりにも貧しい (きちんとした道具が必要)

8/124

Summer School 数理物理 2021 Lecture2

例) 時間によるツイート内容の推移

http://socialmediaexperience.jp/3192 より引用

l 東日本大震災後、2011/3/12のツイートの時間変化– 簡単な分布では表現できない！
9/124

Summer School 数理物理 2021 Lecture2

例) おまけ・台風コロッケ

l 2019年10月11日の台風時の「コロッケ」のツイート

頻度と時間の関係
l 非常に滑らかな関数！
10/124 Summer School 数理物理 2021 Lecture2

例) Twitterの座標と言葉

l 2009年スーパーボウル時のツイートの単語頻度と座標 (New York Times)
11/124 Summer School 数理物理 2021 Lecture2

例) 株価とツイートの感情分析
Facebook社の株価(赤)とTwitterの感情スコア(青)の関係https://www.aclweb.org/anthology/W18-3102/ より引用
Figure 6: This shows normalized tweeter sentiweeter sentil “Causality Analysis of Twitter Sentiments and Stock Market
ments
calculatedby Amazon
MechanicalTurk
and Language
cal Turk Returns”,
andACL
2018 WSin Economics
andNaturalthe FaceBook
Processing
より stock returns.
12/124 Summer School 数理物理 2021 Lecture2

Table 5: Example of Tweets targeting FB

連続値への回帰問題
l これらは、入力 x 7! 出力 y 2 R (連続値) を予測する<latexit sha1_base64="7Gyym562WbYJaw50w504uTPTSS0=">AAAHh3ichZW/b9NAFMdfWyBt+dEWFiSWiKgVU7gEBBVTA1SialWlCWlTNVXlcy6pFcd2bSdNYoUd/gEGJpAYEAMrzCz8Awz9ExBjkVgYeD47chK/to6Su/u+z/fynu985pauOS5jJxOTU5cuX0lMz8xevXb9xtz8ws1tx2zZqiippm7aZa44QtcMUXI1VxdlyxZKk+tihzee+fGdtrAdzTReul1L7DeVuqHVNFVxUTqYT1XanUpTsRzXTFaOjlpK9VWym6xoRhJV95Bzr9BHiqWZvJLxTibspCC88ubC1BJUoAomqNCCJggwwMW+Dgo4+NmDDDCwUNsHDzUbe5qMC+jDLHpbSAkkFFQb+FvH0V6oGjj253SkW8V/0fFrozMJi+wn+8RO2Q/2mf1i/86cy5Nz+Ll0seWBV1gHc29uF/9e6Gpi68Jh5DrHwZE+vyYXarAsa9GwNksqfpVqMH+79/a0+KSw6C2xD+w31veenbDvWKHR/qN+3BKFd3L2Knpq2LaxVSGFd9jD0SBTjjFPqv0wm4jvEGSH4LoE1yW4HsH1CO6Y4I4JrkZwNYJrEFyD4HIElyO4NYJbI7h1glsnuDLBlQlul+B2CY4RHItxPFxffx9S68rDdQ3i8fXkchcP4oKIN3HHRkRFjuOUhTlqI1ygUKQTIx2S9HNzsf5hdqDFaf8uDfhhR6T3CZcun3mOyrhvOEI587GK82dUXES1LldymI7UuGNjJKvIszGW07BrM8xfxZiOJ8tmjFgd21Ecx6sxqkBQhRj1XJ42KchKuobns3/uBas5OPGDPIJTiY5k/Xnx/ZMZf9vEO9vZdOZBOrv1MLXyNHwTTcMduAv3cP7HsAIv8P6XMIfX8AW+wrfETOJ+4lFiOUAnJ0LPLRi5Ern/I3OPDw==</latexit>

問題 (回帰問題、regression)– 分類器では対応できない– 通常のガウス分布など単純な分布も使えない
l モデルがないと、無理矢理ニューラルネットに入れても解けない：動作がまったく保証されない道具を増やす必要がある

13/124 Summer School 数理物理 2021 Lecture2

ガウス過程とは?
y = f (x)3<latexit sha1_base64="u9vYqg89V+1qEr4Gdd1Xu4qInY8=">AAAHgnichZXNbtNAEMenH5BSPprCBQkJRUStClKjTYpUhEBqgUpUrao0IW2qpopsdx2sOLZlO2k+lBsnXoADJ5A4AEd4Ay68AIc+AuJYJC4cGK8dOc1Oi6Nkd2d+/8nM7npXdUzD8xk7HhufmLxwMTF1afrylavXZpKz13c8u+lqvKTZpu2WVcXjpmHxkm/4Ji87Llcaqsl31frTwL/b4q5n2NYLv+Pwg4ZSswzd0BQfTdXk7U6qUueutZhZdvzHcTelL1Ra7bvVZJplmHhScicbddIQPXl7dmIeKnAINmjQhAZwsMDHvgkKePjZhywwcNB2AD20udgzhJ9DH6ZR20SKI6GgtY6/NRztR1YLx0FMT6g1/BcTvy4qUzDHfrCP7IR9Z5/ZT/b3zFg9ESPIpYOtGmq5U515fbP457+qBrY+vIxV5yhUpM+vyQcdHohaDKzNEZagSi2M3+q+OSk+LMz15tl79gvre8eO2Tes0Gr91j5s88JbEf0QNTq2LWw1SOMM93A0yFRFX09Y+1E2Md8myDbBdQiuQ3BdgusS3BHBHRGcTnA6wdUJrk5wqwS3SnDrBLdOcBsEt0FwZYIrE9wewe0RHCM4JnFqtL7BPqTWVY3WNfTL66mKXTzwc8LfwB0bExUxlikHczROcaGFIj2J9EgyyM3H+ofZgU2mg1ka8MOK2N4nVKZ451W0jOqGPZQyL1WcP6PiIlprYiWH6dgqKzZPZRVrNkdyGlZtRflr6DPxZNmSiLWRHaXieE2iCgRVkKhn4rRJQ07QOp7PwbkXrubgxA/zCE8l2pML4uL9kx29beTOTi6TXcrktu+nV55EN9EU3II7sIDxl2EFnuP8lzCHV/AJvsDXxGTiXiKbWArR8bFIcwNOPYlH/wDJT4to</latexit>

2
1

yf 0
<latexit sha1_base64="7/SSaSZsREB6ukgOue0YsIbG7wk=">AAAHY3ichZXNbtNAEMenLdSlBfoBBySEVBG14hRtAhKIUwtUompV5YO0qdqq8rrrYMWxLdtJm0R5AbiCOHACiQPiMbjwAhz6ABwQxyJx4cB47chJdlocJbs78/tPZnbXu9yzrSBk7HRsfOLS5Ult6sr0zNVr12fn5he2A7fpG6JiuLbrV7keCNtyRCW0QltUPV/oDW6LHV5/Gvl3WsIPLNd5EbY9cdDQa45lWoYeoqnYPpzLsCyTz6LaySWdDCRPwZ2fWIZ9OAIXDGhCAwQ4EGLfBh0C/OxBDhh4aDuALtp87FnSL6AH06htIiWQ0NFax98ajvYSq4PjKGYg1Qb+i41fH5WLsMS+s8/sjH1jX9hP9vfcWF0ZI8qljS2PtcI7nH11q/znv6oGtiG8TFUXKDjSF9cUggmPZC0W1uZJS1SlEcdvdd6dlR+XlrrL7CP7hfV9YKfsK1botH4bn4qi9F5GP0KNiW0LWwMyOMNdHPUz5ejrSmsvySblTwjyhODaBNcmuA7BdQjumOCOCc4kOJPg6gRXJ7hVglsluHWCWye4DYLbILgqwVUJbpfgdgmOERxTOJ6sb7QPqXXlybrGfnU9udzFfb8g/A3csSmxL8cq5WGO1hAXWygyUMiAJKPcQqx/kO3bVDqapT4/qEjtPUJly3eeo2VUN+ihlAWl4sI5FZfRWpMrOUinVlWxOZRVqtkcyWlQtZXkb6DPxpNlSyHWRnYUx/GaQpUIqqRQz+Rpk4G8pE08n6NzL17N/okf5xGfSrQnH8XF+yc3etuone18Nnc/my8+yKw8SW6iKbgNd+Eexn8IK/Ac57+COQh4DW/g7eQPbUZb0G7G6PhYorkBQ4925x/gtIGS</latexit>

-1
-2
-3
-1

<latexit sha1_base64="s+gglNENGzc2w4yaDTBuW3xdn+w=">AAAHZXichZXNbtNAEMenLTSlUPoBQkgcqIhacYo2AQnUUwtUompVpQlpUzVV5XXXwYpjW7aT5kN5ASSu9MAJJA6Ix+DCC3DoEyDEsUhcODBeO3KSnRZHye7O/P6Tmd31Lnct0w8YOxsbn7hydTI1dW36+o2Zm7Nz8wu7vtPwdFHSHcvxylzzhWXaohSYgSXKrie0OrfEHq89D/17TeH5pmO/CtquOKxrVds0TF0L0FSsNFtHc2mWYfJZVDvZuJOG+Mk78xPLUIFjcECHBtRBgA0B9i3QwMfPAWSBgYu2Q+iizcOeKf0CejCN2gZSAgkNrTX8reLoILbaOA5j+lKt479Y+PVQuQhL7Dv7zM7ZN/aF/WR/L4zVlTHCXNrY8kgr3KPZN3eLf/6rqmMbwOtEdYmCI315TQEY8FTWYmJtrrSEVepR/Gbn9Ly4UljqLrOP7BfW94Gdsa9Yod38rX/aEYX3Mvoxagxsm9jqkMYZ7uKonylHX1dae3E2Cd8iyBbBtQmuTXAdgusQ3AnBnRCcQXAGwdUIrkZwawS3RnAbBLdBcJsEt0lwZYIrE9w+we0THCM4pnA8Xt9wH1LryuN1jfzqenK5i/t+QfjruGMToiLHKuVijuYQF1ko0ldInyTD3AKsf5Dt21Q6nKU+P6hI7D1CZcl3nqNlVDfooZR5peL8BRUX0VqVKzlIJ1ZVsTWUVaLZGslpULUd56+jz8KTZVsh1kd2FMfxukIVCKqgUC/kaZOGnKQNPJ/Dcy9azf6JH+URnUq0JxfGxfsnO3rbqJ3dXCb7KJPbeZxefRbfRFNwDx7AQ4z/BFbhJc5/CXOowlt4B6eTP1IzqdupOxE6PhZrbsHQk7r/D+0rgnc=</latexit>

0

1

2

3

4

x

x

l 非常に柔軟な回帰関数 f : x 7! yを生成する確率モデル<latexit sha1_base64="bQ1NOQGDcgw5E4ORoug05kKLfxs=">AAAHdHichZXNbtNAEMenLTSlfDSFCxIcIqxUnKJNQAL11AKVqFpVaULaVE0V2e46WPGXbMfNh/ICvAAHuIBUJMRjcOEFOPQREMciuHBgvHbkJJ4WR8nuzvz+k5nd9a7iGLrnM3Y2Mzt35ep8ZuHa4vUbN28tZZdv73l2x1V5TbUN260rsscN3eI1X/cNXndcLpuKwfeV9vPQvx9w19Nt65Xfc/iRKbcsXdNV2UdTM5vVVnONoJtrmLLj+Xau18xKrMDEk0t3inFHgvgp28tzK9CAY7BBhQ6YwMECH/sGyODh5xCKwMBB2xEM0OZiTxd+DkNYRG0HKY6EjNY2/rZwdBhbLRyHMT2hVvFfDPy6qMxBnn1nn9k5+8a+sB/s74WxBiJGmEsPWyXScqe59OZu9c9/VSa2PrxOVJcoFKQvr8kHDZ6KWnSszRGWsEo1ih/0355XVyv5wQr7yH5ifR/YGfuKFVrBL/V0l1feiejHqNGwDbBVQcIZHuBolKmCvoGwDuNsEr5LkF2C6xFcj+D6BNcnuBOCOyE4jeA0gmsTXJvg1gluneA2CW6T4LYIbovg6gRXJ7gDgjsgOEZwLMUp8fqG+5BaVyVe18ifXk9F7OKRnxN+E3dsQjTEOE05mKM+wUUWivRSpEeSYW4+1j/OjmxpOpylET+uSOxDQmWId15By7Ru3EMpy6mKyxdUXEVrS6zkOJ1Y04rtiawSzfZUTuOqnTh/FX0Gniw7KWJjakcpON5IURWCqqSoF+K0kaAkaA3P5/Dci1ZzdOJHeUSnEu0phXHx/ilO3zbpzl6pUHxUKO0+ltaexTfRAtyDB/AQ4z+BNXiJ81/DHAJ4D6fwaf535n5GyuQjdHYm1tyBiSdT+Ac6bIeT</latexit>(関数の確率分布)0l カーネル関数 k(x, x ) によって様々な関数が生成できる(ベイズ的なカーネル法)<latexit sha1_base64="UQ5LWKw2Asos2c1plKW066YAzMc=">AAAHbXichZXPaxNBFMdfW21q/dFWEQRFiqE/BAmTKCieWrVgaSlpYtqUpJTMdjYu2ewuu5s0P8g/4F08CIqCB/HP8OI/4KF/gnjwUMGLB9/Obtgk81o3bGbmvc/35b2ZyQx3TMPzGTseG584d34yMXVh+uKly1dmZueu7nh2w9VEQbNN2y3yiidMwxIF3/BNUXRcUalzU+zy2tPAv9sUrmfY1gu/7Yj9eqVqGbqhVXw0lWrL5WbrHr5Ldw9mkyzF5DOvdtJRJwnRk7XnJhahDIdggwYNqIMAC3zsm1ABDz8lSAMDB2370EWbiz1D+gX0YBq1DaQEEhW01vC7iqNSZLVwHMT0pFrDXzHxdVE5DwvsO/vMTtg39oX9YH9PjdWVMYJc2tjyUCucg5lXN/J//quqY+vDy1h1hoIjfXZNPujwSNZiYG2OtARVamH8ZufNSf5xbqG7yD6yn1jfB3bMvmKFVvO39mlb5N7K6Ieo0bFtYqtBEme4i6N+phx9XWntRdnEfIsgWwTXJrg2wXUIrkNwRwR3RHA6wekEVyO4GsGtEtwqwa0T3DrBbRDcBsEVCa5IcHsEt0dwjOCYwvFofYN9SK0rj9Y19KvryeUu7vsF4a/jjo2JshyrlIM5GkNcaKFITyE9kgxy87H+QbZvU+lglvr8oCK29wiVKf/zHC2jukEPpcwqFWdPqTiP1qpcyUE6tqqKzaGsYs3mSE6Dqq0ofw19Jp4sWwqxNrKjOI7XFCpHUDmFeiZPmyRkJK3j+Ryce+Fq9k/8MI/wVKI9mSAu3j/p0dtG7exkUun7qcz2g+TKk+gmmoKbcAeWMf5DWIHnOP8FeV+8hnfwfvJX4nriVuJ2iI6PRZprMPQklv4BwEeFIA==</latexit>

14/124 Summer School 数理物理 2021 Lecture2

ガウス過程とは? (2)l 入力が2次元の場合のガウス過程からのサンプル＝ランダムな連続曲面l 入力xがもっと高次元な場合も同様のイメージ
15/124 Summer School 数理物理 2021 Lecture2

ガウス過程とは? (3)l 関数のベイズ推定 : データが与えられると、関数の事後分布が得られる
l 青線は期待値
l データのない

場所は分散が大きい
l 通常の最適化では分散は表現できない

16/124 Summer School 数理物理 2021 Lecture2

ガウス過程とは? (4)l 柔軟な回帰関数を使える確率モデルなので、全体を見通しのよい統計モデルとして定義できる– 統計的な振る舞いが保証されている ( $ NN)– ハイパーパラメータも同時に学習できる
l カーネル法なので、多くの入力 x に対して自然に定義できる– 高次元の入力– 文字列、グラフ、木、確率モデル、‥
l ニューラルネットは、素子数! 1でガウス過程になる(Neal 1996)<latexit sha1_base64="yqDw6V2iC5ZNxiM8yakDIpO3Yzw=">AAAHc3ichZXPTxNBFMcfoBTxR4teTLgQKsSLzbSYaDyBSiKBEGgtlFDS7CyzZdPt7mZ329I2/Qf8Bzxw0sjB+Gd48R/wwJ9gPKLx4sG3s9v0xzxwm3Zm3vt8X9+bmZ3hrmX6AWMXE5NTN25OJ2Zuzd6+c/deMjV3f893Gp4uirpjOV6Ja76wTFsUAzOwRMn1hFbnltjntVehf78pPN907LdB2xVHda1qm4apawGaKqlk2RJG4JnVk0DzPKdVSaVZhslnQe1k404a4mfHmZtahjIcgwM6NKAOAmwIsG+BBj5+DiELDFy0HUEXbR72TOkX0INZ1DaQEkhoaK3hbxVHh7HVxnEY05dqHf/Fwq+HygVYYt/ZZ3bJvrEv7Af7e2WsrowR5tLGlkda4VaS7x4W/vxXVcc2gJOB6hoFR/r6mgIw4LmsxcTaXGkJq9Sj+M3O+8vCi/xSd5l9ZD+xvg/sgn3FCu3mL/18V+TPZPRj1BjYNrHVIY0z3MVRP1OOvq609uJsBvwpQZ4SXJvg2gTXIbgOwbUIrkVwBsEZBFcjuBrBrRHcGsFtENwGwW0S3CbBlQiuRHAHBHdAcIzgmMLxeH3DfUitK4/XNfKr68nlLu77BeGv444dEGU5VikXczRHuMhCkb5C+iQZ5hZg/cNs36bS4Sz1+WHFwN4jVJZ85zlaxnXDHkq5o1S8c0XFBbRW5UoO0wOrqtgayWqg2RrLaVi1Heevo8/Ck2VbIdbHdhTH8bpC5Qkqr1Cv5WmThpykDTyfw3MvWs3+iR/lEZ1KtCcXxsX7Jzt+26idvVwmu5LJ7T5Nr76Mb6IZmIdFeIzxn8EqvMH5L8ob6Qw+wfn078R8YjHxKEInJ2LNAxh5Ek/+AV0oiCY=</latexit><latexit sha1_base64="s+gglNENGzc2w4yaDTBuW3xdn+w=">AAAHZXichZXNbtNAEMenLTSlUPoBQkgcqIhacYo2AQnUUwtUompVpQlpUzVV5XXXwYpjW7aT5kN5ASSu9MAJJA6Ix+DCC3DoEyDEsUhcODBeO3KSnRZHye7O/P6Tmd31Lnct0w8YOxsbn7hydTI1dW36+o2Zm7Nz8wu7vtPwdFHSHcvxylzzhWXaohSYgSXKrie0OrfEHq89D/17TeH5pmO/CtquOKxrVds0TF0L0FSsNFtHc2mWYfJZVDvZuJOG+Mk78xPLUIFjcECHBtRBgA0B9i3QwMfPAWSBgYu2Q+iizcOeKf0CejCN2gZSAgkNrTX8reLoILbaOA5j+lKt479Y+PVQuQhL7Dv7zM7ZN/aF/WR/L4zVlTHCXNrY8kgr3KPZN3eLf/6rqmMbwOtEdYmCI315TQEY8FTWYmJtrrSEVepR/Gbn9Ly4UljqLrOP7BfW94Gdsa9Yod38rX/aEYX3Mvoxagxsm9jqkMYZ7uKonylHX1dae3E2Cd8iyBbBtQmuTXAdgusQ3AnBnRCcQXAGwdUIrkZwawS3RnAbBLdBcJsEt0lwZYIrE9w+we0THCM4pnA8Xt9wH1LryuN1jfzqenK5i/t+QfjruGMToiLHKuVijuYQF1ko0ldInyTD3AKsf5Dt21Q6nKU+P6hI7D1CZcl3nqNlVDfooZR5peL8BRUX0VqVKzlIJ1ZVsTWUVaLZGslpULUd56+jz8KTZVsh1kd2FMfxukIVCKqgUC/kaZOGnKQNPJ/Dcy9azf6JH+URnUq0JxfGxfsnO3rbqJ3dXCb7KJPbeZxefRbfRFNwDx7AQ4z/BFbhJc5/CXOowlt4B6eTP1IzqdupOxE6PhZrbsHQk7r/D+0rgnc=</latexit><latexit sha1_base64="iXB7CJqATxphe1LWPWyMcL9MliA=">AAAHdnichZXNbtNAEMenLTSlfDQtFyQkFBGlcIo2AQnEqQUqUbWq0oS0qZoqst11asWxLdv5Vl6AF+CAhAQSAsRjcOEFOPQREMciwYED47UjJ9lpcZTs7szvP5nZXe+qjml4PmOnM7Nzly7PJxauLF69dv3GUnJ5Zc+zW67Gy5pt2m5FVTxuGhYv+4Zv8orjcqWpmnxfbTwL/Ptt7nqGbb30ew4/aip1y9ANTfHRVEuuVF2jfuIrrmt3UlXD0v1eLZlmWSaelNzJRZ00RE/BXp5bhSocgw0atKAJHCzwsW+CAh5+DiEHDBy0HcEAbS72DOHnMIRF1LaQ4kgoaG3gbx1Hh5HVwnEQ0xNqDf/FxK+LyhRk2Hf2mZ2xb+wL+8H+nhtrIGIEufSwVUMtd2pLr26Vfv9X1cTWh5NYdYFCRfrimnzQ4bGoxcDaHGEJqtTC+O3+67PSk2JmsMres59Y3zt2yr5ihVb7l/ZhlxffiOjHqNGxbWOrQRpneICjUaYq+gbCOoyyifkuQXYJrkdwPYLrE1yf4DoE1yE4neB0gmsQXIPg1gluneA2CW6T4LYIbovgKgRXIbgDgjsgOEZwTOLUaH2DfUitqxqta+iX11MVu3jk54S/iTs2JqpiLFMO5mhMcKGFIj2J9EgyyM3H+sfZkU2mg1ka8eOK2D4kVKZ451W0TOvGPZSyIFVcOKfiElrrYiXH6dgqK7Ynsoo121M5jat2ovw19Jl4suxIxMbUjlJxvCFRRYIqStRzcdqkIS9oHc/n4NwLV3N04od5hKcS7ckHcfH+yU3fNnJnL5/NPcjmdx+m155GN9EC3Ia7cB/jP4I1eIHzX8YcuvAWPsKn+T+JO4lM4l6Izs5Empsw8STYP8wViT8=</latexit>

17/124 Summer School 数理物理 2021 Lecture2

線形回帰モデル

18 / 8018

単回帰モデル (simple regression)l 最も単純な回帰： y = a + bx
l aとbをどうやって決める?

19/124 Summer School 数理物理 2021 Lecture2

誤差の最小化l 実際の値 yn と予測値 ŷn = a+b xn の誤差 ✏ を<latexit sha1_base64="R2/EbKBXP4sp6l4utDPeEJgVDcU=">AAAHrnichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJiRN1USV12waK/6S7SRNorwARy4cygUkDojH4MILcOgjII5F4sKB8dpRPnZSbCXenfn9xzO7613uWWYQMnY+N79w5eo1bfH60o2bt24vr6zeKQduyzdEyXAt169wPRCW6YhSaIaWqHi+0G1uiQPefBn5D9rCD0zXeR12PVGz9RPHrJuGHqKpUhVeYFquc7ySYhkmrzW1kU0aKUiuvLu6sAVVeAMuGNACGwQ4EGLbAh0CvI8gCww8tNWgjzYfW6b0CxjAEmpbSAkkdLQ28f8Ee0eJ1cF+FDOQagPfYuHPR+UapNkP9oVdsO/sK/vJ/s6M1Zcxoly6+OSxVnjHy2/vFf/8V2XjM4TGSHWJgiN9eU0h1OGZrMXE2jxpiao04vjt3vuL4vNCuv+QfWK/sL6P7Jx9wwqd9m/j874onGH0tBxvgZGq0ManASkc4z72hrly9PWldZDkM+InFa7s+3ibmGMD1brsudCZGeGUeNcpwXUJrktwPYLrEVyH4DoEVye4OsE1Ca5JcBsEt0Fw2wS3TXA7BLdDcBWCqxDcIcEdEhwjOKZwPJnfaC1T88qTeY396nxy+SUM/YLw27jqR0RV9lXKwxzNCS62UGSgkAFJRrlFK3ycHdpUOhqlIT+uGNkHhMqS+wZHy7Ru3EMp80rF+RkVF+XXak9VMrKqit2JrEaa3amcxlV7Sf4G+izcnfYUYnNqRXHsbypUgaAKCvVK7jYpyEm6jntQtHfGszk8NeI84l2J9uSiuHiGZadPLLVRzmWyjzO5/Sep9RfJabYI9+EBPML4T2EdtnD8S/Kkegdn8EFjWlmraccxOj+XaO7CxKU1/gHgsJHv</latexit><latexit sha1_base64="9LohLkALeohWoe9L6zCvtAqTdE0=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5oq8ppNasWxLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPstNhKvDvz+49ndte73LVMP2DsdGZ27tLlK9r81YVr12/cXFxavrXrO23PEGXDsRyvwnVfWKYtyoEZWKLiekJvcUvs8eaL0L/XEZ5vOvaroOeKw5besM26aegBmkq9ml1bSrEMk9eK2sjGjRTEV95ZntuAKrwBBwxoQwsE2BBg2wIdfLwPIAsMXLQdwgBtHrZM6RcwhAXUtpESSOhobeJ/A3sHsdXGfhjTl2oD32Lhz0PlCqTZD/aFnbHv7Cv7yf6eG2sgY4S59PDJI61wa4tv75T+/FfVwmcAR4nqAgVH+uKaAqjDU1mLibW50hJWaUTxO/2Ts9KzYnpwn31iv7C+j+yUfcMK7c5v43NBFD9g9LQcb4GRqtDBpwEpHOMB9ka5cvQNpHUY55PwkwpH9j28TczxCNW67DnQPTfCMfGuY4LrEVyP4PoE1ye4LsF1Ca5OcHWCaxJck+DWCG6N4DYJbpPgtghui+AqBFchuH2C2yc4RnBM4Xg8v+FapuaVx/Ma+dX55PJLGPkF4W/hqk+IquyrlIs5mhNcZKFIXyF9kgxzC1f4ODuyqXQ4SiN+XJHYh4TKkvsGR8u0btxDKfNKxflzKi7Jr7U1VUliVRXbE1klmu2pnMZVO3H+Bvos3J12FGJ9akVx7K8rVJGgigr1Uu42KchJuo57ULh3RrM5OjWiPKJdifbkwrh4hmWnTyy1sZvLZB9lcoXHqdXn8Wk2D3fhHjzA+E9gFTZw/MuYQwPewXs40R5qBa2ivY7Q2ZlYcxsmLs34BwY5j60=</latexit><latexit sha1_base64="4V8ILK3KXNRdVgJP2wuNf1nln6c=">AAAHxHichZXNbtNAEMenLZBSPtrCBQkOCVEQElW0CUggJKQWKGrVqkoT2qaqq8hrNo0Vx7Zs51vhwJEX4MAJJCQQJ3gFLrwAhz4C4lgkLhwYrx3lw5NiK/HuzO8/ntld73Lb0F2PseOp6ZkzZ8/FZs/PXbh46fL8wuKVXdeqO5rY0SzDcopcdYWhm2LH0z1DFG1HqDVuiD1efeL79xrCcXXLfO61bXFYU49Mvaxrqoem0sINpaJ63XavZMaVxCMlEVeVxB0lwZWlVgndSZZm8opHG5mwkYTwylmLM2ugwAuwQIM61ECACR62DVDBxfsAMsDARtshdNHmYEuXfgE9mENtHSmBhIrWKv4fYe8gtJrY92O6Uq3hWwz8OaiMQ4r9YJ/YCfvOPrOf7O/EWF0Zw8+ljU8eaIVdmn99rfDnv6oaPj2oDFSnKDjSp9fkQRkeyFp0rM2WFr9KLYjf6Lw5KTzMp7q32Hv2C+t7x47ZN6zQbPzWPmyL/FuMnpLjLTCSAg18apDEMe5ir58rR19XWnthPgN+VGHJvoO3jjlWUK3KngXNiRFaxLtaBNcmuDbBdQiuQ3BNgmsSXJngygRXJbgqwa0Q3ArBrRPcOsFtENwGwRUJrkhw+wS3T3CM4FiE4+H8+muZmlcezmvgj84nl19C3y8Ifw1X/YBQZD9K2ZijPsIFFop0I6RLkn5u/gofZvu2KO2PUp8fVgzsPUJlyH2Do2VcN+yhlLlIxbkJFRfk11obq2RgjSo2R7IaaDbHchpWbYX5a+gzcHfaihCrYyuKY381QuUJKh+hnsrdJglZSZdxD/L3zmA2+6dGkEewK9GerB8Xz7DM+IkVbexm05m76ez2veTy4/A0m4XrcBNuY/z7sAxrOP47mMMr+Ahf4GvsWcyIubF6gE5PhZqrMHLFXv4DEueYlQ==</latexit>

最小化✏ = yn<latexit sha1_base64="7VSOK3jGYq0LXE1W6jZdmhzP5vM=">AAAH1XichZW/b9NAFMdfWyCl/GgKCxKLRRRUhFpdAhIICakFKrVqVbUpaVI1VeQzl9aKf8l207pRNsTAwsDAwAQSA+IvYKUL/wBD/wTEWCQWBp7PjpzEL8VW4rv3Pt/n9+7Od9wxdM9n7GRkdOzc+QuZ8YsTly5fuTqZnbq26dn7ribKmm3YbpWrnjB0S5R93TdE1XGFanJDVHjzaeivtITr6bb13A8csWOqu5be0DXVR1M9O1MTjqcbtqU8VoK6pcwotT3VbwedemKZVpW7Cj+sW3fq2RybZfJS0o1C3MhBfK3ZU2OLUIMXYIMG+2CCAAt8bBuggof3NhSAgYO2HWijzcWWLv0COjCB2n2kBBIqWpv4v4u97dhqYT+M6Um1hm8x8OeiUoE8+8E+s1P2nX1hP9nfobHaMkaYS4BPHmmFU598fWPjz39VJj592EtUZyg40mfX5EMDHspadKzNkZawSi2K3zp6d7rxqJRv32Yf2S+s7wM7YcdYodX6rX1aF6X3GD0vx1tgpBq08KlBDse4jb1urhx9bWntxPkkfL/Cln0Xbx1z3EO1Kns2HAyNcEi865DgAoILCO6I4I4I7oDgDgiuQXANgmsSXJPg5glunuCWCG6J4JYJbpngqgRXJbgtgtsiOEZwLMXxeH7DtUzNK4/nNfKn55PLL6HrF4TfxFWfEDXZT1MO5qj3cZGFIr0U6ZFkmFu4wnvZri1Nh6PU5XsVib1DqAy5b3C0DOp6PZRyLVXx2pCKN+TXag5UkljTipW+rBLNykBOvarVOH8NfQbuTqspYmFgRXHsL6SoEkGVUtQzudvkoCjpBu5B4d4ZzWb31IjyiHYl2lMM4+IZVhg8sdKNzeJs4d5scf1+bu5JfJqNw024BdMY/wHMwSKOfxlzeAtf4RscZyqZTuZl5lWEjo7EmuvQd2Xe/ANEUp6N</latexit>ŷn = yn(a + bxn )yn<latexit sha1_base64="9LohLkALeohWoe9L6zCvtAqTdE0=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5oq8ppNasWxLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPstNhKvDvz+49ndte73LVMP2DsdGZ27tLlK9r81YVr12/cXFxavrXrO23PEGXDsRyvwnVfWKYtyoEZWKLiekJvcUvs8eaL0L/XEZ5vOvaroOeKw5besM26aegBmkq9ml1bSrEMk9eK2sjGjRTEV95ZntuAKrwBBwxoQwsE2BBg2wIdfLwPIAsMXLQdwgBtHrZM6RcwhAXUtpESSOhobeJ/A3sHsdXGfhjTl2oD32Lhz0PlCqTZD/aFnbHv7Cv7yf6eG2sgY4S59PDJI61wa4tv75T+/FfVwmcAR4nqAgVH+uKaAqjDU1mLibW50hJWaUTxO/2Ts9KzYnpwn31iv7C+j+yUfcMK7c5v43NBFD9g9LQcb4GRqtDBpwEpHOMB9ka5cvQNpHUY55PwkwpH9j28TczxCNW67DnQPTfCMfGuY4LrEVyP4PoE1ye4LsF1Ca5OcHWCaxJck+DWCG6N4DYJbpPgtghui+AqBFchuH2C2yc4RnBM4Xg8v+FapuaVx/Ma+dX55PJLGPkF4W/hqk+IquyrlIs5mhNcZKFIXyF9kgxzC1f4ODuyqXQ4SiN+XJHYh4TKkvsGR8u0btxDKfNKxflzKi7Jr7U1VUliVRXbE1klmu2pnMZVO3H+Bvos3J12FGJ9akVx7K8rVJGgigr1Uu42KchJuo57ULh3RrM5OjWiPKJdifbkwrh4hmWnTyy1sZvLZB9lcoXHqdXn8Wk2D3fhHjzA+E9gFTZw/MuYQwPewXs40R5qBa2ivY7Q2ZlYcxsmLs34BwY5j60=</latexit>ŷn = a + bxn<latexit sha1_base64="RpKJQO0vcFbVMReJuW2uL+4jlmM=">AAAHuHichZXNbtNAEMenLZBSPprCBYlLRRSEhBRtAhKoElILVGrVqkoT0qZqquA1m8SKY1u282nlBXgBDnABqUiIx+DCC3DoIyCOReLCgfHakZN4Umwl3p35/cczu+tdbuma4zJ2Nje/cOnylcTi1aVr12/cXE6u3DpwzLatipJq6qZd5oojdM0QJVdzdVG2bKG0uC4OefOF7z/sCNvRTOOV27fESUupG1pNUxUXTdVkstJQXK8/rBrPlIe8V0VTimWYvFbjjWzYSEF45c2VhS2owBswQYU2tECAAS62dVDAwfsYssDAQtsJeGizsaVJv4AhLKG2jZRAQkFrE//r2DsOrQb2/ZiOVKv4Fh1/NipXIc1+sC/snH1nX9lP9ndmLE/G8HPp45MHWmFVl9/eKf75r6qFTxcakeoCBUf64ppcqMFTWYuGtVnS4lepBvE7g3fnxbVC2rvPPrFfWN9Hdsa+YYVG57d6ui8K7zF6Wo63wEgV6OBThRSOsYe9Ua4cfZ60DsN8In5SYcq+jbeGOTZQrcieCd2ZEXrEu3oE1ye4PsENCG5AcF2C6xJcjeBqBNckuCbBbRDcBsFtE9w2we0Q3A7BlQmuTHBHBHdEcIzgWIzj4fz6a5maVx7Oa+CPzyeXX8LILwh/C1d9RFRkP05ZmKM2wQUWinRipEOSfm7+Ch9nR7Y47Y/SiB9XRPYhodLlvsHRMq0b91DKfKzi/IyKi/JrbU1VElnjit2JrCLN7lRO46q9MH8VfTruTnsxYnNqRXHsb8aoAkEVYtRLudukICfpGu5B/t4ZzObo1AjyCHYl2pPz4+IZlp0+seKNg1wm+yiT23+cWn8enmaLcBfuwQOM/wTWYQvHv4Q5dOADnMLnxFridaKe0AJ0fi7U3IaJK2H/A+LHlWE=</latexit>

20/124 Summer School 数理物理 2021 Lecture2

単回帰モデル (2)l データ D = {(x1 , y1 ), (x2 , y2 ), · · · , (xN , yN )}があったとする.
l 各 xn に対する予測値 ŷn は、一次式

ŷn = a + bxn
l 観測値との差はyn ŷn = yn (a + bxn )– これを最小にしたい！

21/124 Summer School 数理物理 2021 Lecture2

単回帰モデル (3)l n=1,2,...,N について、

誤差＝ ynŷn → 誤差の総和を最小にしたい

l 誤差は負のこともあるので、二乗した二乗誤差を最小化 (最小二乗法)：

E=

N
X

(yn

ŷn )2 =

n=1

N
X

n=1

を最小にする a,b を求める

22/124 Summer School 数理物理 2021 Lecture2(yn (a + bxn ))2

単回帰モデル (4)l Eの極小点ではa,bについての偏微分は0になるので、
N
@E
@ X
=
(yn (a + bxn ))2@a@a n=1
N@ X 2=(yn + a2 + b2 x2n 2ayn 2abxn + 2bxn yn ) = 0@a n=1N@E@ X=(yn (a + bxn ))2@b@b n=1
N@ X 2=(yn + a2 + b2 x2n 2ayn 2abxn + 2bxn yn ) = 0@b n=1
P 2P
P
P
x
yn
n xn
n x n ynPa = n n Pn 2l これを解いて、N n xn ( n xn ) 2
PPP
N n x n yn
n xn
n yn
P 2
P
b=
N n xn ( n xn ) 2

23/124 Summer School 数理物理 2021 Lecture2

単回帰モデルの計算例
y = 1.77 + 0.42x

l 公式に代入して、

a = 1.77, b= 0.42
24/124 Summer School 数理物理 2021 Lecture2

一番単純な場合：
データD={(3,2),(2,4),(-1,1)}

重回帰モデルl 入力xが多次元なら? → 重回帰 (multiple regression)x = (x1 , x2 , · · · , xD )T のとき、
y = w 0 + w 1 x1 + w 2 x2 + · · · + w D xD
l 二乗誤差は、(yŷ)2 = (yl これを最小化(w0 + w1 x1 + w2 x2 + · · · + wD xD ))2

PN→ E = n=1 (yn ŷn )2 を w0 , w1 , · · · , wD について微分して0とおき、連立方程式を解けばよい.

25/124 Summer School 数理物理 2021 Lecture2

もっと見通しよく!
l x を新しく x = (1, x1 , x2 , · · · , xD ) 、

重みベクトルを w = (w0 , w1 , w2 , · · · , wD ) と表せば、

ŷ = w0 + w1 x1 + w2 x2 + · · · + wD xD
0 1= (w0 , w1 , w2 , · · · , wD ) 1
B x1 CB C
B .. C
@ . A
T

=w x

26/124 Summer School 数理物理 2021 Lecture2

xD

もっと見通しよく! (2)l よって、n=1,2,...,N について縦に並べれば、

0

0 T1
w x1
x1
ŷ1
B ŷ2 C B wT x2 C B xT2 C
B C B
C B C
B .. C = B .. C = B .. C w@ . A @ . A @ . A
ŷN

1

0

T

w T xNl つまり、ŷ = Xwと書ける!

27/124 Summer School 数理物理 2021 Lecture2

1

xTN

計画行列
という

行列・ベクトル表現
E=

N
X

n=1

2

(yn ŷn ) = (y1 ŷ1 , · · · , yN1y1 ŷ1
B
C
..
@
A
.
yN

l なので、E = (yŷN )0Xw)T (yXw)= yT (yXw)= yT y2wT (XT y) + wT XT Xw

28/124 Summer School 数理物理 2021 Lecture2(Xw)T (yXw)

ŷN重回帰モデルの解E = yT y

2wT (XT y) + wT XT Xw

l を w で微分して、

@E
=
@w

2XT y + 2XT Xw = 0

l よって(正規方程式)XT Xw = XT y
) w = (XT X) 1 XT y .
重回帰モデルの解

29/124 Summer School 数理物理 2021 Lecture2

重回帰モデルの計算例
l データが下のとき、30/124 Summer School 数理物理 2021 Lecture2

0
1
B1
X=B
@1
1

1
1
3
2

1

!2
1C
C
0A
2

重回帰モデルの計算例 (2)l よって、重みベクトルwの解は

y = 1.202 0.016x1 +1.209x2

31/124 Summer School 数理物理 2021 Lecture2

もっと複雑にしたい!

l 直線や平面で表せない関係も多いのでは?

関数をもっと複雑にすればよい！
32/124 Summer School 数理物理 2021 Lecture2

線形回帰モデル
y = w0 + w1 x + w2 sin(x)y = w 0 + w 1 x + w 2 x2
01
0 1
w0 w1 w21
w0 w1 w2
1
@ x A
@xA
=
=
sin(x)x2|{z} | {z }|{z} | {z }
(x)
wT
(x)
wT

l どれも、係数ベクトルの線形式として書ける！
T– y = w (x) … 線形回帰モデル (linear regression
model)l これをシグモイド関数に通したのがロジスティック
T
y
=
(w
(x))
回帰モデル
<latexit sha1_base64="R6dNB6qxcdhKJVVsyuqO/fHCSek=">AAAHf3ichZXPTxNBFMdfQV3EHxS9mHBpbEC4NNNq4o/EBFQSCYRAS6GEYrOzzJZNt+2mu/2dHrz6D3jwpIkxxov+DV78BzzwJxiPmHjx4NvZbfpjHrBNOzPvfb6v783MznDHtlyPsZPIxOSly1e0qavT167fuDkTnb2161brNUNkjapdreW47grbqoisZ3m2yDk1oZe5LfZ46bnv32uImmtVKzte2xGHZb1YsUzL0D00FaJz7djTWN61imV9Md9ovtrJc+fYwm5raakQjbMEk09M7STDThzCZ6s6O7kAeTiCKhhQhzIIqICHfRt0cPFzAElg4KDtELpoq2HPkn4BPZhGbR0pgYSO1hL+FnF0EForOPZjulJt4L/Y+K2hMgbz7Cf7zE7ZD/aF/WL/zozVlTH8XNrY8kArnMLMmzuZvxeqyth6cDxQnaPgSJ9fkwcmPJK1WFibIy1+lUYQv9F5e5p5kp7vLrAP7DfW956dsO9YYaXxx/i4LdLvZPQj1JjYNrA1II4z3MVRP1OOvq609sJsBnyLIFsE1ya4NsF1CK5DcE2CaxKcSXAmwZUIrkRwKwS3QnBrBLdGcOsEt05wOYLLEdw+we0THCM4pnA8XF9/H1LrysN1DfzqenK5i/t+QfjLuGMHRF6OVcrBHK0RLrBQpKuQLkn6uXlY/zDbt6m0P0t9flgxsPcIlS3feY6Wcd2wh1JuKRVvnVFxBq1FuZLD9MCqKjZGshpoNsZyGlZthvkb6LPxZNlUiNWxHcVxvKpQaYJKK9QLedrEISVpE89n/9wLVrN/4gd5BKcS7Un5cfH+SY7fNmpnN5VI3k+kth/El5+FN9EUzMFdWMT4D2EZXuL8ZzGH1/AJvsI3LaLd0xIaC9CJSKi5DSOP9vg/UIWKmA==</latexit>

33/124 Summer School 数理物理 2021 Lecture2

線形回帰モデル (2)y = wT (x) (= (x)T w)
T
y
=
w
x
l は、xがφ(x)に変わっただけで重回帰モデル
2
3
(x)
=
(1,
x,
x
,
x
)
と同じなので、たとえばのとき、上をN個並べれば
0 1 0
1
0
10 1
T
2
3
(x1 )
ŷ1
1 x1 x1 x1
w0
B ŷ2 C B (x2 )T C
B1 x2 x22 x32 C B C
B C B
C
B
C Bw1 CB .. C = B
C w = B ..
.. C @w A..
@ . A @
A
@.
2
. A
.
w3
ŷN
1 xN x2N x3N(xN )T|| {z }{z} | {z }wŷ

新しい計画行列34/124 Summer School 数理物理 2021 Lecture2

線形回帰モデル (3)l つまり一般に、線形回帰モデルは以下のように書けるb=l 計画行列Φを使って、yl X 7!

w と書ける以外は重回帰と同じなので、w = (35/124 Summer School 数理物理 2021 Lecture2

T

) 1

T

y

線形回帰モデルの例(x) = (1, x, x2 , sin x, cos x)Tとして先ほどのデータに適用すると、
w = ( 0.065, 0.068, 0.022, 0.333, 0.863)T が解

l 特徴ベクトルを

y = 0.07+0.07x+0.02x2 +0.33 sin x 0.86 cos x
36/124 Summer School 数理物理 2021 Lecture2

線形回帰モデルと基底関数
y = wT (x)w = (w0 , w1 , w2 , · · · , wH )(x) = ( 0 (x),| {z }

1 (x),

2 (x), · · · ,H (x))=1l よって、y = w0 +w1 1 (x)+w2 2 (x)+· · ·+wH H (x) は関数 y = 0 (x) (= 1)y = 1 (x)y = 2 (x)..
.
y = H (x)

基底関数
という

の重みつき和 (線形結合) とみなせる
37/124 Summer School 数理物理 2021 Lecture2

線形回帰モデルと基底関数

l 2次関数y=-1.7x2-2x+0.9 は、関数 y=1, y=x, y=x2の線形和38/124 Summer School 数理物理 2021 Lecture2動径基底関数回帰l それなら、 h (x) = exp✓(x µh )
2

2◆をたくさん用意すれば、任意の関数が表せるのでは?
動径基底関数回帰 (radial basis function regression)
39/124 Summer School 数理物理 2021 Lecture2

次元の呪いl しかし…

l 動径基底関数回帰に必要な基底関数の数(=パラメータの数)は、xの次元が増えると指数的に増加– 1おきに基底関数をとると、[-10,10]で1次元では21個– 2次元では212=441個– 10次元では2110=16,679,880,978,201個！
→ 次元の呪い (curse of dimensionality)
40/124 Summer School 数理物理 2021 Lecture2

ガウス分布とガウス過程

41 / 8041

ガウス分布l ガウス分布 (Gaussian distribution) または

正規分布 (normal distribution)：最も基本的な確率分布

p(x) = N (x|µ,

2

1
)= p
exp
2⇡

42/124 Summer School 数理物理 2021 Lecture2

✓

(x

µ)
2 2

2

◆

多変量ガウス分布

l p(x|µ, ⌃) = p

1✓µ = E[x]

p
exp
|⌃|

⌃ = E[xxT ]E[x]E[x]T(2⇡)D

43/124 Summer School 数理物理 2021 Lecture2

1(x µ)T ⌃ 1 (x µ)2(平均ベクトル)(共分散行列)◆多変量ガウス分布からのサンプル

l 共分散行列の要素の値が大きい(共分散が大)と、類似した値がサンプルされる– 負では逆相関、0ならば、無相関
44/124 Summer School 数理物理 2021 Lecture2

多変量ガウス分布からのサンプル (2)

0
1
1 0 0
⌃ = @0 1 0A
0 0 1
l 3次元の場合

01
0
1
1 0.5 0.2
1 0.8 0.6
⌃ = @0.5 1 0.5A ⌃ = @0.8 1 0.8A
0.2 0.5 1
0.6 0.8 1

45/124 Summer School 数理物理 2021 Lecture2

多変量ガウス分布の線形変換
l x が多変量ガウス分布に従っていて<latexit sha1_base64="3PJSfo9WPR2PXWJ+NPyjTsCIZQs=">AAAHsHichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJqRNaarKa9aNib9kO9/KC3BG4oBAAokD4jG48AIc+giIY5G4cGC8duQknhRbiXdnfv/xzO56l7um4QeMnc7Mzl24eEmZv7xw5eq164tLyzf2fKfpaaKiOabjVbnqC9OwRSUwAlNUXU+oFjfFPm88Df37LeH5hmM/D7quOLLUE9vQDU0N0PSiZqlBnev9zuB4KcNyTF4r6UY+bmQgvorO8twG1OAlOKBBEywQYEOAbRNU8PE+hDwwcNF2BH20edgypF/AABZQ20RKIKGitYH/J9g7jK029sOYvlRr+BYTfx4qVyDLfrAv7Ix9Z1/ZT/Z3aqy+jBHm0sUnj7TCPV58fav8578qC58B1BPVOQqO9Pk1BaDDI1mLgbW50hJWqUXxW723Z+XHpWz/LvvEfmF9H9kp+4YV2q3f2uddUXqH0bNyvAVGqkELnxpkcIz72BvmytHXl9ZBnE/Cjysc2ffwNjDHOqpV2XOgPTVCh3hXh+C6BNcluB7B9QiuTXBtgtMJTie4BsE1CG6N4NYIbpPgNglui+C2CK5KcFWCOyC4A4JjBMdSHI/nN1zL1LzyeF4jf3o+ufwShn5B+C1c9QlRk/005WKOxhgXWSjST5E+SYa5hSt8lB3a0nQ4SkN+VJHYB4TKlPsGR8ukbtRDKYupiotTKi7Lr9WaqCSxphXbY1klmu2JnEZVO3H+GvpM3J12UsT6xIri2F9PUSWCKqWoZ3K3yUBB0jruQeHeGc3m8NSI8oh2JdpTCOPiGZafPLHSjb1CLn8/V9h9kFl9Ep9m83Ab7sA9jP8QVmEDx7+COdjwBt7DB6WgVJVjRY3Q2ZlYcxPGLuXVP9IIkus=</latexit><latexit sha1_base64="RafrNbWluIAqaT3HvAoOIfhOyYE=">AAAIBHichZW/b9NAFMdfWyAl/GgLCxJLRFTUDq0uAQnE1AKVWrWq2qQ/UtVtZJtzYsWOLdtNk1peGfgHGJhAYkCwIVYGWPgHGPofgJigSCwMPJ8dOYlfi63Ed+99vs/v3Z3vFNvQXY+x46HhkXPnL2RGL2YvXb5ydWx84tqWax04Kt9ULcNyKorsckNv8k1P9wxesR0um4rBt5XGo9C/3eKOq1vNDa9j8z1TrjV1TVdlD03VccmekkzZqyua3w6mc5LtWLZn5STetiWDa95UdkbSHFn1C4FfDHIJu78hKaYvlfWaKQf7/kwhSHxZydFrdW+6Op5ns0xcuXSjEDfyEF9r1sTIIkjwBCxQ4QBM4NAED9sGyODivQsFYGCjbQ98tDnY0oWfQwBZ1B4gxZGQ0drA/xr2dmNrE/thTFeoVXyLgT8HlTmYZF/ZG3bCvrC37Dv7e2osX8QIc+ngU4m03K6OPbtR/vNflYlPD+qJ6gyFgvTZNXmgwX1Ri4612cISVqlG8VtHz0/KD0qT/m32iv3A+l6yY/YZK2y2fquv13npBUafFOPNMZIELXyqkMcx9rHXzVVBny+sQZxPwvcrLNF38NYxxzqqZdGz4PDUCG3iXW2C6xBch+COCO6I4A4J7pDgNILTCK5BcA2Cmye4eYJbIrglglsmuGWCqxBcheB2CG6H4BjBsRSnxPMbrmVqXpV4XiN/ej4V8SV0/Zzwm7jqE0IS/TRlY456HxdZKNJNkS5JhrmFK7yX7drSdDhKXb5XkdgDQmWIfUNBy6Cu10Mp11IVr51ScVl8reZAJYk1rVjpyyrRrAzk1KtajfNX0Wfg7rSaIhYGVpSC/YUUVSKoUop6LHabPBQFreEeFO6d0Wx2T40oj2hXoj3FMC6eYYXBEyvd2CrOFu7MFtfv5ucexqfZKNyEWzCF8e/BHCzi+G9iDp/gG/yEX5mnmXeZ95kPETo8FGuuQ9+V+fgPOXi0yg==</latexit>

p(x) / exp✓

1 T
x ⌃ 1x2◆

x = A 1yのとき、
l x を行列 Aで変換した y = Ax の分布は
✓
◆
1
@y
1
T
1
1
p(y) / exp(A y) ⌃ (A y)
2
@x
✓
◆
1 T
/ exp
y ⇤y
⇤ = (A 1 )T ⌃ 1 A 1 y
2<latexit sha1_base64="qnHg2EttvIwBsOv5z2lf7Gn717k=">AAAHvXichZXNbtNAEMenLZBSPpqCkJC4pERBXIg2AQmEhGiASq1aVWlC2lR1iWyzaa04tmU7zoeVF+AFOHAqEgfoY3DhBTj0ERDHInHhwHjtKB+etLYS7878/uOZ3fWuYuma4zJ2OjM7d+nylcT81YVr12/cXEwu3dpxzJat8opq6qZdVWSH65rBK67m6rxq2VxuKjrfVRqvA/+ux21HM423btfiB0350NDqmiq7aKol70heJyUtv5CWU5JXeOc/yvUlr1tLplmWiSsVb+SiRhqiq2guza2BBO/BBBVa0AQOBrjY1kEGB+99yAEDC20H4KPNxpYm/Bz6sIDaFlIcCRmtDfw/xN5+ZDWwH8R0hFrFt+j4s1GZggz7yb6yM/aDnbBf7N/UWL6IEeTSxacSarlVW/xwt/z3QlUTny4cDVXnKBSkz6/JhTo8E7VoWJslLEGVahjf6308Kz8vZfwH7DP7jfUds1P2HSs0vD/ql21e+oTRM2K8OUaSwMOnCmkcYx97g1wV9PnC2o/yGfLjClP0bbw1zPEI1bLomdCeGqFDvKtDcF2C6xJcj+B6BNcmuDbB1QmuTnANgmsQXIHgCgS3TnDrBLdBcBsEVyW4KsHtEdwewTGCYzFOieY3WMvUvCrRvIb++Hwq4ksY+Dnhb+KqHxKS6McpC3PUxrjQQpFOjHRIMsgtWOGj7MAWp4NRGvCjiqG9T6h0sW8oaJnUjXooZTFWcXFKxWXxtTYnKhla44rNsayGms2JnEZVW1H+Kvp03J22YsTqxIpSsL8ao0oEVYpRb8Ruk4a8oOu4BwV7Zzibg1MjzCPclWhPPoiLZ1hu8sSKN3by2dzjbH77SXrlVXSazcM9uA8PMf5TWIE1HP+KyOEYvsFJ4mWCJ/SEEaKzM5HmNoxdifZ/woaWHw==</latexit><latexit sha1_base64="3PJSfo9WPR2PXWJ+NPyjTsCIZQs=">AAAHsHichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJqRNaarKa9aNib9kO9/KC3BG4oBAAokD4jG48AIc+giIY5G4cGC8duQknhRbiXdnfv/xzO56l7um4QeMnc7Mzl24eEmZv7xw5eq164tLyzf2fKfpaaKiOabjVbnqC9OwRSUwAlNUXU+oFjfFPm88Df37LeH5hmM/D7quOLLUE9vQDU0N0PSiZqlBnev9zuB4KcNyTF4r6UY+bmQgvorO8twG1OAlOKBBEywQYEOAbRNU8PE+hDwwcNF2BH20edgypF/AABZQ20RKIKGitYH/J9g7jK029sOYvlRr+BYTfx4qVyDLfrAv7Ix9Z1/ZT/Z3aqy+jBHm0sUnj7TCPV58fav8578qC58B1BPVOQqO9Pk1BaDDI1mLgbW50hJWqUXxW723Z+XHpWz/LvvEfmF9H9kp+4YV2q3f2uddUXqH0bNyvAVGqkELnxpkcIz72BvmytHXl9ZBnE/Cjysc2ffwNjDHOqpV2XOgPTVCh3hXh+C6BNcluB7B9QiuTXBtgtMJTie4BsE1CG6N4NYIbpPgNglui+C2CK5KcFWCOyC4A4JjBMdSHI/nN1zL1LzyeF4jf3o+ufwShn5B+C1c9QlRk/005WKOxhgXWSjST5E+SYa5hSt8lB3a0nQ4SkN+VJHYB4TKlPsGR8ukbtRDKYupiotTKi7Lr9WaqCSxphXbY1klmu2JnEZVO3H+GvpM3J12UsT6xIri2F9PUSWCKqWoZ3K3yUBB0jruQeHeGc3m8NSI8oh2JdpTCOPiGZafPLHSjb1CLn8/V9h9kFl9Ep9m83Ab7sA9jP8QVmEDx7+COdjwBt7DB6WgVJVjRY3Q2ZlYcxPGLuXVP9IIkus=</latexit><latexit sha1_base64="Rnf2v3enTatqUa8YTMYpvnBuqHc=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwNUatWqygdpU5qq8ppNasWOLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPstNhKvDvz+49ndte73LVMP2DsdGZ27tLlK9r81YVr12/cXFxavrXjO23PEBXDsRyvynVfWGZLVAIzsETV9YRuc0vs8uaL0L/bEZ5vOq1XQc8VB7beaJl109ADNJVrnfzhUoplmLxW1EY2bqQgvgrO8tw61OANOGBAG2wQ0IIA2xbo4OO9D1lg4KLtAAZo87BlSr+AISygto2UQEJHaxP/G9jbj60t7Icxfak28C0W/jxUrkCa/WBf2Bn7zr6yn+zvubEGMkaYSw+fPNIK93Dx7Z3yn/+qbHwGcJSoLlBwpC+uKYA6PJW1mFibKy1hlUYUv9M/OSs/K6UH99kn9gvr+8hO2TessNX5bXwuitIHjJ6W4y0wUg06+DQghWM8wN4oV46+gbQO43wSflLhyL6Ht4k5HqFalz0HuudGOCbedUxwPYLrEVyf4PoE1yW4LsHVCa5OcE2CaxJcnuDyBLdBcBsEt0lwmwRXJbgqwe0R3B7BMYJjCsfj+Q3XMjWvPJ7XyK/OJ5dfwsgvCL+Nqz4harKvUi7maE5wkYUifYX0STLMLVzh4+zIptLhKI34cUViHxIqS+4bHC3TunEPpSwoFRfOqbgsv1Z7qpLEqiq2JrJKNFtTOY2rtuP8DfRZuDttK8Ta1Iri2F9TqBJBlRTqpdxtUpCTdB33oHDvjGZzdGpEeUS7Eu3JhXHxDMtOn1hqYyeXyT7K5IqPU6vP49NsHu7CPXiA8Z/AKqzj+Fcwhwa8g/dwoj3UilpVex2hszOx5jZMXJrxD5Ofj3o=</latexit><latexit sha1_base64="4MBZGRALBMbZVKw59qADGeWLQhs=">AAAHtHichZXNbtNAEMenLeBSPtrCBYlLSxTEKdoEBAgJqQEqtWpVpQlpU9VtZZtNa8Vfsh3nS3kBXgAkTq3EAfEYXHgBDn0ExLFIXDgwXjtyEk+KrcS7M7//eGZ3vas6hu75jJ1PTc9cuXpNmr0+d+PmrdvzC4t3djy76Wq8qtmG7dZUxeOGbvGqr/sGrzkuV0zV4Ltq43Xo3w246+m29dbvOPzAVI4tva5rio+mQznoyMsv5WU5KMpB+2ghw3JMXEvpRj5uZCC+SvbizBrI8A5s0KAJJnCwwMe2AQp4eO9DHhg4aDuAHtpcbOnCz6EPc6htIsWRUNDawP9j7O3HVgv7YUxPqDV8i4E/F5VLkGU/2Bd2wb6zr+wn+zsxVk/ECHPp4FONtNw5mn9/r/LnvyoTnz6cJKpLFCrSl9fkQx2ei1p0rM0RlrBKLYofdD9cVF6Us72H7Iz9wvpO2Tn7hhVawW/t8zYvf8LoWTHeHCPJEOBTgwyOcQ97g1xV9PWEtR/nk/CjClv0Xbx1zPEE1Yro2dCaGKFNvKtNcB2C6xBcl+C6BNciuBbB1QmuTnANgmsQXJHgigS3TnDrBLdBcBsEVyO4GsHtEdwewTGCYylOjec3XMvUvKrxvEb+9Hyq4ksY+DnhN3HVJ4Qs+mnKwRz1ES6yUKSXIj2SDHMLV/gwO7Cl6XCUBvywIrH3CZUh9g0VLeO6YQ+lLKUqLk2ouCK+VnOsksSaVmyOZJVoNsdyGlZtxflr6DNwd9pKEatjK0rF/mqKKhNUOUW9EbtNBgqCruMeFO6d0WwOTo0oj2hXoj2FMC6eYfnxEyvd2Cnk8o9zhe0nmZVX8Wk2C/fhATzC+M9gBdZw/KuYgwsf4RTOpKeSLGkSj9DpqVhzF0YuyfoHGyGTtA==</latexit><latexit sha1_base64="ski3/L4QaZRjqCefepBv6RTsNkQ=">AAAIsnichZXLbtNQEIYnLZfUXNrCBolNRNQqSFCdhEogJKQWqNSqVdUmvaTUbWW7x6kVJza2kyY1fgEeABYsuEgsEI/Bhhdg0UdALIvEhgXjY4dcPG1tJT4z8/1zZnzsY9U2Dddj7Dg1NHzh4qXL6RHpytVr10fHxm9suFbD0fi6ZpmWU1YVl5tGna97hmfysu1wpaaafFOtPgvjm03uuIZVX/PaNt+pKZW6oRua4qFrbzw1YufkZvuuNCnbjmV7liTzli2bXPdy0n1ZdxTNzwd+IZAQm9317+eDEN9dk9WSUakpwtMXkmTHqBx4eA2TvJKiHLKtOJ6hmBlEgl6rFcQCJOVzq5DvoT6cfAlb3FfQ6E73sqHsR5PmOvHMk8z/2gZq7ik5zrA3lmVTTByZ5CAfD7IQHyvW+PA8yLAPFmjQgBpwqIOHYxMUcPHchjwwsNG3Az76HBwZIs4hAAm1DaQ4Egp6q/hfQWs79tbRDnO6Qq3hLCb+HFRmYIL9YF/YCfvOvrKf7O+puXyRI6yljVc10nJ7b/T1rdKfc1U1vHpw0FWdoVCRPrsnD3R4JHoxsDdbeMIutSh/8+jtSelxccKfZJ/YL+zvIztm37DDevO39nmVF99h9glxvzlmkqGJVw2yeI99tDq1qhjzhTeI6+ny/QpL2A6eBtZ4gGpFWBYcnpqhRczVIrg2wbUJ7ojgjgjukOAOCU4nOJ3gqgRXJbhZgpsluAWCWyC4RYJbJLgywZUJbovgtgiOERxLcGq8vuGzTK2rGq9rFE+upyrehE6cE/EaPvVdQhZ2krKxRqOPizwU6SZIlyTD2sInvJft+JJ0eJc6fK+i6w8IlSn2DRU9g7reCKVcSXS8ckrHJfG21gY66XqTiqW+qrqapYGaelXLcf0axkzcnZYTxNzAE6WiPZegigRVTFDPxW6ThYKgddyDwr0zWs3OVyOqI9qV6EghzIvfsPzgFys52ChM5R9MFVanszNP469ZGm7DHchh/ocwA/N4/9dBS9mpN6n3qQ/p6fSLtJLWInQoFWtuQt+RNv8BKVzseg==</latexit>– よって、y もガウス分布に従う<latexit sha1_base64="cBy7SP6mgyWfvD8VPfcTFpsZ06Q=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5qq8ppNasWOLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPspNhKvDvz+49ndte73LVMP2DsbGZ27tLlK9r81YVr12/cXFxavrXrOy3PEGXDsRyvwnVfWGZTlAMzsETF9YRuc0vs8caL0L/XFp5vOs1XQdcVh7Zeb5o109ADNJWq7e7RUoplmLxW1EY2bqQgvvLO8twGVOENOGBAC2wQ0IQA2xbo4ON9AFlg4KLtEPpo87BlSr+AASygtoWUQEJHawP/69g7iK1N7Icxfak28C0W/jxUrkCa/WBf2Dn7zr6yn+zv1Fh9GSPMpYtPHmmFe7T49k7pz39VNj4DOE5UFyg40hfXFEANnspaTKzNlZawSiOK3+6dnpeeFdP9++wT+4X1fWRn7BtW2Gz/Nj4XRPEDRk/L8RYYqQptfBqQwjHuY2+YK0dfX1oHcT4JP65wZN/D28Qcj1Gty54DnakRToh3nRBcl+C6BNcjuB7BdQiuQ3A1gqsRXIPgGgS3RnBrBLdJcJsEt0VwWwRXIbgKwe0T3D7BMYJjCsfj+Q3XMjWvPJ7XyK/OJ5dfwtAvCL+Nqz4hqrKvUi7maI5xkYUifYX0STLMLVzho+zQptLhKA35UUViHxAqS+4bHC2TulEPpcwrFeenVFySX6s9UUliVRXbY1klmu2JnEZVO3H+Bvos3J12FGJ9YkVx7K8rVJGgigr1Uu42KchJuoZ7ULh3RrM5PDWiPKJdifbkwrh4hmUnTyy1sZvLZB9lcoXHqdXn8Wk2D3fhHjzA+E9gFTZw/MuYQx3ewXs41R5qBa2ivY7Q2ZlYcxvGLs34Byplj7I=</latexit>

46/124 Summer School 数理物理 2021 Lecture2

線形回帰モデルふたたびl 線形回帰モデル y =がガウス分布w において、重みベクトル w

w ⇠ N (0, ↵I)に従っているとする

47/124 Summer School 数理物理 2021 Lecture2

重みwの積分消去
l このとき、Φは定数行列なので、wを定数行列で変換

した y = w もガウス分布に従い、– 平均µ = E[y] = E[ w] = E[w] = 0– 共分散 ⌃ = E[yyT ] E[y]E[y]T= E[( w)( w)T ] ==↵Tl すなわち、yは全体として、y ⇠ N (0, ↵のガウス分布に従う.

48/124 Summer School 数理物理 2021 Lecture2

T)

E[wwT ]

T

重みwの積分消去 (2)01

00 1 0y1
0
B y2 CBB0C BB C
BB C B⇠N
B .. CBB .. C, ↵ B@ . A@@ . A @yN<latexit sha1_base64="ZpgalaZCId0eK3EaSKstpzvK+/k=">AAAJc3iclZVLb9NAEMenKY8mPNrCBamXtKGolaBaByQQpxaolKpV1Kb0pW4VbHeTWHFiy3bTpFG+AF+AAyeQOCA+BhfEnUM/AuJYEBcOjNdO48STIpyHd2d+//HM7npXs03D9Rg7HUmMXrp85epYMnXt+o2b4xOTt7Zd68jRxZZumZazq6muMI262PIMzxS7tiPUmmaKHa363PfvNITjGlb9pdeyxUFNLdeNkqGrHpqKkyNfuSbKRr1t11TPMZqdVKuopDlPt4pZ/8Ybh5bnBoZ8iov6YY/krlFLtbmumul8h5ui5M2lBqMxX8r6I7HBONP3Uxy/qmlX1FgEbleMIpvjjWZRmU9zPYjiG3PnRh6lshSVDahuDlE+f85HBfn5/iz59H9k1gscPkmhKCVKycy6eG54uSHeP4COUa5488WJDFtg8krHG0rYyEB4rVuTozngcAgW6HAENRBQBw/bJqjg4mcfFGBgo+0A2mhzsGVIv4AOpFB7hJRAQkVrFf/L2NsPrXXs+zFdqdbxKSb+HFSmYZZ9Yx/ZGfvCPrHv7M/QWG0Zw8+lhXct0Aq7OP76zubvf6pqePeg0lNdoNCQvrgmD0rwRNZiYG22tPhV6kH8xsmbs82nhdn2Pfae/cD63rFT9hkrrDd+6h82ROEtRp+V4y0wEocG3nXI4Bi3sdfNVUNfW1o7YT49vl9hyb6DHwNzrKBalT0LjodGaBLPahJci+BaBHdCcCcEd0xwxwRXIrgSwVUJrkpwSwS3RHArBLdCcKsEt0pwuwS3S3B7BLdHcIzgWIzTwvn11zI1r1o4r4E/Pp+afBO6fkH4a7jqewSX/ThlY45GHxdYKNKNkS5J+rn5KzzKdm1x2h+lLh9V9OwdQmXKfUNDy6Au6qGU67GK14dUvCnf1tpAJT1rXLHWl1VPszaQU1SVD/PX0Wfi7pSPEcsDK0rD/nKMKhBUIUa9kLtNBrKSLuEe5O+dwWx2T40gj2BXoj1ZPy6eYcrgiRVvbGcXlIcL2Y1HmcVn4Wk2BlMwA3MY/zEsQg7Hfwv0hJLYSbxKqGO/klPJmeTdAE2MhJrb0HclH/wF0j0ryA==</latexit><latexit sha1_base64="UQk+J0T7yvQSgstNA5m1qkerP+A=">AAAHxHichZXNbtNAEMenLeBSPprCBQkOFVEQp2gTkECcWqCoVasqTUibqqkir7tprPhLtpM0scKBIy/AgRNISCBO8ApceAEOfQTEsUhcODBeO8qHp62txLszv/94Zne9yx1D93zGjqemZy5cvKTMXp67cvXa9fnUwo1tz265mihrtmG7Fa56wtAtUfZ13xAVxxWqyQ2xw5vPQv9OW7ieblsv/a4j9k310NLruqb6aKql7lRb1oFwuatqIqg2PCd85oXZ79eCfi2VZlkmr8VkIxc30hBfBXthZhWqcAA2aNACEwRY4GPbABU8vPcgBwwctO1DgDYXW7r0C+jDHGpbSAkkVLQ28f8Qe3ux1cJ+GNOTag3fYuDPReUiZNhP9pmdsB/sC/vF/p0aK5Axwly6+OSRVji1+Te3Sn/PVZn49KExVJ2h4EifXZMPdXgsa9GxNkdawiq1KH679/ak9KSYCe6xD+w31veeHbPvWKHV/qN93BLFdxg9I8dbYKQqtPGpQRrHOMDeIFeOvkBa+3E+Q35cYcu+i7eOOTZQrcqeDZ1TIxwR7zoiuC7BdQmuR3A9gusQXIfg6gRXJ7gmwTUJbpnglglujeDWCG6d4NYJrkJwFYLbJbhdgmMExxIcj+c3XMvUvPJ4XiN/cj65/BIGfkH4TVz1Q6Iq+0nKwRz1MS6yUKSXID2SDHMLV/goO7Al6XCUBvyoYmjvEypD7hscLZO6UQ+lLCQqLpxScUl+reZEJUNrUrExltVQszGR06hqM85fQ5+Bu9NmgliZWFEc+ysJqkhQxQT1XO42achLuo57ULh3RrM5ODWiPKJdifbkw7h4huUmT6xkYzufzT3I5rceppeexqfZLNyGu3Af4z+CJVjF8S9jDq/hE3yFb8oLxVA8pRWh01Ox5iaMXcqr/8G5myQ=</latexit>

|{z}
y

0

<latexit sha1_base64="UMn9wfJ8bWg9f7KM6Nl/F8sSAyk=">AAAHxnichZXNbtNAEMenLZBSPprCBQkJVURBnKJNABVxaoFKqVpVaULaVE0Ved1NY8Vfsp2kiRWJAydegAMnkDgAR3gDLrwAhz4C4lgkLhwYrx3lw5NiK/HuzO8/ntld73Jb11yPsdOZ2bkLFy8l5i8vXLl67fpicunGrmu1HFWUVUu3nApXXKFrpih7mqeLiu0IxeC62OPNZ4F/ry0cV7PMF17XFoeGcmxqdU1VPDTVkneqLfNIONxRVOFXG64dPFcyj4TR79f8fi2ZYhkmr+V4Ixs1UhBdBWtpLg9VOAILVGiBAQJM8LCtgwIu3geQBQY22g7BR5uDLU36BfRhAbUtpAQSClqb+H+MvYPIamI/iOlKtYpv0fHnoHIZ0uwH+8jO2Hf2mf1kf6fG8mWMIJcuPnmoFXZt8fWt0p//qgx8etAYqs5RcKTPr8mDOjyWtWhYmy0tQZVqGL/de3NWelJM+/fYe/YL63vHTtk3rNBs/1Y/7IjiW4yeluMtMFIV2vhUIYVj7GNvkCtHny+t/SifIT+usGTfwVvDHBuoVmTPgs7UCCfEu04IrktwXYLrEVyP4DoE1yG4OsHVCa5JcE2CWyO4NYLbILgNgtskuE2CqxBcheD2CW6f4BjBsRjHo/kN1jI1rzya19Afn08uv4SBXxB+A1f9kKjKfpyyMUdtjAstFOnGSJckg9yCFT7KDmxxOhilAT+qGNr7hEqX+wZHy6Ru1EMpC7GKC1MqLsmv1ZioZGiNK7bGshpqtiZyGlVtR/mr6NNxd9qOEesTK4pjfz1GFQmqGKOey90mBTlJ13EPCvbOcDYHp0aYR7gr0Z5cEBfPsOzkiRVv7OYy2QeZ3M7D1OrT6DSbh9twF+5j/BVYhTyOfxlzeAWf4At8TeQTZqKV6ITo7EykuQljV+LlPx8Nm6A=</latexit><latexit sha1_base64="cBy7SP6mgyWfvD8VPfcTFpsZ06Q=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5qq8ppNasWOLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPspNhKvDvz+49ndte73LVMP2DsbGZ27tLlK9r81YVr12/cXFxavrXrOy3PEGXDsRyvwnVfWGZTlAMzsETF9YRuc0vs8caL0L/XFp5vOs1XQdcVh7Zeb5o109ADNJWq7e7RUoplmLxW1EY2bqQgvvLO8twGVOENOGBAC2wQ0IQA2xbo4ON9AFlg4KLtEPpo87BlSr+AASygtoWUQEJHawP/69g7iK1N7Icxfak28C0W/jxUrkCa/WBf2Dn7zr6yn+zv1Fh9GSPMpYtPHmmFe7T49k7pz39VNj4DOE5UFyg40hfXFEANnspaTKzNlZawSiOK3+6dnpeeFdP9++wT+4X1fWRn7BtW2Gz/Nj4XRPEDRk/L8RYYqQptfBqQwjHuY2+YK0dfX1oHcT4JP65wZN/D28Qcj1Gty54DnakRToh3nRBcl+C6BNcjuB7BdQiuQ3A1gqsRXIPgGgS3RnBrBLdJcJsEt0VwWwRXIbgKwe0T3D7BMYJjCsfj+Q3XMjWvPJ7XyK/OJ5dfwtAvCL+Nqz4hqrKvUi7maI5xkYUifYX0STLMLVzho+zQptLhKA35UUViHxAqS+4bHC2TulEPpcwrFeenVFySX6s9UUliVRXbY1klmu2JnEZVO3H+Bvos3J12FGJ9YkVx7K8rVJGgigr1Uu42KchJuoZ7ULh3RrM5PDWiPKJdifbkwrh4hmUnTyy1sZvLZB9lcoXHqdXn8Wk2D3fhHjzA+E9gFTZw/MuYQx3ewXs41R5qBa2ivY7Q2ZlYcxvGLs34Byplj7I=</latexit>

l 線形回帰モデル|

0 (x1 ) · · ·0 (x2 ) · · ·..
.
0 (xN ) · · ·{z

H (x1 )

<latexit sha1_base64="NBwupxqlhEt7DXDOxySJYdiYuWM=">AAAHtHichZXNbtNAEMenLeBSPtrCBYlLRBTEKdoEBAgJqQUqtWpVpQlpU9Vt5TWbxoq/ZDtJkygvwAuAxKmVOCAegwsvwKGPgDgWiQsHxmtHTuJpsZV4d+b3H8/srne5axp+wNjZ1PTMlavXlNnrczdu3ro9v7B4Z9t3Wp4uqrpjOl6Na74wDVtUAyMwRc31hGZxU+zw5uvQv9MWnm849tug64p9SzuyjbqhawGaDtR2N/Myo/JSw1DbncOFLMszeWXSjULcyEJ8lZzFmVVQ4R04oEMLLBBgQ4BtEzTw8d6DAjBw0bYPfbR52DKkX8AA5lDbQkogoaG1if9H2NuLrTb2w5i+VOv4FhN/HiozkGM/2Bd2zr6zr+wn+3thrL6MEebSxSePtMI9nH9/r/LnvyoLnwE0EtUlCo705TUFUIfnshYDa3OlJaxSj+K3ex/OKy/Kuf5Ddsp+YX0n7Ix9wwrt9m/985Yof8LoOTneAiOp0ManDlkc4z72hrly9PWldRDnk/DjCkf2PbwNzLGBak32HOhcGOGYeNcxwXUJrktwPYLrEVyH4DoEVye4OsE1Ca5JcMsEt0xwawS3RnDrBLdOcDWCqxHcLsHtEhwjOJbieDy/4Vqm5pXH8xr50/PJ5Zcw9AvCb+GqTwhV9tOUizkaY1xkoUg/RfokGeYWrvBRdmhL0+EoDflRRWIfECpT7hscLZO6UQ+lLKUqLl1QcUV+rdZEJYk1rdgYyyrRbEzkNKrajPPX0Wfi7rSZIlYmVhTH/kqKKhNUOUW9kbtNFoqSruMeFO6d0WwOT40oj2hXoj3FMC6eYYXJEyvd2C7mC4/zxa0n2aVX8Wk2C/fhATzC+M9gCVZx/KuYgwcf4QROlaeKquiKiNDpqVhzF8Yuxf4HldqTxQ==</latexit>

CB
CB
A@

H (x2 ) CB
H (xN )

<latexit sha1_base64="dyqS5Q5Y6iIb++TwsWA8ydYDFV4=">AAAHr3ichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJqRJ1JTiNZvGir9kO0kTKy/AlQMH4AASB8RjcOEFOPQREMciceHAeO3ISTwtthLvzvz+45nd9S53DN3zGTuZmZ27cPGSMn954crVa9cXl5Zv7Hl2x9VERbMN261x1ROGbomKr/uGqDmuUE1uiCpvPw391a5wPd22nvt9RxyY6pGlN3VN9dFUb3AzaBRb+vBwKcNyTF4r6UY+bmQgvor28twGNOAV2KBBB0wQYIGPbQNU8PDehzwwcNB2AAHaXGzp0i9gCAuo7SAlkFDR2sb/I+ztx1YL+2FMT6o1fIuBPxeVK5BlP9gXdsq+s6/sJ/t7ZqxAxghz6eOTR1rhHC6+vlX+81+ViU8fWonqHAVH+vyafGjCI1mLjrU50hJWqUXxu4O3p+XHpWxwl31iv7C+j+yEfcMKre5v7fOuKL3D6Fk53gIjNaCLTw0yOMYB9ka5cvQF0jqM80n4SYUt+y7eOubYQrUqezb0zoxwTLzrmOD6BNcnuAHBDQiuR3A9gmsSXJPg2gTXJrg1glsjuE2C2yS4LYLbIrgawdUIrk5wdYJjBMdSHI/nN1zL1LzyeF4jf3o+ufwSRn5B+E1c9QnRkP005WCO+gQXWSjSS5EeSYa5hSt8nB3Z0nQ4SiN+XJHYh4TKkPsGR8u0btxDKYupiotnVFyWX6s5VUliTSu2J7JKNNtTOY2rduL8NfQZuDvtpIj1qRXFsb+eokoEVUpRz+Ruk4GCpJu4B4V7ZzSbo1MjyiPalWhPIYyLZ1h++sRKN/YKufz9XGH3QWb1SXyazcNtuAP3MP5DWIUNHP8K5mDCG3gPH5S8UlVeKC8jdHYm1tyEiUvR/wEnqJJD</latexit>

y=

10

w

}

<latexit sha1_base64="UMn9wfJ8bWg9f7KM6Nl/F8sSAyk=">AAAHxnichZXNbtNAEMenLZBSPprCBQkJVURBnKJNABVxaoFKqVpVaULaVE0Ved1NY8Vfsp2kiRWJAydegAMnkDgAR3gDLrwAhz4C4lgkLhwYrx3lw5NiK/HuzO8/ntld73Jb11yPsdOZ2bkLFy8l5i8vXLl67fpicunGrmu1HFWUVUu3nApXXKFrpih7mqeLiu0IxeC62OPNZ4F/ry0cV7PMF17XFoeGcmxqdU1VPDTVkneqLfNIONxRVOFXG64dPFcyj4TR79f8fi2ZYhkmr+V4Ixs1UhBdBWtpLg9VOAILVGiBAQJM8LCtgwIu3geQBQY22g7BR5uDLU36BfRhAbUtpAQSClqb+H+MvYPIamI/iOlKtYpv0fHnoHIZ0uwH+8jO2Hf2mf1kf6fG8mWMIJcuPnmoFXZt8fWt0p//qgx8etAYqs5RcKTPr8mDOjyWtWhYmy0tQZVqGL/de3NWelJM+/fYe/YL63vHTtk3rNBs/1Y/7IjiW4yeluMtMFIV2vhUIYVj7GNvkCtHny+t/SifIT+usGTfwVvDHBuoVmTPgs7UCCfEu04IrktwXYLrEVyP4DoE1yG4OsHVCa5JcE2CWyO4NYLbILgNgtskuE2CqxBcheD2CW6f4BjBsRjHo/kN1jI1rzya19Afn08uv4SBXxB+A1f9kKjKfpyyMUdtjAstFOnGSJckg9yCFT7KDmxxOhilAT+qGNr7hEqX+wZHy6Ru1EMpC7GKC1MqLsmv1ZioZGiNK7bGshpqtiZyGlVtR/mr6NNxd9qOEesTK4pjfz1GFQmqGKOey90mBTlJ13EPCvbOcDYHp0aYR7gr0Z5cEBfPsOzkiRVv7OYy2QeZ3M7D1OrT6DSbh9twF+5j/BVYhTyOfxlzeAWf4At8TeQTZqKV6ITo7EykuQljV+LlPx8Nm6A=</latexit>|

0 (x1 ) · · ·1 (x1 ) · · ·..
.
H (x1 ) · · ·が、ガウス分布に従う重みwを積分消去してTy ⇠ N 0, ↵になった
<latexit sha1_base64="Jrk7lSRfSgV2TOMz6D0XCCJ+vys=">AAAH7XichZXPa9RAFMdfrbq1/mirF8HL6lKpIGV2FRRPrVpoaSnb7a8tnVoycdINm18k6bbbsP+Ad/GgCAoi0j/Di1cPHqp/gXis4MWDL5Ms+yOvNWGTmfc+37fvzUxmhGeZQcjY0cCZwbPnzueGLgxfvHT5ysjo2NW1wN31dbmqu5brV4UWSMt05Gpohpaser7UbGHJdVF/EvvXG9IPTNdZCZue3LK1Hcc0TF0L0bQ9OsMbzTwPTDsfcV2z8ostfpNb0ggnuK2FNWFErHWXa5ZX03hd+k7RC7ko10z1eLbCfXOnFt7ZHi2wSaaufLZRTBsFSK+yOzY4Cxyegws67IINEhwIsW2BBgHem1AEBh7atiBCm48tU/kltGAYtbtISSQ0tNbxuYO9zdTqYD+OGSi1jv9i4c9HZR7G2Tf2iR2zL+yQ/WR/T4wVqRhxLk18i0Qrve2RF9eX//xXZeM7hFpHdYpCIH16TSEY8FDVYmJtnrLEVepJ/MbBq+PlR5Xx6DZ7z35hfe/YEfuMFTqN3/qHJVl5jdHH1XhLjMShgW8dCjjGEfbauQr0RcraSvPp8L0KV/V9vE3MsYZqTfVc2Dsxwj7xX/sE1yS4JsEdENwBwe0R3B7BGQRnEFyd4OoEN01w0wQ3R3BzBDdPcPMEVyW4KsFtENwGwTGCYxlOpPMbr2VqXkU6r4k/O59CfQltvyT8Nq76DsFVP0t5mKPZwyUWigwyZECScW7xCu9m27YsHY9Sm+9WdOwtQmWpfUOgpV/X7aGU5UzF5RMqXlZfq91XSceaVSz0ZNXRLPTl1K1aTPPX0Wfh7rSYIWb6VpTA/kyGqhBUJUM9VbtNAUqKNnAPivfOZDbbp0aSR7Ir0Z5SHBfPsGL/iZVtrJUmi/cmS0v3C1OP09NsCG7ALZjA+A9gCmZx/Fcxh0P4Ct/hR87Nvcy9yb1N0DMDqeYa9Fy5j/8Aw2Oqsg==</latexit>

49/124 Summer School 数理物理 2021 Lecture2{z<latexit sha1_base64="6F4/44a+IiMZ2oG30QJBtr8YG+o=">AAAHsXichZXNbtNAEMenLeBSPtrCBYlLRRTEKdqkSCBOLVCpVasqTZo2UVKK12waK/6S7SRNorwAd8QBCQQSB8RjcOEFOPQREMciceHAeO3ISTwpthLvzvz+45nd9S53DN3zGTubmZ27dPmKMn914dr1GzcXl5ZvHXh2y9VESbMN2y1z1ROGbomSr/uGKDuuUE1uiEPefBb4D9vC9XTb2ve7jjgy1RNLr+ua6qOpWuNmv5Zv6IMX+8dLKZZh8lpJNrJRIwXRlbeX5zahBq/ABg1aYIIAC3xsG6CCh3cVssDAQdsR9NHmYkuXfgEDWEBtCymBhIrWJv6fYK8aWS3sBzE9qdbwLQb+XFSuQJr9YF/YOfvOvrKf7O/UWH0ZI8ili08eaoVzvPj6TvHPf1UmPn1oxKoLFBzpi2vyoQ6PZS061uZIS1ClFsZv996eF58U0v377BP7hfV9ZGfsG1ZotX9rn/dE4R1GT8vxFhipBm18apDCMe5jb5grR19fWgdRPjE/rrBl38VbxxwbqFZlz4bO1AinxLtOCa5LcF2C6xFcj+A6BNchuDrB1QmuSXBNglsnuHWC2yK4LYLbJrhtgisTXJngKgRXIThGcCzB8Wh+g7VMzSuP5jX0J+eTyy9h6BeE38RVHxM12U9SDuaoj3GhhSK9BOmRZJBbsMJH2aEtSQejNORHFbF9QKgMuW9wtEzqRj2UMp+oOD+l4qL8Ws2JSmJrUrEzllWs2ZnIaVS1G+Wvoc/A3Wk3QWxMrCiO/Y0EVSCoQoJ6LnebFOQkXcc9KNg7w9kcnhphHuGuRHtyQVw8w7KTJ1aycZDLZFczub2HqbWn0Wk2D3fhHjzA+I9gDTZx/EvyzHkD7+GDsqpUlJcKD9HZmUhzG8YupfkPkY+TCQ==</latexit>

0 (xN )

11
CC
CC
AA

1 (xN ) CC
H (xN )

T}

ガウス過程p(y|X) = N (0, ↵T)

は、どんな入力 X = {xn }N
n=1 についても成り立つ
à ガウス過程l どんな入力 (x1 , x2 , · · · , xN ) についても、対応する
出力 y = (y1 , y2 , · · · , yN ) がガウス分布に従うとき、y はガウス過程に従う、という– ガウス過程＝無限次元のガウス分布– 線形回帰モデルで、重みwを積分消去したもの
Tの要素を与えるカーネル関数
Kij = k(xi , xj ) = ↵ (xi )T (xj )だけでガウス分布が定まる (カーネル法, SVMと同じ)

l K=↵

50/124 Summer School 数理物理 2021 Lecture2

ガウス過程 (2)l xnとxn’が近ければ、共分散行列Kの要素Knn’も大きい↓
yn, yn’が近い値をとる

l ガウス過程は、xが似ていればyも似ている ことを数学的に表すための確率過程.
51/124 Summer School 数理物理 2021 Lecture2

カーネルと特徴ベクトルl ガウス過程では、Kij = ↵(xi )T (xj ) だけが必要↓(xi ), (xj ) を直接求める必要はない
02– 例： k(x, x0 ) = (x1 xp01 + x2 xp
+
1)
2
p

のとき
(x) = (x21 , x22 , 2x1 x2 , 2x1 , 2x2 , 1) となるが、この (x) を計算する必要はない
l (x) を求めると、無限次元になることもある(＝無限次元の線形回帰モデルに相当)
l これをカーネルトリックという

52/124 Summer School 数理物理 2021 Lecture2

さまざまなカーネル
k(x, x0 ) = xT x0– (x) = x を意味する→ガウス過程は、重回帰を包む✓◆
l ガウスカーネル:0 2|x x |
0k(x, x ) = exp✓✓◆
l 指数カーネル:
0|x x |
0k(x, x ) = exp✓l 周期カーネル:✓✓◆◆
0|x x |
0k(x, x ) = exp cos ✓1✓2
l 線形カーネル:<latexit sha1_base64="hKUxHzt1f3DVClMWBfqdkAHBWWc=">AAAH5XichZVLb9NAEMenLZBSHm3hgsSlJQqkUlttAhIIqVILVGrVqmoT2qaqS2SbdWLFiS3bedXNF0DihuDAAQHigPgYXOAK4tCPgDgWiQsHxmtHeXhSbCXenfn9xzO7613FMnTHZex4aHjkzNlzsdHzYxcuXro8PjF5Zccxq7bKt1XTMO2cIjvc0Ct829Vdg+csm8tlxeC7Sumh79+tcdvRzcpjt2nxg7JcqOiarsoumvITC6WkVGvM4u/WzILEG5ZkcM1NzkmaLaveEdql6Tlp2vcfPUm3PMktclduSbZeKLoz+Yk4m2fimoo2UmEjDuG1aU6OrIAET8EEFapQBg4VcLFtgAwO3vuQAgYW2g7AQ5uNLV34ObRgDLVVpDgSMlpL+F/A3n5orWDfj+kItYpvMfBno3IKEuwH+8hO2Bf2if1kfwfG8kQMP5cmPpVAy638+LNr2T//VZXx6UKxozpFoSB9ek0uaHBP1KJjbZaw+FWqQfza4auT7P1MwrvJ3rFfWN9bdsw+Y4WV2m/1wxbPvMboCTHeHCNJUMOnCnEcYw977VwV9HnC2grz6fC9ClP0bbx1zLGIaln0TKgPjNAg3tUguCbBNQnukOAOCa5OcHWC0whOI7gSwZUIbonglghuleBWCW6N4NYILkdwOYLbI7g9gmMExyKcEs6vv5apeVXCeQ380flUxJfQ9nPCX8ZV3yEk0Y9SFuao93CBhSKdCOmQpJ+bv8K72bYtSvuj1Oa7FR17i1AZYt9Q0NKv6/ZQys1IxZsDKs6Kr7XcV0nHGlWs92TV0az35dSt2gjzV9Fn4O60ESGW+1aUgv3lCJUhqEyEeiR2mzikBa3hHuTvncFstk+NII9gV6I9aT8unmGp/hMr2thJz6duz6e37sQXH4Sn2ShchxuQxPh3YRFWcPy3MYf38BW+wfdYIfY89iL2MkCHh0LNVei5Ym/+ASwOpyg=</latexit>

53/124 Summer School 数理物理 2021 Lecture2

さまざまなカーネル (2)54/124 Summer School 数理物理 2021 Lecture2

直感的理解l 相関のある多変量ガウス分布

55/124 Summer School 数理物理 2021 Lecture2

直感的理解l 相関のある多変量ガウス分布

56/124 Summer School 数理物理 2021 Lecture2

直感的理解l 相関のある多変量ガウス分布

57/124 Summer School 数理物理 2021 Lecture2

「基底関数」の消去
l 点hでの基底関数
h (x) = ⌧ exp

✓

(x

h/H)
r2

2

◆

h
<latexit sha1_base64="PcqcH8xoANut5SR4xdK1Tw0sB6o=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1YlHyRN1VSV12waK3Zs2U7SJMoLwBXEgRNIHBCPwYUX4NBHQByLxIUD47UjJ9lpsZV4d+b3H8/srne5a5l+wNjJzOzchYuXtPnLC1euXru+uLR8o+I7bc8QZcOxHK/KdV9YZkuUAzOwRNX1hG5zS+zy5rPQv9sRnm86rZdBzxUHtn7UMuumoQdoKjQOl1Isw+S1ojaycSMF8ZV3luc2oAavwAED2mCDgBYE2LZABx/vfcgCAxdtBzBAm4ctU/oFDGEBtW2kBBI6Wpv4f4S9/djawn4Y05dqA99i4c9D5Qqk2Q/2hZ2y7+wr+8n+nhlrIGOEufTwySOtcA8XX98q/fmvysZnAI1EdY6CI31+TQHU4bGsxcTaXGkJqzSi+J3++9PSk2J6cJd9Yr+wvo/shH3DClud38bngih+wOhpOd4CI9Wgg08DUjjGA+yNcuXoG0jrMM4n4ScVjux7eJuYYwPVuuw50D0zwjHxrmOC6xFcj+D6BNcnuC7BdQmuTnB1gmsSXJPg1ghujeA2CW6T4LYIbovgqgRXJbg9gtsjOEZwTOF4PL/hWqbmlcfzGvnV+eTySxj5BeG3cdUnRE32VcrFHM0JLrJQpK+QPkmGuYUrfJwd2VQ6HKURP65I7ENCZcl9g6NlWjfuoZR5peL8GRWX5NdqT1WSWFXF9kRWiWZ7Kqdx1U6cv4E+C3enHYVYn1pRHPvrClUkqKJCPZe7TQpykq7jHhTundFsjk6NKI9oV6I9uTAunmHZ6RNLbVRymeyDTK7wMLX6ND7N5uE23IF7GP8RrMIGjn8ZcxDwBt7CO+2+9kKraNUInZ2JNTdh4tL0f2Mdjrs=</latexit>

を考えてみるl H→∞にしてグリッドを無限に細かくすると、
PH 200k(x, x ) = lim
(x)
(x
)
2
h
h
h=
H
H!1
✓
◆
✓
◆
Z 1
2
0
2
(x h)
(x
h)
2
!
⌧ exp
exp
dh
2
2
r
r
1
✓
◆
p
1
2
0 2
2
=⌧
⇡r /2 exp
(x
x
)
2
2r
✓
◆
1
⌘ ✓1 exp(x x0 )2
ガウスカーネル！
✓2
58/124 Summer School 数理物理 2021 Lecture2

ガウス過程回帰モデル
l 新しい入力点x*での出力y*の分布はどうなるか？

59/124 Summer School 数理物理 2021 Lecture2

ガウス過程回帰モデル (2)l 学習データのyにy*を加えた y’=(y, y*) が、

学習データのXにx*を加えたX’から計算される行列を共分散行列としたガウス分布に従うのでここでk⇤ = (k(x⇤ , x1 ), k(x⇤ , x2 ), · · · , k(x⇤ , xN ))k⇤⇤ = k(x⇤ , x⇤ )60/124 Summer School 数理物理 2021 Lecture2

ガウス過程回帰モデル (3)l 数式で簡潔に書くと

✓ ◆
✓ ✓
K
y
⇠ N 0,
⇤
y
kT⇤

k⇤
k⇤⇤

◆◆

l なので、多変量ガウス分布の条件つき分布の公式から

p(y ⇤ |x⇤, X, y) = N kT⇤ K 1 y, k⇤⇤ kT⇤ K 1 k⇤
– よって、その期待値はE[y ⇤ |x⇤, X, y] = kT⇤ K 1 y
61/124 Summer School 数理物理 2021 Lecture2

条件付きガウス分布の公式
l 多変量ガウス分布の条件付き分布は、のとき

l 証明は教科書を参照してください

62/124 Summer School 数理物理 2021 Lecture2

条件付きガウス分布のイメージ<latexit sha1_base64="bYXOmILt/IhMc7N8ACyPLA8iLUA=">AAAHbHichZXPaxNBFMdfW21q/dFWPQhFKIaGChImUVA8tWrB0lLSxLSpbQk720lcstlddjdpfpB/wLPgQRQUPIh/hhf/AQ/9E0TwUsGLB9/Obtgk89puSGbmvc/35b2Z2RnumIbnM3Y8Nj5x4eJkYurS9OUrV6/NzM5d3/bshquLom6btlvimidMwxJF3/BNUXJcodW5KXZ47Wng32kK1zNs64XfdsRBXataRsXQNR9NL52lVjlzr1XO3i3PJlmayWdB7WSiThKiJ2fPTaRgHw7BBh0aUAcBFvjYN0EDDz97kAEGDtoOoIs2F3uG9AvowTRqG0gJJDS01vC3iqO9yGrhOIjpSbWO/2Li10XlAiyyH+wLO2Hf2Vf2k/07NVZXxghyaWPLQ61wyjOvbxX+nquqY+vDq1h1hoIjfXZNPlTgkazFwNocaQmq1MP4zc7bk8Lj/GI3xT6xX1jfR3bMvmGFVvOP/nlL5N/J6IeoqWDbxFaHJM5wF0f9TDn6utLai7KJ+RZBtgiuTXBtgusQXIfgjgjuiOAqBFchuBrB1QhuheBWCG6N4NYIbp3g1gmuRHAlgtsluF2CYwTHFI5H6xvsQ2pdebSuoV9dTy53cd8vCH8dd2xM7MuxSjmYozHEhRaK9BTSI8kgNx/rH2T7NpUOZqnPDypie49QmfKd52gZ1Q16KGVOqTh3SsUFtFblSg7SsVVVbAxlFWs2RnIaVG1G+evoM/Fk2VSI1ZEdxXG8qlB5gsor1DN52iQhK+kKns/BuReuZv/ED/MITyXakw3i4v2TGb1t1M52Np25n85uPUguP4luoimYhzuwhPEfwjI8x/kvYg4WvIH38GHyd+JmYj5xO0THxyLNDRh6Eqn/DqmEcQ==</latexit>p(x1 , x2 )<latexit sha1_base64="YhiHKi13lWia+/DZesZFus7jg/U=">AAAHbHichZXPaxNBFMdfW21q/dFWPQhFKIaGegmTKCieWrVgaSlpYtrUtoSd7Wxcstlddjdpfph/wLPgQRQUPIh/hhf/AQ/9E0TwUsGLB9/ObsiPeW03JDPz3uf78t7M7Ax3LdMPGDseG5+4cHEyMXVp+vKVq9dmZueub/tO3dNFUXcsxytxzReWaYtiYAaWKLme0GrcEju8+iT07zSE55uO/TxoueKgplVs0zB1LUDTC3epWc6+apYzd8uzSZZm8llQO5m4k4T4yTlzEynYh0NwQIc61ECADQH2LdDAx88eZICBi7YD6KDNw54p/QK6MI3aOlICCQ2tVfyt4Ggvtto4DmP6Uq3jv1j49VC5AIvsB/vCTth39pX9ZP9OjdWRMcJcWtjySCvc8szrW4W/56pq2Abwsq86Q8GRPrumAAx4KGsxsTZXWsIq9Sh+o/32pPAov9hJsU/sF9b3kR2zb1ih3fijf94S+Xcy+iFqDGwb2OqQxBnu4KiXKUdfR1q7cTZ9vkmQTYJrEVyL4NoE1ya4I4I7IjiD4AyCqxJcleBWCG6F4NYIbo3g1gluneBKBFciuF2C2yU4RnBM4Xi8vuE+pNaVx+sa+dX15HIX9/yC8Ndwx/aJfTlWKRdzNIe4yEKRvkL6JBnmFmD9g2zPptLhLPX4QUXf3iVUlnznOVpGdYMeSplTKs6dUnEBrRW5koN036oqNoay6ms2RnIaVG3G+evos/Bk2VSI1ZEdxXG8qlB5gsor1FN52iQhK2kDz+fw3ItWs3fiR3lEpxLtyYZx8f7JjN42amc7m87cS2e37ieXH8c30RTMwx1YwvgPYBme4fwXMQcb3sB7+DD5O3EzMZ+4HaHjY7HmBgw9idR/P7uEwQ==</latexit>p(x2 |x1 )l 多変量ガウス分布 p(x1 , x2 ) を x1 で条件づけると、<latexit sha1_base64="HEw7b8vMzH1qejvjxAKvcEA62ZE=">AAAHZXichZXPTxNBFMcfoBQRBNQYEw8SGoinZlpNNJ5AJZFASGktlABpdpbZuul2d7O7Lf2R/gMmXuXgSRMPxj/Di/+AB/4CYzxi4sWDb2e32bbzwG3amXnv8319b2Z2hruW6QeMnY2NT1y5OpmaujZ9fWb2xtz8ws1d32l4uijpjuV4Za75wjJtUQrMwBJl1xNanVtij9eeh/69pvB807FfBW1XHNW1qm0apq4FaCq2KtnKfJplmHwW1U427qQhfvLOwsQKHMIxOKBDA+ogwIYA+xZo4OPnALLAwEXbEXTR5mHPlH4BPZhGbQMpgYSG1hr+VnF0EFttHIcxfanW8V8s/HqoXIRl9p19ZufsG/vCfrK/F8bqyhhhLm1seaQVbmXuzd3in/+q6tgG8DpRXaLgSF9eUwAGPJG1mFibKy1hlXoUv9k5PS8+LSx3V9hH9gvr+8DO2Fes0G7+1j/tiMJ7Gf0YNQa2TWx1SOMMd3HUz5SjryutvTibhG8RZIvg2gTXJrgOwXUI7oTgTgjOIDiD4GoEVyO4NYJbI7gNgtsguE2C2yS4MsGVCW6f4PYJjhEcUzger2+4D6l15fG6Rn51PbncxX2/IPx13LEJcSjHKuVijuYQF1ko0ldInyTD3AKsf5Dt21Q6nKU+P6hI7D1CZcl3nqNlVDfooZR5peL8BRUX0VqVKzlIJ1ZVsTWUVaLZGslpULUd56+jz8KTZVsh1kd2FMfxukIVCKqgUC/kaZOGnKQNPJ/Dcy9azf6JH+URnUq0JxfGxfsnO3rbqJ3dXCb7MJPbeZRefRbfRFNwD5bgAcZ/DKvwEue/hDlU4S28g9PJH6nZ1O3UnQgdH4s1t2DoSd3/Bx9/gjU=</latexit><latexit sha1_base64="bYXOmILt/IhMc7N8ACyPLA8iLUA=">AAAHbHichZXPaxNBFMdfW21q/dFWPQhFKIaGChImUVA8tWrB0lLSxLSpbQk720lcstlddjdpfpB/wLPgQRQUPIh/hhf/AQ/9E0TwUsGLB9/Obtgk89puSGbmvc/35b2Z2RnumIbnM3Y8Nj5x4eJkYurS9OUrV6/NzM5d3/bshquLom6btlvimidMwxJF3/BNUXJcodW5KXZ47Wng32kK1zNs64XfdsRBXataRsXQNR9NL52lVjlzr1XO3i3PJlmayWdB7WSiThKiJ2fPTaRgHw7BBh0aUAcBFvjYN0EDDz97kAEGDtoOoIs2F3uG9AvowTRqG0gJJDS01vC3iqO9yGrhOIjpSbWO/2Li10XlAiyyH+wLO2Hf2Vf2k/07NVZXxghyaWPLQ61wyjOvbxX+nquqY+vDq1h1hoIjfXZNPlTgkazFwNocaQmq1MP4zc7bk8Lj/GI3xT6xX1jfR3bMvmGFVvOP/nlL5N/J6IeoqWDbxFaHJM5wF0f9TDn6utLai7KJ+RZBtgiuTXBtgusQXIfgjgjuiOAqBFchuBrB1QhuheBWCG6N4NYIbp3g1gmuRHAlgtsluF2CYwTHFI5H6xvsQ2pdebSuoV9dTy53cd8vCH8dd2xM7MuxSjmYozHEhRaK9BTSI8kgNx/rH2T7NpUOZqnPDypie49QmfKd52gZ1Q16KGVOqTh3SsUFtFblSg7SsVVVbAxlFWs2RnIaVG1G+evoM/Fk2VSI1ZEdxXG8qlB5gsor1DN52iQhK+kKns/BuReuZv/ED/MITyXakw3i4v2TGb1t1M52Np25n85uPUguP4luoimYhzuwhPEfwjI8x/kvYg4WvIH38GHyd+JmYj5xO0THxyLNDRh6Eqn/DqmEcQ==</latexit>「切り口」p(x2 |x1 ) はまたガウス分布になる<latexit sha1_base64="YhiHKi13lWia+/DZesZFus7jg/U=">AAAHbHichZXPaxNBFMdfW21q/dFWPQhFKIaGegmTKCieWrVgaSlpYtrUtoSd7Wxcstlddjdpfph/wLPgQRQUPIh/hhf/AQ/9E0TwUsGLB9/ObsiPeW03JDPz3uf78t7M7Ax3LdMPGDseG5+4cHEyMXVp+vKVq9dmZueub/tO3dNFUXcsxytxzReWaYtiYAaWKLme0GrcEju8+iT07zSE55uO/TxoueKgplVs0zB1LUDTC3epWc6+apYzd8uzSZZm8llQO5m4k4T4yTlzEynYh0NwQIc61ECADQH2LdDAx88eZICBi7YD6KDNw54p/QK6MI3aOlICCQ2tVfyt4Ggvtto4DmP6Uq3jv1j49VC5AIvsB/vCTth39pX9ZP9OjdWRMcJcWtjySCvc8szrW4W/56pq2Abwsq86Q8GRPrumAAx4KGsxsTZXWsIq9Sh+o/32pPAov9hJsU/sF9b3kR2zb1ih3fijf94S+Xcy+iFqDGwb2OqQxBnu4KiXKUdfR1q7cTZ9vkmQTYJrEVyL4NoE1ya4I4I7IjiD4AyCqxJcleBWCG6F4NYIbo3g1gluneBKBFciuF2C2yU4RnBM4Xi8vuE+pNaVx+sa+dX15HIX9/yC8Ndwx/aJfTlWKRdzNIe4yEKRvkL6JBnmFmD9g2zPptLhLPX4QUXf3iVUlnznOVpGdYMeSplTKs6dUnEBrRW5koN036oqNoay6ms2RnIaVG3G+evos/Bk2VSI1ZEdxXG8qlB5gsor1FN52iQhK2kDz+fw3ItWs3fiR3lEpxLtyYZx8f7JjN42amc7m87cS2e37ieXH8c30RTMwx1YwvgPYBme4fwXMQcb3sB7+DD5O3EzMZ+4HaHjY7HmBgw9idR/P7uEwQ==</latexit>

63/124 Summer School 数理物理 2021 Lecture2

ガウス過程回帰モデル (4)l 注：ガウス過程回帰の期待値

E[y ⇤ |x⇤, X, y] = kT⇤ K 1 y

はカーネルリッジ回帰と実は同じだが、カーネル法と異なりベイズ推定なので、– 分散も使って分布を推定することができ、
p(y ⇤ |x⇤, X, y) = N kT⇤ K 1 y, k⇤⇤ kT⇤ K 1 k⇤

– カーネル法と違って完全な確率モデルなので、カーネル自体を学習することも可能(通常のカーネル法ではクロスバリデーションに頼る必要があり、最適化は難しい)64/124 Summer School 数理物理 2021 Lecture2

ガウス過程回帰のアルゴリズム

l (xtrain,ytrain)が与えられたとき、xtestの各点について

平均muと分散varを出力
65/124 Summer School 数理物理 2021 Lecture2

ガウス過程回帰の例

l 新しい入力x*での予測値y*の分布は、ガウス分布l 青線は期待値、水色の領域は66/124 Summer School 数理物理 2021 Lecture2

2σのエリアマウナロアCO2濃度データ

5.4 Model Selection for GP Regression

2

CO concentration, ppm

420
400
380
360
340
320
1960

1970

1980

1990
year

2000

2010

2020

l GPML p.119より引用

Figure 5.6: The 545 observations of monthly averages of the atmospheric concentra-

tionof CO2 made between 1958 and the end of 2003, together with 95% predictive
l 周期カーネルを使うことで、非常に正確に周期データ
confidence region for a Gaussian process regression model, 20 years into the future.
にフィットして予測することができる!
Risingtrend and seasonal variations are clearly visible. Note also that the confidence
interval gets wider the further the predictions are extrapolated.
67/124 Summer School 数理物理 2021 Lecture2

as insights into its properties by interpretation of the adapted hyperparame-

Gaussian processes on galaxy(ICML 2015)68/124 Summer School 数理物理 2021 Lecture2

Gaussian processes on galaxy (2)l Astronomers want to find DLA(Damped Lyman-α systems):– Large gaseous objects with neutralhydrogen gas– Emits little light and cannot be
observed directly.

69/124 Summer School 数理物理 2021 Lecture2

Finding DLAsl DLAs can be found by quaser emission spectruml Usually: by astronomers looking at it

70/124 Summer School 数理物理 2021 Lecture2

Finding DLAsl Problem: Quaser spectrums are huge in number!

– Sloan digital survey: 300 000– Millions of quasers observedl How to automate discoveries?

71/124 Summer School 数理物理 2021 Lecture2

Finding DLAsl Solution: compare probabilities ofp(spectrum | DLA exists)p(spectrum | DLA does not exist)
l How to define these non-trivial probabilities?

72/124 Summer School 数理物理 2021 Lecture2

Case of no DLAsl Gaussian process + noise
l log p(spectrum|¬DLA) = -2589.

73/124 Summer School 数理物理 2021 Lecture2

Case of DLAsl Gaussian process + unknown absorption
l log p(spectrum|DLA) = -2453 > -2589= p(spectrum|¬DLA)l DLA exists!

74/124 Summer School 数理物理 2021 Lecture2

Technically..
l If DLA does not existp(y|⇥, ¬DLA) = N(y|µ, K +⌦+N )l If DLA exist

Absorption frequency
dependent!

y( ) = f ( )e ⌧ (z,N ) + ✏#p(y|⇥, DLA, z, N ) = N(y|a µ, A(K +⌦)A+N )75/124 Summer School 数理物理 2021 Lecture2

Technically..
l If DLA does not existp(y|⇥, ¬DLA) = N(y|µ, K +⌦+N )l If DLA existp(y|⇥, DLA) ==Z
Z

Unknown frequenciesp(y, z, N |⇥, DLA)dzdNp(y|⇥, DLA, z, N )p(z, N )dzdN= (numerical integration).
– Requires understanding of Gaussian processmachinery and statistics!

76/124 Summer School 数理物理 2021 Lecture2

ガウス過程とカーネルの学習

77 / 8077ハイパーパラメータの最適化
k(xi , xj ) = ✓1 exp✓|xi xj |✓22◆+ ✓3 (i, j)l カーネルのハイパーパラメータを ✓ = (✓1 , ✓2 , ✓3 )とおくと、yの確率はガウス分布なので、θに依存してp(y|X, ✓) = N (y|0, K✓ )
✓
◆
1
1
1 T
1
=
exp
y
K
✓ y
N/2
1/2
2
(2⇡)
|K✓ |
l すなわち、log p(y|X, ✓) = log |K✓ |yT K✓ 1 y + const.

– これを最大にするθを求めればよい.
78/124 Summer School 数理物理 2021 Lecture2

ハイパーパラメータの最適化 (2)L = log p(y|X, ✓) = log |K✓ |yT K✓ 1 y + const.

l ある ✓ 2 ✓ について、微分の連鎖則から
N X
N
X
@L
@L @K✓@L @Kij==@✓@K✓ @✓@Kij @✓
i=1 j=1

l ここで

✓

◆

@
1 @K✓
log |K✓ | = tr K✓
@✓
@✓
@
1
1 @K✓
K✓ = K✓K✓ 1@✓@✓
なので、後は @Kij を使っているカーネルごとに計算すればよい.@✓79/124 Summer School 数理物理 2021 Lecture2

ハイパーパラメータの最適化 (3)l @K✓ は?

@✓
→ 各Kijをθで微分して並べた行列.
✓
◆
l 例:
2
|xi xj |+ ✓3 (i, j)
Kij = k(xi , xj ) = ✓1 exp✓2のとき、⌧ = log ✓1 とおけば、
θ1>0 なので ✓1 = e⌧✓◆2@k(xi , xj )|xi xj |⌧⌧= e exp
=
k(x
,
x
)
e
(i, j)ij⌧@⌧e– ✓2 , ✓3 についても同様
80/124 Summer School 数理物理 2021 Lecture2

ハイパーパラメータの最適化 (4)l 最適化アルゴリズム (Python, BFGSの場合)
from scipy.optimize import minimize
def optimize (xtrain, ytrain, kernel, kgrad, init):
res = minimize (loglik, init,
args = (xtrain,ytrain,kernel,kgrad),jac = gradient, method = 'BFGS’,
callback = printparam,
options = {'gtol' : 1e-4, 'disp' : True})
print res.message
return res.x– loglik で目的関数(負の対数尤度)を、gradientで偏微分を並べたベクトルを計算

81/124 Summer School 数理物理 2021 Lecture2

最適化ルーチンは最小化問題を解いているため

ハイパーパラメータの最適化 (5)l カーネルk(xi , xj ) = ✓1 exp✓|xi xj |✓22◆+ ✓3 (i, j)

で、✓1 = 1 としてみる
l 上の画像は、観測点が少ないので若干オーバーフィット
82/124 Summer School 数理物理 2021 Lecture2

ハイパーパラメータの最適化 (6)log ✓3log ✓2l log ✓2 , log ✓3の関数
l 複数の峰があり、最適化は初期値に依存する
83/124 Summer School 数理物理 2021 Lecture2

カーネルの組み合わせl カーネル k1(x,x’) と k2(x,x’) の和や積も、正しい

カーネル関数になる– ✓1 k1 (x, x0 ) + ✓2 k2 (x, x0 )– k1 (x, x0 )p · k2 (x, x0 )q (p, q 2 N)などは、また有効なカーネル関数→GPML4章を参照
l カーネルとして、たとえばを使って ✓1 , ✓2 , ✓3 , ✓4 を最適化すれば、線形性と周期性を自動的に調節した回帰モデルが得られる!
84/124 Summer School 数理物理 2021 Lecture2

ガウス分布以外の観測モデル(離散値を含む)

85 / 8085

さまざまな観測モデル
l 論文ではがガウス分布だと仮定されることが多いが、現実の観測値yはガウス分布とは限らない– 離散観測値：y=2,1,4,3,1,0,1,…
– 外れ値の存在 (ガウス分布では外れ値を扱えない)– 点過程データ：イベントがfに基づいてランダムに生起

86/124 Summer School 数理物理 2021 Lecture2

ポアソン観測モデル
l ポアソン分布: 自然数上の確率分布

87/124 Summer School 数理物理 2021 Lecture2

ポアソン観測モデル (2)

l

なので、
とおく
(場所 におけるポアソン分布の期待値)
場所xにおける
植物の個体数の架空データ(久保拓弥「データ解析のための
統計モデリング入門」より)

88/124 Summer School 数理物理 2021 Lecture2

ポアソン観測モデル (3)l ここでも、事後分布はガウス分布ではないl たとえばGPyで計算

89/124 Summer School 数理物理 2021 Lecture2

ポアソン観測モデル (4)l たとえばGPyで計算
import GPy
def gpr_poisson (data):
N = len(data)
xx = np.linspace (1,N,N)
model = GPy.core.GP (X=xx[:,None], Y=data[:,None],¥kernel=GPy.kern.RBF(1),¥inference_method=GPy.inference.latent_
function_inference.Laplace(),¥likelihood=GPy.likelihoods.Poisson())
model.optimize ()mu,var = model._raw_predict (xx[:,None])plt.plot (xx, np.exp(mu))
plt.fill_between (xx, exp(mu[:,0] + 3*sqrt(var[:,0])),¥
exp(mu[:,0] - 3*sqrt(var[:,0])),¥
color='#ccccff')
plt.plot (xx, data, 'xb', markersize=8)90/124 Summer School 数理物理 2021 Lecture2

ポアソン観測モデル (5)l 推定した結果 (ラプラス近似; 分散が小さめ)– 空間的に滑らかな推定結果が得られる
緑本11章と比較！

91/124 Summer School 数理物理 2021 Lecture2

ガウス過程とニューラルネット

92 / 8092

ニューラルネットとガウス過程
l Neal (1996)は、ニューラル

ネットワークは素子数→∞の極限でガウス過程と等価であることを示した
l ニューラルネットの多数のパラメータを学習する必要がない!
l 以下、その説明を簡単に紹介

93/124 Summer School 数理物理 2021 Lecture2

ニューラルネットとガウス過程 (2)l 入力 xn に対して yn を出力する、

yn<latexit sha1_base64="9LohLkALeohWoe9L6zCvtAqTdE0=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5oq8ppNasWxLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPstNhKvDvz+49ndte73LVMP2DsdGZ27tLlK9r81YVr12/cXFxavrXrO23PEGXDsRyvwnVfWKYtyoEZWKLiekJvcUvs8eaL0L/XEZ5vOvaroOeKw5besM26aegBmkq9ml1bSrEMk9eK2sjGjRTEV95ZntuAKrwBBwxoQwsE2BBg2wIdfLwPIAsMXLQdwgBtHrZM6RcwhAXUtpESSOhobeJ/A3sHsdXGfhjTl2oD32Lhz0PlCqTZD/aFnbHv7Cv7yf6eG2sgY4S59PDJI61wa4tv75T+/FfVwmcAR4nqAgVH+uKaAqjDU1mLibW50hJWaUTxO/2Ts9KzYnpwn31iv7C+j+yUfcMK7c5v43NBFD9g9LQcb4GRqtDBpwEpHOMB9ka5cvQNpHUY55PwkwpH9j28TczxCNW67DnQPTfCMfGuY4LrEVyP4PoE1ye4LsF1Ca5OcHWCaxJck+DWCG6N4DYJbpPgtghui+AqBFchuH2C2yc4RnBM4Xg8v+FapuaVx/Ma+dX55PJLGPkF4W/hqk+IquyrlIs5mhNcZKFIXyF9kgxzC1f4ODuyqXQ4SiN+XJHYh4TKkvsGR8u0btxDKfNKxflzKi7Jr7U1VUliVRXbE1klmu2pnMZVO3H+Bvos3J12FGJ9akVx7K8rVJGgigr1Uu42KchJuo57ULh3RrM5OjWiPKJdifbkwrh4hmWnTyy1sZvLZB9lcoXHqdXn8Wk2D3fhHjzA+E9gFTZw/MuYQwPewXs40R5qBa2ivY7Q2ZlYcxsmLs34BwY5j60=</latexit><latexit sha1_base64="f8SHKOPxCgw9+pknyUjVbpReW2I=">AAAHq3ichZXNbtNAEMenLeBSPtrCBYlLRRTEhWgTkECcWqBSq1ZVmpA2pakir9m0Vhzbsp1v5QW4cATECSQOiMfgwgtw6CMgjkXiwoHx2pGT7KTYSrw78/uPZ3bXu9y1TD9g7HRmdu7CxUva/OWFK1evXV9cWr6x5ztNzxAlw7Ecr8x1X1imLUqBGVii7HpCb3BL7PP6s9C/3xKebzr2i6DriqOGfmybNdPQAzSVKq1O1a4upViGyWtFbWTjRgriK+8sz21ABV6BAwY0oQECbAiwbYEOPt6HkAUGLtqOoI82D1um9AsYwAJqm0gJJHS01vH/GHuHsdXGfhjTl2oD32Lhz0PlCqTZD/aFnbHv7Cv7yf5OjdWXMcJcuvjkkVa41cXXt4p//qtq4DOAk0R1joIjfX5NAdTgsazFxNpcaQmrNKL4rd7bs+KTQrp/l31iv7C+j+yUfcMK7dZv4/OuKHzA6Gk53gIjVaCFTwNSOMZ97A1z5ejrS+sgzifhxxWO7Ht4m5jjCap12XOgPTVCh3hXh+C6BNcluB7B9QiuTXBtgqsRXI3g6gRXJ7g1glsjuE2C2yS4LYLbIrgywZUJ7oDgDgiOERxTOB7Pb7iWqXnl8bxGfnU+ufwShn5B+Bu46hOiIvsq5WKO5hgXWSjSV0ifJMPcwhU+yg5tKh2O0pAfVST2AaGy5L7B0TKpG/VQyrxScX5KxUX5tTYmKkmsqmJ7LKtEsz2R06hqJ87fQJ+Fu9OOQqxPrCiO/XWFKhBUQaGey90mBTlJ13APCvfOaDaHp0aUR7Qr0Z5cGBfPsOzkiaU29nKZ7INMbvdhavVpfJrNw224A/cw/iNYhQ0c/xLmYMIbeAfvtftaUXupVSJ0dibW3ISxSxP/AExmkJI=</latexit>

右図のような1層のニューラル
ネットを考えるl 式で書くと、(PHyn = j=1 wj hj (xn )PD
hj (xn ) = ( k=0 vjk xnk )wj<latexit sha1_base64="/xaxpPWeaydpO9+ZwBc5wsGVf0U=">AAAHsHichZXNbtNAEMenLeBSPtrCBYlLRRRULtEmIIE4tUClVq2qNCFtSlNFXrNp3PhLtvOtvABnJA4IJJA4IB6DCy/AoY+AOBaJCwfGa0dO4mlrK/HuzO8/ntld73LH0D2fsZOp6ZlLl68os1fnrl2/cXN+YfHWrmc3XU2UNNuw3TJXPWHolij5um+IsuMK1eSG2OON54F/ryVcT7etl37XEYememTpNV1TfTS9qlePlyutTtV6UF1IsQyT11KykY0aKYiuvL04sw4VeA02aNAEEwRY4GPbABU8vA8gCwwctB1CH20utnTpFzCAOdQ2kRJIqGht4P8R9g4iq4X9IKYn1Rq+xcCfi8olSLOf7Cs7ZT/YN/aL/TszVl/GCHLp4pOHWuFU59/cKf69UGXi04d6rDpHwZE+vyYfavBE1qJjbY60BFVqYfxW791p8Wkh3b/PPrPfWN8ndsK+Y4VW64/2ZUcU3mP0tBxvgZEq0MKnBikc4z72hrly9PWldRDlE/PjClv2Xbx1zLGOalX2bGifGaFDvKtDcF2C6xJcj+B6BNcmuDbB1QiuRnANgmsQ3CrBrRLcBsFtENwmwW0SXJngygS3T3D7BMcIjiU4Hs1vsJapeeXRvIb+5Hxy+SUM/YLwm7jqY6Ii+0nKwRz1MS60UKSXID2SDHILVvgoO7Ql6WCUhvyoIrYPCJUh9w2OlkndqIdS5hMV58+ouCi/VnOiktiaVGyNZRVrtiZyGlVtR/lr6DNwd9pOEGsTK4pjfy1BFQiqkKBeyN0mBTlJ13APCvbOcDaHp0aYR7gr0Z5cEBfPsOzkiZVs7OYy2YeZ3M6j1Mqz6DSbhbtwD5Yx/mNYgXUc/xLmYMFb+AAflZxSVqqKGqLTU5HmNoxdyvF/IhiSRg==</latexit><latexit sha1_base64="PnBHKpcTUoEuzEcRLV+OijS07Uo=">AAAHp3ichZVLb9NAEMenLeBSHm3hgsSlIgqCS7QJSCBOLVCpVauSB0lTNVXlNZvWxC/ZTtIkyheAK4gDJ5A4ID4GF74Ah34ExLFIXDgwXjvKY6etrcS7M7//eGZ3vcs9ywxCxo6npmcuXLykzV6eu3L12vX5hcUblcBt+oYoG67l+lWuB8IyHVEOzdASVc8Xus0tsc0bzyL/dkv4gek6L8OOJ/Zs/cAx66ahh2gqvN5fSLEMk9eS2sgmjRQkV95dnFmDGrwCFwxogg0CHAixbYEOAd67kAUGHtr2oIc2H1um9Avowxxqm0gJJHS0NvD/AHu7idXBfhQzkGoD32Lhz0flEqTZT/aVnbAf7Bv7xf6dGqsnY0S5dPDJY63w9uff3Cr9PVdl4zOEw6HqDAVH+uyaQqjDY1mLibV50hJVacTxW90PJ6UnxXTvLvvMfmN9n9gx+44VOq0/xpeCKH7E6Gk53gIj1aCFTwNSOMY97A1y5ejrSWs/yWfIjytc2ffxNjHHQ1TrsudC+9QIR8S7jgiuQ3AdgusSXJfg2gTXJrg6wdUJrkFwDYJbIbgVglsnuHWC2yC4DYKrElyV4HYIbofgGMExhePJ/EZrmZpXnsxr7Ffnk8svYeAXhN/GVT8karKvUh7maI5xsYUiA4UMSDLKLVrho+zAptLRKA34UcXQ3idUltw3OFomdaMeSplXKs6fUnFJfq32RCVDq6rYHMtqqNmcyGlUtZXkb6DPwt1pSyFWJ1YUx/6qQhUJqqhQz+Vuk4KcpOu4B0V7Zzybg1MjziPelWhPLoqLZ1h28sRSG5VcJvsgkys8TC0/TU6zWbgNd+Aexn8Ey7CG41/GHAS8hXfwXruvvdAqWjVGp6cSzU0YuzT9P3Gjjr0=</latexit>j<latexit sha1_base64="lpcqcnLbERbmBH8f/i4+04dwJ4I=">AAAH73ichZXPTxNBFMcfoC7iD0AvJl4aGwwkpE6ricYTqEQIhJRWoITCZnedwtL95e7SUjb1D/Af4GBi1MQDMf4VXrwbDyT6BxiPmHjx4NvZbfpjX3E37c689/m+fW9mdkZ1DN3zGTsZGBw6d/6CNHxx5NLlK1dHx8avrXn2vqvxVc02bLekKh43dIuv+rpv8JLjcsVUDb6uVh+H/vUadz3dtp75DYdvmcqOpVd0TfHRJI89rct7qbKnm6mgrClGark5yaaxv2Mqcn07NzWdejlSk4O9arMfVdvO3ZmfksfSLMPElUo2snEjDfGVt8eH5qEMz8EGDfbBBA4W+Ng2QAEP703IAgMHbVsQoM3Fli78HJowgtp9pDgSClqr+L+Dvc3YamE/jOkJtYZvMfDnojIFE+wbO2an7Av7yH6yv31jBSJGmEsDn2qk5Y48+upG8c9/VSY+fdhtq85QqEifXZMPFXggatGxNkdYwiq1KH7t8Oi0+LAwEdxm79kvrO8dO2GfsUKr9lv7sMILrzH6hBhvjpHKUMOnBmkc4wB7rVxV9AXC2ozzafPdClv0Xbx1zHEX1Yro2VDvG+GAeNcBwTUIrkFwhwR3SHB1gqsTXIXgKgRXJbgqwc0S3CzBLRDcAsEtEtwiwZUIrkRwGwS3QXCM4FiCU+P5DdcyNa9qPK+RPzmfqvgSWn5O+E1c9W2iLPpJysEc9S4uslCklyA9kgxzC1d4J9uyJelwlFp8p6JtbxIqQ+wbKlp6dZ0eSplPVJzvU3FRfK1mTyVta1Kx1JVVW7PUk1OnajnOX0OfgbvTcoKY61lRKvbnElSBoAoJ6onYbdKQE3QF96Bw74xms3VqRHlEuxLtyYVx8QzL9p5YycZaLpO9m8mt3EvPPIpPs2G4CbdgEuPfhxmYx/FfxRw+wVf4Dj+kF9KR9EZ6G6GDA7HmOnRd0vE/612pGg==</latexit>

vjk ⇠ N (0,に従うとする94/124 Summer School 数理物理 2021 Lecture2···<latexit sha1_base64="TrCYO938AN9SldJ7ZwZrTCx2d5w=">AAAHrHichZXNbtNAEMenLZBSPtrCBYlLRRTEKdqkSCBOLVCpVasqTZo0VRJVtrtuTfwl20maRHkBTtwQcAKJA+IxuPACHPoIiGORuHBgvHbkJDspthLvzvz+45nd9a7qmoYfMHY+Mzt35eq11Pz1hRs3b91eXFq+U/GdlqfxsuaYjldVFZ+bhs3LgRGYvOp6XLFUkx+ozReh/6DNPd9w7P2g6/KGpZzYhm5oSoCmSl07dgL/aCnNskxcK3IjFzfSEF8FZ3luE+pwDA5o0AILONgQYNsEBXy8a5ADBi7aGtBHm4ctQ/g5DGABtS2kOBIKWpv4f4K9Wmy1sR/G9IVaw7eY+PNQuQIZ9oN9YRfsO/vKfrK/U2P1RYwwly4+1UjL3aPF1/dKf/6rsvAZwGmiukShIn15TQHo8FTUYmBtrrCEVWpR/Hbv7UXpWTHTf8g+sV9Y30d2zr5hhXb7t/Z5jxc/YPSMGG+OkerQxqcGaRzjPvaGuaro6wvrIM4n4ccVjuh7eBuY4ymqFdFzoDM1whnxrjOC6xJcl+B6BNcjuA7BdQhOJzid4JoE1yS4dYJbJ7gtgtsiuG2C2ya4KsFVCe6Q4A4JjhEckzg1nt9wLVPzqsbzGvnl+VTFlzD0c8Jv4apPiLroy5SLORpjXGShSF8ifZIMcwtX+Cg7tMl0OEpDflSR2AeEyhT7hoqWSd2oh1IWpIoLUyouia/VmqgkscqKnbGsEs3ORE6jqt04fw19Ju5OuxKxMbGiVOxvSFSRoIoS9VLsNmnIC1rHPSjcO6PZHJ4aUR7RrkR78mFcPMNykyeW3Kjks7nVbH7vcXrteXyazcN9eACPMP4TWINNHP8y5vAK3sA7eJ/KpvZTtVQjQmdnYs1dGLtS+j9AmZD+</latexit>

vjk<latexit sha1_base64="VyogNH+mkG17LDChTI0pUzKy71c=">AAAHrHichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJk2aKokq2123bvwl20maWHkBTtwQcAKJA+IxuPACHPoIiGORuHBgvHbkJJ4UW4l3Z37/8czueldxDN3zGTufmZ27cvWaNH994cbNW7cXl5bvVD277aq8otqG7dYU2eOGbvGKr/sGrzkul03F4PtK62Xo3+9w19Nta8/vObxpyseWrumq7KOp2jkMTluDw6UMyzFxraQb+biRgfgq2stzG9CAI7BBhTaYwMECH9sGyODhXYc8MHDQ1oQAbS62dOHnMIAF1LaR4kjIaG3h/zH26rHVwn4Y0xNqFd9i4M9F5Qpk2Q/2hV2w7+wr+8n+To0ViBhhLj18KpGWO4eLr++V//xXZeLTh5NEdYlCQfrymnzQ4JmoRcfaHGEJq1Sj+J3+24vy81I2eMg+sV9Y30d2zr5hhVbnt/p5l5c+YPSsGG+OkRrQwacKGRzjAHvDXBX0BcI6iPNJ+HGFLfou3jrmeIJqWfRs6E6NcEa864zgegTXI7g+wfUJrktwXYLTCE4juBbBtQhujeDWCG6T4DYJbovgtgiuRnA1gjsguAOCYwTHUpwSz2+4lql5VeJ5jfzp+VTElzD0c8Jv4qpPiIbopykHc9THuMhCkV6K9EgyzC1c4aPs0Jamw1Ea8qOKxD4gVIbYNxS0TOpGPZSymKq4OKXisvhazYlKEmtasT2WVaLZnshpVLUT56+iz8DdaSdFrE+sKAX76ymqRFClFPVK7DYZKAhawz0o3Duj2RyeGlEe0a5EewphXDzD8pMnVrpRLeTyj3OF3SeZ1RfxaTYP9+EBPML4T2EVNnD8K5jDKbyBd/Beykl7Ul1qRujsTKy5C2OXpP0DaweRJw==</latexit>k<latexit sha1_base64="i4PsFJxuscb3On436ezMeEzl0eo=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1YlHyRN1VSV12xaK45t2U7SJMoLwBXEgRNIHBCPwYUX4NBHQByLxIUD47UjJ9lJsZV4d+b3H8/srne5a5l+wNjpzOzchYuXtPnLC1euXru+uLR8o+I7Lc8QZcOxHK/KdV9Ypi3KgRlYoup6Qm9yS+zyxrPQv9sWnm869sug64qDpn5km3XT0AM0FRqHSymWYfJaURvZuJGC+Mo7y3MbUINX4IABLWiCABsCbFugg4/3PmSBgYu2A+ijzcOWKf0CBrCA2hZSAgkdrQ38P8Lefmy1sR/G9KXawLdY+PNQuQJp9oN9YWfsO/vKfrK/U2P1ZYwwly4+eaQV7uHi61ulP/9VNfEZwHGiOkfBkT6/pgDq8FjWYmJtrrSEVRpR/Hbv/VnpSTHdv8s+sV9Y30d2yr5hhXb7t/G5IIofMHpajrfASDVo49OAFI5xH3vDXDn6+tI6iPNJ+HGFI/se3ibmeIxqXfYc6EyNcEK864TgugTXJbgewfUIrkNwHYKrE1yd4BoE1yC4NYJbI7hNgtskuC2C2yK4KsFVCW6P4PYIjhEcUzgez2+4lql55fG8Rn51Prn8EoZ+QfibuOoToib7KuVijuYYF1ko0ldInyTD3MIVPsoObSodjtKQH1Uk9gGhsuS+wdEyqRv1UMq8UnF+SsUl+bU2JypJrKpieyyrRLM9kdOoaifO30CfhbvTjkKsT6wojv11hSoSVFGhnsvdJgU5SddxDwr3zmg2h6dGlEe0K9GeXBgXz7Ds5ImlNiq5TPZBJld4mFp9Gp9m83Ab7sA9jP8IVmEDx7+MOQh4A2/hnXZfe6FVtGqEzs7Empswdmn6P3jmjr4=</latexit>– ただし重み w, v はi.i.d.に2w ),hj (xn )<latexit sha1_base64="+snKFaxrKOt7Jurze6eJStKqFCY=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5oqst1NauIv2U7SJMoLIHGlB04gcUA8BhdegEMfAXEsEhcOjNeOnGQnxVbi3Znffzyzu97VXNPwA8bOZmbnLl2+osxfXbh2/cbNxaXlW7u+0/J0XtYd0/Eqmupz07B5OTACk1dcj6uWZvI9rfki9O+1uecbjv0q6Lr80FIbtlE3dDVAU6lTe1NbSrEME9eK3MjGjRTEV95ZntuAKhyBAzq0wAIONgTYNkEFH+8DyAIDF22H0Eebhy1D+DkMYAG1LaQ4Eipam/jfwN5BbLWxH8b0hVrHt5j481C5Amn2g31h5+w7+8p+sr9TY/VFjDCXLj61SMvd2uLbO6U//1VZ+AzgOFFdoNCQvrimAOrwVNRiYG2usIRV6lH8du/0vPSsmO7fZ5/YL6zvIztj37BCu/1b/1zgxQ8YPS3Gm2OkKrTxqUMKx7iPvWGuGvr6wjqI80n4cYUj+h7eBuZ4jGpV9BzoTI1wQrzrhOC6BNcluB7B9QiuQ3AdgqsTXJ3gmgTXJLg1glsjuE2C2yS4LYLbIrgKwVUIbp/g9gmOERyTOC2e33AtU/OqxfMa+eX51MSXMPRzwm/hqk+IqujLlIs5GmNcZKFIXyJ9kgxzC1f4KDu0yXQ4SkN+VJHYB4TKFPuGhpZJ3aiHUualivNTKi6Jr9WaqCSxyortsawSzfZETqOqnTh/HX0m7k47ErE+saI07K9LVJGgihL1Uuw2KcgJuo57ULh3RrM5PDWiPKJdifbkwrh4hmUnTyy5sZvLZB9lcoXHqdXn8Wk2D3fhHjzA+E9gFTZw/MuYQwPewXs4VR4qBaWivI7Q2ZlYcxvGLkX/B9qUj6c=</latexit><latexit sha1_base64="Ac9yLM6ntNDV5XzC/KEEhM/ExkM=">AAAIaHichZXLbtNAFIZPKNBgLk1hAYiN1ahVuyCaBBAVUqUWWqlVq6pNSZuq01q2O0lcXxLZzq2WX4AXYMEKJBaIl0Biwwuw6CMAyyKxYcHxJcrF02Ir8cw53398zsx4RmkYmuMScpa6Mnb12vXx9A3h5q3bdyYyk3d3nXrTVllJrRt1u6zIDjM0i5VczTVYuWEz2VQMtqforwL/XovZjla3XrvdBjs05aqlVTRVdtEkZb5Qg1Vc6glUYVXN8mTblru+Zxu+0JUscWZBpC7ruI7bNRh1TNmpedRpmpJ3spD3j1Z9sS2diDXpZJa2OpI1J1J6kGedQ4HWnIasMu8xyT1jHV/oI1RntoXmeWYOhxepo1VNeTaKry8Q/2hZbOGbdL8jeZbuzwmUWcdxigK1tWrNzUmZLMmR8BKTjXzcyEJ8bdUnx1aBwjHUQYUmmMDAAhfbBsjg4H0AeSDQQNsheGizsaWFfgY+CKhtIsWQkNGq438Vewex1cJ+ENMJ1Sq+xcCfjUoRpsl38omck2/kM/lB/l4YywtjBLl08alEWtaQJt482PnzX5WJTxdqfdUlCgXpy2tyoQLzYS0a1tYILUGVahS/dfr2fOdFcdqbIR/IL6zvPTkjX7FCq/Vb/bjNiu8w+nQ43gwjUWjhU4UsjrGHvV6uCvq80OrH+fT5YUU97Nt4a5hjDdVy2KtD+8IIHc67Ohyuy+G6HO6Uw51yuDaHa3O4CoercDidw+kcbonDLXG4NQ63xuHWOdw6hytzuDKH2+dw+xyOcDiS4JR4foO1zJtXJZ7XyJ+cTyX8Enp+xvGbuOr7BA37SaqBOWpDXGThkU6CdLhkkFuwwgfZni1JB6PU4wcVfbvPURnhvqGgZVQ36OEptxIVb11Q8U74tZojlfStScXGUFZ9zcZIToOqzTh/FX0G7k6bCWJlZEUp2F9JUEUOVUxQy+Fuk4VCSFdwDwr2zmg2e6dGlEe0K/E9hSAunmH50RMr2dgt5PJPcoXtp9nFl/FploZHMAWzGP85LMIqjn8J1NRMaiNVSu2O/0xn0vfTDyP0SirW3IOhKz31D7JM1AI=</latexit>

wj ⇠ N (0,<latexit sha1_base64="9LohLkALeohWoe9L6zCvtAqTdE0=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5oq8ppNasWxLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPstNhKvDvz+49ndte73LVMP2DsdGZ27tLlK9r81YVr12/cXFxavrXrO23PEGXDsRyvwnVfWKYtyoEZWKLiekJvcUvs8eaL0L/XEZ5vOvaroOeKw5besM26aegBmkq9ml1bSrEMk9eK2sjGjRTEV95ZntuAKrwBBwxoQwsE2BBg2wIdfLwPIAsMXLQdwgBtHrZM6RcwhAXUtpESSOhobeJ/A3sHsdXGfhjTl2oD32Lhz0PlCqTZD/aFnbHv7Cv7yf6eG2sgY4S59PDJI61wa4tv75T+/FfVwmcAR4nqAgVH+uKaAqjDU1mLibW50hJWaUTxO/2Ts9KzYnpwn31iv7C+j+yUfcMK7c5v43NBFD9g9LQcb4GRqtDBpwEpHOMB9ka5cvQNpHUY55PwkwpH9j28TczxCNW67DnQPTfCMfGuY4LrEVyP4PoE1ye4LsF1Ca5OcHWCaxJck+DWCG6N4DYJbpPgtghui+AqBFchuH2C2yc4RnBM4Xg8v+FapuaVx/Ma+dX55PJLGPkF4W/hqk+IquyrlIs5mhNcZKFIXyF9kgxzC1f4ODuyqXQ4SiN+XJHYh4TKkvsGR8u0btxDKfNKxflzKi7Jr7U1VUliVRXbE1klmu2pnMZVO3H+Bvos3J12FGJ9akVx7K8rVJGgigr1Uu42KchJuo57ULh3RrM5OjWiPKJdifbkwrh4hmWnTyy1sZvLZB9lcoXHqdXn8Wk2D3fhHjzA+E9gFTZw/MuYQwPewXs40R5qBa2ivY7Q2ZlYcxsmLs34BwY5j60=</latexit>

xn<latexit sha1_base64="f8SHKOPxCgw9+pknyUjVbpReW2I=">AAAHq3ichZXNbtNAEMenLeBSPtrCBYlLRRTEhWgTkECcWqBSq1ZVmpA2pakir9m0Vhzbsp1v5QW4cATECSQOiMfgwgtw6CMgjkXiwoHx2pGT7KTYSrw78/uPZ3bXu9y1TD9g7HRmdu7CxUva/OWFK1evXV9cWr6x5ztNzxAlw7Ecr8x1X1imLUqBGVii7HpCb3BL7PP6s9C/3xKebzr2i6DriqOGfmybNdPQAzSVKq1O1a4upViGyWtFbWTjRgriK+8sz21ABV6BAwY0oQECbAiwbYEOPt6HkAUGLtqOoI82D1um9AsYwAJqm0gJJHS01vH/GHuHsdXGfhjTl2oD32Lhz0PlCqTZD/aFnbHv7Cv7yf5OjdWXMcJcuvjkkVa41cXXt4p//qtq4DOAk0R1joIjfX5NAdTgsazFxNpcaQmrNKL4rd7bs+KTQrp/l31iv7C+j+yUfcMK7dZv4/OuKHzA6Gk53gIjVaCFTwNSOMZ97A1z5ejrS+sgzifhxxWO7Ht4m5jjCap12XOgPTVCh3hXh+C6BNcluB7B9QiuTXBtgqsRXI3g6gRXJ7g1glsjuE2C2yS4LYLbIrgywZUJ7oDgDgiOERxTOB7Pb7iWqXnl8bxGfnU+ufwShn5B+Bu46hOiIvsq5WKO5hgXWSjSV0ifJMPcwhU+yg5tKh2O0pAfVST2AaGy5L7B0TKpG/VQyrxScX5KxUX5tTYmKkmsqmJ7LKtEsz2R06hqJ87fQJ+Fu9OOQqxPrCiO/XWFKhBUQaGey90mBTlJ13APCvfOaDaHp0aUR7Qr0Z5cGBfPsOzkiaU29nKZ7INMbvdhavVpfJrNw224A/cw/iNYhQ0c/xLmYMIbeAfvtftaUXupVSJ0dibW3ISxSxP/AExmkJI=</latexit>

xnk<latexit sha1_base64="6WHVIAH7ZCstTetqTAVqZsuoqmI=">AAAHrHichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJk2aKokq26xbE3/Jdj6tvAAnbgg4gcQB8RhceAEOfQTEsUhcODBeO3IST4qtxLszv/94Zne9qziG7vmMnc3NL1y6fEVavLp07fqNm8srq7eqnt12VV5RbcN2a4rscUO3eMXXfYPXHJfLpmLwQ6X1PPQfdrjr6bZ14Pcd3jTlE0vXdFX20VTtHQdWa3i8kmE5Jq61dCMfNzIQX0V7dWELGvASbFChDSZwsMDHtgEyeHjXIQ8MHLQ1IUCbiy1d+DkMYQm1baQ4EjJaW/h/gr16bLWwH8b0hFrFtxj4c1G5Bln2g31h5+w7+8p+sr8zYwUiRphLH59KpOXO8fLrO+U//1WZ+PThNFFdoFCQvrgmHzR4ImrRsTZHWMIq1Sh+Z/D2vPy0lA3us0/sF9b3kZ2xb1ih1fmtft7npQ8YPSvGm2OkBnTwqUIGxzjA3ihXBX2BsA7jfBJ+UmGLvou3jjmeoloWPRu6MyP0iHf1CK5PcH2CGxDcgOC6BNclOI3gNIJrEVyL4DYIboPgtglum+B2CG6H4GoEVyO4I4I7IjhGcCzFKfH8hmuZmlclntfIn55PRXwJIz8n/Cau+oRoiH6acjBHfYKLLBTppUiPJMPcwhU+zo5saTocpRE/rkjsQ0JliH1DQcu0btxDKYupioszKi6Lr9WcqiSxphW7E1klmt2pnMZVe3H+KvoM3J32UsTm1IpSsL+ZokoEVUpRL8Ruk4GCoDXcg8K9M5rN0akR5RHtSrSnEMbFMyw/fWKlG9VCLv8wV9h/lFl/Fp9mi3AX7sEDjP8Y1mELx7+CObyCN/AO3ks56UCqS80InZ+LNbdh4pK0f5arkS0=</latexit>

2
v /H)

ニューラルネットとガウス過程 (3)l このとき、yn の期待値は?

yn<latexit sha1_base64="9LohLkALeohWoe9L6zCvtAqTdE0=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5oq8ppNasWxLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPstNhKvDvz+49ndte73LVMP2DsdGZ27tLlK9r81YVr12/cXFxavrXrO23PEGXDsRyvwnVfWKYtyoEZWKLiekJvcUvs8eaL0L/XEZ5vOvaroOeKw5besM26aegBmkq9ml1bSrEMk9eK2sjGjRTEV95ZntuAKrwBBwxoQwsE2BBg2wIdfLwPIAsMXLQdwgBtHrZM6RcwhAXUtpESSOhobeJ/A3sHsdXGfhjTl2oD32Lhz0PlCqTZD/aFnbHv7Cv7yf6eG2sgY4S59PDJI61wa4tv75T+/FfVwmcAR4nqAgVH+uKaAqjDU1mLibW50hJWaUTxO/2Ts9KzYnpwn31iv7C+j+yUfcMK7c5v43NBFD9g9LQcb4GRqtDBpwEpHOMB9ka5cvQNpHUY55PwkwpH9j28TczxCNW67DnQPTfCMfGuY4LrEVyP4PoE1ye4LsF1Ca5OcHWCaxJck+DWCG6N4DYJbpPgtghui+AqBFchuH2C2yc4RnBM4Xg8v+FapuaVx/Ma+dX55PJLGPkF4W/hqk+IquyrlIs5mhNcZKFIXyF9kgxzC1f4ODuyqXQ4SiN+XJHYh4TKkvsGR8u0btxDKfNKxflzKi7Jr7U1VUliVRXbE1klmu2pnMZVO3H+Bvos3J12FGJ9akVx7K8rVJGgigr1Uu42KchJuo57ULh3RrM5OjWiPKJdifbkwrh4hmWnTyy1sZvLZB9lcoXHqdXn8Wk2D3fhHjzA+E9gFTZw/MuYQwPewXs40R5qBa2ivY7Q2ZlYcxsmLs34BwY5j60=</latexit>

l まず、

wj

<latexit sha1_base64="9LohLkALeohWoe9L6zCvtAqTdE0=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5oq8ppNasWxLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPstNhKvDvz+49ndte73LVMP2DsdGZ27tLlK9r81YVr12/cXFxavrXrO23PEGXDsRyvwnVfWKYtyoEZWKLiekJvcUvs8eaL0L/XEZ5vOvaroOeKw5besM26aegBmkq9ml1bSrEMk9eK2sjGjRTEV95ZntuAKrwBBwxoQwsE2BBg2wIdfLwPIAsMXLQdwgBtHrZM6RcwhAXUtpESSOhobeJ/A3sHsdXGfhjTl2oD32Lhz0PlCqTZD/aFnbHv7Cv7yf6eG2sgY4S59PDJI61wa4tv75T+/FfVwmcAR4nqAgVH+uKaAqjDU1mLibW50hJWaUTxO/2Ts9KzYnpwn31iv7C+j+yUfcMK7c5v43NBFD9g9LQcb4GRqtDBpwEpHOMB9ka5cvQNpHUY55PwkwpH9j28TczxCNW67DnQPTfCMfGuY4LrEVyP4PoE1ye4LsF1Ca5OcHWCaxJck+DWCG6N4DYJbpPgtghui+AqBFchuH2C2yc4RnBM4Xg8v+FapuaVx/Ma+dX55PJLGPkF4W/hqk+IquyrlIs5mhNcZKFIXyF9kgxzC1f4ODuyqXQ4SiN+XJHYh4TKkvsGR8u0btxDKfNKxflzKi7Jr7U1VUliVRXbE1klmu2pnMZVO3H+Bvos3J12FGJ9akVx7K8rVJGgigr1Uu42KchJuo57ULh3RrM5OjWiPKJdifbkwrh4hmWnTyy1sZvLZB9lcoXHqdXn8Wk2D3fhHjzA+E9gFTZw/MuYQwPewXs40R5qBa2ivY7Q2ZlYcxsmLs34BwY5j60=</latexit>

hj (xn )<latexit sha1_base64="+snKFaxrKOt7Jurze6eJStKqFCY=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5oqst1NauIv2U7SJMoLIHGlB04gcUA8BhdegEMfAXEsEhcOjNeOnGQnxVbi3Znffzyzu97VXNPwA8bOZmbnLl2+osxfXbh2/cbNxaXlW7u+0/J0XtYd0/Eqmupz07B5OTACk1dcj6uWZvI9rfki9O+1uecbjv0q6Lr80FIbtlE3dDVAU6lTe1NbSrEME9eK3MjGjRTEV95ZntuAKhyBAzq0wAIONgTYNkEFH+8DyAIDF22H0Eebhy1D+DkMYAG1LaQ4Eipam/jfwN5BbLWxH8b0hVrHt5j481C5Amn2g31h5+w7+8p+sr9TY/VFjDCXLj61SMvd2uLbO6U//1VZ+AzgOFFdoNCQvrimAOrwVNRiYG2usIRV6lH8du/0vPSsmO7fZ5/YL6zvIztj37BCu/1b/1zgxQ8YPS3Gm2OkKrTxqUMKx7iPvWGuGvr6wjqI80n4cYUj+h7eBuZ4jGpV9BzoTI1wQrzrhOC6BNcluB7B9QiuQ3AdgqsTXJ3gmgTXJLg1glsjuE2C2yS4LYLbIrgKwVUIbp/g9gmOERyTOC2e33AtU/OqxfMa+eX51MSXMPRzwm/hqk+IqujLlIs5GmNcZKFIXyJ9kgxzC1f4KDu0yXQ4SkN+VJHYB4TKFPuGhpZJ3aiHUualivNTKi6Jr9WaqCSxyortsawSzfZETqOqnTh/HX0m7k47ErE+saI07K9LVJGgihL1Uuw2KcgJuo57ULh3RrM5PDWiPKJdifbkwrh4hmUnTyy5sZvLZB9lcoXHqdXn8Wk2D3fhHjzA+E9gFTZw/MuYQwPewXs4VR4qBaWivI7Q2ZlYcxvGLkX/B9qUj6c=</latexit>E[hj (xn )] = E[ (<latexit sha1_base64="g6Sz+u8OMtQNHkNo4gWo0ktj8wI=">AAAH5XichZXPT9RAFMcfoC7iD0AvJl6IGwxcyOxqojEhAWUTCITAIrBkuzZtnV1Kf6btLrs0+w+YeDN68GDUeDD+GV70qvHAn2A8YuLFg6/TbvZHH9hmtzPvfb6v781MZ1TX1P2AseOh4ZFz5y9kRi+OXbp85er4xOS1Hd+pexrf1hzT8Uqq4nNTt/l2oAcmL7keVyzV5Luq8Sjy7za45+uO/ThoubxiKTVbr+qaEqBJnpiXAt4M/KBl8rFCeV8+mJEaTdmercwXypKv1yxlRvLrlhwa86z9ZGmqIYcHRrsph7bRnq3IE1k2x8Q1lW7kkkYWkmvDmRxZBgmeggMa1MECDjYE2DZBAR/vMuSAgYu2CoRo87ClCz+HNoyhto4UR0JBq4H/NeyVE6uN/SimL9QavsXEn4fKKZhmP9hHdsK+sE/sJ/t7aqxQxIhyaeFTjbXclcef3dj681+Vhc8A9ruqMxQq0mfXFEAV7otadKzNFZaoSi2O3zh6dbL1oDgd3mbv2C+s7y07Zp+xQrvxW/uwyYuvMfq0GG+OkSRo4FODLI5xiL1Orir6QmFtJ/l0+X6FI/oe3jrmuI9qRfQcODw1QpN4V5PgWgTXIrgjgjsiuEOCOyS4KsFVCc4gOIPgFglukeBWCG6F4FYJbpXgSgRXIrg9gtsjOEZwLMWpyfxGa5maVzWZ19ifnk9VfAkdPyf8Fq76LiGJfppyMUe9j4stFOmnSJ8ko9yiFd7LdmxpOhqlDt+r6NrbhMoU+4aKlkFdr4dSbqQq3jil4i3xtVoDlXStacVaX1ZdzdpATr2q9SR/DX0m7k7rKaIwsKJU7BdSVJGgiilqSew2WcgLuop7ULR3xrPZOTXiPOJdifbko7h4huUGT6x0Yyc/l7szl9+8m114mJxmo3ATbsEMxr8HC7CM47+NObyHr/ANvmdqmeeZF5mXMTo8lGiuQ9+VefMPM/Cnsg==</latexit>PD

k=0 vjk xnk )]は j に関わらず同じ分布l よって、中心極限定理よりPH
yn = j=1 wj hj (xn )<latexit sha1_base64="kpe6Pp4O3i1aSHEgMdygeITsvoU=">AAAH1HichZVLb9NAEMenLZBSHk3hgsTFIioqEqo2AQmEVKkFKrVqVaUpbVM1rWWbTePEL9nOqyYnxIUDBy4cOIHEAfEFuAIXvgCHfgTEsUhcODBeO8rD02Ir8e7M7z+e2V3vqo6hez5jRyOjY2fOnkuNn5+4cPHS5cn01JUtz667Gt/UbMN2i6ricUO3+Kav+wYvOi5XTNXg22rtUejfbnDX023rid92+J6pHFh6WdcUH01y+nbJ5y3f89sGn2jLljQnlby6KQfVuWxnf0lqylWpIldnSo2WbN2S0xk2y8QlJRvZuJGB+MrbU2NLUIKnYIMGdTCBgwU+tg1QwMN7F7LAwEHbHgRoc7GlCz+HDkygto4UR0JBaw3/D7C3G1st7IcxPaHW8C0G/lxUSjDNfrCP7Jh9Z5/YT/b3xFiBiBHm0sanGmm5I0++vLbx578qE58+VHqqUxQq0qfX5EMZ7otadKzNEZawSi2K3zh8c7zxoDAd3GTv2S+s7x07Yt+wQqvxW/uwzgtvMfq0GG+OkUrQwKcGGRzjAHvdXFX0BcLaifPp8YMKW/RdvHXMsYJqRfRsaJ4YoUW8q0VwbYJrE9whwR0SXJPgmgRXJrgywdUIrkZwCwS3QHDLBLdMcCsEt0JwRYIrEtwOwe0QHCM4luDUeH7DtUzNqxrPa+RPzqcqvoSunxN+E1d9jyiJfpJyMEd9gIssFOklSI8kw9zCFd7Pdm1JOhylLt+v6Nk7hMoQ+4aKlmFdv4dS5hMV50+oeEN8reZQJT1rUrE6kFVPszqUU79qLc5fQ5+Bu9NaglgcWlEq9hcTVIGgCgnqsdhtMpATdBn3oHDvjGaze2pEeUS7Eu3JhXHxDMsOn1jJxlZuNntnNrd+NzP/MD7NxuE63IAZjH8P5mEJx38Tc3gNn+ELfE1tpZ6lnqdeROjoSKy5CgNX6tU/gw6f8A==</latexit><latexit sha1_base64="/xaxpPWeaydpO9+ZwBc5wsGVf0U=">AAAHsHichZXNbtNAEMenLeBSPtrCBYlLRRRULtEmIIE4tUClVq2qNCFtSlNFXrNp3PhLtvOtvABnJA4IJJA4IB6DCy/AoY+AOBaJCwfGa0dO4mlrK/HuzO8/ntld73LH0D2fsZOp6ZlLl68os1fnrl2/cXN+YfHWrmc3XU2UNNuw3TJXPWHolij5um+IsuMK1eSG2OON54F/ryVcT7etl37XEYememTpNV1TfTS9qlePlyutTtV6UF1IsQyT11KykY0aKYiuvL04sw4VeA02aNAEEwRY4GPbABU8vA8gCwwctB1CH20utnTpFzCAOdQ2kRJIqGht4P8R9g4iq4X9IKYn1Rq+xcCfi8olSLOf7Cs7ZT/YN/aL/TszVl/GCHLp4pOHWuFU59/cKf69UGXi04d6rDpHwZE+vyYfavBE1qJjbY60BFVqYfxW791p8Wkh3b/PPrPfWN8ndsK+Y4VW64/2ZUcU3mP0tBxvgZEq0MKnBikc4z72hrly9PWldRDlE/PjClv2Xbx1zLGOalX2bGifGaFDvKtDcF2C6xJcj+B6BNcmuDbB1QiuRnANgmsQ3CrBrRLcBsFtENwmwW0SXJngygS3T3D7BMcIjiU4Hs1vsJapeeXRvIb+5Hxy+SUM/YLwm7jqY6Ii+0nKwRz1MS60UKSXID2SDHILVvgoO7Ql6WCUhvyoIrYPCJUh9w2OlkndqIdS5hMV58+ouCi/VnOiktiaVGyNZRVrtiZyGlVtR/lr6DNwd9pOEGsTK4pjfy1BFQiqkKBeyN0mBTlJ13APCvbOcDaHp0aYR7gr0Z5cEBfPsOzkiZVs7OYy2YeZ3M6j1Mqz6DSbhbtwD5Yx/mNYgXUc/xLmYMFb+AAflZxSVqqKGqLTU5HmNoxdyvF/IhiSRg==</latexit><latexit sha1_base64="PnBHKpcTUoEuzEcRLV+OijS07Uo=">AAAHp3ichZVLb9NAEMenLeBSHm3hgsSlIgqCS7QJSCBOLVCpVauSB0lTNVXlNZvWxC/ZTtIkyheAK4gDJ5A4ID4GF74Ah34ExLFIXDgwXjvKY6etrcS7M7//eGZ3vcs9ywxCxo6npmcuXLykzV6eu3L12vX5hcUblcBt+oYoG67l+lWuB8IyHVEOzdASVc8Xus0tsc0bzyL/dkv4gek6L8OOJ/Zs/cAx66ahh2gqvN5fSLEMk9eS2sgmjRQkV95dnFmDGrwCFwxogg0CHAixbYEOAd67kAUGHtr2oIc2H1um9Avowxxqm0gJJHS0NvD/AHu7idXBfhQzkGoD32Lhz0flEqTZT/aVnbAf7Bv7xf6dGqsnY0S5dPDJY63w9uff3Cr9PVdl4zOEw6HqDAVH+uyaQqjDY1mLibV50hJVacTxW90PJ6UnxXTvLvvMfmN9n9gx+44VOq0/xpeCKH7E6Gk53gIj1aCFTwNSOMY97A1y5ejrSWs/yWfIjytc2ffxNjHHQ1TrsudC+9QIR8S7jgiuQ3AdgusSXJfg2gTXJrg6wdUJrkFwDYJbIbgVglsnuHWC2yC4DYKrElyV4HYIbofgGMExhePJ/EZrmZpXnsxr7Ffnk8svYeAXhN/GVT8karKvUh7maI5xsYUiA4UMSDLKLVrho+zAptLRKA34UcXQ3idUltw3OFomdaMeSplXKs6fUnFJfq32RCVDq6rYHMtqqNmcyGlUtZXkb6DPwt1pSyFWJ1YUx/6qQhUJqqhQz+Vuk4KcpOu4B0V7Zzybg1MjziPelWhPLoqLZ1h28sRSG5VcJvsgkys8TC0/TU6zWbgNd+Aexn8Ey7CG41/GHAS8hXfwXruvvdAqWjVGp6cSzU0YuzT9P3Gjjr0=</latexit>j···<latexit sha1_base64="TrCYO938AN9SldJ7ZwZrTCx2d5w=">AAAHrHichZXNbtNAEMenLZBSPtrCBYlLRRTEKdqkSCBOLVCpVasqTZo0VRJVtrtuTfwl20maRHkBTtwQcAKJA+IxuPACHPoIiGORuHBgvHbkJDspthLvzvz+45nd9a7qmoYfMHY+Mzt35eq11Pz1hRs3b91eXFq+U/GdlqfxsuaYjldVFZ+bhs3LgRGYvOp6XLFUkx+ozReh/6DNPd9w7P2g6/KGpZzYhm5oSoCmSl07dgL/aCnNskxcK3IjFzfSEF8FZ3luE+pwDA5o0AILONgQYNsEBXy8a5ADBi7aGtBHm4ctQ/g5DGABtS2kOBIKWpv4f4K9Wmy1sR/G9IVaw7eY+PNQuQIZ9oN9YRfsO/vKfrK/U2P1RYwwly4+1UjL3aPF1/dKf/6rsvAZwGmiukShIn15TQHo8FTUYmBtrrCEVWpR/Hbv7UXpWTHTf8g+sV9Y30d2zr5hhXb7t/Z5jxc/YPSMGG+OkerQxqcGaRzjPvaGuaro6wvrIM4n4ccVjuh7eBuY4ymqFdFzoDM1whnxrjOC6xJcl+B6BNcjuA7BdQhOJzid4JoE1yS4dYJbJ7gtgtsiuG2C2ya4KsFVCe6Q4A4JjhEckzg1nt9wLVPzqsbzGvnl+VTFlzD0c8Jv4apPiLroy5SLORpjXGShSF8ifZIMcwtX+Cg7tMl0OEpDflSR2AeEyhT7hoqWSd2oh1IWpIoLUyouia/VmqgkscqKnbGsEs3ORE6jqt04fw19Ju5OuxKxMbGiVOxvSFSRoIoS9VLsNmnIC1rHPSjcO6PZHJ4aUR7RrkR78mFcPMNykyeW3Kjks7nVbH7vcXrteXyazcN9eACPMP4TWINNHP8y5vAK3sA7eJ/KpvZTtVQjQmdnYs1dGLtS+j9AmZD+</latexit>

vjk<latexit sha1_base64="VyogNH+mkG17LDChTI0pUzKy71c=">AAAHrHichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJk2aKokq2123bvwl20maWHkBTtwQcAKJA+IxuPACHPoIiGORuHBgvHbkJJ4UW4l3Z37/8czueldxDN3zGTufmZ27cvWaNH994cbNW7cXl5bvVD277aq8otqG7dYU2eOGbvGKr/sGrzkul03F4PtK62Xo3+9w19Nta8/vObxpyseWrumq7KOp2jkMTluDw6UMyzFxraQb+biRgfgq2stzG9CAI7BBhTaYwMECH9sGyODhXYc8MHDQ1oQAbS62dOHnMIAF1LaR4kjIaG3h/zH26rHVwn4Y0xNqFd9i4M9F5Qpk2Q/2hV2w7+wr+8n+To0ViBhhLj18KpGWO4eLr++V//xXZeLTh5NEdYlCQfrymnzQ4JmoRcfaHGEJq1Sj+J3+24vy81I2eMg+sV9Y30d2zr5hhVbnt/p5l5c+YPSsGG+OkRrQwacKGRzjAHvDXBX0BcI6iPNJ+HGFLfou3jrmeIJqWfRs6E6NcEa864zgegTXI7g+wfUJrktwXYLTCE4juBbBtQhujeDWCG6T4DYJbovgtgiuRnA1gjsguAOCYwTHUpwSz2+4lql5VeJ5jfzp+VTElzD0c8Jv4qpPiIbopykHc9THuMhCkV6K9EgyzC1c4aPs0Jamw1Ea8qOKxD4gVIbYNxS0TOpGPZSymKq4OKXisvhazYlKEmtasT2WVaLZnshpVLUT56+iz8DdaSdFrE+sKAX76ymqRFClFPVK7DYZKAhawz0o3Duj2RyeGlEe0a5EewphXDzD8pMnVrpRLeTyj3OF3SeZ1RfxaTYP9+EBPML4T2EVNnD8K5jDKbyBd/Beykl7Ul1qRujsTKy5C2OXpP0DaweRJw==</latexit>k<latexit sha1_base64="i4PsFJxuscb3On436ezMeEzl0eo=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1YlHyRN1VSV12xaK45t2U7SJMoLwBXEgRNIHBCPwYUX4NBHQByLxIUD47UjJ9lJsZV4d+b3H8/srne5a5l+wNjpzOzchYuXtPnLC1euXru+uLR8o+I7Lc8QZcOxHK/KdV9Ypi3KgRlYoup6Qm9yS+zyxrPQv9sWnm869sug64qDpn5km3XT0AM0FRqHSymWYfJaURvZuJGC+Mo7y3MbUINX4IABLWiCABsCbFugg4/3PmSBgYu2A+ijzcOWKf0CBrCA2hZSAgkdrQ38P8Lefmy1sR/G9KXawLdY+PNQuQJp9oN9YWfsO/vKfrK/U2P1ZYwwly4+eaQV7uHi61ulP/9VNfEZwHGiOkfBkT6/pgDq8FjWYmJtrrSEVRpR/Hbv/VnpSTHdv8s+sV9Y30d2yr5hhXb7t/G5IIofMHpajrfASDVo49OAFI5xH3vDXDn6+tI6iPNJ+HGFI/se3ibmeIxqXfYc6EyNcEK864TgugTXJbgewfUIrkNwHYKrE1yd4BoE1yC4NYJbI7hNgtskuC2C2yK4KsFVCW6P4PYIjhEcUzgez2+4lql55fG8Rn51Prn8EoZ+QfibuOoToib7KuVijuYYF1ko0ldInyTD3MIVPsoObSodjtKQH1Uk9gGhsuS+wdEyqRv1UMq8UnF+SsUl+bU2JypJrKpieyyrRLM9kdOoaifO30CfhbvTjkKsT6wojv11hSoSVFGhnsvdJgU5SddxDwr3zmg2h6dGlEe0K9GeXBgXz7Ds5ImlNiq5TPZBJld4mFp9Gp9m83Ab7sA9jP8IVmEDx7+MOQh4A2/hnXZfe6FVtGqEzs7Empswdmn6P3jmjr4=</latexit>

xn<latexit sha1_base64="f8SHKOPxCgw9+pknyUjVbpReW2I=">AAAHq3ichZXNbtNAEMenLeBSPtrCBYlLRRTEhWgTkECcWqBSq1ZVmpA2pakir9m0Vhzbsp1v5QW4cATECSQOiMfgwgtw6CMgjkXiwoHx2pGT7KTYSrw78/uPZ3bXu9y1TD9g7HRmdu7CxUva/OWFK1evXV9cWr6x5ztNzxAlw7Ecr8x1X1imLUqBGVii7HpCb3BL7PP6s9C/3xKebzr2i6DriqOGfmybNdPQAzSVKq1O1a4upViGyWtFbWTjRgriK+8sz21ABV6BAwY0oQECbAiwbYEOPt6HkAUGLtqOoI82D1um9AsYwAJqm0gJJHS01vH/GHuHsdXGfhjTl2oD32Lhz0PlCqTZD/aFnbHv7Cv7yf5OjdWXMcJcuvjkkVa41cXXt4p//qtq4DOAk0R1joIjfX5NAdTgsazFxNpcaQmrNKL4rd7bs+KTQrp/l31iv7C+j+yUfcMK7dZv4/OuKHzA6Gk53gIjVaCFTwNSOMZ97A1z5ejrS+sgzifhxxWO7Ht4m5jjCap12XOgPTVCh3hXh+C6BNcluB7B9QiuTXBtgqsRXI3g6gRXJ7g1glsjuE2C2yS4LYLbIrgywZUJ7oDgDgiOERxTOB7Pb7iWqXnl8bxGfnU+ufwShn5B+Bu46hOiIvsq5WKO5hgXWSjSV0ifJMPcwhU+yg5tKh2O0pAfVST2AaGy5L7B0TKpG/VQyrxScX5KxUX5tTYmKkmsqmJ7LKtEsz2R06hqJ87fQJ+Fu9OOQqxPrCiO/XWFKhBUQaGey90mBTlJ13APCvfOaDaHp0aUR7Qr0Z5cGBfPsOzkiaU29nKZ7INMbvdhavVpfJrNw224A/cw/iNYhQ0c/xLmYMIbeAfvtftaUXupVSJ0dibW3ISxSxP/AExmkJI=</latexit>

xnk<latexit sha1_base64="6WHVIAH7ZCstTetqTAVqZsuoqmI=">AAAHrHichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJk2aKokq26xbE3/Jdj6tvAAnbgg4gcQB8RhceAEOfQTEsUhcODBeO3IST4qtxLszv/94Zne9qziG7vmMnc3NL1y6fEVavLp07fqNm8srq7eqnt12VV5RbcN2a4rscUO3eMXXfYPXHJfLpmLwQ6X1PPQfdrjr6bZ14Pcd3jTlE0vXdFX20VTtHQdWa3i8kmE5Jq61dCMfNzIQX0V7dWELGvASbFChDSZwsMDHtgEyeHjXIQ8MHLQ1IUCbiy1d+DkMYQm1baQ4EjJaW/h/gr16bLWwH8b0hFrFtxj4c1G5Bln2g31h5+w7+8p+sr8zYwUiRphLH59KpOXO8fLrO+U//1WZ+PThNFFdoFCQvrgmHzR4ImrRsTZHWMIq1Sh+Z/D2vPy0lA3us0/sF9b3kZ2xb1ih1fmtft7npQ8YPSvGm2OkBnTwqUIGxzjA3ihXBX2BsA7jfBJ+UmGLvou3jjmeoloWPRu6MyP0iHf1CK5PcH2CGxDcgOC6BNclOI3gNIJrEVyL4DYIboPgtglum+B2CG6H4GoEVyO4I4I7IjhGcCzFKfH8hmuZmlclntfIn55PRXwJIz8n/Cau+oRoiH6acjBHfYKLLBTppUiPJMPcwhU+zo5saTocpRE/rkjsQ0JliH1DQcu0btxDKYupioszKi6Lr9WcqiSxphW7E1klmt2pnMZVe3H+KvoM3J32UsTm1IpSsL+ZokoEVUpRL8Ruk4GCoDXcg8K9M5rN0akR5RHtSrSnEMbFMyw/fWKlG9VCLv8wV9h/lFl/Fp9mi3AX7sEDjP8Y1mELx7+CObyCN/AO3ks56UCqS80InZ+LNbdh4pK0f5arkS0=</latexit>

は、H ! 1 で平均0のガウス分布に収束
l 共分散 V [yn ym ] = E[yn ym ] の計算も同様
à y = (y1 , y2 , . . . , yN ) は多変量ガウス分布に収束<latexit sha1_base64="VIxxuRi7NT2e/bvUPasnJ4hXpwo=">AAAHuHichZXNbtNAEMenLeBSPprCBYlLShTEKdoEJFBPLVApVasqTUibqq6CbdapFX/JdpImUV6AF+AAF5CKhHgMLrwAhz4C4lgkLhwYrx3lw5NiK/HuzO8/ntld76quafgBY+dz8wtXrl6TFq8v3bh56/ZyauXOvu+0PI1XNcd0vJqq+Nw0bF4NjMDkNdfjiqWa/EBtvgj9B23u+YZjvwq6Lj+2lIZt6IamBGiqp1LFtLwqB468mpYNWw+69VSG5Zi40slGPm5kIL5KzspCEWR4Aw5o0AILONgQYNsEBXy8jyAPDFy0HUMfbR62DOHnMIAl1LaQ4kgoaG3ifwN7R7HVxn4Y0xdqDd9i4s9DZRqy7Af7wi7Yd/aV/WR/Z8bqixhhLl18qpGWu/Xlt/cqf/6rsvAZwMlIdYlCRfrymgLQ4ZmoxcDaXGEJq9Si+O3eu4vKWjnbf8g+sV9Y30d2zr5hhXb7t3a2x8vvMXpWjDfHSDK08alBBse4j71hrir6+sI6iPMZ8ZMKR/Q9vA3M8QTViug50JkZ4ZR41ynBdQmuS3A9gusRXIfgOgSnE5xOcE2CaxLcBsFtENwWwW0R3DbBbRNcjeBqBHdIcIcExwiOJTg1nt9wLVPzqsbzGvmT86mKL2Ho54TfwlU/ImTRT1Iu5mhMcJGFIv0E6ZNkmFu4wsfZoS1Jh6M05McVI/uAUJli31DRMq0b91DKUqLi0oyKK+JrtaYqGVmTip2JrEaanamcxlW7cf4a+kzcnXYTxObUilKxv5mgygRVTlAvxW6TgYKgddyDwr0zms3hqRHlEe1KtKcQxsUzLD99YiUb+4Vc/nGusPcks/48Ps0W4T48gEcY/ymsQxHHv4o5tOEDnMFnaU16LTUkI0Ln52LNXZi4JO8fkiuUYQ==</latexit><latexit sha1_base64="SeDyNf2bCmx4VxiOUrmq7REqnyw=">AAAHwXichZXNbtNAEMenLeBSPprCBcQlJQriVG0CEggJqaVEatWqShOSpkqiyDab1oq/ZDtJkyjizgtw4ARSDwgJHoILL8Chj4A4FokLB8ZrR07iSbGVeHfm9x/P7K53FVvXXI+xs7n5hUuXr0iLV5euXb9xczmxcqvsWm1H5SXV0i2nosgu1zWTlzzN03nFdrhsKDo/UFqbvv+gwx1Xs8xXXs/mdUM+MrWmpsoemhqJu+Vqr2H2GkY9WVt9XltN5kb9RiLF1pi4kvFGJmykILzy1srCFtTgNVigQhsM4GCCh20dZHDxrkIGGNhoq8MAbQ62NOHnMIQl1LaR4kjIaG3h/xH2qqHVxL4f0xVqFd+i489BZRLS7Af7xM7Zd/aZ/WR/Z8YaiBh+Lj18KoGW243lt3eKf/6rMvDpwXGkukChIH1xTR404amoRcPabGHxq1SD+J3+u/Pis0J68IB9ZL+wvg/sjH3DCs3Ob/V0nxfeY/S0GG+OkWrQwacKKRzjAfZGuSroGwjrMMwn4icVlug7eGuY4zGqZdGzoDszwgnxrhOC6xFcj+D6BNcnuC7BdQmuSXBNgmsRXIvgNghug+C2CW6b4HYIbofgKgRXIbhDgjskOEZwLMYp4fz6a5maVyWc18Afn09FfAkjPyf8Bq76iKiJfpyyMUdtggssFOnGSJck/dz8FT7Ojmxx2h+lET+uiOxDQqWLfUNBy7Ru3EMp87GK8zMqLoqv1ZiqJLLGFbsTWUWa3amcxlV7Yf4q+nTcnfZiRG5qRSnYz8WoAkEVYtRLsdukICvoJu5B/t4ZzObo1AjyCHYl2pP14+IZlpk+seKNcnYt82gtu/84tf4iPM0W4R7ch4cY/wmswxaOfwlzeAOn8AW+SpuSJtmSE6Dzc6HmNkxc0uAfbyeYDA==</latexit><latexit sha1_base64="nQdqgGOpJhxyqpUxBGi90Ss49bM=">AAAHw3ichZXNbtNAEMenLeBSPprCBamXliioSFW0CUggJKQWiNSqVZQmpE1VV5HtblordmzZTlInyoErL8CBE0gIIS7wDFx4AQ59BMSxSFw4MF47yocnxVbi3Znffzyzu95VbUN3PcbOpqZnLl2+Is1enbt2/cbN+cTCrV3XajoaL2uWYTkVVXG5oTd42dM9g1dshyumavA9tf488O+1uOPqVuOl59v80FSOG3pN1xQPTdXEotzy5eWn8vKKX82s+tXsqnxkeS628veriSRLM3EtxRuZqJGE6CpYCzMbIMMRWKBBE0zg0AAP2wYo4OJ9ABlgYKPtELpoc7ClCz+HHsyhtokUR0JBax3/j7F3EFkb2A9iukKt4VsM/DmoXIIU+8E+sXP2nX1mP9nfibG6IkaQi49PNdRyuzr/+k7pz39VJj49OBmoLlCoSF9ckwc1eCxq0bE2W1iCKrUwfqvz5rz0pJjq3mPv2S+s7x07Y9+wwkbrt/ZhhxffYvSUGG+OkWRo4VODJI5xF3v9XFX0dYW1F+Uz4EcVlug7eOuY4wmqFdGzoD0xwinxrlOC8wnOJ7gOwXUIrk1wbYKrEVyN4OoEVye4dYJbJ7hNgtskuC2C2yK4CsFVCG6f4PYJjhEci3FqNL/BWqbmVY3mNfTH51MVX0Lfzwm/iat+QMiiH6dszFEf4UILRbox0iXJILdghQ+zfVucDkapzw8rBvYeoTLEvqGiZVw37KGUhVjFhQkVl8TXao5VMrDGFdsjWQ0022M5DavyUf4a+gzcnfIxIje2olTs52JUkaCKMeqF2G2SkBV0DfegYO8MZ7N/aoR5hLsS7ckGcfEMy4yfWPHGbjadeZDO7jxMrj2LTrNZWIS7sILxH8EabOD4lzGHV/ARvsBXKSfVJUfyQnR6KtLchpFL6v0DM5WYLg==</latexit>– 詳しい計算は、Neal (1996)を参照のこと
95/124 Summer School 数理物理 2021 Lecture2

ニューラルネットとガウス過程 (4)l 事前分布からランダムに生成したニューラルネット

について、入力 (x1 , x2 ) = ( 0.2, 0.4) に対する出力(y1 , y2 ) をプロット
y2 15
y2 15
<latexit sha1_base64="4g1wu9DOVIOn3QUzfGN31a/Unzw=">AAAHfHichZXNbtNAEMenLTSlfDSFSyUuLaFVKtpoYypASEgtUImqVZUmpE3VVJHtboIVx7ZsJ82HwgPwAhw4geCAELwEF16AQx8BcSwSF4QYrx05iafFUbK7M7//ZGZ3vatYuua4jJ2MjI5duDgem7g0efnK1WtT8enru45Zt1WeV03dtAuK7HBdM3je1VydFyybyzVF53tK9Ynn32tw29FM47nbsvhhTa4YWllTZRdNpfhMsllKLzVL0mJx7lFxLrnMUtISS60sluIJlmLimY120kEnAcGTMafHFqAIR2CCCnWoAQcDXOzrIIODnwNIAwMLbYfQQZuNPU34OXRhErV1pDgSMlqr+FvB0UFgNXDsxXSEWsV/0fFro3IW5tl39pGdsm/sE/vB/pwZqyNieLm0sFV8LbdKU69mcr//q6ph68KLUHWOQkH6/JpcKMMDUYuGtVnC4lWp+vEb7denuYfZ+c4Ce8d+Yn1v2Qn7ihUajV/qhx2efSOiH6GmjG0DWxUSOMMdHPUyVdDXEdZukE3INwmySXAtgmsRXJvg2gR3THDHBFcmuDLBVQmuSnBrBLdGcBsEt0FwmwS3SXAFgisQ3D7B7RMcIzgW4ZRgfb19SK2rEqyr74+upyJ2cc/PCX8Nd2xIFMU4SlmYozbA+RaKdCKkQ5Jebi7W38/2bFHam6Ue368I7V1CpYt3XkHLsK7fQykzkYozZ1ScQ2tFrGQ/HVqjiq2BrELN1lBO/artIH8VfTqeLNsRYn1oRyk4Xo9QWYLKRqin4rRJgCToMp7P3rnnr2bvxPfz8E8l2iN5cfH+SQ/fNtHOrpRK301JOyuJ1cfBTTQBN+EWJDH+fViFZzj/eczhJbyHz/Bl/G/sduxObNlHR0cCzQ0YeGL3/gHBFYfB</latexit><latexit sha1_base64="63X19ZaHK9udL9Vd/RPcT+N6Dw4=">AAAHa3ichZXPTxNBFMcfoBTxByAXox6ITQkmpplWE40nUEkkEFJaCyVAmt1ltm66v7K7LWyb/gNePXhQD5p4MP4ZXvwHPPAnGOMJEy8efDu7zbadB27Tzsx7n+/rezOzM6prGn7A2MnY+MSFi5OZqUvTl69cvTYzO3d923dansarmmM6Xk1VfG4aNq8GRmDymutxxVJNvqM2n0b+nTb3fMOxXwShyw8spWEbuqEpAZp2l8J64V5YL96tz2ZZnolnQe4Ukk4WkqfkzE0swj4cggMatMACDjYE2DdBAR8/e1AABi7aDqCLNg97hvBz6ME0altIcSQUtDbxt4GjvcRq4ziK6Qu1hv9i4tdD5QLk2Hf2mZ2yb+wL+8H+nhmrK2JEuYTYqrGWu/WZVzcqf/6rsrAN4GWqOkehIn1+TQHo8EjUYmBtrrBEVWpx/HbnzWnlcTnXXWQf2U+s7wM7YV+xQrv9W/u0xctvRfRD1OjYtrHVIIsz3MVRP1MVfV1h7SXZpPwxQR4TXEhwIcF1CK5DcEcEd0RwOsHpBNckuCbBrRDcCsGtEdwawa0T3DrB1QiuRnC7BLdLcIzgmMSpyfpG+5BaVzVZ19gvr6cqdnHfzwm/hTs2JfbFWKZczNEY4mILRfoS6ZNklFuA9Q+yfZtMR7PU5wcVqb1HqEzxzqtoGdUNeihlSaq4dEbFFbQ2xEoO0qlVVmwMZZVqNkZyGlRtJvlr6DPxZNmUiNWRHaXieFWiygRVlqhn4rTJQlHQOp7P0bkXr2b/xI/ziE8l2lOM4uL9Uxi9beTOdjFfuJ8vbj3ILj9JbqIpuAV3YAnjP4RleI7zX8UcLHgN7+D95K/MfOZm5naMjo8lmnkYejK5f96Gg/k=</latexit><latexit sha1_base64="zrBSiDb8zxRR5mW8U9uZKF37rqU=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5oq8ppNasWOLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPstNhKvDvz+49ndte73LVMP2DsdGZ27tLlK9r81YVr12/cXFxavrXrO23PEGXDsRyvwnVfWGZLlAMzsETF9YRuc0vs8eaL0L/XEZ5vOq1XQc8Vh7beaJl109ADNJV6tVxtKcUyTF4raiMbN1IQX3lneW4DqvAGHDCgDTYIaEGAbQt08PE+gCwwcNF2CAO0edgypV/AEBZQ20ZKIKGjtYn/DewdxNYW9sOYvlQb+BYLfx4qVyDNfrAv7Ix9Z1/ZT/b33FgDGSPMpYdPHmmFW1t8e6f0578qG58BHCWqCxQc6YtrCqAOT2UtJtbmSktYpRHF7/RPzkrPiunBffaJ/cL6PrJT9g0rbHV+G58LovgBo6fleAuMVIUOPg1I4RgPsDfKlaNvIK3DOJ+En1Q4su/hbWKOR6jWZc+B7rkRjol3HRNcj+B6BNcnuD7BdQmuS3B1gqsTXJPgmgS3RnBrBLdJcJsEt0VwWwRXIbgKwe0T3D7BMYJjCsfj+Q3XMjWvPJ7XyK/OJ5dfwsgvCL+Nqz4hqrKvUi7maE5wkYUifYX0STLMLVzh4+zIptLhKI34cUViHxIqS+4bHC3TunEPpcwrFefPqbgkv1Z7qpLEqiq2J7JKNNtTOY2rduL8DfRZuDvtKMT61Iri2F9XqCJBFRXqpdxtUpCTdB33oHDvjGZzdGpEeUS7Eu3JhXHxDMtOn1hqYzeXyT7K5AqPU6vP49NsHu7CPXiA8Z/AKmzg+Jcxhwa8g/dwoj3UClpFex2hszOx5jZMXJrxD1Jnj3E=</latexit><latexit sha1_base64="zrBSiDb8zxRR5mW8U9uZKF37rqU=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5oq8ppNasWOLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPstNhKvDvz+49ndte73LVMP2DsdGZ27tLlK9r81YVr12/cXFxavrXrO23PEGXDsRyvwnVfWGZLlAMzsETF9YRuc0vs8eaL0L/XEZ5vOq1XQc8Vh7beaJl109ADNJV6tVxtKcUyTF4raiMbN1IQX3lneW4DqvAGHDCgDTYIaEGAbQt08PE+gCwwcNF2CAO0edgypV/AEBZQ20ZKIKGjtYn/DewdxNYW9sOYvlQb+BYLfx4qVyDNfrAv7Ix9Z1/ZT/b33FgDGSPMpYdPHmmFW1t8e6f0578qG58BHCWqCxQc6YtrCqAOT2UtJtbmSktYpRHF7/RPzkrPiunBffaJ/cL6PrJT9g0rbHV+G58LovgBo6fleAuMVIUOPg1I4RgPsDfKlaNvIK3DOJ+En1Q4su/hbWKOR6jWZc+B7rkRjol3HRNcj+B6BNcnuD7BdQmuS3B1gqsTXJPgmgS3RnBrBLdJcJsEt0VwWwRXIbgKwe0T3D7BMYJjCsfj+Q3XMjWvPJ7XyK/OJ5dfwsgvCL+Nqz4hqrKvUi7maE5wkYUifYX0STLMLVzh4+zIptLhKI34cUViHxIqS+4bHC3TunEPpcwrFefPqbgkv1Z7qpLEqiq2J7JKNNtTOY2rduL8DfRZuDvtKMT61Iri2F9XqCJBFRXqpdxtUpCTdB33oHDvjGZzdGpEeUS7Eu3JhXHxDMtOn1hqYzeXyT7K5AqPU6vP49NsHu7CPXiA8Z/AKmzg+Jcxhwa8g/dwoj3UClpFex2hszOx5jZMXJrxD1Jnj3E=</latexit>

10

10

5

5

0

0

-5

-5

-10

-10

-15
-15

-10

-5

0

5

10

H=1の場合96/124 Summer School 数理物理 2021 Lecture2

15

y1<latexit sha1_base64="LplDJDBX/79ZxxV6sBREkaOwObQ=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5oq8ppNasWOLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPstNhKvDvz+49ndte73LVMP2DsdGZ27tLlK9r81YVr12/cXFxavrXrO23PEGXDsRyvwnVfWGZLlAMzsETF9YRuc0vs8eaL0L/XEZ5vOq1XQc8Vh7beaJl109ADNJV6tWxtKcUyTF4raiMbN1IQX3lneW4DqvAGHDCgDTYIaEGAbQt08PE+gCwwcNF2CAO0edgypV/AEBZQ20ZKIKGjtYn/DewdxNYW9sOYvlQb+BYLfx4qVyDNfrAv7Ix9Z1/ZT/b33FgDGSPMpYdPHmmFW1t8e6f0578qG58BHCWqCxQc6YtrCqAOT2UtJtbmSktYpRHF7/RPzkrPiunBffaJ/cL6PrJT9g0rbHV+G58LovgBo6fleAuMVIUOPg1I4RgPsDfKlaNvIK3DOJ+En1Q4su/hbWKOR6jWZc+B7rkRjol3HRNcj+B6BNcnuD7BdQmuS3B1gqsTXJPgmgS3RnBrBLdJcJsEt0VwWwRXIbgKwe0T3D7BMYJjCsfj+Q3XMjWvPJ7XyK/OJ5dfwsgvCL+Nqz4hqrKvUi7maE5wkYUifYX0STLMLVzh4+zIptLhKI34cUViHxIqS+4bHC3TunEPpcwrFefPqbgkv1Z7qpLEqiq2J7JKNNtTOY2rduL8DfRZuDvtKMT61Iri2F9XqCJBFRXqpdxtUpCTdB33oHDvjGZzdGpEeUS7Eu3JhXHxDMtOn1hqYzeXyT7K5AqPU6vP49NsHu7CPXiA8Z/AKmzg+Jcxhwa8g/dwoj3UClpFex2hszOx5jZMXJrxD0skj3A=</latexit>

-15
-15

-10

-5

0

5

H=2の場合1015y1<latexit sha1_base64="LplDJDBX/79ZxxV6sBREkaOwObQ=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5oq8ppNasWOLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPstNhKvDvz+49ndte73LVMP2DsdGZ27tLlK9r81YVr12/cXFxavrXrO23PEGXDsRyvwnVfWGZLlAMzsETF9YRuc0vs8eaL0L/XEZ5vOq1XQc8Vh7beaJl109ADNJV6tWxtKcUyTF4raiMbN1IQX3lneW4DqvAGHDCgDTYIaEGAbQt08PE+gCwwcNF2CAO0edgypV/AEBZQ20ZKIKGjtYn/DewdxNYW9sOYvlQb+BYLfx4qVyDNfrAv7Ix9Z1/ZT/b33FgDGSPMpYdPHmmFW1t8e6f0578qG58BHCWqCxQc6YtrCqAOT2UtJtbmSktYpRHF7/RPzkrPiunBffaJ/cL6PrJT9g0rbHV+G58LovgBo6fleAuMVIUOPg1I4RgPsDfKlaNvIK3DOJ+En1Q4su/hbWKOR6jWZc+B7rkRjol3HRNcj+B6BNcnuD7BdQmuS3B1gqsTXJPgmgS3RnBrBLdJcJsEt0VwWwRXIbgKwe0T3D7BMYJjCsfj+Q3XMjWvPJ7XyK/OJ5dfwsgvCL+Nqz4hqrKvUi7maE5wkYUifYX0STLMLVzh4+zIptLhKI34cUViHxIqS+4bHC3TunEPpcwrFefPqbgkv1Z7qpLEqiq2J7JKNNtTOY2rduL8DfRZuDvtKMT61Iri2F9XqCJBFRXqpdxtUpCTdB33oHDvjGZzdGpEeUS7Eu3JhXHxDMtOn1hqYzeXyT7K5AqPU6vP49NsHu7CPXiA8Z/AKmzg+Jcxhwa8g/dwoj3UClpFex2hszOx5jZMXJrxD0skj3A=</latexit>

ニューラルネットとガウス過程 (5)
y2 15

y215

10

10

5

5

0

0

-5

-5

-10

-10

<latexit sha1_base64="zrBSiDb8zxRR5mW8U9uZKF37rqU=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5oq8ppNasWOLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPstNhKvDvz+49ndte73LVMP2DsdGZ27tLlK9r81YVr12/cXFxavrXrO23PEGXDsRyvwnVfWGZLlAMzsETF9YRuc0vs8eaL0L/XEZ5vOq1XQc8Vh7beaJl109ADNJV6tVxtKcUyTF4raiMbN1IQX3lneW4DqvAGHDCgDTYIaEGAbQt08PE+gCwwcNF2CAO0edgypV/AEBZQ20ZKIKGjtYn/DewdxNYW9sOYvlQb+BYLfx4qVyDNfrAv7Ix9Z1/ZT/b33FgDGSPMpYdPHmmFW1t8e6f0578qG58BHCWqCxQc6YtrCqAOT2UtJtbmSktYpRHF7/RPzkrPiunBffaJ/cL6PrJT9g0rbHV+G58LovgBo6fleAuMVIUOPg1I4RgPsDfKlaNvIK3DOJ+En1Q4su/hbWKOR6jWZc+B7rkRjol3HRNcj+B6BNcnuD7BdQmuS3B1gqsTXJPgmgS3RnBrBLdJcJsEt0VwWwRXIbgKwe0T3D7BMYJjCsfj+Q3XMjWvPJ7XyK/OJ5dfwsgvCL+Nqz4hqrKvUi7maE5wkYUifYX0STLMLVzh4+zIptLhKI34cUViHxIqS+4bHC3TunEPpcwrFefPqbgkv1Z7qpLEqiq2J7JKNNtTOY2rduL8DfRZuDvtKMT61Iri2F9XqCJBFRXqpdxtUpCTdB33oHDvjGZzdGpEeUS7Eu3JhXHxDMtOn1hqYzeXyT7K5AqPU6vP49NsHu7CPXiA8Z/AKmzg+Jcxhwa8g/dwoj3UClpFex2hszOx5jZMXJrxD1Jnj3E=</latexit><latexit sha1_base64="zrBSiDb8zxRR5mW8U9uZKF37rqU=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5oq8ppNasWOLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPstNhKvDvz+49ndte73LVMP2DsdGZ27tLlK9r81YVr12/cXFxavrXrO23PEGXDsRyvwnVfWGZLlAMzsETF9YRuc0vs8eaL0L/XEZ5vOq1XQc8Vh7beaJl109ADNJV6tVxtKcUyTF4raiMbN1IQX3lneW4DqvAGHDCgDTYIaEGAbQt08PE+gCwwcNF2CAO0edgypV/AEBZQ20ZKIKGjtYn/DewdxNYW9sOYvlQb+BYLfx4qVyDNfrAv7Ix9Z1/ZT/b33FgDGSPMpYdPHmmFW1t8e6f0578qG58BHCWqCxQc6YtrCqAOT2UtJtbmSktYpRHF7/RPzkrPiunBffaJ/cL6PrJT9g0rbHV+G58LovgBo6fleAuMVIUOPg1I4RgPsDfKlaNvIK3DOJ+En1Q4su/hbWKOR6jWZc+B7rkRjol3HRNcj+B6BNcnuD7BdQmuS3B1gqsTXJPgmgS3RnBrBLdJcJsEt0VwWwRXIbgKwe0T3D7BMYJjCsfj+Q3XMjWvPJ7XyK/OJ5dfwsgvCL+Nqz4hqrKvUi7maE5wkYUifYX0STLMLVzh4+zIptLhKI34cUViHxIqS+4bHC3TunEPpcwrFefPqbgkv1Z7qpLEqiq2J7JKNNtTOY2rduL8DfRZuDvtKMT61Iri2F9XqCJBFRXqpdxtUpCTdB33oHDvjGZzdGpEeUS7Eu3JhXHxDMtOn1hqYzeXyT7K5AqPU6vP49NsHu7CPXiA8Z/AKmzg+Jcxhwa8g/dwoj3UClpFex2hszOx5jZMXJrxD1Jnj3E=</latexit>

-15
-15

-10

-5

0

5

H=10の場合

1015

y1<latexit sha1_base64="LplDJDBX/79ZxxV6sBREkaOwObQ=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5oq8ppNasWOLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPstNhKvDvz+49ndte73LVMP2DsdGZ27tLlK9r81YVr12/cXFxavrXrO23PEGXDsRyvwnVfWGZLlAMzsETF9YRuc0vs8eaL0L/XEZ5vOq1XQc8Vh7beaJl109ADNJV6tWxtKcUyTF4raiMbN1IQX3lneW4DqvAGHDCgDTYIaEGAbQt08PE+gCwwcNF2CAO0edgypV/AEBZQ20ZKIKGjtYn/DewdxNYW9sOYvlQb+BYLfx4qVyDNfrAv7Ix9Z1/ZT/b33FgDGSPMpYdPHmmFW1t8e6f0578qG58BHCWqCxQc6YtrCqAOT2UtJtbmSktYpRHF7/RPzkrPiunBffaJ/cL6PrJT9g0rbHV+G58LovgBo6fleAuMVIUOPg1I4RgPsDfKlaNvIK3DOJ+En1Q4su/hbWKOR6jWZc+B7rkRjol3HRNcj+B6BNcnuD7BdQmuS3B1gqsTXJPgmgS3RnBrBLdJcJsEt0VwWwRXIbgKwe0T3D7BMYJjCsfj+Q3XMjWvPJ7XyK/OJ5dfwsgvCL+Nqz4hqrKvUi7maE5wkYUifYX0STLMLVzh4+zIptLhKI34cUViHxIqS+4bHC3TunEPpcwrFefPqbgkv1Z7qpLEqiq2J7JKNNtTOY2rduL8DfRZuDvtKMT61Iri2F9XqCJBFRXqpdxtUpCTdB33oHDvjGZzdGpEeUS7Eu3JhXHxDMtOn1hqYzeXyT7K5AqPU6vP49NsHu7CPXiA8Z/AKmzg+Jcxhwa8g/dwoj3UClpFex2hszOx5jZMXJrxD0skj3A=</latexit>

-15
-15

-10

-5

0

5

10

H=100の場合

l (y1,y2)の同時分布は多変量ガウス分布に漸近する！

– ノード数H=10ですでにほぼガウス分布と等価– 中心極限定理の効果
97/124 Summer School 数理物理 2021 Lecture2

15

y1<latexit sha1_base64="LplDJDBX/79ZxxV6sBREkaOwObQ=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatWqygdpU5oq8ppNasWOLdtJmkR5ASSu9MAJJA6Ix+DCC3DoIyCOReLCgfHakZPstNhKvDvz+49ndte73LVMP2DsdGZ27tLlK9r81YVr12/cXFxavrXrO23PEGXDsRyvwnVfWGZLlAMzsETF9YRuc0vs8eaL0L/XEZ5vOq1XQc8Vh7beaJl109ADNJV6tWxtKcUyTF4raiMbN1IQX3lneW4DqvAGHDCgDTYIaEGAbQt08PE+gCwwcNF2CAO0edgypV/AEBZQ20ZKIKGjtYn/DewdxNYW9sOYvlQb+BYLfx4qVyDNfrAv7Ix9Z1/ZT/b33FgDGSPMpYdPHmmFW1t8e6f0578qG58BHCWqCxQc6YtrCqAOT2UtJtbmSktYpRHF7/RPzkrPiunBffaJ/cL6PrJT9g0rbHV+G58LovgBo6fleAuMVIUOPg1I4RgPsDfKlaNvIK3DOJ+En1Q4su/hbWKOR6jWZc+B7rkRjol3HRNcj+B6BNcnuD7BdQmuS3B1gqsTXJPgmgS3RnBrBLdJcJsEt0VwWwRXIbgKwe0T3D7BMYJjCsfj+Q3XMjWvPJ7XyK/OJ5dfwsgvCL+Nqz4hqrKvUi7maE5wkYUifYX0STLMLVzh4+zIptLhKI34cUViHxIqS+4bHC3TunEPpcwrFefPqbgkv1Z7qpLEqiq2J7JKNNtTOY2rduL8DfRZuDvtKMT61Iri2F9XqCJBFRXqpdxtUpCTdB33oHDvjGZzdGpEeUS7Eu3JhXHxDMtOn1hqYzeXyT7K5AqPU6vP49NsHu7CPXiA8Z/AKmzg+Jcxhwa8g/dwoj3UClpFex2hszOx5jZMXJrxD0skj3A=</latexit>ガウス過程による教師なし学習

98 / 8098

ガウス過程と教師なし学習l ガウス過程回帰モデルでは、入力xと出力yのペア(x, y)が与えられていた
l 観測値 y しかない場合はどうする?
l 非常によくある設定 (教師なし学習)– y＝あるユーザーのクリック履歴– y＝ロボットの姿勢ベクトル (各関節角のベクトル)– y＝ある星の吸収線スペクトル– ここでは、yが連続値の場合を考える

99/124 Summer School 数理物理 2021 Lecture2

確率的主成分分析
低次元

xn

W 高次元

yn

l Probabilistic PCA (Tipping & Bishop 1999)⇢

yn = Wxn + ✏✏ ⇠ N(0, 2 I)l よって、L =PN

PN

n=1 log p(yn ) =n=1 log N (yn |Wxn ,N=
log 2⇡ + log |C| + tr(C 1 S)
2
(C = WWT + 2 I, S = YYT /N )<latexit sha1_base64="6s9MZNRYX6OTA38+uzzRmxHkKNk=">AAAIlnichZVLb9NAEMcnLY8QHm3hUomLRdQqVWnZBCQQUlFLqGjVKmqbPtVtIttsUqt+yXbTJq65wIkvwIETSBwQH4MLX4BDPwJC4lIkLhwYrx2SxtviPLw78/vPzuyu14qta65HyHGqr//CxUuX01cyV69dvzEwOHRz3bX2HZWtqZZuOZuK7DJdM9map3k627QdJhuKzjaUvWLo32gwx9Usc9Vr2mzHkOumVtNU2UNTdfDnYmZ0inrs0HO9ps4k6u4bVd+cygeVkkR1qy7ZOdpoVs0xaSqTdPpUlXWpFETMETVkb1ep+RsBbRxWzbvU1eqGXCnQxvxYhlIcSpqgNUdW/VLgFwKqs5qX44EK1NakcR70qHiELZ86huQ5Qa5Y8SfyQXmMOlp91+NhckUJI3UN9q9VWR3vGvPuS6ksIdjYwm9l9V5prDqYJZOEX1KykY8bWYivJWuofw4ovAALVNgHAxiY4GFbBxlc/GxDHgjYaNsBH20OtjTuZxBABrX7SDEkZLTu4X8de9ux1cR+GNPlahVH0fHnoFKCEfKNfCIn5Cv5TL6TP2fG8nmMMJcm3pVIy+zqwJvh8u//qgy8e7DbUZ2jUJA+vyYPavCI16JhbTa3hFWqUfxG6+1J+fHKiD9KPpAfWN97cky+YIVm45f6cZmtvMPoI3y+GUai0MC7ClmcYx977VwV9PncGsT5dPjTCov3HfxomOMuqmXes+DgzAiHgrEOBVxTwDUFXEvAtQTcgYA7EHA1AVcTcHsCbk/AzQi4GQE3L+DmBdyCgFsQcJsCblPAbQm4LQFHBBxJcEq8vuFeFq2rEq9r5E+up8KfhLafCfwG7voOQXk/SdmYo3aKiywi0k2QrpAMcwt3eDfbtiXpcJbafLeiYw8EKp2fGwpaenXdHpFyKVHx0hkVl/nTavRU0rEmFYunsupoFnty6laV4vxV9Ol4OpUSxGzPjlKwP5ugVgTUSoJ6xk+bLBQ4XcMzKDw7o9VsvzWiPKJTSewphHHxHZbvfWMlG+uFyfz9ycLyg+z00/htlobbcAdyGP8hTMMczv8aqKnVVCv1KvU6PZx+kp5NP4/QvlSsuQWnrvTSX0VT4BQ=</latexit>

100/124 Summer School 数理物理 2021 Lecture22I)確率的主成分分析 (2)
低次元

xn

W

高次元

yn

l @L/@W = 0 より、データの尤度Lを最大にするWの最尤推定値はW = Uq (⇤q

2

I)1/2

– ⇤q , Uq： YYT の最大q個の固有値・固有ベクトルを並べた行列– 2 = 0 で通常の主成分分析と一致

101/124 Summer School 数理物理 2021 Lecture2

確率的PCAからGPLVMへ
l 確率的PCA：xàyへの射影行列Wを最適化

xn

W

yn

l Wは巨大、xの次元に依存する

à Wの方に事前分布を与えて積分消去
p(W) =

DY

d=1

p(Y|X) =Z

N(wd |0, ↵ 1 I)

p(Y|X, W)p(W)dW
✓
◆
1
1
1
T
=
exp
tr(K
YY
)
DN/2
D/2
2
(2⇡)
|K|

102/124 Summer School 数理物理 2021 Lecture2

GPLVMl よって、

DN
log p(Y|X) =log(2⇡)
2
KX = ↵XXT + 1 I

D
log |KX |2

1
tr(KX1 YYT )2

X = (x1 , · · · , xN )l これを最大化する潜在的なXを見つければよい
l Gaussian Process Latent Variable Model(GPLVM)という (Lawrence+, NIPS 2003)

103/124 Summer School 数理物理 2021 Lecture2

GPLVMのイメージ

l xが1次元の場合: 各観測値 (×,△,○)の背後にxが存在
104/124 Summer School 数理物理 2021 Lecture2

GPLVMの最適化
DN
log p(Y|X) =log(2⇡)
2
KX = ↵XXT + 1 I

D
log |KX |2

1
tr(KX1 YYT )2

X = (x1 , · · · , xN )l 自然にカーネル化されているà任意のカーネルを導入

k(xn , xm ) = ↵ exp|xn xm |2 + (n, m)@L= KX1 YYT KX1 D KX1l@KX@L@L @KX=–を適用して微分@xnj @Kx @xnj
105/124 Summer School 数理物理 2021 Lecture2

1

(RBF)

GPLVMの最適化 (2)l Python実装：『ガウス過程と機械学習』サポートページ– http://chasen.org/~daiti-m/gpbook/l Neil Lawrence によるMATLAB原実装– http://inverseprobability.com/gplvm/

106/124 Summer School 数理物理 2021 Lecture2

GPLVM：計算例
l Oil flowデータ (PRML掲載と同じもの)
0.3
2.4
0.2

0.2

2.2
0.15
2

0.1

0.1
1.8
0.05

0

1.6
0

−0.1

1.4
−0.05
1.2

−0.2

−0.1
1
−0.15

−0.3

0.8

−0.2

−0.4

0.6

−0.25

−0.5
−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

−0.2

−0.1

0

0.1

0.2

0.3

0.4

Figure 1: Visualisation of the Oil data with (a) PCA (a linear GPLVM) and (b) A GPLVM whichuses an RBF kernel. Crosses, circles and plus signs represent stratifi ed, annular and homogeneousflows respectively. The greyscales in plot (b) indicate the precision with which the manifold was
expressed in data-space for that latent point. The optimised parameters of the kernel were
,
and
.

– 左:
GPLVM
2.2 A確率的PCA、右:Practical Algorithm for GPLVMs– GPLVMは分散を使ってconfidenceの分布も得られる
There are three main components to our revised, computationally efficient, optimisation

process:
107/124 Summer School 数理物理 2021 Lecture2
Sparsification. Kernel methods may be sped up through sparsification, i.e. representing

(Hertzmann 2004, SIGGRAPH)

Style-based Inverse Kinematics
l 低次元(たとえば2次元)の潜在空間から、人間の関節角ベクトルへの写像をGPLVMで学習– 観測ベクトルを並べた行列Yから、潜在座標Xを下の式で最適化

l プロジェクトWebページ：

https://grail.cs.washington.edu/projects/styleik/

108/124 Summer School 数理物理 2021 Lecture2

GPLVM: 3次元の場合
l 松浦さん“Statmodelingmemorandum”
http://statmodeling.hatena
blog.com/entry/gaussianprocess-latent-variablemodel-2 によるl Stan言語による推定

109/124 Summer School 数理物理 2021 Lecture2

ガウス過程の様々な分野への
応用

110 /
80

ガウス過程とロボティクス(IROS 2019)(「システム/制御/情報」2016)

111/124 Summer School 数理物理 2021 Lecture2

ガウス過程とロボティクス (1)l ガウス過程状態空間モデル

xt+1 = f (xt , ut ) + w,
<latexit sha1_base64="es684xJQlpPhQbda6XaZNrVyKX8=">AAAHknichZXNbtNAEMenLZBSPtICByQuFlFRq6JqE5BAlZAaSiWqVlWb0DZVXUVed51asRMTO1+1/AK8AAdOIHFAvAVceAEOfQMQJygSFw6M146SxtOyUby7M7//ZMa72eWOZboeY8cjo2MXLl5KjV+euHL12vX05NSNbbfebOhiS69b9UaJa66wzJrY8kzPEiWnITSbW2KHV5dC/05LNFyzXnvhdR2xb2uVmmmYuuahqTypqq1O2ffmsoHyRDFmwpl3X1FtzTvkht8Myt6sMqeorTYaXza1gwkcKqpr2oqv6pqlrAczPZgFyHDbV4tmxdaCcnu2PJlh80w2JTnIxoMMxG2jPjWWBhUOoA46NMEGATXwcGyBBi5+9iALDBy07YOPtgaOTOkXEMAEaptICSQ0tFbxWcHZXmyt4TyM6Uq1jr9i4beBSgWm2Vf2gZ2wL+wj+87+nhnLlzHCXLrY80grnHL61e3in/+qbOw9OOyrzlFwpM+vyQMDHstaTKzNkZawSj2K3zp6fVJcKEz799g79gPre8uO2WessNb6rb/fFIU3MvoBagzsW9jrkME37OOslylHny+tQZxNn+8QZIfgugTXJbgjgjsiuDbBtQnOIDiD4KoEVyW4PMHlCW6F4FYIbpXgVgmuRHAlgtsluF2CYwTHEhyP1zfch9S68nhdI39yPbncxT2/IPw27tg+ocp5knIwR/MUF1ko0k2QLkmGdQt8akgN8n17QKgs+S/maBnWDXoo5Uaiho0zaiiitSLXZpDuW5OKtVNZ9TVrQzkNqtbj/HX0WXhWrCeI5aE9wnG+nKAKBFVIUM/k+ZGBnKQNPHHDkyxan94ZHuURnTO0JxfGxRslO3x/JAfbufnsg/nc5sPM4tP4bhmHO3AXZjD+I1iE5/j+tzCHT/ANfsKv1K3UQiqfWorQ0ZFYcxNOtdTaP0wbmgA=</latexit>– x：状態ベクトルt– ut：制御入力– w：システムノイズw ⇠ N (0, ⌃w )<latexit sha1_base64="du4X50jkOitaT/waNe2SS/5JNUc=">AAAHOXichZVLb9NAEMenLbilPNLCBYlLRVTEKdoEJCpOLVCJqlWVJqRN1VSR110HK45t2U6ah/IFuHAExAkkDoiPwJELX4BDj9x4HIPEhQPjtSMn8bTdKN7dmd9/MuPd7HLHNDyfsZOp6ZkLF5XZuUvzl69cvZZaWLy+69lNVxMlzTZtt8xVT5iGJUq+4Zui7LhCbXBT7PH648C/1xKuZ9jWM7/jiMOGWrMM3dBUH02lSqtd9asLaZZhsi0lB9lokIao5e3FmRRU4Ahs0KAJDRBggY9jE1Tw8HMAWWDgoO0QemhzcWRIv4A+zKO2iZRAQkVrHZ81nB1EVgvnQUxPqjX8FRO/LiqXYJl9Yx/ZgH1ln9hP9u/UWD0ZI8ilgz0PtcKppl7cLP49V9XA3ofnseoMBUf67Jp80GFF1mJgbY60BFVqYfxW99Wg+LCw3LvD3rPfWN87dsK+YIVW64/2YUcU3sroR6jRsW9hr0Ea33APZ8NMOfp60tqPson5NkG2Ca5DcB2C6xJcl+COCe6Y4HSC0wmuTnB1glsjuDWC2yC4DYLbJLhNgisTXJng9glun+AYwbEEx6P1DfYhta48WtfQn1xPLnfx0C8IfwN3bExU5DxJOZijMcaFFor0EqRHkkHdAp8qUqN8bO8TKlP+izlaJnWjHkqZT9SQP6WGIlprcm1G6diaVGyNZRVrtiZyGlVtR/lr6DPxrNhOEOsTe4TjfD1BFQiqkKCeyPMjDTlJ63jiBidZuD7DMzzMIzxnaE8uiIs3Snby/kgOdnOZ7L1Mbud+evVRdLfMwS24DXcx/gNYhaf4/kuYgwEv4TW8UT4r35Ufyq8QnZ6KNDdgrCmD/9oLfVs=</latexit><latexit sha1_base64="SCr2vuTKYLF8B5uuRbsbn9m9qSs=">AAAHQHichZXNbtNAEMenLZhSPtLCBYlLRVTEKdoEJBCnFqhE1apKE9KmaqrI665TK3Zs2U7axMoL8ABw4MCHxAHxFnDhBTj0DUAcECoSEuLAeO3ISTxtN4p3d+b3n8x4N7vcMQ3PZ+xoYnLq3HnlwvTFmUuXr1zNzM5d2/TstquJimabtlvlqidMoyUqvuGbouq4QrW4KbZ483Ho3+oI1zPs1jO/64hdS220DN3QVB9NtZql+vtcD9r9ul+fzbIck20+PcjHgyzErWjPTWWgBntggwZtsEBAC3wcm6CCh58dyAMDB227EKDNxZEh/QL6MIPaNlICCRWtTXw2cLYTW1s4D2N6Uq3hr5j4dVE5DwvsK/vAjtkX9pF9Z/9OjBXIGGEuXex5pBVOPfP8RvnPmSoLex/2E9UpCo706TX5oMMDWYuBtTnSElapRfE7vZfH5YelheA2e8d+YH1v2RH7jBW2Or+19xui9EpG30ONjn0Hew2y+IYDnA0y5egLpLUfZ5PwhwR5SHBdgusSXI/gegR3QHAHBKcTnE5wTYJrEtwSwS0R3ArBrRDcKsGtElyV4KoEt01w2wTHCI6lOB6vb7gPqXXl8bpG/vR6crmLB35B+C3csQlRk/M05WCOxggXWSjSS5EeSYZ1C3yqSA3zib1PqEz5L+ZoGdcNeyhlMVVD8YQaymhtyLUZphNrWrE2klWiWRvLaVi1Huevoc/Es2I9RSyP7RGO8+UUVSKoUop6Is+PLBQkreOJG55k0foMzvAoj+icoT2FMC7eKPnx+yM92Czk8ndzhY172cVH8d0yDTfhFtzB+PdhEZ7i+69gDg68gNfwRvmkfFN+Kr8idHIi1lyHkab8/Q8ms4CS</latexit><latexit sha1_base64="Me59KkzDSkCQx4uJ/HgsElmx9vM=">AAAHPnichZXPaxNBFMdfW11r/ZFWL4KXYqh4CpMoKJ5atWBpKWli2tSmlJ3tbLpkd7PsbpImS/4Bz4IHUVDwIP4Tghf/AQ/9BwQR8VBBDx58O7thk+xrnZCdmfc+35f3diYz3DENz2fsaGJy6sxZ5dz0+ZkLFy9dzszOXdn0mi1XExWtaTbdKlc9YRq2qPiGb4qq4wrV4qbY4o2HoX+rLVzPaNpP/K4jdi21bhu6oak+mp7WLNU/4HrQ6e/NZlmOyTafHuTjQRbiVmzOTWWgBvvQBA1aYIEAG3wcm6CCh58dyAMDB227EKDNxZEh/QL6MIPaFlICCRWtDXzWcbYTW22chzE9qdbwV0z8uqichwX2hb1nx+wz+8C+sb8nxgpkjDCXLvY80gpnL/PsWvn3f1UW9j4cJKpTFBzp02vyQYd7shYDa3OkJaxSi+K3ey+Oy/dLC8FN9pZ9x/resCP2CSu027+0dxui9FJG30eNjn0bew2y+IYDnA0y5egLpLUfZ5PwhwR5SHBdgusSXI/gegTXIbgOwekEpxNcg+AaBLdEcEsEt0JwKwS3SnCrBFcluCrBbRPcNsExgmMpjsfrG+5Dal15vK6RP72eXO7igV8Qfgt3bELU5DxNOZijMcJFFor0UqRHkmHdAp8qUsN8Yu8TKlP+izlaxnXDHkpZTNVQPKGGMlrrcm2G6cSaVqyNZJVo1sZyGlatx/lr6DPxrFhPEctje4TjfDlFlQiqlKIeyfMjCwVJ63jihidZtD6DMzzKIzpnaE8hjIs3Sn78/kgPNgu5/O1cYeNOdvFBfLdMw3W4Abcw/l1YhMf4/iuYgw3P4RW8Vj4qX5Ufys8InZyINVdhpCl//gE4XX+t</latexit>

l 状態が、各次元ごとにガウス過程に従って時間発展すると考えているà (x1 , x2 , · · · , xT ) のp次元目だけを取ったベクトル(x1p , x2p , · · · , xT p )<latexit sha1_base64="0KUEPhydB6kAnkzjc6dc/9MO1XM=">AAAHUHichZXPaxNBFMdfW01q/JFUL6KXYqhUKGESBcVTqxYsLSVNmjalKWFnMxuXbHaX7CbNDwKe/Qc8eFJQEA/6P3jxLHjonyAeK+hB0LezGzbNvtYNmR/vfb4v7+1MZrht6I7L2NHU9My587H47IXExUuXryRTc1d3HKvdUkVJtQyrVeaKIwzdFCVXdw1RtltCaXJD7PLGY8+/2xEtR7fMbbdni4OmUjd1TVcVF03V1I3FSqdbzS55bW6potYs15GT7TvVVJplmHzmo4NsMEhD8OStuZkkVKAGFqjQhiYIMMHFsQEKOPjZhywwsNF2AAO0tXCkS7+AISRQ20ZKIKGgtYFtHWf7gdXEuRfTkWoVf8XAbwuV87DAvrH37Jh9YR/Yd/bn1FgDGcPLpYc997XCriZfXC/++q+qib0Lz0LVGQqO9Nk1uaDBA1mLjrXZ0uJVqfrxO/2Xx8WHhYXBbfaG/cD6XrMj9hkrNDs/1bdbovBKRq+hRsO+g70KaXzDA5yNMuXoG0jrMMgm5LsE2SW4HsH1CK5PcH2COyS4Q4LTCE4juAbBNQhuheBWCG6N4NYIbp3g1gmuTHBlgtsjuD2CYwTHIhwP1tfbh9S68mBdfX90PbncxSO/IPxN3LEhUZHzKGVjjvoJzrdQpBMhHZL06hbYKkiN86F9SKgM+S/maJnUjXsoZT5SQ/6UGoporcu1GadDa1SxcSKrULMxkdO4ajPIX0WfgWfFZoRYndgjHOerEapAUIUI9USeH2nISVrDE9c7yfz1GZ3hfh7+OUN7cl5cvFGyk/dHdLCTy2TvZnJb99LLj4K7ZRZuwi1YxPj3YRme4vsvYQ7P4R18hE+xr7Hfsb/xKR+dDnq4BieeeOIfrRKDQw==</latexit><latexit sha1_base64="QvkFaxv/lGDOex9Hgrauj2J+Ohk=">AAAHU3ichZXBbtNAEIanaUlKoSSFCxJCqoiKioSiTUACcWqBSlStqjRp2lRtFdnOOlhxbMt20iRWTtx4AQ6cQOJQcYNH4MIL9NBHQByLxAEOjNeOnMTTslG8uzPfP5nxbnZlS9ccl7GzqcT0zJVkavbq3LXr8zfSmYWbu47ZthVeUUzdtKuy5HBdM3jF1VydVy2bSy1Z53ty84Xv3+tw29FMY8ftWfyoJTUMTdUUyUVTLXN3uVvz8tbgIXYF7A6Vuuk6/mzHGjyoZbIsx0RbjA/y4SALYSuaC9NpOIQ6mKBAG1rAwQAXxzpI4ODnAPLAwELbEXhos3GkCT+HAcyhto0UR0JCaxOfDZwdhFYD535MR6gV/BUdvzYqF2GJnbITds6+s8/sB/t7YSxPxPBz6WEvB1pu1dJvb5d//1fVwt6F15HqEoWM9OU1uaDCU1GLhrVZwuJXqQTxO/135+VnpSXvPvvIfmJ9H9gZ+4YVGp1fyqdtXnovotdRo2LfwV6BLL5hD2fDTGX0ecI6CLOJ+C5BdgmuR3A9gusTXJ/gjgnumOBUglMJrklwTYJbJbhVglsnuHWC2yC4DYKrElyV4PYJbp/gGMGxGCeH6+vvQ2pd5XBdA398PWWxi4d+TvhbuGMj4lDM45SFOWpjXGChSCdGOiTp183xKSE1ykf2AaHSxb9YRsukbtRDKYuxGooX1FBGa0OszSgdWeOKzbGsIs3mRE6jqq0wfwV9Op4VWzFibWKPyDhfi1ElgirFqJfi/MhCQdAqnrj+SRasz/AMD/IIzhnaU/Dj4o2Sn7w/4oPdQi7/KFfYfpxdeR7eLbNwB+7BMsZ/AivwCt9/BXN4AyfwBb4mT5N/UonUTIAmpkLNLRhrqfl/K4WEJA==</latexit>

がガウス過程に従う

112/124 Summer School 数理物理 2021 Lecture2

ガウス過程とロボティクス (2)l ロボティクスにおいては、将来の状態の分散が重要(最適な経路を一つだけ出しても意味がない！)
l 強化学習とガウス過程を組み合わせた船舶の制御
113/124 Summer School 数理物理 2021 Lecture2

ガウス過程行列分解(ICML 2009)

114/124 Summer School 数理物理 2021 Lecture2

ガウス過程行列分解
l 協調フィルタリングなどで、観測されたデータ(例えばユーザがサイトを見た回数)は行列Yとして
表現できる
l 例：
ユーザー (N次元)

Web
サイト
(D次元)

0
B
B
B
B
@

Y
<latexit sha1_base64="Pz12kUJWoII+pq2g7SiC1KC5muQ=">AAAHP3ichZXPaxNBFMdfW401/kirF8FLMVQ8hUkUFE+tWrC0lDQxbUJSys52Nl2yv9jdpE2W/APexYOgKHgQ/wrx4j/goX+BiCBCBRE8+HZ2Q37sazshOzPvfb4v7+1MZrhj6J7P2NHU9My586kLsxfTly5fuZqZm7+25dltVxUV1TZst8oVTxi6JSq+7hui6rhCMbkhtnnrcejf7gjX023rmd91xI6pNC1d01XFR1O9YSr+PteCWj+9O5dlOSbbQnKQjwdZiFvRnp/JQAP2wAYV2mCCAAt8HBuggIefOuSBgYO2HQjQ5uJIl34BfUijto2UQEJBawufTZzVY6uF8zCmJ9Uq/oqBXxeVC7DIvrIP7Jh9YR/Zd/bvxFiBjBHm0sWeR1rh7Gae3yj/OVNlYu/D/lB1ioIjfXpNPmjwQNaiY22OtIRVqlH8Tu/lcflhaTG4zd6xH1jfW3bEPmOFVue3+n5TlF7J6Huo0bDvYK9CFt9wgLNBphx9gbT242yG/CFBHhJcl+C6BNcjuB7BHRDcAcFpBKcRXIvgWgS3THDLBLdKcKsEt0ZwawRXJbgqwdUIrkZwjOBYguPx+ob7kFpXHq9r5E+uJ5e7eOAXhN/EHTskGnKepBzMUR/jIgtFegnSI8mwboFPBalRfmjvEypD/os5WiZ1ox5KWUzUUDyhhjJam3JtRumhNalYH8tqqFmfyGlUtRHnr6LPwLNiI0GsTOwRjvOVBFUiqFKCeiLPjywUJK3hiRueZNH6DM7wKI/onKE9hTAu3ij5yfsjOdgq5PJ3c4XNe9mlR/HdMgs34Rbcwfj3YQme4vuvyBvgBbyGN6lPqW+pn6lfETo9FWuuw1hL/f0P3j5/ow==</latexit><latexit sha1_base64="aHPA7SqKo0KiuBlVAUSfnPsY0d0=">AAAHdnichZXNbtNAEMenLTTFfCSFCxISqogKnKpN+BSnFKhE1apqE9qmaqrK666DFX/JdtKkVl6AF+DACSSQEI/BhQPXIvUREMciwYED47Ujp/G0bBTv7szvP5nxbna5axp+wNjR2PjEufOTuakLysVLl6/kC9NXN3yn7WliXXNMx6tz1RemYYv1wAhMUXc9oVrcFJu89Szyb3aE5xuO/TLouWLHUpu2oRuaGqBpt1BRGlw0DTt0LTXwjG5faXhtU4QPhNUPmRsM5tEwfIhGpSHsvZRWdgtFNsdkm8kOSsmgCElbdaYn8tCAPXBAgzZYIMCGAMcmqODjZxtKwMBF2w6EaPNwZEi/gD4oqG0jJZBQ0drCZxNn24nVxnkU05dqDX/FxK+HyhmYZYfsEztmX9ln9oP9PTVWKGNEufSw57FWuLv519drv/+rsrAP4FWqOkPBkT67pgB0eCxrMbA2V1qiKrU4fufgzXHtSXU2vM3es59Y3zt2xL5ghXbnl/ZhTVTfyuh7qNGx72CvQRHfcIizQaYcfaG09pNsUr5LkF2C6xFcj+AOCO6A4PYJbp/gdILTCa5FcC2Cmye4eYJbJLhFglsiuCWCqxNcneC2CG6L4BjBsQzHk/WN9iG1rjxZ19ifXU8ud/HALwi/hTs2JRpynqVczNE4wcUWivQzpE+SUd0CnypSw3xq7xMqU/6LOVpGdcMeSrmaqWH1lBpqaG3KtRmmU2tWsXwiq1SzPJLTsGolyV9Dn4lnxUqGWBjZIxznCxmqSlDVDPVcnh9FKEtaxxM3Osni9Rmc4XEe8TlDe8pRXLxRSqP3R3awUZ4r3Zsrr90vVp4md8sU3IBbcBfjP4IKvMD3v445fIRvcAjfJ//kbuZmc3didHws0VyDEy3H/gFlrpIq</latexit>

115/124 Summer School 数理物理 2021 Lecture2

1
C
C
C
C
A

ydn：ユーザーnが<latexit sha1_base64="dqP6aOd7vlAGAQuumMYz8F9kfNY=">AAAHOnichZW/b9NAFMdfWzCl/EgLCxJLRVTEFF0CEoipBSpRtarShKSp2qryOedg4tiW7aRNrPwDTGwImEBiQPwLbCz8Awxd2aqOrcTCwPPZkZP4tb0ovrv3Pt+X93yXO+6YhuczdjgxOXXpsnJl+urMtes3bmZm525VPbvtaqKi2abt1rjqCdOwRMU3fFPUHFeoLW6KTd58Hvo3O8L1DNt65XcdsdtSG5ahG5rqo6na3QvqVn9vNstyTLb59CAfD7IQt6I9N5WBHaiDDRq0oQUCLPBxbIIKHn62IQ8MHLTtQoA2F0eG9Avowwxq20gJJFS0NvHZwNl2bLVwHsb0pFrDXzHx66JyHhbYb/aNnbBf7Ds7Yv/OjBXIGGEuXex5pBXOXubtnfLfC1Ut7H14najOUXCkz6/JBx2eyFoMrM2RlrBKLYrf6b0/KT8tLQT32Rd2jPV9ZofsJ1ZodU61rxui9ElGr6NGx76DvQZZfMMBzgaZcvQF0tqPs0n4A4I8ILguwXUJrkdwPYLbJ7h9gtMJTie4JsE1CW6J4JYIboXgVghuleBWCa5GcDWC2yK4LYJjBMdSHI/XN9yH1LryeF0jf3o9udzFA78g/C3csQmxI+dpysEcjREuslCklyI9kgzrFvhUkRrmE3ufUJnyX8zRMq4b9lDKYqqG4hk1lNHakGszTCfWtGJtJKtEszaW07BqPc5fQ5+JZ8V6ilge2yMc58spqkRQpRT1Qp4fWShIWscTNzzJovUZnOFRHtE5Q3sKYVy8UfLj90d6UC3k8g9zhY1H2cVn8d0yDXfhHjzA+I9hEV7i+69gDm/gHXyAj8oP5Y9ypBxH6ORErLkNI005/Q+RdH3q</latexit>

サイトdを見た
回数 (例えば)ガウス過程行列分解 (2)l Yは多くの場合高次元かつ非常にスパースなので、低次元の基底Wと負荷量Xの積として表現
0
1
0

B
B
B
B
@

Y

<latexit sha1_base64="NyjJpibpWp1R/z24b755D/S1kwc=">AAAIR3ichZVdT9NgFMdPQR3OF0BvTLwhDgwmsDwboMYrUEkkEAJDYGSdpC1PR7O2a9pubDT9An4BL4wXmqgxfgxv9AN4wUcwXniBUS+88PRl7KUH7LL2ec75/f87p0/3VLZ0zXEZOxIGBs+dv5Aaupi+dPnK1eGR0WtbTq1uK3xTqek1uyhLDtc1k2+6mqvzomVzyZB1vi1XHwX57Qa3Ha1mPnVbFi8bUsXUVE2RXAztjgojoswrmulZhuTaWtNPi4ZU5XKtWZrjRrmklD3Rruu8ND2DU49Zru/d5YY/jpi7L6vejj+OGm7udTk4mpH2REXSx1Z9UeeqO5kWVbT0Trzzvd757NyJ+1x2ptt/G/3Rcqrfoae6SJkLXPyJtGhLmhNAHsvOYsQ7MSsGZoHbFNZYMaRn+XZmCaO2Vtl37+yOZFiWhcdYcpCLBxmIj7Xa6OAwiLAHNVCgDgZwMMHFsQ4SOPgpQQ4YWBgrg4cxG0damOfgQxq1daQ4EhJGq3iu4KwUR02cB55OqFbwV3T82qgcgwn2lX1gx+wz+8i+sb+nenmhR1BLC69ypOXW7vDzGxu//6sy8OrCfkd1hkJG+uyeXFDhftiLhr1ZYSToUon8G4cvjjceFCa82+wN+479vWZH7BN2aDZ+Km/XeeFl6L6HGhWvDbwqkME77OGsXamMOS+M+nE1Hb5JkE2CaxFci+AOCe6Q4A4I7oDgVIJTCa5KcFWCWyC4BYJbIrglglsmuGWCKxJckeB2CG6H4BjBsQQnx+sbPIfUusrxukb55HrK4VPcznMib+AT2yHEcJ6kLKxR6+GiCEU6CdIhyaBvjmcJqW6+E/cJlR7+i2WM9Ou6M5RyLdHD2ik9bGC0Eq5NN92JJhUrPVV1NCt9NXWrVuP6FczpuFesJojFvmdExvligioQVCFBPQ73jwzkQ1rFHTfYyaL1ae/hUR3RPkNn8oEvvlFy/e+P5GArn83NZPPrs5n5h/G7ZQhuwi2YRP97MA9P8P5vgiIcCK+Ed8L71JfUj9Sv1J8IHRBizXXoOYaEf35N3+8=</latexit>

C
B
C
B
C⇠NB W
C
B
A
@

X

1

C
C
2 C
, IC
A

l ここでWにガウス事前分布を与えて積分消去すると、上式はガウス過程として表現できる<latexit sha1_base64="CuH7NsT69l9ycjredvgUrF1O6qY=">AAAHeXichZXPbxJBFMdfWy21/mirFxMvKKlBY8hATTSeWmsTGxpCQVpIqWRnO0s37MKGXWhhs/+A/4AHT5p4aPwzvHjzZCJ/gvFYEw968O3sEij7WoewM/Pe5/t4b2eY4Zah2w5jg6npmUuXZ2NzV+avXrt+Y2Fx6eaO3eq0VVFSW0arXeaKLQy9KUqO7hiibLWFYnJD7PLGuu/f7Yq2rbear5yeJfZNpd7UNV1VHDTVFterpuIccs2tePGqrZtxt6oqRjznJYeOrFcbDsveIx+qm8rrzNC26T2oLSZYiskWjw7S4SABYcu3lmYWoAoH0AIVOmCCgCY4ODZAARs/e5AGBhba9sFFWxtHuvQL8GAetR2kBBIKWhv4rONsL7Q2ce7HtKVaxV8x8NtGZRyW2Td2wk7ZF/aJ/WB/z43lyhh+Lj3seaAVVm3hze3i7/+qTOwdOBypLlBwpC+uyQENnspadKzNkha/SjWI3+2/PS0+Kyy799kH9hPre88G7DNW2Oz+Uj9ui8I7Gf0ANRr2XexVSOAbdnE2zJSjz5VWL8xmxB8T5DHB9QiuR3B9gusT3BHBHRGcRnAawTUIrkFwawS3RnCbBLdJcFmCyxJcmeDKBFchuArBMYJjEY6H6+vvQ2pdebiugT+6nlzu4qFfEH4Td+yIqMp5lLIwR/0MF1go0o6QNkn6dQt8KkiN8yO7R6gM+S/maJnUjXsoZT5SQ/6cGoporcu1GadH1qhi60xWI83WRE7jqlyYv4o+A8+KXITYmNgjHOcbEapAUIUI9UKeHwnISFrDE9c/yYL1GZ7hQR7BOUN7Mn5cvFHSk/dHdLCTSaVXUpntx4nV5+HdMgd34B4kMf4TWIWX+P5LmMMJfIXvMJj9E7sbS8YeBuj0VKi5BWdabOUfV96TDA==</latexit>

Y ⇠ N (KX ,2I)

KX はXの列間のカーネル行列<latexit sha1_base64="1s1Q4Hh9G/Duf8gidp0dxM/deaA=">AAAHS3ichZXPaxNBFMdfW5vW+iOpIgheiqHiKUyioHhq1YIlpaSJaVPaEna2s+mS/UV2kzZZ8g/4D3jwVMGD9s/w4sWjh/4J4jGCID34dnbDJtnXOiE7M+99vi/v7UxmuGPorsfY+dT0zLXZ1Nz89YUbN2/dTmcW72y7druliqpqG3arxhVXGLolqp7uGaLmtIRickPs8OarwL/TES1Xt623XtcRB6bSsHRNVxUPTfXMvX1T8Y645hf79eGw1q9nsizHZFtKDvLRIAtRK9mLM2nYh0OwQYU2mCDAAg/HBijg4mcP8sDAQdsB+Ghr4UiXfgF9WEBtGymBhILWJj4bONuLrBbOg5iuVKv4KwZ+W6hcgmX2g31mA/aNnbGf7OLSWL6MEeTSxZ6HWuHU0+/uV/78V2Vi78FRrLpCwZG+uiYPNHgua9GxNkdagirVMH6n935QeVFe9h+xj+wX1nfKztlXrNDq/FY/bYnyBxn9EDUa9h3sVcjiG/ZxNsyUo8+X1n6UTcyfEOQJwXUJrktwPYLrEdwxwR0TnEZwGsE1Ca5JcKsEt0pw6wS3TnBFgisSXI3gagS3S3C7BMcIjiU4Hq1vsA+pdeXRuob+5HpyuYuHfkH4TdyxMbEv50nKwRz1MS60UKSbIF2SDOoW+FSQGuVje59QGfJfzNEyqRv1UMpSoobSJTVU0NqQazNKx9akYmMsq1izMZHTqGozyl9Fn4FnxWaCWJvYIxznawmqTFDlBPVanh9ZKEhawxM3OMnC9Rme4WEe4TlDewpBXLxR8pP3R3KwXcjln+QKW0+zKy+ju2UeHsBDeIzxn8EKvMH3X5U5nMIXOEt9Tw1Sf1MXITo9FWnuwlibm/0H1cmDng==</latexit>– 理論については、ガウス過程鹿本7章を参照のこと
116/124 Summer School 数理物理 2021 Lecture2

ガウス過程行列分解 (3)<latexit sha1_base64="CuH7NsT69l9ycjredvgUrF1O6qY=">AAAHeXichZXPbxJBFMdfWy21/mirFxMvKKlBY8hATTSeWmsTGxpCQVpIqWRnO0s37MKGXWhhs/+A/4AHT5p4aPwzvHjzZCJ/gvFYEw968O3sEij7WoewM/Pe5/t4b2eY4Zah2w5jg6npmUuXZ2NzV+avXrt+Y2Fx6eaO3eq0VVFSW0arXeaKLQy9KUqO7hiibLWFYnJD7PLGuu/f7Yq2rbear5yeJfZNpd7UNV1VHDTVFterpuIccs2tePGqrZtxt6oqRjznJYeOrFcbDsveIx+qm8rrzNC26T2oLSZYiskWjw7S4SABYcu3lmYWoAoH0AIVOmCCgCY4ODZAARs/e5AGBhba9sFFWxtHuvQL8GAetR2kBBIKWhv4rONsL7Q2ce7HtKVaxV8x8NtGZRyW2Td2wk7ZF/aJ/WB/z43lyhh+Lj3seaAVVm3hze3i7/+qTOwdOBypLlBwpC+uyQENnspadKzNkha/SjWI3+2/PS0+Kyy799kH9hPre88G7DNW2Oz+Uj9ui8I7Gf0ANRr2XexVSOAbdnE2zJSjz5VWL8xmxB8T5DHB9QiuR3B9gusT3BHBHRGcRnAawTUIrkFwawS3RnCbBLdJcFmCyxJcmeDKBFchuArBMYJjEY6H6+vvQ2pdebiugT+6nlzu4qFfEH4Td+yIqMp5lLIwR/0MF1go0o6QNkn6dQt8KkiN8yO7R6gM+S/maJnUjXsoZT5SQ/6cGoporcu1GadH1qhi60xWI83WRE7jqlyYv4o+A8+KXITYmNgjHOcbEapAUIUI9UKeHwnISFrDE9c/yYL1GZ7hQR7BOUN7Mn5cvFHSk/dHdLCTSaVXUpntx4nV5+HdMgd34B4kMf4TWIWX+P5LmMMJfIXvMJj9E7sbS8YeBuj0VKi5BWdabOUfV96TDA==</latexit>

Y ⇠ N (KX ,2

I)l 別の書き方をすると、以下の表現とも等価p(Y|X) =D Y
NY

d=1 n=1<latexit sha1_base64="wohGGqksQWAoBMe27z9wiOM43lU=">AAAHqHichZXfT9NQFMfPQC3OH4C+mPjSuECGMeRummhMSEAhSiDLGAJbKCxtdzuatV3TlsEo/Qf8B3zwSRMfjH+GL/4DJvInGB8x8cUYT38sG+sBuqz3nnM+37Nzeu9uFdvQXY+xk8zI6JWr14Sx69kbN2/dHp+YvLPptvcdlW+obaPtVBXZ5YZu8Q1P9wxetR0um4rBt5TWyzC+1eGOq7etN17X5jum3LR0TVdlD131CcfOS53asdSpzojTc1nJdtqNut+YKwS7iz3LCq1S1pdU2RBLQb6LgBUca/VGXjJlb0/R/MOgbs08EiVXb5ryblHqLM+IkpTVxGl0mSh1TPFVOciHv1OfyLFZFl1ielJIJjlIrnJ7cnQcJGhAG1TYBxM4WODh3AAZXPxsQwEY2OjbAR99Ds70KM4hgCxq95HiSMjobeG9idZ24rXQDnO6kVrFXzHw66BShCn2nX1mp+wb+8J+sr/n5vKjHGEtXRyVWMvt+vjbe+t/LlWZOHqw11ddoFCQvrgnDzR4FvWiY2925Am7VOP8naN3p+vPK1P+NPvIfmF/H9gJ+4odWp3f6qc1XnkfZW+gRsOxg6MKOXzCPlq9ShWM+ZE3SKrp84cEeUhwXYLrEtwRwR0R3AHBHRCcRnAawbUIrkVwCwS3QHDLBLdMcCsEt0JwVYKrElyN4GoExwiOpTglWd9wH1LrqiTrGsfT66lEu7gX50TcxB3bJ6TITlM21qif4WIPRbop0iXJsG+OdxmpQb7vDwiVEf2LFfQM6wYjlLKc6qF8Tg/r6G1GazNI971pxeqZqvqa1aGaBlWlpH4VYwaeFaUUsTS0RxS0l1JUhaAqKWoxOj9yUIxoDU/c8CSL16d3hsd1xOcMHSmGefGNUhh+f6Qnm8XZwuPZ4tqT3PyL5N0yBvfhAeQx/1OYh9f4/Dewhh/wLyNkxoSHQlnYEmoxOpJJNHfhzCUo/wHij5+j</latexit>

N (ydn |fd (xn ),2

I)f ⇠ GP(X)– 各ユーザーを表す潜在変数xn間のカーネル行列を使ったガウス過程fから、観測値yが生成– xnは未知なので、p(Y|X)を最大化するように最適化– ガウス過程行列分解は、非常に精度が高いことが知られている
GPLVMの一種

117/124 Summer School 数理物理 2021 Lecture2

空間統計学l 空間統計学においては、ガウス過程はクリギングとして知られる基本的な手法
l 狭義の空間統計以外にも、応用が広がっている

118/124 Summer School 数理物理 2021 Lecture2

空間統計学 (2)l ダイヤモンド鉱山での疎らな地形観測データからの復元

119/124 Summer School 数理物理 2021 Lecture2

空間統計学 (3)l 火星ランドローバーでの、地形ラベルの空間的推定“Curiosity”と火星の地形
120/124 Summer School 数理物理 2021 Lecture2

ガウス過程回帰で予測された地形ラベル (空間を考慮)

空間統計学 (4)l 生態学での生物種の観測データと、ガウス過程回帰

による空間的補間– Geothlypis philadelpha (ウグイスの一種)の分布
121/124 Summer School 数理物理 2021 Lecture2

空間統計学 (5)l BRT (Boosted Regression Tree)、GAM (Generalized

Additive Model)との密度推定結果の比較
l 論文からの図122/124 Summer School 数理物理 2021 Lecture2

まとめl ガウス過程：連続的に変化する関数を生成する

確率過程– ガウス過程回帰＝カーネル法に基づくベイズ的な非線形回帰モデル– 空間データやロボティクスなど、自然言語処理でもこれから連続値を扱う必要– 教師なし学習も含め、さまざまな応用
l 深層学習はノード数→∞でガウス過程に一致
3l 計算量はナイーブには O(N ) だが、様々な計算量削減法があり、実質 O(N 2 )l 詳しくは、『ガウス過程と機械学習』を参照のこと<latexit sha1_base64="mCUP+zb/4rZRvDkeKtjwvGqy814=">AAAHrHichZXNbtNAEMenLeBSPtrCBYlLRRRULtEmRQJxaoFKrVqFNGnSVEmpbLNuTfwl20maRHkBTtwQcAKJA+IxuPACHPoIiGORuHBgvHbkJJ60thLvzvz+45nd9a7iGLrnM3Y6NT1z6fIVafbq3LXrN27OLyzeqnh201V5WbUN260qsscN3eJlX/cNXnVcLpuKwfeUxrPAv9firqfb1q7fcfiBKR9Zuqarso+myovl/MuVB4cLKZZh4lpKNrJRIwXRVbAXZzagDq/ABhWaYAIHC3xsGyCDh3cNssDAQdsB9NDmYksXfg59mENtEymOhIzWBv4fYa8WWS3sBzE9oVbxLQb+XFQuQZr9ZF/ZGfvBvrFf7N/EWD0RI8ilg08l1HLncP7NndLfC1UmPn04jlXnKBSkz6/JBw0ei1p0rM0RlqBKNYzf6r47Kz0ppnv32Wf2G+v7xE7Zd6zQav1Rv+zw4keMnhbjzTFSHVr4VCGFY9zD3iBXBX09Ye1H+cT8qMIWfRdvHXM8RrUseja0J0Y4Id51QnAdgusQXJfgugTXJrg2wWkEpxFcg+AaBLdGcGsEt0lwmwS3RXBbBFcluCrB7RPcPsExgmMJTonmN1jL1Lwq0byG/uR8KuJLGPg54Tdx1cdEXfSTlIM56iNcaKFIL0F6JBnkFqzwYXZgS9LBKA34YUVs7xMqQ+wbClrGdcMeSllIVFyYUHFJfK3mWCWxNanYHskq1myP5TSsykf5q+gzcHfKJ4j1sRWlYH89QRUJqpignovdJgU5QWu4BwV7Zzibg1MjzCPclWhPLoiLZ1h2/MRKNiq5THYlk9t5mFp9Gp1ms3AX7sEyxn8Eq7CB41/GHF7DW3gPH6SMtCvVpIMQnZ6KNLdh5JK0/ybwkAQ=</latexit><latexit sha1_base64="9RZvK5sLEfyKilwG/A45nciJQk0=">AAAHrHichZXNbtNAEMenLeBSPtrCBYlLRRRULtEmIIE4tUClVq1CmjRpqiRUtlm3Jv6S7SRNorwAJ24IOIHEAfEYXHgBDn0ExLFIXDgwXjtyEk+KrcS7M7//eGZ3vas4hu75jJ3OzM5duHhJmr+8cOXqteuLS8s3Kp7dclVeVm3DdquK7HFDt3jZ132DVx2Xy6Zi8H2l+TTw77e56+m2ted3Hd4w5SNL13RV9tFUeb6af5G7d7iUYhkmrpVkIxs1UhBdBXt5bhPq8BJsUKEFJnCwwMe2ATJ4eNcgCwwctDWgjzYXW7rwcxjAAmpbSHEkZLQ28f8Ie7XIamE/iOkJtYpvMfDnonIF0uwH+8LO2Hf2lf1kf6fG6osYQS5dfCqhljuHi69vlf78V2Xi04fjWHWOQkH6/Jp80OCRqEXH2hxhCapUw/jt3tuz0uNiun+XfWK/sL6P7JR9wwqt9m/18y4vfsDoaTHeHCPVoY1PFVI4xn3sDXNV0NcX1kGUT8yPK2zRd/HWMcdjVMuiZ0NnaoQT4l0nBNcluC7B9QiuR3AdgusQnEZwGsE1Ca5JcOsEt05wWwS3RXDbBLdNcFWCqxLcAcEdEBwjOJbglGh+g7VMzasSzWvoT86nIr6EoZ8TfhNXfUzURT9JOZijPsaFFor0EqRHkkFuwQofZYe2JB2M0pAfVcT2AaEyxL6hoGVSN+qhlIVExYUpFZfE12pOVBJbk4qdsaxizc5ETqOqfJS/ij4Dd6d8gtiYWFEK9jcSVJGgignqmdhtUpATtIZ7ULB3hrM5PDXCPMJdifbkgrh4hmUnT6xko5LLZO9ncrsPUmtPotNsHm7DHVjF+A9hDTZx/MuYwyt4A+/gvZSR9qSa1AjR2ZlIcxPGLkn7Bx+skAM=</latexit>

123/124 Summer School 数理物理 2021 Lecture2

終わり

124/124 Summer School 数理物理 2021 Lecture2