ノンパラメトリックベイズ統計と自然言語処理持橋大地
統計数理研究所
daichi@ism.ac.jp
Summer School 数理物理
2020-8-29 (日)持橋担当分のスケジュール
l 1日目：概要と目次、ノンパラメトリックベイズ法(離散的な場合; 無限モデル)
l 2日目：ガウス過程とその適用(連続的な場合; ベイズ的関数回帰)
l 3日目：ノンパラメトリックベイズ法と自然言語処理への応用(研究紹介)

1 / 81

Summer School 数理物理2021 Lecture3

はじめに: 雑談l 数理と実装について– パッケージ化と理解、データ構造

2 / 81

Summer School 数理物理2021 Lecture3

無限木構造HMM (iTHMM)

0l HTSSBにより、無限木構造上の状態遷移確率とその事後分布が計算できる→ HTSSB-HMM = Infinite Tree HMM (iTHMM)

TSSB

3 / 81

Summer School 数理物理2021 Lecture3

rr<latexit sha1_base64="N8UiF8I7aRKBFnQwEchSntCV4D8=">AAAHp3ichZXNbtNAEMenLZBSPtrCBYlLRBQEl2jjRk3h1AKVWrUqTULSVE1V2WbTWrFjy3aSJlFeAK4gDpxA4oB4DC68AIc+AuJYJC4cGK9tOYknra3EuzO//3hmd72rWLrmuIydTU3PXLl6LTF7fe7GzVu35xcW71Qcs2WrvKyaumlXFdnhutbkZVdzdV61bC4bis73lMZzz7/X5rajmc1Xbtfih4Z83NTqmiq7aCrYRwsplnmysizllpMsw1g+K2W9hpTPLeWSWbR4VwqCa9dcnNmAGrwGE1RogQEcmuBiWwcZHLwPIAsMLLQdQh9tNrY04ecwgDnUtpDiSMhobeD/MfYOAmsT+15MR6hVfIuOPxuVSUizn+wrO2c/2Df2i/2bGKsvYni5dPGp+FpuHc2/uVf6e6nKwKcLJ5HqAoWC9MU1uVCHFVGLhrVZwuJVqfrx270P56WnxXT/IfvMfmN9n9gZ+44VNtt/1C8FXvyI0dNivDlGqkEbnyqkcIz72AtzVdDXF9ZBkE/EjypM0bfx1jDHE1TLomdCZ2KEU+JdpwTXJbguwfUIrkdwHYLrEFyd4OoE1yC4BsGtEdwawW0S3CbBbRHcFsFVCa5KcPsEt09wjOBYjFOC+fXWMjWvSjCvvj8+n4r4EkI/J/wGrvqIqIl+nLIwR22E8y0U6cRIhyS93LwVPsyGtjjtjVLIDysi+4BQ6WLfUNAyrhv2UMrdWMW7Eyouia/VGKskssYV2yNZRZrtsZyGVTtB/ir6dNyddmLE+tiKUrC/HqOKBFWMUS/EbpMCSdB13IO8vdOfzfDU8PPwdyXaI3lx8QwLD6rk5EZFymSXMlIhl1p9Fpxms3AfHsAjjJ+HVdjA8S9jDhzewjt4n3iceJmoJKo+Oj0VaO7CyJWQ/wOHJo8H</latexit>

1ディリクレ過程(DP)とガウス過程(GP)l 二者の定義は、ほとんど一緒– GP: どんな入力の集合 (x1 , · · · , xN ) をとってきても、対応する (y1 , · · · , yN ) がガウス分布に従う– DP: 空間のどんな離散化 (X1 , · · · , Xk ) についても、対応する離散分布がディリクレ分布
Dir(↵(X1 ), · · · , ↵(Xk )) に従う<latexit sha1_base64="3atjQpJyeWL8njDl4bND/nedEb0=">AAAHvHichZXNbtNAEMenLeBSPpoCByQuFVFQkapoE5BAHFALVGrVKkoT0qZqqsh2Nq0Vf8l2nC/lBXgBDlwAiUPhMbjwAhz6CIhjkbhwYLx2lA9PWluJd2d+//HM7npXsXXN9Rg7m5mdu3L1mjR/feHGzVu3FxNLd/Zcq+movKRauuWUFdnlumbykqd5Oi/bDpcNRef7SuN14N/3ueNqlvnW69j8yJCPTa2uqbKHpmri3krFb1czqxW1ZnnuatDJPa4mkizNxLUcb2SiRhKiK28tzW1CBWpggQpNMICDCR62dZDBxfsQMsDARtsR9NDmYEsTfg59WEBtEymOhIzWBv4fY+8wsprYD2K6Qq3iW3T8OahchhT7yU7ZOfvBvrFf7N/UWD0RI8ilg08l1HK7uvjufvHvpSoDnx6cDFUXKBSkL67Jgzo8F7VoWJstLEGVahjf774/L74opHqP2Gf2G+v7xM7Yd6zQ9P+oX3Z54QNGT4nx5hipAj4+VUjiGPewN8hVQV9PWPtRPkN+XGGJvoO3hjmeoFoWPQtaUyO0iXe1Ca5DcB2C6xJcl+BaBNciuDrB1QmuQXANglsnuHWC2yK4LYLbJrhtgisTXJngDgjugOAYwbEYp0TzG6xlal6VaF5Df3w+FfElDPyc8Bu46odERfTjlI05amNcaKFIN0a6JBnkFqzwUXZgi9PBKA34UcXQ3idUutg3FLRM6kY9lDIfqzg/peKi+FqNiUqG1rhiZyyroWZnIqdRVS7KX0WfjrtTLkZsTKwoBfsbMapAUIUY9UbsNknICrqOe1Cwd4azOTg1wjzCXYn2ZIO4eIZlJk+seGMvm848SWd3nybXXkWn2Tw8gIewgvGfwRps4viXMIcufIRT+Cq9lGpSQzJCdHYm0tyFsUvy/wOA8JY1</latexit><latexit sha1_base64="+mPd8ikf29YTDCmRnBsJ3jeg9Do=">AAAHuHichZXNbtNAEMenLeBSPprCBYlLRRRUpCraBCRQTy1QqVWrKE1Im6qpUtvdpFb8JdtJ6kR5AV6AA1xAKhLiMbjwAhz6CIhjkbhwYLx25CSeFluJd2d+//HM7npXsXXN9Rg7n5qeuXb9hjR7c+7W7Tt351ML93Zdq+2ovKJauuVUFdnlumbyiqd5Oq/aDpcNRed7SutV4N/rcMfVLPON59v80JCbptbQVNlDUz2VWvLrueWaemx57rJfLzypp9Isy8S1mGzkokYaoqtoLcxsQA2OwQIV2mAABxM8bOsgg4v3AeSAgY22Q+ijzcGWJvwcBjCH2jZSHAkZrS38b2LvILKa2A9iukKt4lt0/DmoXIQM+8G+sAv2nX1lP9nfS2P1RYwgFx+fSqjldn3+7YPyn/+qDHx6cBKrrlAoSF9dkwcNeCFq0bA2W1iCKtUwfqf37qK8Usr0H7NP7BfW95Gds29Yodn5rZ7t8NJ7jJ4R480xUg06+FQhjWPcx94wVwV9fWEdRPnE/LjCEn0Hbw1zPEG1LHoWdC+NcEq865TgfILzCa5HcD2C6xJcl+AaBNcguBbBtQhujeDWCG6T4DYJbovgtgiuSnBVgtsnuH2CYwTHEpwSzW+wlql5VaJ5Df3J+VTElzD0c8Jv4KqPiZroJykbc9TGuNBCkW6CdEkyyC1Y4aPs0Jakg1Ea8qOK2D4gVLrYNxS0TOpGPZSymKi4eEnFZfG1GhOVxNakYnssq1izPZHTqKoQ5a+iT8fdqZAg1idWlIL99QRVIqhSgnotdps05AXdwD0o2DvD2RyeGmEe4a5Ee/JBXDzDcpMnVrKxm8/mnmbzO8/Sqy+j02wWHsIjWML4z2EVNnD8K5hDBz7AGXyWVqQjqSlpITo9FWnuw9glOf8A3wiUaw==</latexit><latexit sha1_base64="1emHyqnWJcH1eHJlASWGMqDGJbE=">AAAHuHichZXNbtNAEMenLeBSPprCBYlLRRRUpCraBCRQTy1QqVWrKk1I66qpUtvdpFb8JdtJmkR5AV6AA1xAKhLiMbjwAhz6CIhjkbhwYLx25CSeFFuJd2d+//HM7npXdQzd8xm7mJqeuXb9hjR7c+7W7Tt351ML9/Y8u+lqvKzZhu3KquJxQ7d42dd9g8uOyxVTNfi+2ngV+Pdb3PV023rjdxx+ZCp1S6/pmuKjqZpKLcnV3HJFO7F9b1muNp5UU2mWZeJaTDZyUSMN0VWwF2Y2oAInYIMGTTCBgwU+tg1QwMP7EHLAwEHbEfTQ5mJLF34OfZhDbRMpjoSC1gb+17F3GFkt7AcxPaHW8C0G/lxULkKG/WBf2CX7zr6yn+zvxFg9ESPIpYNPNdRypzr/9kHpz39VJj59OI1VVyhUpK+uyYcavBC16FibIyxBlVoYv9V9d1laKWZ6j9kn9gvr+8gu2Des0Gr91s53efE9Rs+I8eYYqQItfGqQxjHuYW+Qq4q+nrD2o3xiflRhi76Lt445nqJaET0b2hMjnBHvOiO4DsF1CK5LcF2CaxNcm+BqBFcjuAbBNQhujeDWCG6T4DYJbovgtghOJjiZ4A4I7oDgGMGxBKdG8xusZWpe1WheQ39yPlXxJQz8nPCbuOpjoiL6ScrBHPURLrRQpJcgPZIMcgtW+DA7sCXpYJQG/LAitvcJlSH2DRUt47phD6UsJCouTKi4JL5Wc6yS2JpUbI9kFWu2x3IaVu1E+WvoM3B32kkQ62MrSsX+eoIqElQxQb0Wu00a8oKu4R4U7J3hbA5OjTCPcFeiPfkgLp5hufETK9nYy2dzT7P53Wfp1ZfRaTYLD+ERLGH857AKGzj+ZcyhBR/gHD5LK9KxVJf0EJ2eijT3YeSS3H/QNpRG</latexit><latexit sha1_base64="jpNApOnHkXBEfJuEInhrH5Zhzrc=">AAAH0XichZW/b9NAFMdfWyCl/GgKCxJLRBSUSFV1CUggphZaqVWrKk1Im6quItu9NFbs2LKdpEkUCTHCwMjABBIDYmaFgYV/gKF/AmIsEgsDz2cHJ/FLsZX47r3P9/m9u/OdYuma4zJ2OjU9c+Hipdjs5bkrV69dn48v3Nh1zKat8pJq6qZdVmSH61qDl1zN1XnZsrlsKDrfU+pPPP9ei9uOZjaeuh2LHxrycUOraqrsoqkSz/Qk20isanY/Lcm6VZPT5Uo2syipR6brLP4z1TOZSjzJlpi4EtFGNmgkIbjy5sLMOkhwBCao0AQDODTAxbYOMjh4H0AWGFhoO4Qe2mxsacLPoQ9zqG0ixZGQ0VrH/2PsHQTWBva9mI5Qq/gWHX82KhOQYt/ZB3bGvrGP7Af7MzFWT8TwcungU/G13KrMv7hV/P1flYFPF2qh6hyFgvT5NblQhYeiFg1rs4TFq1L147e6r8+Kjwqp3l32jv3E+t6yU/YVK2y0fqnvd3jhDUZPifHmGEmCFj5VSOIY97A3yFVBX09Y+0E+IT+qMEXfxlvDHGuolkXPhPbECCfEu04IrkNwHYLrElyX4NoE1ya4KsFVCa5OcHWCWyG4FYLbILgNgtskuE2CKxNcmeD2CW6f4BjBsQinBPPrrWVqXpVgXn1/dD4V8SUM/JzwG7jqQ0IS/ShlYY7aCOdbKNKJkA5Jerl5K3yYHdiitDdKA35YEdr7hEoX+4aClnHdsIdS5iMV5ydUXBRfqzFWSWiNKrZGsgo1W2M5Dau2g/xV9Om4O21HiLWxFaVgfy1CFQiqEKFWxW6ThJygq7gHeXunP5uDU8PPw9+VaE/Oi4tnWHb8xIo2dnNL2XtLuZ37yeXHwWk2C7fhDqQx/gNYhnUc/xLm8Ao+wWf4EivGOrFnsec+Oj0VaG7CyBV7+Rc61p4Y</latexit>l どちらも、無限次元の smoother になっている

4 / 81

Summer School 数理物理2021 Lecture3

(1)「副詞」の学習

5 / 805人間の動作からの副詞の理解
l “しっかりと”、“さっと”、“慎重に”といった副詞の理解は、特に介護などで今後とても重要– これらは静的な画像ではなく、動作の動的な
時系列の性質に関連している– 適当なニューラル手法では、動的な性質を十分捉えることができない
l 『スペクトル混合カーネルとガウス過程に基づく

動画からの副詞の意味理解』(言語処理学会2021)– 谷口巴(お茶大)、持橋大地(統数研) 他

6 / 81

Summer School 数理物理2021 Lecture3

動画からの副詞の理解
l 動作を表す動画に、「そわそわ」「速足で」「こっそりと」など副詞をクラウドソーシングでタグ付けしてもらう
l 動画からOpenPose+前処理で骨格を抽出

l 48次元の関節座標の時系列データが得られる

7 / 81

Summer School 数理物理2021 Lecture3

動画からの副詞の理解 (2)l さまざまな動作を表す関節座標の時系列姿勢データと、
l それを記述する副詞の集合がペアになったデータセットが得られた

サッサッと, 力強く 重々しく, 確実に 素早く, 正確に速く, 重々しく, 鋭く 軽く, ゆっくりと, 忍んで8 / 81

Summer School 数理物理2021 Lecture3

動画からの副詞の理解 (3)l 48次元の姿勢データを、GPLVMでP次元の潜在空間に非線形圧縮 (P=3)

l この潜在空間の時系列データの特徴をどう捉えるか?

– 速い、遅い、微妙な震え、...
– 「関数の特徴を捉える」問題

9 / 81

Summer School 数理物理2021 Lecture3

スペクトル混合カーネル
l “Gaussian Process Kernels for Pattern Discovery

and Extrapolation”, Andrew Gordon Wilson, Ryan
Prescott Adams, ICML 2013.
l ガウス過程で使うカーネルを、RBFのような既存のカーネルおよびその組み合わせに限定せず、フーリエ領域で混合ガウス分布を考えることでデータから自動的に学習できる (！)

10 / 81

Summer School 数理物理2021 Lecture3

スペクトル混合カーネル (2)
0
⌧
=
x
x
l ガウス過程のカーネルとして、値がだけに依存する、定常カーネル k(τ) を考える
<latexit sha1_base64="q59azai5Ts9o5dJKcYkU42AmF4U=">AAAG53ichZRLaxNRFMdPo461PproRnDTGKpuDDexUBGEVi1YWkqamDalKWFmeicOmcwM88iTfAE37lQQFxVUxI/hxi/goivX4rKCGxeeuTMljzmtE3If5/z+J+fcyT2Kbeiux9jhVOLM2XPS+ekLMxcvXb4ym0xd3XIt31F5WbUMy6kosssN3eRlT/cMXrEdLjcVg28rjceBf7vFHVe3zGde1+Z7Tblu6pquyh6aaslU1ZP9avphNd2ppu/ieLuWzLAsE89cfJGLFhmInoKVSvyAKuyDBSr40AQOJni4NkAGFz+7kAMGNtr2oI82B1e68HMYwAxqfaQ4EjJaGzjWcbcbWU3cBzFdoVbxVwz8Oqicg3n2nX1mR+wb+8J+sr8nxuqLGEEuXZyVUMvt2uyL66U//1U1cfbg+VB1ikJB+vSaPNDgvqhFx9psYQmqVMP4rd7ro9KD4nz/FnvPfmF9B+yQfcUKzdZv9cMmL74V0fdRo+HcwlmFDJ5wH3fHmSro6wvrIMpmyHcIskNwXYLrElyP4HoE1ya4NsFpBKcRXIPgGgS3THDLBLdKcKsEt0ZwawRXIbgKwe0Q3A7BMYJjMS74//mCDFahwicoG2PoY1xooUg3RrokGeTFcZSRGuWH9gGhMsQtU9AyqRv1UMpCrIbCCTWU0FoXZzdKD61xxfpYVkPN+kROo6qNKH8VfQbe5Y0YsTLxDhXcr8SoIkEVY9QTcb8zkBe0hh0x6DTh+znusWEeYR+gPfkgLnb83GR/jy+28tncvWx+cyGz9Cjq/dNwA27CHYy/CEvwFM+/jDm04R18hE+SLr2UXklvQjQxFWmuwdgjHfwDIcZs+w==</latexit>– RBF (ガウス)カーネルもこの仲間
l ボホナーの定理により、任意のk(τ)はk(⌧ ) =
<latexit sha1_base64="wkWseKn3C84jqciOv5PvEOS5uqw=">AAAHEnichZTPTxNBFMcfqBXxB6AXEy+rDaYkhkyricZoAgqJBEJKS6GEpc3udlom3W433W2h3fQf8GziwZMkHoz/gTfDhYMXTTxw8mw8YuLFg29nl7R0HzBNd2be+3xf35vtPN02heMydjg0fOHipdjlkSujV69dvzE2PnFzzak3GwbPGXWz3sjrmsNNYfGcK1yT5+0G12q6ydf16kvfv97iDUfUrVW3bfOtmlaxRFkYmoum4vizakJ1teaU8lxRheUWPbWmudu67mW6hbmuwgteSrWFIhT1gVNYVe/6cFe1HZEoOVPF8TibZnIo0UUyXMQhHOn6xPBPUKEEdTCgCTXgYIGLaxM0cPCzCUlgYKNtCzy0NXAlpJ9DF0ZR20SKI6GhtYrPCu42Q6uFez+mI9UG/oqJ3wYqFZhkP9hHdsQO2Cf2i/07NZYnY/i5tHHWAy23i2Ovb2f/nquq4ezCdk91hkJH+uyaXCjDE1mLwNpsafGrNIL4rc7bo+zTzKR3n+2x31jfe3bI9rFCq/XH+LDCM+9k9BJqyji3cDYgjifs4e44Ux19nrR2w2x6/C5B7hJcm+DaBNchuA7B7RDcDsGVCa5McFWCqxLcLMHNEtwCwS0Q3CLBLRJcnuDyBLdBcBsExwiORTj//9eUpL8KFE2CsjGGOMEFFop0IqRDkn5eHJ8aUv18z94lVKa8ZTpaBnX9HkqZjtSQPqWGLFor8uz66Z41qlg6kVVPszSQU79qOczfQJ+Jd3k5QswPvEMd9/MRKkNQmQg1J+93HFKSLmNH9DtN8H6Oe2yQR9AHaE/Kj4sdPznY36OLtdR08uF0auVRfOZF2PtH4A7cgwTGfwwz8ArPP4c57MEBfIPvsTexz7Evsf0AHR4KNbfgxIh9/Q+i7H6s</latexit>

Z

RD

e

2⇡i sT⌧(ds)の形に表せる (逆フーリエ変換)– (s)が、周波数領域でのk(τ)の等価な表現↓
k(τ)を確率分布で表せる！
<latexit sha1_base64="WVESHGB3r106RH/16eDtgG9SxUY=">AAAG23ichZS/b9NQEMevAUwpP/qDBYmlwioqS/QSkECdWqASVasqTUiTqo0q230OVhzb8o+0iZWJrWJhQEh0AokB8Wew8A8wdGJGjEViYeD87MpJfG1fFL/37j7fy91z3qmOaXg+Y8djuUuXr0hXx69NXL9x89bk1PTMpmcHrsarmm3abl1VPG4aFq/6hm/yuuNypa2avKa2nkX+Woe7nmFbL/2uwxttpWkZuqEpPppqO45nzHsPdqdklmdizGYXhWQhQzJK9nTuJ+zAHtigQQBt4GCBj2sTFPDwsw0FYOCgrQEh2lxcGcLPoQ8TqA2Q4kgoaG3hs4m77cRq4T6K6Qm1hr9i4tdF5SzMsR/sCzth39lX9ov9OzNWKGJEuXRxVmMtd3YnD+9U/l6oauPsw6tUdY5CRfr8mnzQ4YmoxcDaHGGJqtTi+J3e+5PKQnkuvM8+sd9Y30d2zL5hhVbnj/Z5g5ePRPQ91Og4d3DWQMYTDnF3mqmKvlBY+0k2KX9AkAcE1yW4LsH1CK5HcPsEt09wOsHpBNciuBbBLRHcEsGtENwKwa0S3CrB1QmuTnBbBLdFcIzgWIaL/n+BIKNVrAgIysEYxhAXWyjSy5AeSUZ5cXwqSA3yqb1PqExxy1S0jOoGPZSylKmhdEYNFbQ2xdkN0qk1q1gbyirVrI3kNKhaT/LX0GfiXV7PEMsj71DF/XKGKhNUOUM9F/dbhqKgdeyIUaeJ389pj43ziPsA7SlGcbHjF0b7e3axWcwXHuaLG4/kxadJ7x+Hu3AP5jH+Y1iEF3j+VdHt3sIHOJIa0mvpUHoTo7mxRHMbhob07j//CWm3</latexit>

11 / 81

Summer School 数理物理2021 Lecture3

The inverse Fourier
The inverse
transform
Fourierof atransform(symmetrised)of a (symmetrised)
non centred non centred
Kernels andpositiveapproximation
measures
Spectral approximation
Gaussian is: Gaussian
Kernels andis:
positive measures
Spectral

スペクトル混合カーネルのイメージ

Spectral approximation
witha mixture (A.
ofGaussian
Spectral approximation
with a mixtureof Gaussian
Wilson, k̃(A.
µ(!)
µ(!)
k̃ (t)(t)Wilson,
ICML 2013) ICML 2013)
!

!

周波数空間
カーネル
F 1 ! F 1
We obtain
a parametrised
kernel that isF
parametrised
and the1 means by
We(確率分布)obtain a kernelthat isby theandthethemeans (関数)Gaussians
bells inspace:
the measure space:bandwidths ofbandwidths
Gaussians ofbells
in the measure<latexit sha1_base64="TJs8yef1QE+y3SNL6ek7eVzsTO0=">AAAHzHichZW/b9NAFMdfWyCl/GgKCxJLRAhiIboEJBBTCxS1ahWlCWlTNSWyzSWxYseW7SRNLK8M/QcYmEBiQCyssLLwDzD0T0CMRWJh4PnsyEn8Umwlvnvv831+7+58J5uaajuMnczNL5w7fyGxeHHp0uUrV5eTK9d2baNrKbyiGJphVWXJ5pra4RVHdTReNS0u6bLG9+T2U9+/1+OWrRqdF87A5Ie61OyoDVWRHDTVk7druuS0DNOtWWqz5UiWZfS9uuvWFElLPfdeuvdynldPplmWiSsVb+TCRhrCq2isLGxADV6BAQp0QQcOHXCwrYEENt4HkAMGJtoOwUWbhS1V+Dl4sITaLlIcCQmtbfxvYu8gtHaw78e0hVrBt2j4s1CZggz7wT6yU/adfWI/2d+ZsVwRw89lgE850HKzvnx8o/znvyodnw60ItUZChnps2tyoAGPRC0q1mYKi1+lEsTvDd+clh+XMu4d9p79wvresRP2DSvs9H4rH3Z46S1Gz4jx5hipBj18KpDGMXaxN8pVRp8rrF6YT8RPKgzRt/BWMccWqiXRM6A/M8IR8a4jghsQ3IDghgQ3JLg+wfUJrkFwDYJrE1yb4NYIbo3gNgluk+C2CG6L4KoEVyW4fYLbJzhGcCzGyeH8+muZmlc5nNfAH59PWXwJIz8n/Dqu+oioiX6cMjFHdYILLBRpx0ibJP3c/BU+zo5scdofpRE/rojsHqHSxL4ho2VaN+6hlMVYxcUZFZfF16pPVRJZ44rtiawizfZUTuOqQpi/gj4Nd6dCjFifWlEy9tdjVImgSjHqmdht0pAXdAP3IH/vDGZzdGoEeQS7Eu3J+3HxDMtNn1jxxm4+m7ufze88SK8+CU+zRbgJt+Auxn8Iq7CB41/BHI7hM3yBr4lCwkm4CS9A5+dCzXWYuBKv/wGKo55Z</latexit>µ(!)
k̃ (t)µ(!)
k̃ (t)
This can be generalisedThis can betogeneraliseda measure to
based
a measureon the based
sum ofonGaussians.
the sum of Gaussians.
!
F 1! F !1FGP Summer School (Sheffield,
GP Summer
June School2013) (Sheffield, JuneKernel2013)Design<latexit sha1_base64="TJs8yef1QE+y3SNL6ek7eVzsTO0=">AAAHzHichZW/b9NAFMdfWyCl/GgKCxJLRAhiIboEJBBTCxS1ahWlCWlTNSWyzSWxYseW7SRNLK8M/QcYmEBiQCyssLLwDzD0T0CMRWJh4PnsyEn8Umwlvnvv831+7+58J5uaajuMnczNL5w7fyGxeHHp0uUrV5eTK9d2baNrKbyiGJphVWXJ5pra4RVHdTReNS0u6bLG9+T2U9+/1+OWrRqdF87A5Ie61OyoDVWRHDTVk7druuS0DNOtWWqz5UiWZfS9uuvWFElLPfdeuvdynldPplmWiSsVb+TCRhrCq2isLGxADV6BAQp0QQcOHXCwrYEENt4HkAMGJtoOwUWbhS1V+Dl4sITaLlIcCQmtbfxvYu8gtHaw78e0hVrBt2j4s1CZggz7wT6yU/adfWI/2d+ZsVwRw89lgE850HKzvnx8o/znvyodnw60ItUZChnps2tyoAGPRC0q1mYKi1+lEsTvDd+clh+XMu4d9p79wvresRP2DSvs9H4rH3Z46S1Gz4jx5hipBj18KpDGMXaxN8pVRp8rrF6YT8RPKgzRt/BWMccWqiXRM6A/M8IR8a4jghsQ3IDghgQ3JLg+wfUJrkFwDYJrE1yb4NYIbo3gNgluk+C2CG6L4KoEVyW4fYLbJzhGcCzGyeH8+muZmlc5nNfAH59PWXwJIz8n/Dqu+oioiX6cMjFHdYILLBRpx0ibJP3c/BU+zo5scdofpRE/rojsHqHSxL4ho2VaN+6hlMVYxcUZFZfF16pPVRJZ44rtiawizfZUTuOqQpi/gj4Nd6dCjFifWlEy9tdjVImgSjHqmdht0pAXdAP3IH/vDGZzdGoEeQS7Eu3J+3HxDMtNn1jxxm4+m7ufze88SK8+CU+zRbgJt+Auxn8Iq7CB41/BHI7hM3yBr4lCwkm4CS9A5+dCzXWYuBKv/wGKo55Z</latexit>

1Kernel Design

N. Durrande

18 / 67 N. Durrande

18 / 67

Gaussian process summer school 2013の資料“Kernel Design” (N. Durrande) より引用
GP Summer School (Sheffield,
June 2013)
KernelGP Summer
School (Sheffield, June2013)Design

12 / 81

Summer School 数理物理2021 Lecture3

Kernel Design

N. Durrande

19 / 67 N. Durrande

19 / 67

スペクトル混合カーネル (3)l 通常のガウスカーネル⌘
1k(x, x0 ) = exp(x x0 )2 /`22の周波数表現は、フーリエ変換すると<latexit sha1_base64="B0o9cuDck6Z4+EB6eZuErz1SUXQ=">AAAHcnichZXPTxNBFMcfoBTrjxa9GL1UG7QkpU5XE43RBFASCYSUVqCEhWZ3ma2bbtvN7lJaNv0H/Ac8eJLEg/HP8MJVEw/8CcYjJF48+HZ2SX/sA6fpzsx7n+/rezvTGdUyDcdl7HhkdOzS5fHYxJX41WvXbySSkzfXneaerfE1rWk27bKqONw0GnzNNVyTly2bK3XV5Btq7ZXv32hx2zGajbdux+LbdaXaMHRDU1w0VZIvapl2tv1wOvUyJfO2Jc8bVTMTn0nJuq1oXr7ryVlJznYz7RmEdqRHMjfNHSnuc/Z0JZlmOSZaKjrIh4M0hK3QnBxLgAy70AQN9qAOHBrg4tgEBRz8bEEeGFho2wYPbTaODOHn0IU4aveQ4kgoaK3hs4qzrdDawLkf0xFqDX/FxK+NyhRMsZ/sCzthR+wr+8X+nhvLEzH8XDrYq4GWW5XE+9ulP/9V1bF34V1PdYFCRfrimlzQ4ZmoxcDaLGHxq9SC+K2DDyel58Up7wE7ZL+xvk/smH3DChutU+3zKi9+FNF3UaNj38JegzS+YQ9nZ5mq6POEtRtm0+PbBNkmuA7BdQjugOAOCG6f4PYJTic4neBqBFcjuDmCmyO4RYJbJLglglsiuDLBlQluk+A2CY4RHItwari+/j6k1lUN1zXwR9dTFbv4zM8Jfx13bI+QxTxKWZijMcAFFop0IqRDkn7dHJ8KUv18z94lVKb4F6toGdb1eyhlIVJD4ZwaSmitirXpp3vWqGJ5IKueZnkop37VSpi/hj4Tz4qVCLEwtEdUnC9EqCJBFSPUa3F+pEEStI4nrn+SBetzdoYHeQTnDO2R/Lh4o+SH74/oYF3K5R/npNUn6dn58G6ZgLtwHzIY/ynMwht8/2uYwyEcwXf4MX4auxO7FwsvotGRUHMLBlos+w/jx4wC</latexit>⇣

S(s) = (2⇡`2 )1/2 exp( 2⇡ 2 `2 s2 )<latexit sha1_base64="AMkHtCj5O3ah+inHtfl2oHc1PzA=">AAAHZHichZXPaxNBFMdfW21qrSa1CIIgwVBJD8bJKiiC0KoFS0tJE9OmNE3Y2c7GJZtk2d2k+UH+Ac+CB08KHsT/wGs9+A946D8giMcKXjz4dnZD0uxrnZCdmfc+35f3diYz3DINx2XseGJy6sLF6cjMpdnLc1euRmPz17adRtPWRF5rmA27wFVHmEZd5F3DNUXBsoVa46bY4dVnnn+nJWzHaNRfuh1L7NfUSt3QDU110VSOpXJJZ+lJUilaRlGYZklZKvXS95R+vCjaVvKuZy8pvsdBZzmWYCkmWzw8SAeDBAQt05ifikIRDqABGjShBgLq4OLYBBUc/OxBGhhYaNuHHtpsHBnSL6APs6htIiWQUNFaxWcFZ3uBtY5zL6Yj1Rr+iolfG5VxWGTf2Sd2wr6xz+wn+3tmrJ6M4eXSwZ77WmGVo69v5P78V1XD3oVXQ9U5Co70+TW5oMMjWYuBtVnS4lWp+fFb3bcnucfZxd4d9oH9wvres2N2hBXWW7+1j1si+05GP0CNjn0Lew0S+IZ7OBtkytHXk9Z+kM2QbxNkm+A6BNchuC7BdQnukOAOCU4nOJ3gqgRXJbgVglshuDWCWyO4dYJbJ7gCwRUIbpfgdgmOERwLcTxYX28fUuvKg3X1/eH15HIXD/yC8Ndwxw6JopyHKQtzNE5xvoUinRDpkKRXt8CnitQoP7T3CZUp/8UcLeO6UQ+lzIRqyJxRQw6tFbk2o/TQGlZsnMpqqNkYy2lUtRnkr6HPxLNiM0Ssju0RjvPVEJUlqGyIei7PjwQoktbxxPVOMn99Bme4n4d/ztAexYuLN0p6/P4ID7aVVPp+Stl6kFh+GtwtM3ATbkMS4z+EZXiB7z+PObyBL3AEX6d/ROYiC5HrPjo5EWgW4FSL3PoHO0aHlA==</latexit>– 中心0のガウス分布!

13 / 81

Summer School 数理物理2021 Lecture3

スペクトル混合カーネル (4)l k(τ)は周波数領域での確率密度(s) と等価なので、(s) に関して混合ガウス分布を考える<latexit sha1_base64="WVESHGB3r106RH/16eDtgG9SxUY=">AAAG23ichZS/b9NQEMevAUwpP/qDBYmlwioqS/QSkECdWqASVasqTUiTqo0q230OVhzb8o+0iZWJrWJhQEh0AokB8Wew8A8wdGJGjEViYeD87MpJfG1fFL/37j7fy91z3qmOaXg+Y8djuUuXr0hXx69NXL9x89bk1PTMpmcHrsarmm3abl1VPG4aFq/6hm/yuuNypa2avKa2nkX+Woe7nmFbL/2uwxttpWkZuqEpPppqO45nzHsPdqdklmdizGYXhWQhQzJK9nTuJ+zAHtigQQBt4GCBj2sTFPDwsw0FYOCgrQEh2lxcGcLPoQ8TqA2Q4kgoaG3hs4m77cRq4T6K6Qm1hr9i4tdF5SzMsR/sCzth39lX9ov9OzNWKGJEuXRxVmMtd3YnD+9U/l6oauPsw6tUdY5CRfr8mnzQ4YmoxcDaHGGJqtTi+J3e+5PKQnkuvM8+sd9Y30d2zL5hhVbnj/Z5g5ePRPQ91Og4d3DWQMYTDnF3mqmKvlBY+0k2KX9AkAcE1yW4LsH1CK5HcPsEt09wOsHpBNciuBbBLRHcEsGtENwKwa0S3CrB1QmuTnBbBLdFcIzgWIaL/n+BIKNVrAgIysEYxhAXWyjSy5AeSUZ5cXwqSA3yqb1PqExxy1S0jOoGPZSylKmhdEYNFbQ2xdkN0qk1q1gbyirVrI3kNKhaT/LX0GfiXV7PEMsj71DF/XKGKhNUOUM9F/dbhqKgdeyIUaeJ389pj43ziPsA7SlGcbHjF0b7e3axWcwXHuaLG4/kxadJ7x+Hu3AP5jH+Y1iEF3j+VdHt3sIHOJIa0mvpUHoTo7mxRHMbhob07j//CWm3</latexit><latexit sha1_base64="WVESHGB3r106RH/16eDtgG9SxUY=">AAAG23ichZS/b9NQEMevAUwpP/qDBYmlwioqS/QSkECdWqASVasqTUiTqo0q230OVhzb8o+0iZWJrWJhQEh0AokB8Wew8A8wdGJGjEViYeD87MpJfG1fFL/37j7fy91z3qmOaXg+Y8djuUuXr0hXx69NXL9x89bk1PTMpmcHrsarmm3abl1VPG4aFq/6hm/yuuNypa2avKa2nkX+Woe7nmFbL/2uwxttpWkZuqEpPppqO45nzHsPdqdklmdizGYXhWQhQzJK9nTuJ+zAHtigQQBt4GCBj2sTFPDwsw0FYOCgrQEh2lxcGcLPoQ8TqA2Q4kgoaG3hs4m77cRq4T6K6Qm1hr9i4tdF5SzMsR/sCzth39lX9ov9OzNWKGJEuXRxVmMtd3YnD+9U/l6oauPsw6tUdY5CRfr8mnzQ4YmoxcDaHGGJqtTi+J3e+5PKQnkuvM8+sd9Y30d2zL5hhVbnj/Z5g5ePRPQ91Og4d3DWQMYTDnF3mqmKvlBY+0k2KX9AkAcE1yW4LsH1CK5HcPsEt09wOsHpBNciuBbBLRHcEsGtENwKwa0S3CrB1QmuTnBbBLdFcIzgWIaL/n+BIKNVrAgIysEYxhAXWyjSy5AeSUZ5cXwqSA3yqb1PqExxy1S0jOoGPZSylKmhdEYNFbQ2xdkN0qk1q1gbyirVrI3kNKhaT/LX0GfiXV7PEMsj71DF/XKGKhNUOUM9F/dbhqKgdeyIUaeJ389pj43ziPsA7SlGcbHjF0b7e3axWcwXHuaLG4/kxadJ7x+Hu3AP5jH+Y1iEF3j+VdHt3sIHOJIa0mvpUHoTo7mxRHMbhob07j//CWm3</latexit>– 0に関して対称なので、正だけ考えて鏡映(s | µ,2) = N (s | µ,2)

S(s) = ( (s) + ( s))/ 2<latexit sha1_base64="7tSfeinyeI0Tc6Psx6IUsmbDAQ8=">AAAHLXichZTPbxJREMenqFjxR6leTLwQSQlEio/VRGPSpFWb2LQhFKSl6Vayu33QDQts2IUWkH/Af8CDJ008GA/+BZ68+A/00MTowYMx6qkmXjw4+3Ybfuy0PsK+92Y+32HmLW9U09Atm7GDicCp02eCZyfPhc5fuHhpKjx9ec1qtJoaL2gNo9EsqorFDb3OC7ZuG7xoNrlSUw2+rlYfOP71Nm9aeqP+2O6YfKumVOp6WdcUG02l8Lps7uhxS04+lZNyrZWULb1SU55IidhcT9YUI5LpU15ZDuXjViISm4vE3QiJG2KetRKJm3JSKoWjLMXEiPgXaW8RBW9kG9OBLyDDNjRAgxbUgEMdbFwboICFn01IAwMTbVvQQ1sTV7rwc+hDCLUtpDgSClqr+KzgbtOz1nHvxLSEWsNfMfDbRGUEZtg+e8MO2Uf2ln1nf4+N1RMxnFw6OKuulpulqWdX83/+q6rhbMPOQHWCQkX65JpsKMNdUYuOtZnC4lSpufHb3eeH+Xu5mV6MvWI/sL6X7IB9wArr7d/a61WeeyGib6OmjHMbZw2ieMI93B1lqqKvJ6x9L5sBv0eQewTXIbgOwXUJrktwuwS3S3BlgisTXJXgqgS3QHALBLdEcEsEt0xwywRXJLgiwW0Q3AbBMYJjPs75/7UE6axcRYugTIyhj3CuhSItH2mRpJMXx6eC1DA/sPcJlSFumYqWcd2wh1JmfTVkj6khj9aKOLthemD1K1ZGshpoVsZyGlZlvPw19Bl4lzM+YnHsHaq4X/RROYLK+aiH4n5HQRJ0GTui02nc93PUY9083D5AeyQnLnb89Hh/9y/WpFT6VkpavR2dv+/1/km4BtchjvHvwDw8wvMvYA7v4Sv8hF/Bd8H94KfgZxcNTHiaKzAygt/+AaZohcw=</latexit>l ガウス分布の各要素は、もとの領域では以下のカーネル関数を考えていることと等価k(⌧ | , µ) = exp( 2⇡ 2 2 ⌧ 2 ) cos(2⇡µ⌧ )<latexit sha1_base64="ukX6dPUh9R956JbHrIIzMCo2ueI=">AAAHFXichZRNb9NAEIanAUIpH03hgsSlIipKJIg2BgmEhGiBSlStqjQhbaq6jWx3E6w4thXbaT7IH+AP9MAJpB4Qv4ETCCFx4EIPPXFGHIvEhQPjtat8eNo6inf3needzKzjVW1Dd1zGDsZiZ86ei58fvzBx8dLlK5OJqaurjuU1NF7ULMNqlFTF4YZu8qKruwYv2Q2u1FWDr6m1p358rckbjm6ZL9y2zTfrStXUK7qmuCiVE49rKdlVvFeyo1frym257qUfybxlp+5Isq1vSYGOI0JbUlrWLCflRxD0pXQ5kWQZJq7p6CQbTpIQXjlrKvYTZNgGCzTwoA4cTHBxboACDn42IAsMbNQ2oYtaA2e6iHPowQR6PaQ4EgqqNbxXcbURqiau/ZyOcGv4KwZ+G+ichhm2z96zQ/aVfWC/2L9jc3VFDr+WNo5q4OV2efL19cLfU111HF142Xed4FCRPrknFyrwQPSiY2+2UPwutSB/s7N7WHiYn+neYu/Yb+zvLTtgn7BDs/lH21vh+Tci+zZ6Kjg2cdQgiTvcxdVRpSrGukLthdX0+RZBtgiuTXBtgusQXIfgdghuh+AqBFchuBrB1QhujuDmCG6B4BYIbpHgFgmuRHAlglsnuHWCYwTHIpz///ME6c8Ch0dQNubQh7hAoUgnQjok6dfF8a4gNcj39R7hMsRbpqIy6huMUM5cpIfcMT0UUK2KvRuk+2rUsTRUVd+zNFLToGs5rF/DmIHv8nKEmB95hiqu5yNUnqDyEeqZeL+TIAm6gieif9IEz+fojA3qCM4BOiL5efHEz46e79HJqpTJ3s1IK/eSs0/Cs38cbsBNSGH++zALz3H/i1jDHnyDH7Af341/jH+OfwnQ2FjouQZDV/z7f1vZgCw=</latexit>

14 / 81

Summer School 数理物理2021 Lecture3

スペクトル混合カーネル (5)l すなわち、これはカーネルとして、次の混合を考えていることになる (Spectral Mixture kernel)
P
D
⇣ X
⌘
X
(p)
k(⌧ ) =
wp cos(2⇡⌧ Tµ(p) ) exp
2⇡ 2 d ⌧d2<latexit sha1_base64="NYCBISOywSc8B3HKgx5Tt5TJmIM=">AAAHVXichZTNbtNAEMcnpSQlBdryISFxCURFyYFqY5BASJXa0kpUrao0/UpVt5HtbIIVO7ZiO21qRdx5AQ6cQOKAeAwuvABCPXHigDgWiQtIjNcuSeNp2Sje2dnffzKzzo5qG7rjMnaUGLowfDGZGrmUHr185erY+MS1TcfyWhrf0CzDapVVxeGG3uQbru4avGy3uGKqBt9SG0+D/a02bzm61Vx3OzbfNZV6U6/pmuKiqzL+opGTVdOXXcXr5qfTsuOZFd+eLnT3ipn9ip2RNcvJSbKt/6P21uU7YmGi7efsfDeflvmBLc/pdSN3P4xQDSLMZwLhniQ7et1UKtWQDoKgLQV8K18Zz7IpJkYmbhQiIwvRKFoTQ19BhipYoIEHJnBogou2AQo4+NmBAjCw0bcLPvpaaOlin0MX0qj1kOJIKOht4LOOq53I28R1ENMRag1/xcBvC5UZmGSf2Xt2zD6xD+w7+31mLF/ECHLp4KyGWm5Xxl7eWvv1X5WJswvPe6pzFCrS59fkQg0ei1p0rM0WnqBKLYzfPnx1vPakNOnfY2/ZD6zvDTtiH7HCZvun9m6Vl16L6FXU1HBu46xBFk/Yx9VJpiru+cLbjbLp8QcEeUBwHYLrENwhwR0S3D7B7RNcjeBqBNcguAbBzRLcLMEtEtwiwS0R3BLBlQmuTHDbBLdNcIzgWIwL/n+eIAMrVHgEZWMM/RQXeijSiZEOSQZ5cXwqSPXzPX+XUBnilqnoGdT171DKYqyG4hk1rKG3Ls6un+5544rlU1n1NMsDOfWrVqL8Ndwz8C6vxIiFgXeo4nohRpUIqhSj5sX9zoIk6Bp2xKDThO/npMeGeYR9gN6RgrjY8QuD/T1ubEpThQdT0urD7Mxc1PtH4DbchRzGfwQz8AzPfwNz+JYYTdxI3Ex+Sf5JDaeSITqUiDTX4dRIjf0FdAaTnw==</latexit>

p=1

d=1

– パラメータw、μ、σは通常のハイパーパラメータ最適化で学習できる– ARD事前分布を使うことで、不要なガウス分布を除去している

15 / 81

Summer School 数理物理2021 Lecture3

Airline Passengerデータ
l 1949-1961の毎月の航空乗客数のうち、最初の8年を学習に使って残りの4年分を予測
実際の
データ
スペクトル
混合
カーネル
Maternカーネル
ガウスカーネル

16 / 81

Summer School 数理物理2021 Lecture3

Airline Passengerデータ (2)ガウスカーネル
スペクトル混合カーネル

l 線形トレンド(一番左)に加え、12, 6, 4, 3ヶ月の周期性とその細かな違いを捉えている– 通常のガウスカーネル(赤)はまったくの大雑把
17 / 81

Summer School 数理物理2021 Lecture3

動画からの副詞の理解 (3)l スペクトル混合カーネルの適用：カーネルの周波数空間での特徴を抽出– 直接フーリエ変換すると、関数の位相に依存＆FFTにはデータが不足

l 一方で、各動作には「しっかり」「堂々と」などの副詞が付与されている
18 / 81

Summer School 数理物理2021 Lecture3

動画からの副詞の理解 (再掲)l さまざまな動作を表す関節座標の時系列姿勢データと、
l それを記述する副詞の集合がペアになったデータセットが得られた

サッサッと, 力強く 重々しく, 確実に素早く, 正確に速く, 重々しく, 鋭く 軽く, ゆっくりと, 忍んで19 / 81

Summer School 数理物理2021 Lecture3

テキストの潜在トピックモデル
l Hofmann (1999), Blei+ (2001) に始まる
l 文章の「意味」を自動的に解釈するためのモデル
l 解釈性の高さから、社会科学などでは特に現在でも使われている

20 / 81

Summer School 数理物理2021 Lecture3

“Bag of words” データ
l 文書の中に同時に現れる単語群には相関がある→ 文書の中に現れた“単語”とその頻度{“情報”:5,“IT”:2,“システム”:1,“開発”:4}{“バッター”:1,“監督”:3,“ドラフト”:2,“打率”:1}l 一般には、文書＝観測ユニット、単語＝離散データと読み替えてOK– 例：文書＝人、単語＝購入した商品

21 / 81

Summer School 数理物理2021 Lecture3

21

トピックモデルの学習例 (1) (Blei+ 2003)より

l それぞれの語がどんな「話題」(トピック)に属するのかが、大量のデータだけからわかる– 実際には、話題の確率がわかる
l 文書=人、単語=その人の買った商品、URL、‥‥

22 / 81

Summer School 数理物理2021 Lecture3

22

トピックモデルの学習例 (2)(Blei+ 2003)より

l コーパスから完全に自動的に関連語を抽出できる

23 / 81

Summer School 数理物理2021 Lecture3

23

LDAとはl トピックモデルの最も基本となるモデル
l Blei+ (NIPS 2001, JMLR 2003)で提案
l 基本的な考え方は、単語ごとに潜在トピックがあること (文書毎の混合モデル)

24 / 81

Summer School 数理物理2021 Lecture3

24

LDAの生成モデル
l 文書

を話題(トピック)の混合で表現

政治

l 混合比

をディリクレ事前分布から生成

×25 / 81スポーツ

×Summer School 数理物理2021 Lecture3

×LDAの生成モデル (2)1. トピック混合比を生成.

2. For n = 1 … N,a. トピック

を選択

b. 単語

を生成.

法案 点 国会

26 / 81

Summer School 数理物理2021 Lecture3

「政治」
トピック

議院

LDAの生成モデル (3)l 「話題」とは?à単語の生起確率分布「政治」
法案 点 国会

議院「スポーツ」
バスケット

27 / 81

Summer School 数理物理2021 Lecture3

点

フォーム 競泳LDAの学習例
l 川端康成「雪国」の冒頭– 2000年度毎日新聞記事全文 (2,887万語) で学習したモデルで分析
l 青色のトピックは冬に関係する
l 緑色のトピックは電車に関係する
l 黒色は地の文28 / 81

Summer School 数理物理2021 Lecture3

LDAのグラフィカルモデル

l

は

l

–

の順で単語 を生成回(文書数)、 は 回(単語数) 生成
はトピック別単語分布
2929 / 81

Summer School 数理物理2021 Lecture3

LDAの確率モデル
観測単語

トピック

よって、文書l パラメータは

30 / 81

トピック分布についてとSummer School 数理物理2021 Lecture3

30

Spectral Mixture LDA
l 動画に付与された副詞 w の集合と、

w：<latexit sha1_base64="EFLs/J8MhiPuYkx3JE7BnPCPzjs=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1YlHyRN1VSV12xaK45t2U7SJMoLwBXEgRNIHBCPwYUX4NBHQByLxIUD47UjJ9lJsZV4d+b3H8/srne5a5l+wNjpzOzchYuXtPnLC1euXru+uLR8o+I7Lc8QZcOxHK/KdV9Ypi3KgRlYoup6Qm9yS+zyxrPQv9sWnm869sug64qDpn5km3XT0AM0FTqHSymWYfJaURvZuJGC+Mo7y3MbUINX4IABLWiCABsCbFugg4/3PmSBgYu2A+ijzcOWKf0CBrCA2hZSAgkdrQ38P8Lefmy1sR/G9KXawLdY+PNQuQJp9oN9YWfsO/vKfrK/U2P1ZYwwly4+eaQV7uHi61ulP/9VNfEZwHGiOkfBkT6/pgDq8FjWYmJtrrSEVRpR/Hbv/VnpSTHdv8s+sV9Y30d2yr5hhXb7t/G5IIofMHpajrfASDVo49OAFI5xH3vDXDn6+tI6iPNJ+HGFI/se3ibmeIxqXfYc6EyNcEK864TgugTXJbgewfUIrkNwHYKrE1yd4BoE1yC4NYJbI7hNgtskuC2C2yK4KsFVCW6P4PYIjhEcUzgez2+4lql55fG8Rn51Prn8EoZ+QfibuOoToib7KuVijuYYF1ko0ldInyTD3MIVPsoObSodjtKQH1Uk9gGhsuS+wdEyqRv1UMq8UnF+SsUl+bU2JypJrKpieyyrRLM9kdOoaifO30CfhbvTjkKsT6wojv11hSoSVFGhnsvdJgU5SddxDwr3zmg2h6dGlEe0K9GeXBgXz7Ds5ImlNiq5TPZBJld4mFp9Gp9m83Ab7sA9jP8IVmEDx7+MOQh4A2/hnXZfe6FVtGqEzs7Empswdmn6P9AKjso=</latexit>z<latexit sha1_base64="2O6M6pKNmBYG0jx166tkBlGGsJA=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1YlHyRN1VSV12xaK45t2U7SJMoLwBXEgRNIHBCPwYUX4NBHQByLxIUD47UjJ9lJsZV4d+b3H8/srne5a5l+wNjpzOzchYuXtPnLC1euXru+uLR8o+I7Lc8QZcOxHK/KdV9Ypi3KgRlYoup6Qm9yS+zyxrPQv9sWnm869sug64qDpn5km3XT0AM0FXqHSymWYfJaURvZuJGC+Mo7y3MbUINX4IABLWiCABsCbFugg4/3PmSBgYu2A+ijzcOWKf0CBrCA2hZSAgkdrQ38P8Lefmy1sR/G9KXawLdY+PNQuQJp9oN9YWfsO/vKfrK/U2P1ZYwwly4+eaQV7uHi61ulP/9VNfEZwHGiOkfBkT6/pgDq8FjWYmJtrrSEVRpR/Hbv/VnpSTHdv8s+sV9Y30d2yr5hhXb7t/G5IIofMHpajrfASDVo49OAFI5xH3vDXDn6+tI6iPNJ+HGFI/se3ibmeIxqXfYc6EyNcEK864TgugTXJbgewfUIrkNwHYKrE1yd4BoE1yC4NYJbI7hNgtskuC2C2yK4KsFVCW6P4PYIjhEcUzgez2+4lql55fG8Rn51Prn8EoZ+QfibuOoToib7KuVijuYYF1ko0ldInyTD3MIVPsoObSodjtKQH1Uk9gGhsuS+wdEyqRv1UMq8UnF+SsUl+bU2JypJrKpieyyrRLM9kdOoaifO30CfhbvTjkKsT6wojv11hSoSVFGhnsvdJgU5SddxDwr3zmg2h6dGlEe0K9GeXBgXz7Ds5ImlNiq5TPZBJld4mFp9Gp9m83Ab7sA9jP8IVmEDx7+MOQh4A2/hnXZfe6FVtGqEzs7Empswdmn6P+XTjs0=</latexit>↵<latexit sha1_base64="dzDxZ4WRWRQQVVENdzDOwGDvgRQ=">AAAHrHichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJk2aKokqr1k3Jv6S7SRNorwAJ24IOIHEAfEYXHgBDn0ExLFIXDgwXjvKhyfFVuLdmd9/PLO73uWuafgBY2dz8wuXLl9RFq8uXbt+4+byyuqtsu+0PE2UNMd0vApXfWEatigFRmCKiusJ1eKmOOTN56H/sC0833Dsg6DrirqlntiGbmhqgKZyTTXdhnq8kmIZJq+1ZCMbN1IQX3lndWELavASHNCgBRYIsCHAtgkq+HhXIQsMXLTVoY82D1uG9AsYwBJqW0gJJFS0NvH/BHvV2GpjP4zpS7WGbzHx56FyDdLsB/vCztl39pX9ZH9nxurLGGEuXXzySCvc4+XXd4p//quy8BlAY6S6QMGRvrimAHR4ImsxsDZXWsIqtSh+u/f2vPi0kO7fZ5/YL6zvIztj37BCu/1b+7wvCh8welqOt8BINWjjU4MUjnEfe8NcOfr60jqI8xnxkwpH9j28DcyxgWpV9hzozIxwSrzrlOC6BNcluB7B9QiuQ3AdgtMJTie4JsE1CW6D4DYIbpvgtgluh+B2CK5CcBWCOyK4I4JjBMcSHI/nN1zL1LzyeF4jf3I+ufwShn5B+C1c9SOiJvtJysUcjQkuslCknyB9kgxzC1f4ODu0JelwlIb8uGJkHxAqU+4bHC3TunEPpcwnKs7PqLgov1ZrqpKRNanYnchqpNmdymlctRfnr6HPxN1pL0FsTq0ojv3NBFUgqEKCeiF3mxTkJK3jHhTundFsDk+NKI9oV6I9uTAunmHZ6RMr2SjnMtmHmdz+o9T6s/g0W4S7cA8eYPzHsA5bOP4lzOEVvIF38F7JKAdKValH6PxcrLkNE5ei/wOZi5Dn</latexit>

wN<latexit sha1_base64="FIRHTJ6y0aQjTcRcNcbK/HAMxJ0=">AAAHqXichZVLb9NAEMenLeBSHm3hgsSlIgpCAkWbgATi1AKVWrWq8iBtSltVXneTWvFLtpM0ifIFkLjSAyeQOCA+Bhe+AId+BMSxSFw4MF47ymMnxVbi3Znffzyzu97lnmUGIWNnU9Mzly5f0Wavzl27fuPm/MLire3AbfiGKBuu5foVrgfCMh1RDs3QEhXPF7rNLbHD6y8j/05T+IHpOq/DticObL3mmFXT0EM0lTqP2ocLKZZh8lpSG9mkkYLkyruLM2uwD0fgggENsEGAAyG2LdAhwHsPssDAQ9sBdNHmY8uUfgE9mENtAymBhI7WOv7XsLeXWB3sRzEDqTbwLRb+fFQuQZr9YF/YOfvOvrKf7O/EWF0ZI8qljU8ea4V3OP/2TunPf1U2PkM4HqguUHCkL64phCo8k7WYWJsnLVGVRhy/2Tk9Lz0vprv32Sf2C+v7yM7YN6zQaf42PhdE8QNGT8vxFhhpH5r4NCCFY9zFXj9Xjr6utPaSfAb8qMKVfR9vE3M8RrUuey60JkY4Id51QnBtgmsTXIfgOgTXIrgWwVUJrkpwdYKrE9wKwa0Q3DrBrRPcBsFtEFyF4CoEt0twuwTHCI4pHE/mN1rL1LzyZF5jvzqfXH4Jfb8g/Dau+gGxL/sq5WGO5ggXWygyUMiAJKPcohU+zPZtKh2NUp8fVgzsPUJlyX2Do2VcN+yhlHml4vyEikvya7XHKhlYVcXmSFYDzeZYTsOqrSR/A30W7k5bCrE6tqI49lcVqkhQRYV6JXebFOQkXcU9KNo749nsnxpxHvGuRHtyUVw8w7LjJ5ba2M5lso8zucKT1PKL5DSbhbtwDx5g/KewDGs4/mXMoQbv4D2cag+1glbR3sTo9FSiuQ0jl2b8A+q1j4Y=</latexit><latexit sha1_base64="EFLs/J8MhiPuYkx3JE7BnPCPzjs=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1YlHyRN1VSV12xaK45t2U7SJMoLwBXEgRNIHBCPwYUX4NBHQByLxIUD47UjJ9lJsZV4d+b3H8/srne5a5l+wNjpzOzchYuXtPnLC1euXru+uLR8o+I7Lc8QZcOxHK/KdV9Ypi3KgRlYoup6Qm9yS+zyxrPQv9sWnm869sug64qDpn5km3XT0AM0FTqHSymWYfJaURvZuJGC+Mo7y3MbUINX4IABLWiCABsCbFugg4/3PmSBgYu2A+ijzcOWKf0CBrCA2hZSAgkdrQ38P8Lefmy1sR/G9KXawLdY+PNQuQJp9oN9YWfsO/vKfrK/U2P1ZYwwly4+eaQV7uHi61ulP/9VNfEZwHGiOkfBkT6/pgDq8FjWYmJtrrSEVRpR/Hbv/VnpSTHdv8s+sV9Y30d2yr5hhXb7t/G5IIofMHpajrfASDVo49OAFI5xH3vDXDn6+tI6iPNJ+HGFI/se3ibmeIxqXfYc6EyNcEK864TgugTXJbgewfUIrkNwHYKrE1yd4BoE1yC4NYJbI7hNgtskuC2C2yK4KsFVCW6P4PYIjhEcUzgez2+4lql55fG8Rn51Prn8EoZ+QfibuOoToib7KuVijuYYF1ko0ldInyTD3MIVPsoObSodjtKQH1Uk9gGhsuS+wdEyqRv1UMq8UnF+SsUl+bU2JypJrKpieyyrRLM9kdOoaifO30CfhbvTjkKsT6wojv11hSoSVFGhnsvdJgU5SddxDwr3zmg2h6dGlEe0K9GeXBgXz7Ds5ImlNiq5TPZBJld4mFp9Gp9m83Ab7sA9jP8IVmEDx7+MOQh4A2/hnXZfe6FVtGqEzs7Empswdmn6P9AKjso=</latexit><latexit sha1_base64="8NIuWjUcVuWpm5f7vJ0xiiQ8VXI=">AAAHp3ichZVLb9NAEMenLeBSHm3hgsSlIgqCS7QJSCBOLVCpVauQB0lTNVXlNZvWil+ynaRJlC8AVxAHTiBxQHwMLnwBDv0IiGORuHBgvHaUx06LrcS7M7//eGZ3vcs9ywxCxk5mZucuXLykzV9euHL12vXFpeUb1cBt+YaoGK7l+jWuB8IyHVEJzdASNc8Xus0tscObzyL/Tlv4gek6L8OuJ/Zt/dAxG6ahh2gq5g+WUizD5LWiNrJJIwXJVXCX5zagDq/ABQNaYIMAB0JsW6BDgPceZIGBh7Z96KPNx5Yp/QIGsIDaFlICCR2tTfw/xN5eYnWwH8UMpNrAt1j481G5Amn2g31hp+w7+8p+sr9nxurLGFEuXXzyWCu8g8XXt8p//quy8RnC0Uh1joIjfX5NITTgsazFxNo8aYmqNOL47d770/KTUrp/l31iv7C+j+yEfcMKnfZv43NRlD5g9LQcb4GR6tDGpwEpHOM+9oa5cvT1pXWQ5DPiJxWu7Pt4m5jjEap12XOhc2aEY+JdxwTXJbguwfUIrkdwHYLrEFyD4BoE1yS4JsGtEdwawW0S3CbBbRHcFsHVCK5GcLsEt0twjOCYwvFkfqO1TM0rT+Y19qvzyeWXMPQLwm/jqh8RddlXKQ9zNCe42EKRgUIGJBnlFq3wcXZoU+lolIb8uGJkHxAqS+4bHC3TunEPpSwoFRfOqLgsv1Z7qpKRVVVsT2Q10mxP5TSuyif5G+izcHfKK8T61Iri2F9XqBJBlRTqudxtUpCTdAP3oGjvjGdzeGrEecS7Eu3JRXHxDMtOn1hqo5rLZB9kcsWHqdWnyWk2D7fhDtzD+I9gFTZw/CuYg4A38Bbeafe1F1pVq8Xo7EyiuQkTl6b/A6ZAjqE=</latexit><latexit sha1_base64="KVgb6x8deLP+hk8QSLdInjWywrc=">AAAHq3ichZXNbtNAEMenLeBSPtrCBYlLRRTEhWgTkECcWqBSq1ZVmpA2pakqr9m0Vvwl20maRHkBLhwBcQKJA+IxuPACHPoIiGORuHBgvHbkJDspthLvzvz+45nd9S73LDMIGTudmp65cPGSNnt57srVa9fnFxZv7ARu0zdExXAt169yPRCW6YhKaIaWqHq+0G1uiV3eeBb5d1vCD0zXeRF2PHFg60eOWTcNPURTpcZFqB8uZFiOyWtJbeSTRgaSq+guzqxBDV6BCwY0wQYBDoTYtkCHAO99yAMDD20H0EObjy1T+gX0YQ61TaQEEjpaG/h/hL39xOpgP4oZSLWBb7Hw56NyCbLsB/vCzth39pX9ZH8nxurJGFEuHXzyWCu8w/nXt8p//quy8RnCcao6R8GRPr+mEOrwWNZiYm2etERVGnH8VvftWflJKdu7yz6xX1jfR3bKvmGFTuu38XlblD5g9Kwcb4GRatDCpwEZHOMe9ga5cvT1pLWf5JPyowpX9n28TczxGNW67LnQnhjhhHjXCcF1CK5DcF2C6xJcm+DaBFcnuDrBNQiuQXArBLdCcOsEt05wGwS3QXBVgqsS3B7B7REcIzimcDyZ32gtU/PKk3mN/ep8cvklDPyC8Nu46lOiJvsq5WGO5ggXWygyUMiAJKPcohU+zA5sKh2N0oAfVqT2PqGy5L7B0TKuG/ZQyqJScXFCxWX5tdpjlaRWVbE5klWq2RzLaVi1leRvoM/C3WlLIVbHVhTH/qpClQiqpFDP5W6TgYKk67gHRXtnPJuDUyPOI96VaE8hiotnWH78xFIbO4Vc/kGusP0ws/w0Oc1m4TbcgXsY/xEswxqOfwVzMOENvIP32n2trL3UajE6PZVobsLIpYl/au2Qcw==</latexit>✓<latexit sha1_base64="ynPS7gJubn89RDmwNQQr7ofE/NE=">AAAHrHichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJk2aKokqr1k3Jv6S7SRNorwAJ24IOIHEAfEYXHgBDn0ExLFIXDgwXjvKhyfFVuLdmd9/PLO73uWuafgBY2dz8wuXLl9RFq8uXbt+4+byyuqtsu+0PE2UNMd0vApXfWEatigFRmCKiusJ1eKmOOTN56H/sC0833Dsg6DrirqlntiGbmhqgKZyLWiIQD1eSbEMk9daspGNGymIr7yzurAFNXgJDmjQAgsE2BBg2wQVfLyrkAUGLtrq0Eebhy1D+gUMYAm1LaQEEipam/h/gr1qbLWxH8b0pVrDt5j481C5Bmn2g31h5+w7+8p+sr8zY/VljDCXLj55pBXu8fLrO8U//1VZ+AygMVJdoOBIX1xTADo8kbUYWJsrLWGVWhS/3Xt7XnxaSPfvs0/sF9b3kZ2xb1ih3f6tfd4XhQ8YPS3HW2CkGrTxqUEKx7iPvWGuHH19aR3E+Yz4SYUj+x7eBubYQLUqew50ZkY4Jd51SnBdgusSXI/gegTXIbgOwekEpxNck+CaBLdBcBsEt01w2wS3Q3A7BFchuArBHRHcEcExgmMJjsfzG65lal55PK+RPzmfXH4JQ78g/Bau+hFRk/0k5WKOxgQXWSjST5A+SYa5hSt8nB3aknQ4SkN+XDGyDwiVKfcNjpZp3biHUuYTFednVFyUX6s1VcnImlTsTmQ10uxO5TSu2ovz19Bn4u60lyA2p1YUx/5mgioQVCFBvZC7TQpyktZxDwr3zmg2h6dGlEe0K9GeXBgXz7Ds9ImVbJRzmezDTG7/UWr9WXyaLcJduAcPMP5jWIctHP8S5vAK3sA7eK9klAOlqtQjdH4u1tyGiUvR/wEOAJD3</latexit><latexit sha1_base64="ynPS7gJubn89RDmwNQQr7ofE/NE=">AAAHrHichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJk2aKokqr1k3Jv6S7SRNorwAJ24IOIHEAfEYXHgBDn0ExLFIXDgwXjvKhyfFVuLdmd9/PLO73uWuafgBY2dz8wuXLl9RFq8uXbt+4+byyuqtsu+0PE2UNMd0vApXfWEatigFRmCKiusJ1eKmOOTN56H/sC0833Dsg6DrirqlntiGbmhqgKZyLWiIQD1eSbEMk9daspGNGymIr7yzurAFNXgJDmjQAgsE2BBg2wQVfLyrkAUGLtrq0Eebhy1D+gUMYAm1LaQEEipam/h/gr1qbLWxH8b0pVrDt5j481C5Bmn2g31h5+w7+8p+sr8zY/VljDCXLj55pBXu8fLrO8U//1VZ+AygMVJdoOBIX1xTADo8kbUYWJsrLWGVWhS/3Xt7XnxaSPfvs0/sF9b3kZ2xb1ih3f6tfd4XhQ8YPS3HW2CkGrTxqUEKx7iPvWGuHH19aR3E+Yz4SYUj+x7eBubYQLUqew50ZkY4Jd51SnBdgusSXI/gegTXIbgOwekEpxNck+CaBLdBcBsEt01w2wS3Q3A7BFchuArBHRHcEcExgmMJjsfzG65lal55PK+RPzmfXH4JQ78g/Bau+hFRk/0k5WKOxgQXWSjST5A+SYa5hSt8nB3aknQ4SkN+XDGyDwiVKfcNjpZp3biHUuYTFednVFyUX6s1VcnImlTsTmQ10uxO5TSu2ovz19Bn4u60lyA2p1YUx/5mgioQVCFBvZC7TQpyktZxDwr3zmg2h6dGlEe0K9GeXBgXz7Ds9ImVbJRzmezDTG7/UWr9WXyaLcJduAcPMP5jWIctHP8S5vAK3sA7eK9klAOlqtQjdH4u1tyGiUvR/wEOAJD3</latexit>

D<latexit sha1_base64="dgnbaQMVfaVGceN9ClP19VhJacI=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWmilVq1KPkiaqqkqr7tprTi2ZTtJkygvAFcQB04gcUA8BhdegEMfAXEsEhcOjNeOnGSnxVbi3Znffzyzu97lrmX6AWOnU9Mzly5f0Wavzl27fuPm/MLirYrvtDxDlA3Hcrwq131hmbYoB2ZgiarrCb3JLbHDGy9C/05beL7p2K+Criv2m/qRbdZNQw/QVFg9WEixDJPXktrIxo0UxFfeWZxZhxocggMGtKAJAmwIsG2BDj7ee5AFBi7a9qGPNg9bpvQLGMAcaltICSR0tDbw/wh7e7HVxn4Y05dqA99i4c9D5RKk2Q/2hZ2x7+wr+8n+nhurL2OEuXTxySOtcA/mX98p/fmvqonPAI4T1QUKjvTFNQVQh6eyFhNrc6UlrNKI4rd7789Kz4rp/n32if3C+j6yU/YNK7Tbv43PBVH8gNHTcrwFRqpBG58GpHCM+9gb5srR15fWQZxPwo8rHNn38DYxx2NU67LnQOfcCCfEu04IrktwXYLrEVyP4DoE1yG4OsHVCa5BcA2CWyG4FYLbILgNgtskuE2CqxJcleB2CW6X4BjBMYXj8fyGa5maVx7Pa+RX55PLL2HoF4S/ias+IWqyr1Iu5miOcZGFIn2F9EkyzC1c4aPs0KbS4SgN+VFFYh8QKkvuGxwtk7pRD6XMKxXnz6m4JL/W5kQliVVVbI1llWi2JnIaVW3H+Rvos3B32laItYkVxbG/plBFgioq1KrcbVKQk3Qd96Bw74xmc3hqRHlEuxLtyYVx8QzLTp5YaqOSy2QfZXKFx6nl5/FpNgt34R48wPhPYBnWcfzLmIOAN/AW3mkPtZdaRatG6PRUrLkNY5em/wNdoo6X</latexit>

l 各トピックは、副詞の確率分布を持つ

31 / 81

副詞
z,：y
トピック
✓：
潜在トピ
ック分布

Summer School 数理物理2021 Lecture3

K<latexit sha1_base64="6JmFPPx+QYs8W91EFceIMmhmqLo=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1QlHyRN1VSV12xaK3Zs2U7SJMoLwBXEgRNIHBCPwYUX4NBHQByLxIUD47UjJ9lpsZV4d+b3H8/srne5a5l+wNjJzOzchYuXtPnLC1euXru+uLR8o+o7bc8QFcOxHK/GdV9YZktUAjOwRM31hG5zS+zw5rPQv9MRnm86rZdBzxX7tn7YMhumoQdoKuYPllIsw+S1ojaycSMF8VVwluc2oA6vwAED2mCDgBYE2LZABx/vPcgCAxdt+zBAm4ctU/oFDGEBtW2kBBI6Wpv4f4i9vdjawn4Y05dqA99i4c9D5Qqk2Q/2hZ2y7+wr+8n+nhlrIGOEufTwySOtcA8WX98q//mvysZnAEeJ6hwFR/r8mgJowGNZi4m1udISVmlE8Tv996flJ6X04C77xH5hfR/ZCfuGFbY6v43PRVH6gNHTcrwFRqpDB58GpHCMB9gb5crRN5DWYZxPwk8qHNn38DYxxyNU67LnQPfMCMfEu44JrkdwPYLrE1yf4LoE1yW4BsE1CK5JcE2CWyO4NYLbJLhNgssTXJ7gagRXI7hdgtslOEZwTOF4PL/hWqbmlcfzGvnV+eTySxj5BeG3cdUnRF32VcrFHM0JLrJQpK+QPkmGuYUrfJwd2VQ6HKURP65I7ENCZcl9g6NlWjfuoZQFpeLCGRWX5ddqT1WSWFXF1kRWiWZrKqdx1Xacv4E+C3enbYVYn1pRHPvrClUiqJJCPZe7TQpykm7gHhTundFsjk6NKI9oV6I9uTAunmHZ6RNLbVRzmeyDTK74MLX6ND7N5uE23IF7GP8RrMIGjn8FcxDwBt7CO+2+9kKrarUInZ2JNTdh4tL0f5B3jp4=</latexit>

Spectral Mixture LDA
l 動画に付与された副詞の集合と、動作の周波数空間での成分の同時分布をモデル化
à スペクトル混合潜在ディリクレ配分法 (SMLDA)w：<latexit sha1_base64="EFLs/J8MhiPuYkx3JE7BnPCPzjs=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1YlHyRN1VSV12xaK45t2U7SJMoLwBXEgRNIHBCPwYUX4NBHQByLxIUD47UjJ9lJsZV4d+b3H8/srne5a5l+wNjpzOzchYuXtPnLC1euXru+uLR8o+I7Lc8QZcOxHK/KdV9Ypi3KgRlYoup6Qm9yS+zyxrPQv9sWnm869sug64qDpn5km3XT0AM0FTqHSymWYfJaURvZuJGC+Mo7y3MbUINX4IABLWiCABsCbFugg4/3PmSBgYu2A+ijzcOWKf0CBrCA2hZSAgkdrQ38P8Lefmy1sR/G9KXawLdY+PNQuQJp9oN9YWfsO/vKfrK/U2P1ZYwwly4+eaQV7uHi61ulP/9VNfEZwHGiOkfBkT6/pgDq8FjWYmJtrrSEVRpR/Hbv/VnpSTHdv8s+sV9Y30d2yr5hhXb7t/G5IIofMHpajrfASDVo49OAFI5xH3vDXDn6+tI6iPNJ+HGFI/se3ibmeIxqXfYc6EyNcEK864TgugTXJbgewfUIrkNwHYKrE1yd4BoE1yC4NYJbI7hNgtskuC2C2yK4KsFVCW6P4PYIjhEcUzgez2+4lql55fG8Rn51Prn8EoZ+QfibuOoToib7KuVijuYYF1ko0ldInyTD3MIVPsoObSodjtKQH1Uk9gGhsuS+wdEyqRv1UMq8UnF+SsUl+bU2JypJrKpieyyrRLM9kdOoaifO30CfhbvTjkKsT6wojv11hSoSVFGhnsvdJgU5SddxDwr3zmg2h6dGlEe0K9GeXBgXz7Ds5ImlNiq5TPZBJld4mFp9Gp9m83Ab7sA9jP8IVmEDx7+MOQh4A2/hnXZfe6FVtGqEzs7Empswdmn6P9AKjso=</latexit>z<latexit sha1_base64="2O6M6pKNmBYG0jx166tkBlGGsJA=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1YlHyRN1VSV12xaK45t2U7SJMoLwBXEgRNIHBCPwYUX4NBHQByLxIUD47UjJ9lJsZV4d+b3H8/srne5a5l+wNjpzOzchYuXtPnLC1euXru+uLR8o+I7Lc8QZcOxHK/KdV9Ypi3KgRlYoup6Qm9yS+zyxrPQv9sWnm869sug64qDpn5km3XT0AM0FXqHSymWYfJaURvZuJGC+Mo7y3MbUINX4IABLWiCABsCbFugg4/3PmSBgYu2A+ijzcOWKf0CBrCA2hZSAgkdrQ38P8Lefmy1sR/G9KXawLdY+PNQuQJp9oN9YWfsO/vKfrK/U2P1ZYwwly4+eaQV7uHi61ulP/9VNfEZwHGiOkfBkT6/pgDq8FjWYmJtrrSEVRpR/Hbv/VnpSTHdv8s+sV9Y30d2yr5hhXb7t/G5IIofMHpajrfASDVo49OAFI5xH3vDXDn6+tI6iPNJ+HGFI/se3ibmeIxqXfYc6EyNcEK864TgugTXJbgewfUIrkNwHYKrE1yd4BoE1yC4NYJbI7hNgtskuC2C2yK4KsFVCW6P4PYIjhEcUzgez2+4lql55fG8Rn51Prn8EoZ+QfibuOoToib7KuVijuYYF1ko0ldInyTD3MIVPsoObSodjtKQH1Uk9gGhsuS+wdEyqRv1UMq8UnF+SsUl+bU2JypJrKpieyyrRLM9kdOoaifO30CfhbvTjkKsT6wojv11hSoSVFGhnsvdJgU5SddxDwr3zmg2h6dGlEe0K9GeXBgXz7Ds5ImlNiq5TPZBJld4mFp9Gp9m83Ab7sA9jP8IVmEDx7+MOQh4A2/hnXZfe6FVtGqEzs7Empswdmn6P+XTjs0=</latexit>↵<latexit sha1_base64="dzDxZ4WRWRQQVVENdzDOwGDvgRQ=">AAAHrHichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJk2aKokqr1k3Jv6S7SRNorwAJ24IOIHEAfEYXHgBDn0ExLFIXDgwXjvKhyfFVuLdmd9/PLO73uWuafgBY2dz8wuXLl9RFq8uXbt+4+byyuqtsu+0PE2UNMd0vApXfWEatigFRmCKiusJ1eKmOOTN56H/sC0833Dsg6DrirqlntiGbmhqgKZyTTXdhnq8kmIZJq+1ZCMbN1IQX3lndWELavASHNCgBRYIsCHAtgkq+HhXIQsMXLTVoY82D1uG9AsYwBJqW0gJJFS0NvH/BHvV2GpjP4zpS7WGbzHx56FyDdLsB/vCztl39pX9ZH9nxurLGGEuXXzySCvc4+XXd4p//quy8BlAY6S6QMGRvrimAHR4ImsxsDZXWsIqtSh+u/f2vPi0kO7fZ5/YL6zvIztj37BCu/1b+7wvCh8welqOt8BINWjjU4MUjnEfe8NcOfr60jqI8xnxkwpH9j28DcyxgWpV9hzozIxwSrzrlOC6BNcluB7B9QiuQ3AdgtMJTie4JsE1CW6D4DYIbpvgtgluh+B2CK5CcBWCOyK4I4JjBMcSHI/nN1zL1LzyeF4jf3I+ufwShn5B+C1c9SOiJvtJysUcjQkuslCknyB9kgxzC1f4ODu0JelwlIb8uGJkHxAqU+4bHC3TunEPpcwnKs7PqLgov1ZrqpKRNanYnchqpNmdymlctRfnr6HPxN1pL0FsTq0ojv3NBFUgqEKCeiF3mxTkJK3jHhTundFsDk+NKI9oV6I9uTAunmHZ6RMr2SjnMtmHmdz+o9T6s/g0W4S7cA8eYPzHsA5bOP4lzOEVvIF38F7JKAdKValH6PxcrLkNE5ei/wOZi5Dn</latexit>

wN<latexit sha1_base64="05i1Qsam0Fm9F/5SK3jUPFG0cuo=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1YlHyRN1VSV12xaK45t2c638gJwBXHgBBIHxGNw4QU49BEQxyJx4cB47chJdlpsJd6d+f3HM7vrXe5aph8wdjIzO3fh4iVt/vLClavXri8uLd+o+E7LM0TZcCzHq3LdF5Zpi3JgBpaoup7Qm9wSu7zxLPTvtoXnm479Mui54qCpH9lm3TT0AE2F7uFSimWYvFbURjZupCC+8s7y3AbU4BU4YEALmiDAhgDbFujg470PWWDgou0ABmjzsGVKv4AhLKC2hZRAQkdrA/+PsLcfW23shzF9qTbwLRb+PFSuQJr9YF/YKfvOvrKf7O+ZsQYyRphLD5880gr3cPH1rdKf/6qa+AzgOFGdo+BIn19TAHV4LGsxsTZXWsIqjSh+u//+tPSkmB7cZZ/YL6zvIzth37BCu/3b+FwQxQ8YPS3HW2CkGrTxaUAKx3iAvVGuHH0DaR3G+ST8pMKRfQ9vE3M8RrUuew50zozQJd7VJbgewfUIrk9wfYLrEFyH4OoEVye4BsE1CG6N4NYIbpPgNglui+C2CK5KcFWC2yO4PYJjBMcUjsfzG65lal55PK+RX51PLr+EkV8Q/iau+oSoyb5KuZijOcFFFor0FdInyTC3cIWPsyObSoejNOLHFYl9SKgsuW9wtEzrxj2UMq9UnD+j4pL8WptTlSRWVbE9kVWi2Z7KaVy1E+dvoM/C3WlHIdanVhTH/rpCFQmqqFDP5W6Tgpyk67gHhXtnNJujUyPKI9qVaE8ujItnWHb6xFIblVwm+yCTKzxMrT6NT7N5uA134B7GfwSrsIHjX8YcBLyBt/BOu6+90CpaNUJnZ2LNTZi4NP0f102Oyw==</latexit><latexit sha1_base64="EFLs/J8MhiPuYkx3JE7BnPCPzjs=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1YlHyRN1VSV12xaK45t2U7SJMoLwBXEgRNIHBCPwYUX4NBHQByLxIUD47UjJ9lJsZV4d+b3H8/srne5a5l+wNjpzOzchYuXtPnLC1euXru+uLR8o+I7Lc8QZcOxHK/KdV9Ypi3KgRlYoup6Qm9yS+zyxrPQv9sWnm869sug64qDpn5km3XT0AM0FTqHSymWYfJaURvZuJGC+Mo7y3MbUINX4IABLWiCABsCbFugg4/3PmSBgYu2A+ijzcOWKf0CBrCA2hZSAgkdrQ38P8Lefmy1sR/G9KXawLdY+PNQuQJp9oN9YWfsO/vKfrK/U2P1ZYwwly4+eaQV7uHi61ulP/9VNfEZwHGiOkfBkT6/pgDq8FjWYmJtrrSEVRpR/Hbv/VnpSTHdv8s+sV9Y30d2yr5hhXb7t/G5IIofMHpajrfASDVo49OAFI5xH3vDXDn6+tI6iPNJ+HGFI/se3ibmeIxqXfYc6EyNcEK864TgugTXJbgewfUIrkNwHYKrE1yd4BoE1yC4NYJbI7hNgtskuC2C2yK4KsFVCW6P4PYIjhEcUzgez2+4lql55fG8Rn51Prn8EoZ+QfibuOoToib7KuVijuYYF1ko0ldInyTD3MIVPsoObSodjtKQH1Uk9gGhsuS+wdEyqRv1UMq8UnF+SsUl+bU2JypJrKpieyyrRLM9kdOoaifO30CfhbvTjkKsT6wojv11hSoSVFGhnsvdJgU5SddxDwr3zmg2h6dGlEe0K9GeXBgXz7Ds5ImlNiq5TPZBJld4mFp9Gp9m83Ab7sA9jP8IVmEDx7+MOQh4A2/hnXZfe6FVtGqEzs7Empswdmn6P9AKjso=</latexit><latexit sha1_base64="8NIuWjUcVuWpm5f7vJ0xiiQ8VXI=">AAAHp3ichZVLb9NAEMenLeBSHm3hgsSlIgqCS7QJSCBOLVCpVauQB0lTNVXlNZvWil+ynaRJlC8AVxAHTiBxQHwMLnwBDv0IiGORuHBgvHaUx06LrcS7M7//eGZ3vcs9ywxCxk5mZucuXLykzV9euHL12vXFpeUb1cBt+YaoGK7l+jWuB8IyHVEJzdASNc8Xus0tscObzyL/Tlv4gek6L8OuJ/Zt/dAxG6ahh2gq5g+WUizD5LWiNrJJIwXJVXCX5zagDq/ABQNaYIMAB0JsW6BDgPceZIGBh7Z96KPNx5Yp/QIGsIDaFlICCR2tTfw/xN5eYnWwH8UMpNrAt1j481G5Amn2g31hp+w7+8p+sr9nxurLGFEuXXzyWCu8g8XXt8p//quy8RnC0Uh1joIjfX5NITTgsazFxNo8aYmqNOL47d770/KTUrp/l31iv7C+j+yEfcMKnfZv43NRlD5g9LQcb4GR6tDGpwEpHOM+9oa5cvT1pXWQ5DPiJxWu7Pt4m5jjEap12XOhc2aEY+JdxwTXJbguwfUIrkdwHYLrEFyD4BoE1yS4JsGtEdwawW0S3CbBbRHcFsHVCK5GcLsEt0twjOCYwvFkfqO1TM0rT+Y19qvzyeWXMPQLwm/jqh8RddlXKQ9zNCe42EKRgUIGJBnlFq3wcXZoU+lolIb8uGJkHxAqS+4bHC3TunEPpSwoFRfOqLgsv1Z7qpKRVVVsT2Q10mxP5TSuyif5G+izcHfKK8T61Iri2F9XqBJBlRTqudxtUpCTdAP3oGjvjGdzeGrEecS7Eu3JRXHxDMtOn1hqo5rLZB9kcsWHqdWnyWk2D7fhDtzD+I9gFTZw/CuYg4A38Bbeafe1F1pVq8Xo7EyiuQkTl6b/A6ZAjqE=</latexit><latexit sha1_base64="KVgb6x8deLP+hk8QSLdInjWywrc=">AAAHq3ichZXNbtNAEMenLeBSPtrCBYlLRRTEhWgTkECcWqBSq1ZVmpA2pakqr9m0Vvwl20maRHkBLhwBcQKJA+IxuPACHPoIiGORuHBgvHbkJDspthLvzvz+45nd9S73LDMIGTudmp65cPGSNnt57srVa9fnFxZv7ARu0zdExXAt169yPRCW6YhKaIaWqHq+0G1uiV3eeBb5d1vCD0zXeRF2PHFg60eOWTcNPURTpcZFqB8uZFiOyWtJbeSTRgaSq+guzqxBDV6BCwY0wQYBDoTYtkCHAO99yAMDD20H0EObjy1T+gX0YQ61TaQEEjpaG/h/hL39xOpgP4oZSLWBb7Hw56NyCbLsB/vCzth39pX9ZH8nxurJGFEuHXzyWCu8w/nXt8p//quy8RnCcao6R8GRPr+mEOrwWNZiYm2etERVGnH8VvftWflJKdu7yz6xX1jfR3bKvmGFTuu38XlblD5g9Kwcb4GRatDCpwEZHOMe9ga5cvT1pLWf5JPyowpX9n28TczxGNW67LnQnhjhhHjXCcF1CK5DcF2C6xJcm+DaBFcnuDrBNQiuQXArBLdCcOsEt05wGwS3QXBVgqsS3B7B7REcIzimcDyZ32gtU/PKk3mN/ep8cvklDPyC8Nu46lOiJvsq5WGO5ggXWygyUMiAJKPcohU+zA5sKh2N0oAfVqT2PqGy5L7B0TKuG/ZQyqJScXFCxWX5tdpjlaRWVbE5klWq2RzLaVi1leRvoM/C3WlLIVbHVhTH/qpClQiqpFDP5W6TgYKk67gHRXtnPJuDUyPOI96VaE8hiotnWH78xFIbO4Vc/kGusP0ws/w0Oc1m4TbcgXsY/xEswxqOfwVzMOENvIP32n2trL3UajE6PZVobsLIpYl/au2Qcw==</latexit>

µ, ⌃

✓

<latexit sha1_base64="AITDTYeYszXZQqwAw+WBt2OUmSM=">AAAHsHichZXNbtNAEMenLeBSPtrCBYlLRRTEAUWbgATi1AKVWrWq0oS0KU0Vec0mNfGXbCdpEuUFOCNxQCCBxAHxGFx4AQ59BMSxSFw4MF47ysdOW1uJd2d+//HM7nqXe5YZhIwdT03PXLh4SZu9PHfl6rXr8wuLN3YCt+kbomS4luuXuR4Iy3REKTRDS5Q9X+g2t8QubzyL/Lst4Qem67wIO544sPW6Y9ZMQw/R9LJiN+9Ximbd1qsLKZZh8lpSG9mkkYLkyruLM2tQgVfgggFNsEGAAyG2LdAhwHsfssDAQ9sB9NDmY8uUfgF9mENtEymBhI7WBv7XsbefWB3sRzEDqTbwLRb+fFQuQZr9ZF/ZCfvBvrFf7N+psXoyRpRLB5881gqvOv/mVvHvuSobnyEcDlVnKDjSZ9cUQg0ey1pMrM2TlqhKI47f6r47KT4ppHt32Wf2G+v7xI7Zd6zQaf0xvmyLwnuMnpbjLTBSBVr4NCCFY9zD3iBXjr6etPaTfIb8uMKVfR9vE3M8RLUuey60T41wRLzriOA6BNchuC7BdQmuTXBtgqsRXI3gGgTXILgVglshuHWCWye4DYLbILgywZUJbo/g9giOERxTOJ7Mb7SWqXnlybzGfnU+ufwSBn5B+G1c9UOiIvsq5WGO5hgXWygyUMiAJKPcohU+yg5sKh2N0oAfVQztfUJlyX2Do2VSN+qhlHml4vwpFRfl12pPVDK0qorNsayGms2JnEZVW0n+Bvos3J22FGJ1YkVx7K8qVIGgCgr1XO42KchJuoZ7ULR3xrM5ODXiPOJdifbkorh4hmUnTyy1sZPLZB9kctsPU8tPk9NsFm7DHbiH8R/BMqzh+JcwBwfewgf4qOW0slbV9Bidnko0N2Hs0l7/B/uxkmQ=</latexit><latexit sha1_base64="ynPS7gJubn89RDmwNQQr7ofE/NE=">AAAHrHichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJk2aKokqr1k3Jv6S7SRNorwAJ24IOIHEAfEYXHgBDn0ExLFIXDgwXjvKhyfFVuLdmd9/PLO73uWuafgBY2dz8wuXLl9RFq8uXbt+4+byyuqtsu+0PE2UNMd0vApXfWEatigFRmCKiusJ1eKmOOTN56H/sC0833Dsg6DrirqlntiGbmhqgKZyLWiIQD1eSbEMk9daspGNGymIr7yzurAFNXgJDmjQAgsE2BBg2wQVfLyrkAUGLtrq0Eebhy1D+gUMYAm1LaQEEipam/h/gr1qbLWxH8b0pVrDt5j481C5Bmn2g31h5+w7+8p+sr8zY/VljDCXLj55pBXu8fLrO8U//1VZ+AygMVJdoOBIX1xTADo8kbUYWJsrLWGVWhS/3Xt7XnxaSPfvs0/sF9b3kZ2xb1ih3f6tfd4XhQ8YPS3HW2CkGrTxqUEKx7iPvWGuHH19aR3E+Yz4SYUj+x7eBubYQLUqew50ZkY4Jd51SnBdgusSXI/gegTXIbgOwekEpxNck+CaBLdBcBsEt01w2wS3Q3A7BFchuArBHRHcEcExgmMJjsfzG65lal55PK+RPzmfXH4JQ78g/Bau+hFRk/0k5WKOxgQXWSjST5A+SYa5hSt8nB3aknQ4SkN+XDGyDwiVKfcNjpZp3biHUuYTFednVFyUX6s1VcnImlTsTmQ10uxO5TSu2ovz19Bn4u60lyA2p1YUx/5mgioQVCFBvZC7TQpyktZxDwr3zmg2h6dGlEe0K9GeXBgXz7Ds9ImVbJRzmezDTG7/UWr9WXyaLcJduAcPMP5jWIctHP8S5vAK3sA7eK9klAOlqtQjdH4u1tyGiUvR/wEOAJD3</latexit>

y<latexit sha1_base64="7M9nOU5+dnI0VTl7I55Qzi3u36c=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1YlHyRN1VSV12xaK45t2U7SJMoLwBXEgRNIHBCPwYUX4NBHQByLxIUD47UjJ9lJsZV4d+b3H8/srne5a5l+wNjpzOzchYuXtPnLC1euXru+uLR8o+I7Lc8QZcOxHK/KdV9Ypi3KgRlYoup6Qm9yS+zyxrPQv9sWnm869sug64qDpn5km3XT0AM0FbqHSymWYfJaURvZuJGC+Mo7y3MbUINX4IABLWiCABsCbFugg4/3PmSBgYu2A+ijzcOWKf0CBrCA2hZSAgkdrQ38P8Lefmy1sR/G9KXawLdY+PNQuQJp9oN9YWfsO/vKfrK/U2P1ZYwwly4+eaQV7uHi61ulP/9VNfEZwHGiOkfBkT6/pgDq8FjWYmJtrrSEVRpR/Hbv/VnpSTHdv8s+sV9Y30d2yr5hhXb7t/G5IIofMHpajrfASDVo49OAFI5xH3vDXDn6+tI6iPNJ+HGFI/se3ibmeIxqXfYc6EyNcEK864TgugTXJbgewfUIrkNwHYKrE1yd4BoE1yC4NYJbI7hNgtskuC2C2yK4KsFVCW6P4PYIjhEcUzgez2+4lql55fG8Rn51Prn8EoZ+QfibuOoToib7KuVijuYYF1ko0ldInyTD3MIVPsoObSodjtKQH1Uk9gGhsuS+wdEyqRv1UMq8UnF+SsUl+bU2JypJrKpieyyrRLM9kdOoaifO30CfhbvTjkKsT6wojv11hSoSVFGhnsvdJgU5SddxDwr3zmg2h6dGlEe0K9GeXBgXz7Ds5ImlNiq5TPZBJld4mFp9Gp9m83Ab7sA9jP8IVmEDx7+MOQh4A2/hnXZfe6FVtGqEzs7Empswdmn6P96Qjsw=</latexit>

副詞
x：
周波数
z, y：
トピック
✓：
潜在トピ
ック分布

xM P

<latexit sha1_base64="ynPS7gJubn89RDmwNQQr7ofE/NE=">AAAHrHichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJk2aKokqr1k3Jv6S7SRNorwAJ24IOIHEAfEYXHgBDn0ExLFIXDgwXjvKhyfFVuLdmd9/PLO73uWuafgBY2dz8wuXLl9RFq8uXbt+4+byyuqtsu+0PE2UNMd0vApXfWEatigFRmCKiusJ1eKmOOTN56H/sC0833Dsg6DrirqlntiGbmhqgKZyLWiIQD1eSbEMk9daspGNGymIr7yzurAFNXgJDmjQAgsE2BBg2wQVfLyrkAUGLtrq0Eebhy1D+gUMYAm1LaQEEipam/h/gr1qbLWxH8b0pVrDt5j481C5Bmn2g31h5+w7+8p+sr8zY/VljDCXLj55pBXu8fLrO8U//1VZ+AygMVJdoOBIX1xTADo8kbUYWJsrLWGVWhS/3Xt7XnxaSPfvs0/sF9b3kZ2xb1ih3f6tfd4XhQ8YPS3HW2CkGrTxqUEKx7iPvWGuHH19aR3E+Yz4SYUj+x7eBubYQLUqew50ZkY4Jd51SnBdgusSXI/gegTXIbgOwekEpxNck+CaBLdBcBsEt01w2wS3Q3A7BFchuArBHRHcEcExgmMJjsfzG65lal55PK+RPzmfXH4JQ78g/Bau+hFRk/0k5WKOxgQXWSjST5A+SYa5hSt8nB3aknQ4SkN+XDGyDwiVKfcNjpZp3biHUuYTFednVFyUX6s1VcnImlTsTmQ10uxO5TSu2ovz19Bn4u60lyA2p1YUx/5mgioQVCFBvZC7TQpyktZxDwr3zmg2h6dGlEe0K9GeXBgXz7Ds9ImVbJRzmezDTG7/UWr9WXyaLcJduAcPMP5jWIctHP8S5vAK3sA7eK9klAOlqtQjdH4u1tyGiUvR/wEOAJD3</latexit><latexit sha1_base64="05i1Qsam0Fm9F/5SK3jUPFG0cuo=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1YlHyRN1VSV12xaK45t2c638gJwBXHgBBIHxGNw4QU49BEQxyJx4cB47chJdlpsJd6d+f3HM7vrXe5aph8wdjIzO3fh4iVt/vLClavXri8uLd+o+E7LM0TZcCzHq3LdF5Zpi3JgBpaoup7Qm9wSu7zxLPTvtoXnm479Mui54qCpH9lm3TT0AE2F7uFSimWYvFbURjZupCC+8s7y3AbU4BU4YEALmiDAhgDbFujg470PWWDgou0ABmjzsGVKv4AhLKC2hZRAQkdrA/+PsLcfW23shzF9qTbwLRb+PFSuQJr9YF/YKfvOvrKf7O+ZsQYyRphLD5880gr3cPH1rdKf/6qa+AzgOFGdo+BIn19TAHV4LGsxsTZXWsIqjSh+u//+tPSkmB7cZZ/YL6zvIzth37BCu/3b+FwQxQ8YPS3HW2CkGrTxaUAKx3iAvVGuHH0DaR3G+ST8pMKRfQ9vE3M8RrUuew50zozQJd7VJbgewfUIrk9wfYLrEFyH4OoEVye4BsE1CG6N4NYIbpPgNglui+C2CK5KcFWC2yO4PYJjBMcUjsfzG65lal55PK+RX51PLr+EkV8Q/iau+oSoyb5KuZijOcFFFor0FdInyTC3cIWPsyObSoejNOLHFYl9SKgsuW9wtEzrxj2UMq9UnD+j4pL8WptTlSRWVbE9kVWi2Z7KaVy1E+dvoM/C3WlHIdanVhTH/rpCFQmqqFDP5W6Tgpyk67gHhXtnNJujUyPKI9qVaE8ujItnWHb6xFIblVwm+yCTKzxMrT6NT7N5uA134B7GfwSrsIHjX8YcBLyBt/BOu6+90CpaNUJnZ2LNTZi4NP0f102Oyw==</latexit><latexit sha1_base64="Vwn3ZZVFURP9o9lIJ7xfmnETmuQ=">AAAHp3ichZVLb9NAEMenLeBSHm3hgsSlIgqCS7QJSCBOLVCpVUvJg6Sp2qrymk1rxS/ZTtIkyheAK4gDJ5A4ID4GF74Ah34ExLFIXDgwXjvKYyetrcS7M7//eGZ3vcs9ywxCxk6mpmcuXLykzV6eu3L12vX5hcUblcBt+IYoG67l+lWuB8IyHVEOzdASVc8Xus0tsc3rzyL/dlP4gek6r8K2J/Zt/dAxa6ahh2gqvDhYSLEMk9eS2sgmjRQkV95dnFmDPXgNLhjQABsEOBBi2wIdArx3IQsMPLTtQxdtPrZM6RfQgznUNpASSOhoreP/IfZ2E6uD/ShmINUGvsXCn4/KJUizn+wrO2U/2Df2i/2bGKsrY0S5tPHJY63wDubf3Cr9PVdl4zOEo4HqDAVH+uyaQqjBY1mLibV50hJVacTxm50Pp6UnxXT3LvvMfmN9n9gJ+44VOs0/xpeCKH7E6Gk53gIj7UETnwakcIy72OvnytHXldZeks+AH1W4su/jbWKOR6jWZc+F1sQIx8S7jgmuTXBtgusQXIfgWgTXIrgawdUIrk5wdYJbIbgVglsnuHWC2yC4DYKrElyV4HYIbofgGMExhePJ/EZrmZpXnsxr7Ffnk8svoe8XhN/GVT8g9mRfpTzM0RzhYgtFBgoZkGSUW7TCh9m+TaWjUerzw4qBvUeoLLlvcLSM64Y9lDKvVJyfUHFJfq32WCUDq6rYHMlqoNkcy2lYtZXkb6DPwt1pSyFWx1YUx/6qQhUJqqhQz+Vuk4KcpGu4B0V7Zzyb/VMjziPelWhPLoqLZ1h2/MRSG5VcJvsgkys8TC0/TU6zWbgNd+Aexn8Ey7CG41/GHAS8hXfwXruvvdQqWjVGp6cSzU0YuTT9P579jqA=</latexit><latexit sha1_base64="Lyc9+QmYu4Z587ey+fucMquNi1A=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1YlHyRN1VSV12xaK45t2U7SJMoLwBXEgRNIHBCPwYUX4NBHQByLxIUD47UjJ9lJsZV4d+b3H8/srne5a5l+wNjpzOzchYuXtPnLC1euXru+uLR8o+I7Lc8QZcOxHK/KdV9Ypi3KgRlYoup6Qm9yS+zyxrPQv9sWnm869sug64qDpn5km3XT0AM0FfKHSymWYfJaURvZuJGC+Mo7y3MbUINX4IABLWiCABsCbFugg4/3PmSBgYu2A+ijzcOWKf0CBrCA2hZSAgkdrQ38P8Lefmy1sR/G9KXawLdY+PNQuQJp9oN9YWfsO/vKfrK/U2P1ZYwwly4+eaQV7uHi61ulP/9VNfEZwHGiOkfBkT6/pgDq8FjWYmJtrrSEVRpR/Hbv/VnpSTHdv8s+sV9Y30d2yr5hhXb7t/G5IIofMHpajrfASDVo49OAFI5xH3vDXDn6+tI6iPNJ+HGFI/se3ibmeIxqXfYc6EyNcEK864TgugTXJbgewfUIrkNwHYKrE1yd4BoE1yC4NYJbI7hNgtskuC2C2yK4KsFVCW6P4PYIjhEcUzgez2+4lql55fG8Rn51Prn8EoZ+QfibuOoToib7KuVijuYYF1ko0ldInyTD3MIVPsoObSodjtKQH1Uk9gGhsuS+wdEyqRv1UMq8UnF+SsUl+bU2JypJrKpieyyrRLM9kdOoaifO30CfhbvTjkKsT6wojv11hSoSVFGhnsvdJgU5SddxDwr3zmg2h6dGlEe0K9GeXBgXz7Ds5ImlNiq5TPZBJld4mFp9Gp9m83Ab7sA9jP8IVmEDx7+MOQh4A2/hnXZfe6FVtGqEzs7Empswdmn6P7TGjqM=</latexit><latexit sha1_base64="FIRHTJ6y0aQjTcRcNcbK/HAMxJ0=">AAAHqXichZVLb9NAEMenLeBSHm3hgsSlIgpCAkWbgATi1AKVWrWq8iBtSltVXneTWvFLtpM0ifIFkLjSAyeQOCA+Bhe+AId+BMSxSFw4MF47ymMnxVbi3Znffzyzu97lnmUGIWNnU9Mzly5f0Wavzl27fuPm/MLire3AbfiGKBuu5foVrgfCMh1RDs3QEhXPF7rNLbHD6y8j/05T+IHpOq/DticObL3mmFXT0EM0lTqP2ocLKZZh8lpSG9mkkYLkyruLM2uwD0fgggENsEGAAyG2LdAhwHsPssDAQ9sBdNHmY8uUfgE9mENtAymBhI7WOv7XsLeXWB3sRzEDqTbwLRb+fFQuQZr9YF/YOfvOvrKf7O/EWF0ZI8qljU8ea4V3OP/2TunPf1U2PkM4HqguUHCkL64phCo8k7WYWJsnLVGVRhy/2Tk9Lz0vprv32Sf2C+v7yM7YN6zQaf42PhdE8QNGT8vxFhhpH5r4NCCFY9zFXj9Xjr6utPaSfAb8qMKVfR9vE3M8RrUuey60JkY4Id51QnBtgmsTXIfgOgTXIrgWwVUJrkpwdYKrE9wKwa0Q3DrBrRPcBsFtEFyF4CoEt0twuwTHCI4pHE/mN1rL1LzyZF5jvzqfXH4Jfb8g/Dau+gGxL/sq5WGO5ggXWygyUMiAJKPcohU+zPZtKh2NUp8fVgzsPUJlyX2Do2VcN+yhlHml4vyEikvya7XHKhlYVcXmSFYDzeZYTsOqrSR/A30W7k5bCrE6tqI49lcVqkhQRYV6JXebFOQkXcU9KNo749nsnxpxHvGuRHtyUVw8w7LjJ5ba2M5lso8zucKT1PKL5DSbhbtwDx5g/KewDGs4/mXMoQbv4D2cag+1glbR3sTo9FSiuQ0jl2b8A+q1j4Y=</latexit>

D<latexit sha1_base64="dgnbaQMVfaVGceN9ClP19VhJacI=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWmilVq1KPkiaqqkqr7tprTi2ZTtJkygvAFcQB04gcUA8BhdegEMfAXEsEhcOjNeOnGSnxVbi3Znffzyzu97lrmX6AWOnU9Mzly5f0Wavzl27fuPm/MLirYrvtDxDlA3Hcrwq131hmbYoB2ZgiarrCb3JLbHDGy9C/05beL7p2K+Criv2m/qRbdZNQw/QVFg9WEixDJPXktrIxo0UxFfeWZxZhxocggMGtKAJAmwIsG2BDj7ee5AFBi7a9qGPNg9bpvQLGMAcaltICSR0tDbw/wh7e7HVxn4Y05dqA99i4c9D5RKk2Q/2hZ2x7+wr+8n+nhurL2OEuXTxySOtcA/mX98p/fmvqonPAI4T1QUKjvTFNQVQh6eyFhNrc6UlrNKI4rd7789Kz4rp/n32if3C+j6yU/YNK7Tbv43PBVH8gNHTcrwFRqpBG58GpHCM+9gb5srR15fWQZxPwo8rHNn38DYxx2NU67LnQOfcCCfEu04IrktwXYLrEVyP4DoE1yG4OsHVCa5BcA2CWyG4FYLbILgNgtskuE2CqxJcleB2CW6X4BjBMYXj8fyGa5maVx7Pa+RX55PLL2HoF4S/ias+IWqyr1Iu5miOcZGFIn2F9EkyzC1c4aPs0KbS4SgN+VFFYh8QKkvuGxwtk7pRD6XMKxXnz6m4JL/W5kQliVVVbI1llWi2JnIaVW3H+Rvos3B32laItYkVxbG/plBFgioq1KrcbVKQk3Qd96Bw74xmc3hqRHlEuxLtyYVx8QzLTp5YaqOSy2QfZXKFx6nl5/FpNgt34R48wPhPYBnWcfzLmIOAN/AW3mkPtZdaRatG6PRUrLkNY5em/wNdoo6X</latexit>

K<latexit sha1_base64="6JmFPPx+QYs8W91EFceIMmhmqLo=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1QlHyRN1VSV12xaK3Zs2U7SJMoLwBXEgRNIHBCPwYUX4NBHQByLxIUD47UjJ9lpsZV4d+b3H8/srne5a5l+wNjJzOzchYuXtPnLC1euXru+uLR8o+o7bc8QFcOxHK/GdV9YZktUAjOwRM31hG5zS+zw5rPQv9MRnm86rZdBzxX7tn7YMhumoQdoKuYPllIsw+S1ojaycSMF8VVwluc2oA6vwAED2mCDgBYE2LZABx/vPcgCAxdt+zBAm4ctU/oFDGEBtW2kBBI6Wpv4f4i9vdjawn4Y05dqA99i4c9D5Qqk2Q/2hZ2y7+wr+8n+nhlrIGOEufTwySOtcA8WX98q//mvysZnAEeJ6hwFR/r8mgJowGNZi4m1udISVmlE8Tv996flJ6X04C77xH5hfR/ZCfuGFbY6v43PRVH6gNHTcrwFRqpDB58GpHCMB9gb5crRN5DWYZxPwk8qHNn38DYxxyNU67LnQPfMCMfEu44JrkdwPYLrE1yf4LoE1yW4BsE1CK5JcE2CWyO4NYLbJLhNgssTXJ7gagRXI7hdgtslOEZwTOF4PL/hWqbmlcfzGvnV+eTySxj5BeG3cdUnRF32VcrFHM0JLrJQpK+QPkmGuYUrfJwd2VQ6HKURP65I7ENCZcl9g6NlWjfuoZQFpeLCGRWX5ddqT1WSWFXF1kRWiWZrKqdx1Xacv4E+C3enbYVYn1pRHPvrClUiqJJCPZe7TQpykm7gHhTundFsjk6NKI9oV6I9uTAunmHZ6RNLbVRzmeyDTK74MLX6ND7N5uE23IF7GP8RrMIGjn8FcxDwBt7CO+2+9kKrarUInZ2JNTdh4tL0f5B3jp4=</latexit>

l 各トピックは、副詞の確率分布と周波数空間でのガウス分布の両方を持つ
32 / 81

Summer School 数理物理2021 Lecture3

動画からの副詞の理解 (再掲)l スペクトル混合カーネルの適用：カーネルの周波数空間での特徴を抽出

x<latexit sha1_base64="05i1Qsam0Fm9F/5SK3jUPFG0cuo=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1YlHyRN1VSV12xaK45t2c638gJwBXHgBBIHxGNw4QU49BEQxyJx4cB47chJdlpsJd6d+f3HM7vrXe5aph8wdjIzO3fh4iVt/vLClavXri8uLd+o+E7LM0TZcCzHq3LdF5Zpi3JgBpaoup7Qm9wSu7zxLPTvtoXnm479Mui54qCpH9lm3TT0AE2F7uFSimWYvFbURjZupCC+8s7y3AbU4BU4YEALmiDAhgDbFujg470PWWDgou0ABmjzsGVKv4AhLKC2hZRAQkdrA/+PsLcfW23shzF9qTbwLRb+PFSuQJr9YF/YKfvOvrKf7O+ZsQYyRphLD5880gr3cPH1rdKf/6qa+AzgOFGdo+BIn19TAHV4LGsxsTZXWsIqjSh+u//+tPSkmB7cZZ/YL6zvIzth37BCu/3b+FwQxQ8YPS3HW2CkGrTxaUAKx3iAvVGuHH0DaR3G+ST8pMKRfQ9vE3M8RrUuew50zozQJd7VJbgewfUIrk9wfYLrEFyH4OoEVye4BsE1CG6N4NYIbpPgNglui+C2CK5KcFWC2yO4PYJjBMcUjsfzG65lal55PK+RX51PLr+EkV8Q/iau+oSoyb5KuZijOcFFFor0FdInyTC3cIWPsyObSoejNOLHFYl9SKgsuW9wtEzrxj2UMq9UnD+j4pL8WptTlSRWVbE9kVWi2Z7KaVy1E+dvoM/C3WlHIdanVhTH/rpCFQmqqFDP5W6Tgpyk67gHhXtnNJujUyPKI9qVaE8ujItnWHb6xFIblVwm+yCTKzxMrT6NT7N5uA134B7GfwSrsIHjX8YcBLyBt/BOu6+90CpaNUJnZ2LNTZi4NP0f102Oyw==</latexit>– 直接フーリエ変換すると、関数の位相に依存＆FFTにはデータが不足

l 一方で、各動作には「しっかり」「堂々と」などの副詞が付与されている
33 / 81

Summer School 数理物理2021 Lecture3

Spectral Mixture LDA (2)l 推定された副詞のトピックと、対応する対数周波数
Topic 1

Topic 2

Topic 3Topic 4

Topic 5Topic 6

Topic 7

硬く

だらだら

大股で

痛そうに

楽しそうに

堂々と

ゆっくりと

奇妙に

元気なく

威嚇して クタクタな リズミカル サッサッ じっくりとふざけて気だるく どっしりと 足が痛い

普通に

静々と

ぎこちなく 脱力して

忍んで

辛そうに

きびきび

力強く

確実に

不自然に

偉そうな

よろよろ

軽快に

颯爽と

一歩ずつ

辛い

GPLVM 1次元目
34 / 81

調子よく

GPLVM 2次元目

Summer School 数理物理2021 Lecture3

GPLVM 3次元目

Spectral Mixture LDA (3)l テストデータの動画からトピック分布と副詞を予測
確率

動作の
潜在
トピック
分布 ✓
<latexit sha1_base64="ynPS7gJubn89RDmwNQQr7ofE/NE=">AAAHrHichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJk2aKokqr1k3Jv6S7SRNorwAJ24IOIHEAfEYXHgBDn0ExLFIXDgwXjvKhyfFVuLdmd9/PLO73uWuafgBY2dz8wuXLl9RFq8uXbt+4+byyuqtsu+0PE2UNMd0vApXfWEatigFRmCKiusJ1eKmOOTN56H/sC0833Dsg6DrirqlntiGbmhqgKZyLWiIQD1eSbEMk9daspGNGymIr7yzurAFNXgJDmjQAgsE2BBg2wQVfLyrkAUGLtrq0Eebhy1D+gUMYAm1LaQEEipam/h/gr1qbLWxH8b0pVrDt5j481C5Bmn2g31h5+w7+8p+sr8zY/VljDCXLj55pBXu8fLrO8U//1VZ+AygMVJdoOBIX1xTADo8kbUYWJsrLWGVWhS/3Xt7XnxaSPfvs0/sF9b3kZ2xb1ih3f6tfd4XhQ8YPS3HW2CkGrTxqUEKx7iPvWGuHH19aR3E+Yz4SYUj+x7eBubYQLUqew50ZkY4Jd51SnBdgusSXI/gegTXIbgOwekEpxNck+CaBLdBcBsEt01w2wS3Q3A7BFchuArBHRHcEcExgmMJjsfzG65lal55PK+RPzmfXH4JQ78g/Bau+hFRk/0k5WKOxgQXWSjST5A+SYa5hSt8nB3aknQ4SkN+XDGyDwiVKfcNjpZp3biHUuYTFednVFyUX6s1VcnImlTsTmQ10uxO5TSu2ovz19Bn4u60lyA2p1YUx/5mgioQVCFBvZC7TQpyktZxDwr3zmg2h6dGlEe0K9GeXBgXz7Ds9ImVbJRzmezDTG7/UWr9WXyaLcJduAcPMP5jWIctHP8S5vAK3sA7eK9klAOlqtQjdH4u1tyGiUvR/wEOAJD3</latexit>

潜在トピック
副詞

予測パープレキシティ：
149 (ユニグラム)
à 32 (スペクトル混合LDA)

35 / 81

Summer School 数理物理2021 Lecture3

予測確率

ぎこちなく

0.056

痛そうに

0.054

不自然に

0.048

ゆっくり

0.036

ふざけて

0.035

(2) ガウス過程と階層ディリクレ過程による「動作」の学習

36 / 80

From Movements to Actions

観測値

Movements
観測シーン

Observation

単位系列1

Motion 1

単位系列2

Motion 2

時間
Timel Crucial for high-level recognition and planningof actions of robots– Planning for preparing dishes: wash->cut->boil->...
37 / 81

Summer School 数理物理2021 Lecture3

Model of movements

観測値

Movements
観測シーン

Observation

単位系列1

Motion 1

単位系列2

Motion 2

時間
Timel Observations are continuousà Stochastic models for

trajectories are necessaryl We leverage the Gaussian processes

38 / 81

Summer School 数理物理2021 Lecture3

Actual data of movementsl Time series from two angles (knee and shoulder)l How to induce “motion” from this data?

39 / 81

Summer School 数理物理2021 Lecture3

Model using Gaussian processes

Only the observationl Hidden state (motion class):l Generate trajectory:

cj ⇠ Mult(c|cj 1 )xj ⇠ GP(x | Xcj )l HMM, but segmentations are unknown
40 / 81

Summer School 数理物理2021 Lecture3

Application to Roboticsl Hidden semi-Markov model with Gaussian process

observations to model the time series of joint angles,
Forward-Backward Bayesian learning (infer “words”)
l Segmentation results using simple arm motions :Segmentation result
41 / 81

Summer School 数理物理2021 Lecture3

Dynamic programming MCMC
l Forward filtering:l Backward sampling:

42 / 81

Summer School 数理物理2021 Lecture3

Lattice representation
BOS

1

2

3

4

5

6

p(x[3 : 5]|x[1 : 2])

EOS

segment
lengthl Semi-Markov HMM (Murphy 02, Ostendorf 96)– We do not know “correct path” beforehand (as HMM)
l Draw high-probability path via dynamic programming
43 / 81

Summer School 数理物理2021 Lecture3

The first experimentl Unsupervised segmentation of arm movements

measured by Kinectl 2D coordinates (x,y) of the right hand
– Assume x and y are independently generated:
x ~ GP(c), y ~ GP(c)l Motions involved:– Move the hand to right– Raise the hand high– Raise the hand slightly

44 / 81

Summer School 数理物理2021 Lecture3

Results} Correctly recovered thethree motions} Learned

Gaussian
processes:

1000
1000

1000

500

0

𝑥

0

0

-500

-500
-500

-1000
-1000

-500

𝑦

45 / 81500

500

Summer School 数理物理2021 Lecture3

0

500

1000

-1000
-1000

-500

0

500

1000

-1000
-1000

-500

0

5

Motion segmentationl Correctly recognized Karate motions from observations
46 / 81

Summer School 数理物理2021 Lecture3

Inferred motions (excerpt)l Class 0: Right punch with an additional stepl Class 6: Left lower guarding

47 / 81

Summer School 数理物理2021 Lecture3

Comparison with other methodsl GP-HSMM vs. HDP-HMM, HDP-HMM+NPYLM
Ground truth
Proposed
HDP-HMMHDP-HMM
NPYLM

Error

Error
0

20

40

60

0

20

40

60

0

Error
(confuse right punchwith right
guard)
20
40
60

Ground truth
ProposedError (1 class
1 motion）

HDP-HMMHDP-HMM+
NPYLM

48 / 8102040Summer School 数理物理2021 Lecture3

60

Number of Motions
l Number of hidden states (motions) can also be

estimated using hierarchical Dirichlet processes– HDP-GP-HSMM– “Sequence Pattern Extraction by Segmenting Time
Series Data Using GP-HSMM with Hierarchical Dirichlet
Process”, Nagano+, IROS 2018
Groundtruth: 11

49 / 81

Summer School 数理物理2021 Lecture3

Number of Motions (2)l We leveraged infinite HMM (Beal+ 2001, Teh+ 2006)

with our semi-Markov structurel Since the state space is infinite-dimensional,
also used a Beam sampling (van Gael+2007)
for slice sampling+dynamic programming– Internally,using a stick-breaking representation
for possibly infinite state spaces

50 / 81

Summer School 数理物理2021 Lecture3

High-dimensional regime
l Actually, robot movements are quite high-dimensional– In our case, # of joint angles = 93
– Cannot apply the method to whole data

51 / 81

Summer School 数理物理2021 Lecture3

Strategy (to appear at IROS 2019)l Solution: dimensionality reductionl Linear PCA à NG
l GPLVM (Gaussian process LVM) à OK, butinference is not stablel Using VAE as a surrogate of GPLVM

52 / 81

Summer School 数理物理2021 Lecture3

Simultaneous optimization

Segmentation runs in
the latent space
53 / 81

Summer School 数理物理2021 Lecture3

Note
l In this case, each cluster (=motion) has its own

VAE for compression– VAE priors are different for each cluster

54 / 81

Summer School 数理物理2021 Lecture3

Graphical modell We observe only s (high-dimensional time series)

Hierarchical Dirichlet
process-variational
autoencoder-Gaussian
process-hidden semiMarkov model(Long!)=
HVGH
Observation
55 / 81

Summer School 数理物理2021 Lecture3

Experimentsl Dance exercises which include four and sevenunit motions (labels are not used in learning)

56 / 81

Summer School 数理物理2021 Lecture3

Results (1)l Exercise containing four unit motions:

VAE as a preprocessing

57 / 81

Summer School 数理物理2021 Lecture3

Results (2)l Exercise containing seven unit motions:

VAE as a preprocessing

58 / 81

Summer School 数理物理2021 Lecture3

Estimated latent space in VAEl When VAE is separately learned for compressionl Motions (in color) are mixed and not separated

59 / 81

Summer School 数理物理2021 Lecture3

Estimated latent space in VAE (2)l When simultaneously learned in HVGHl Motions (in color) are clearly separated

60 / 81

Summer School 数理物理2021 Lecture3

Zoology and Brain science (Marmoset)l Can recognize ape’s motions completely automaticallyl joint work with Koki Mimura (NCAP), JSAI 201961 / 81

Summer School 数理物理2021 Lecture3

(3) Nonparametric Bayesian
Deep Visualization(ブリヂストン(株)との共同研究, IBISML 43 (2021)で発表済)

62 / 8062データ可視化
l 高次元データを2次元または3次元に変換して可視化– データ科学の最初のステップとして、依然として不可欠
個体ID

Y1

Y2

…

Y_D

１
２
・・・
N

l 主成分分析(PCA)では限界がある…線形変換では捉えられない非線形な変換が必要な場合が多い

63 / 81

Summer School 数理物理2021 Lecture3

NPDV: NP Bayesian Deep Visualizationl Y→X, X →Vへの圧縮を同時に最適化
l Y→X：NN-iWMM
ガウス過程による
ニューラルネットl X →V：t-SNEによる可視化

64 / 81

Summer School 数理物理2021 Lecture3

無限ワープ混合モデル (iWMM)

推論

X(Iwata+ 2013)より引用

Y<latexit sha1_base64="T1QB0mKOGtjE1L6Pq0zjYHc4ECM=">AAAHsHichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJqRNaarIa9aNib9kO0mTKC/AGYkDAgkkDojH4MILcOgjII5F4sKB8dqRk3hSbCXenfn9xzO7613umoYfMHY6Mzt34eIlZf7ywpWr164vLi3f2POdlqeJiuaYjlflqi9MwxaVwAhMUXU9oVrcFPu8+TT077eF5xuO/TzouuLIUo9tQzc0NUDTi5qlBg2u96uD+lKG5Zi8VtKNfNzIQHwVneW5DajBS3BAgxZYIMCGANsmqODjfQh5YOCi7Qj6aPOwZUi/gAEsoLaFlEBCRWsT/4+xdxhbbeyHMX2p1vAtJv48VK5Alv1gX9gZ+86+sp/s79RYfRkjzKWLTx5phVtffH2r/Oe/KgufATQS1TkKjvT5NQWgwyNZi4G1udISVqlF8du9t2flx6Vs/y77xH5hfR/ZKfuGFdrt39rnXVF6h9GzcrwFRqpBG58aZHCM+9gb5srR15fWQZxPwo8rHNn38DYwxwaqVdlzoDM1wgnxrhOC6xJcl+B6BNcjuA7BdQhOJzid4JoE1yS4NYJbI7hNgtskuC2C2yK4KsFVCe6A4A4IjhEcS3E8nt9wLVPzyuN5jfzp+eTySxj6BeG3cNUnRE3205SLORpjXGShSD9F+iQZ5hau8FF2aEvT4SgN+VFFYh8QKlPuGxwtk7pRD6UspiouTqm4LL9Wa6KSxJpWbI9llWi2J3IaVe3E+WvoM3F32kkR6xMrimN/PUWVCKqUop7J3SYDBUnruAeFe2c0m8NTI8oj2pVoTyGMi2dYfvLESjf2Crn8/Vxh90Fm9Ul8ms3DbbgD9zD+Q1iFDRz/CuZgwxt4Dx+UglJV6ooaobMzseYmjF3Kq3/peZLL</latexit><latexit sha1_base64="p+MlzADJVLtSx0+TRZ5ORfiBDS4=">AAAHsHichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJqRNaKrKa9aNib9kO0mTKC/AGYkDAgkkDojH4MILcOgjII5F4sKB8dqRk3hSbCXenfn9xzO7613umoYfMHY6Mzt34eIlZf7ywpWr164vLi3f2POdlqeJiuaYjlflqi9MwxaVwAhMUXU9oVrcFPu8+TT077eF5xuO/TzouuLQUo9tQzc0NUDTi7qlBg2u92uDo6UMyzF5raQb+biRgfgqOstzG1CHl+CABi2wQIANAbZNUMHH+wDywMBF2yH00eZhy5B+AQNYQG0LKYGEitYm/h9j7yC22tgPY/pSreFbTPx5qFyBLPvBvrAz9p19ZT/Z36mx+jJGmEsXnzzSCvdo8fWt8p//qix8BtBIVOcoONLn1xSADo9kLQbW5kpLWKUWxW/33p6VH5ey/bvsE/uF9X1kp+wbVmi3f2ufd0XpHUbPyvEWGKkObXxqkMEx7mNvmCtHX19aB3E+CT+ucGTfw9vAHBuoVmXPgc7UCCfEu04IrktwXYLrEVyP4DoE1yE4neB0gmsSXJPg1ghujeA2CW6T4LYIbovgqgRXJbgawdUIjhEcS3E8nt9wLVPzyuN5jfzp+eTySxj6BeG3cNUnRF3205SLORpjXGShSD9F+iQZ5hau8FF2aEvT4SgN+VFFYh8QKlPuGxwtk7pRD6UspiouTqm4LL9Wa6KSxJpWbI9llWi2J3IaVe3E+WvoM3F32kkR6xMrimN/PUWVCKqUop7J3SYDBUnruAeFe2c0m8NTI8oj2pVoTyGMi2dYfvLESjf2Crn8/Vxh90Fm9Ul8ms3DbbgD9zD+Q1iFDRz/CuZgwxt4Dx+UglJVjhQ1QmdnYs1NGLuUV/8A8L2SzA==</latexit>l 潜在空間での無限混合ガウス分布àガウス過程で観測値へ変換l 観測データYのd次元目 Yd が、潜在空間Xでのカーネルを使ったGPに従う
Yd ⇠ GP(µ, KX )<latexit sha1_base64="0wQNDylLUmvjz1qx3x5EqYX9jIc=">AAAH0HichZW/b9NAFMdfWyCl/GgKCxJLRFRUJFRdAhKIqQUqWqWq0oS2qeoq8rmXxoodW7aTNLEixNqBlYEJJAbEygoLC/8AQ/8ExFgkFgaez47ywy+trcR3732+z+/dne+4beiux9jJxOTUhYuXEtOXZ65cvXZ9Njl3Y9u1Go4mtjTLsJwSV11h6HWx5emeIUq2I1STG2KH154F/p2mcFzdqr/02rbYN9XDul7RNdVDUzm5sFs+SCmubqZ8xTFTL/LdBYWbvmI2uvdzZcVUvSqv+KXuvXIyzRaZvFLxRiZqpCG68tbc1CoocAAWaNAAEwTUwcO2ASq4eO9BBhjYaNsHH20OtnTpF9CFGdQ2kBJIqGit4f8h9vYiax37QUxXqjV8i4E/B5UpmGc/2Sd2yn6wz+wX+zc2li9jBLm08clDrbDLs8e3in/PVZn49KDaV52h4EifXZMHFXgsa9GxNltagiq1MH6z8/a0+KQw799lH9hvrO89O2HfscJ684/2cVMU3mH0eTneAiMp0MSnBmkcYx97vVw5+nxp7Ub59PlhhSX7Dt465lhFtSp7FrTGRjgi3nVEcG2CaxNch+A6BNciuBbBVQiuQnA1gqsR3DLBLRPcGsGtEVyO4HIEVyK4EsHtEtwuwTGCYzGOR/MbrGVqXnk0r6E/Pp9cfgk9vyD8Jq76PqHIfpyyMUd9iAstFOnGSJckg9yCFT7I9mxxOhilHj+o6Nu7hMqQ+wZHy6hu0EMp87GK82MqLsqv1RyppG+NK9aHsupr1kdyGlRtRPlr6DNwd9qIESsjK4pjfyVGFQiqEKOey90mDVlJV3APCvbOcDZ7p0aYR7gr0Z5sEBfPsMzoiRVvbGcXMw8Ws5sP00tPo9NsGm7DHVjA+I9gCVZx/LcwhzfwBb7Ct0QhcZR4lXgdopMTkeYmDF2J4/8EH56b</latexit>

65 / 81

Summer School 数理物理2021 Lecture3

NNGPカーネルとニューラルネット
l 1層のニューラルネットで、入力を x とする<latexit sha1_base64="05i1Qsam0Fm9F/5SK3jUPFG0cuo=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1YlHyRN1VSV12xaK45t2c638gJwBXHgBBIHxGNw4QU49BEQxyJx4cB47chJdlpsJd6d+f3HM7vrXe5aph8wdjIzO3fh4iVt/vLClavXri8uLd+o+E7LM0TZcCzHq3LdF5Zpi3JgBpaoup7Qm9wSu7zxLPTvtoXnm479Mui54qCpH9lm3TT0AE2F7uFSimWYvFbURjZupCC+8s7y3AbU4BU4YEALmiDAhgDbFujg470PWWDgou0ABmjzsGVKv4AhLKC2hZRAQkdrA/+PsLcfW23shzF9qTbwLRb+PFSuQJr9YF/YKfvOvrKf7O+ZsQYyRphLD5880gr3cPH1rdKf/6qa+AzgOFGdo+BIn19TAHV4LGsxsTZXWsIqjSh+u//+tPSkmB7cZZ/YL6zvIzth37BCu/3b+FwQxQ8YPS3HW2CkGrTxaUAKx3iAvVGuHH0DaR3G+ST8pMKRfQ9vE3M8RrUuew50zozQJd7VJbgewfUIrk9wfYLrEFyH4OoEVye4BsE1CG6N4NYIbpPgNglui+C2CK5KcFWC2yO4PYJjBMcUjsfzG65lal55PK+RX51PLr+EkV8Q/iau+oSoyb5KuZijOcFFFor0FdInyTC3cIWPsyObSoejNOLHFYl9SKgsuW9wtEzrxj2UMq9UnD+j4pL8WptTlSRWVbE9kVWi2Z7KaVy1E+dvoM/C3WlHIdanVhTH/rpCFQmqqFDP5W6Tgpyk67gHhXtnNJujUyPKI9qVaE8ujItnWHb6xFIblVwm+yCTKzxMrT6NT7N5uA134B7GfwSrsIHjX8YcBLyBt/BOu6+90CpaNUJnZ2LNTZi4NP0f102Oyw==</latexit>

l i番目の出力 zi (x) は1<latexit sha1_base64="gMVPXwKsUrY/Py15IDMa+Rz/xkI=">AAAHrnichZVLb9NAEMenLeBSHm3hgsSlIgoql2gTkECcWqBSq1ZVmpA0VRMir7tprPgl23krX4AjFw7lAhIHxMfgwhfg0I+AOBaJCwfGa0d5eFJsJd6d+f3HM7vrXe4Yuuczdj43v3Dl6jVl8frSjZu3bi+vrN4penbT1URBsw3bLXHVE4ZuiYKv+4YoOa5QTW6IQ954GfgPW8L1dNt67XcdUTHVU0uv6Zrqo6nUe5Ou6uudR9WVBEsxea3FG+mokYDoytqrC9tQhhOwQYMmmCDAAh/bBqjg4X0MaWDgoK0CfbS52NKlX8AAllDbREogoaK1gf+n2DuOrBb2g5ieVGv4FgN/LirXIMl+sC/sgn1nX9lP9ndmrL6MEeTSxScPtcKpLr+9l//zX5WJTx/qI9UlCo705TX5UINnshYda3OkJahSC+O3eu8v8s9zyf5D9on9wvo+snP2DSu0Wr+1zwcid4bRk3K8BUYqQwufGiRwjPvYG+bK0deX1kGUz4ifVNiy7+KtY451VKuyZ0N7ZoQO8a4OwXUJrktwPYLrEVyb4NoEVyO4GsE1CK5BcJsEt0lwOwS3Q3C7BLdLcCWCKxHcEcEdERwjOBbjeDS/wVqm5pVH8xr64/PJ5Zcw9AvCb+KqHxFl2Y9TDuaoT3ChhSK9GOmRZJBbsMLH2aEtTgejNOTHFSP7gFAZct/gaJnWjXsoZTZWcXZGxXn5tZpTlYysccXeRFYjzd5UTuOq/Sh/DX0G7k77MWJrakVx7G/FqBxB5WLUK7nbJCAj6RruQcHeGc7m8NQI8wh3JdqTCeLiGZaePrHijWImlX6cyhw8SWy8iE6zRbgPD2Ad4z+FDdjG8S/Ik+odnMEHhSlFpaJUQ3R+LtLchYlLqf8Dic+RMw==</latexit>

zi1 (x) = b1i +N1X<latexit sha1_base64="be+5Foc0l/Cr4ohUtAHWc5/6x0A=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatUqygdpU5oq8ppNasVfsp2kSZQXQOJKD5xA4oB4DC68AIc+AuJYJC4cGK8dOYknxVbi3Znffzyzu97ljqF7PmNnc/MLly5fURavLl27fuPm8srqrT3PbruaqGi2YbtVrnrC0C1R8XXfEFXHFarJDbHPWy8C/35HuJ5uW6/8niOOTLVp6Q1dU300lfN1Vl9JsQyT11qykY0aKYiugr26sAU1eAM2aNAGEwRY4GPbABU8vA8hCwwctB3BAG0utnTpFzCEJdS2kRJIqGht4X8Te4eR1cJ+ENOTag3fYuDPReUapNkP9oWds+/sK/vJ/s6MNZAxglx6+OShVjj15bd3yn/+qzLx6cNxrLpAwZG+uCYfGvBU1qJjbY60BFVqYfxO//S8/KyUHtxnn9gvrO8jO2PfsEKr81v7XBSlDxg9LcdbYKQadPCpQQrHeIC9Ua4cfQNpHUb5xPykwpZ9F28dczxGtSp7NnRnRjgh3nVCcD2C6xFcn+D6BNcluC7BNQiuQXAtgmsR3AbBbRDcNsFtE9wOwe0QXJXgqgR3QHAHBMcIjiU4Hs1vsJapeeXRvIb+5Hxy+SWM/ILwm7jqY6Im+0nKwRz1CS60UKSXID2SDHILVvg4O7Il6WCURvy4IrYPCZUh9w2OlmnduIdSFhIVF2ZUXJZfqzlVSWxNKnYnsoo1u1M5javyUf4a+gzcnfIJYnNqRXHsbyaoEkGVEtRLudukICfpBu5Bwd4Zzubo1AjzCHcl2pML4uIZlp0+sZKNvVwm+yiTKz5OrT+PTrNFuAv34AHGfwLrsIXjX8EcmvAO3sOp8lApKlXldYjOz0Wa2zBxKdo/CzuPRA==</latexit>

Wij1 x1j (x), x1j (x) =

b0j +

j=1

<latexit sha1_base64="LMYTwKSxLTup9cVMeObg85dJ7Rc=">AAAIIXichZXNbtNAEMcnlJISoB9wQeJiERW1AlWbgARCqtQClVq1qtqUtqma1orNpnFix5btpEks9wF4AQ6cAHFAPAYXuCEkDhUHzohjkbhwYLx2yIenxVbi3Znffzyzu95VLF1zXMaOE+eGzg9fSI5cTF26fGV0bHzi6pZj1m2Vb6qmbtp5pehwXavxTVdzdZ63bF40FJ1vK9XHgX+7wW1HM2tP3ZbF94ziQU0raWrRRZM83mrvZ2RtqjktzUpK0JRuSwWnbsheZTbj73urcsaXttHhaRW/ic8Ksneko6NUp4PCglXWpILOS+5UStlncuVfkGoQZBkDMAxY9aWmXE0VbO2g7E7L42k2w8QlxRuZqJGG6FozJ4YWoQDPwAQV6mAAhxq42NahCA7eu5ABBhba9sBDm40tTfg5+JBCbR0pjkQRrVX8P8DebmStYT+I6Qi1im/R8WejUoJJ9pW9YyfsI3vPfrA/p8byRIwglxY+lVDLLXns+fWN3/9VGfh0odxVnaFQkD67JhdK8EDUomFtlrAEVaph/Eb7xcnGw9ykd4u9Zj+xvlfsmH3ACmuNX+rbdZ57idEnxXhzjFSABj5VSOMYe9jr5KqgzxNWP8qny/crTNG38dYwxzKqi6JnwuGpEZrEu5oE1yK4FsG1Ca5NcIcEd0hwJYIrEVyV4KoEN09w8wS3RHBLBLdMcMsElye4PMHtENwOwTGCYzFOieY3WMvUvCrRvIb++Hwq4kvo+DnhN3DVd4mC6McpC3PU+rjQQpFOjHRIMsgtWOG9bMcWp4NR6vC9iq7dJ1S62DcUtAzqej2Uci1W8dopFW+Ir9UYqKRrjStW+rLqalYGcupVrUb5q+jTcXdajRELAytKwf5CjMoRVC5GPRG7TRqygi7hHhTsneFsdk6NMI9wV6I92SAunmGZwRMr3tjKzmTuzmTX76XnHkWn2QjcgJswhfHvwxws4vhvYg7fE8OJ0cRY8k3yU/Jz8kuInktEmmvQdyW//QWPvLeb</latexit><latexit sha1_base64="G6XmFjgUM/Lat9F/OGdVLLxN0U0=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatUqygdpU5oq8ppNasVfsp2kSZQXQOJKD5xA4oB4DC68AIc+AuJYJC4cGK8dOYknxVbi3Znffzyzu97ljqF7PmNnc/MLly5fURavLl27fuPm8srqrT3PbruaqGi2YbtVrnrC0C1R8XXfEFXHFarJDbHPWy8C/35HuJ5uW6/8niOOTLVp6Q1dU300lfP1XH0lxTJMXmvJRjZqpCC6CvbqwhbU4A3YoEEbTBBggY9tA1Tw8D6ELDBw0HYEA7S52NKlX8AQllDbRkogoaK1hf9N7B1GVgv7QUxPqjV8i4E/F5VrkGY/2Bd2zr6zr+wn+zsz1kDGCHLp4ZOHWuHUl9/eKf/5r8rEpw/HseoCBUf64pp8aMBTWYuOtTnSElSphfE7/dPz8rNSenCffWK/sL6P7Ix9wwqtzm/tc1GUPmD0tBxvgZFq0MGnBikc4wH2Rrly9A2kdRjlE/OTClv2Xbx1zPEY1ars2dCdGeGEeNcJwfUIrkdwfYLrE1yX4LoE1yC4BsG1CK5FcBsEt0Fw2wS3TXA7BLdDcFWCqxLcAcEdEBwjOJbgeDS/wVqm5pVH8xr6k/PJ5Zcw8gvCb+Kqj4ma7CcpB3PUJ7jQQpFegvRIMsgtWOHj7MiWpINRGvHjitg+JFSG3Dc4WqZ14x5KWUhUXJhRcVl+reZUJbE1qdidyCrW7E7lNK7KR/lr6DNwd8oniM2pFcWxv5mgSgRVSlAv5W6TgpykG7gHBXtnOJujUyPMI9yVaE8uiItnWHb6xEo29nKZ7KNMrvg4tf48Os0W4S7cgwcY/wmswxaOfwVzaMI7eA+nykOlqFSV1yE6PxdpbsPEpWj/ABnBj0Y=</latexit><latexit sha1_base64="R8ZqzoF4SDVnDSmgi4CIMGySDBw=">AAAHuHichZXNbtNAEMenLeBSPprCBYlLIAriVG0CEqinFqjUqlWVJqRN1ZTgNevUimNbtpM0ifICvAAHuIBUJMRjcOEFOPQREMciceHAeO3ISTwpthLvzvz+45nd9S53TMPzGTubmZ27dPmKMn914dr1GzcXU0u39jy75WqirNmm7Va46gnTsETZN3xTVBxXqE1uin3eeB7499vC9Qzbeul3HXHUVOuWoRua6qOplkpVTaH7quvaneq9NH+Vq6UybJnJK51s5KJGBqKrYC/NbUAV3oANGrSgCQIs8LFtggoe3oeQAwYO2o6gjzYXW4b0CxjAAmpbSAkkVLQ28L+OvcPIamE/iOlJtYZvMfHnojINWfaDfWHn7Dv7yn6yv1Nj9WWMIJcuPnmoFU5t8e2d0p//qpr49OE4Vl2g4EhfXJMPOjyVtRhYmyMtQZVaGL/de3deWilm+w/YJ/YL6/vIztg3rNBq/9ZOd0XxPUbPyvEWGKkKbXxqkMEx7mNvmCtHX19aB1E+MT+usGXfxdvAHI9RrcqeDZ2pEU6Id50QXJfgugTXI7gewXUIrkNwOsHpBNcguAbBrRHcGsFtEtwmwW0R3BbBVQiuQnAHBHdAcIzgWILj0fwGa5maVx7Na+hPzieXX8LQLwh/E1d9TFRlP0k5mKMxxoUWivQSpEeSQW7BCh9lh7YkHYzSkB9VxPYBoTLlvsHRMqkb9VDKQqLiwpSKS/JrbU5UEluTiu2xrGLN9kROo6qdKH8NfSbuTjsJYn1iRXHsryeoIkEVE9QLudtkIC9pHfegYO8MZ3N4aoR5hLsS7ckHcfEMy02eWMnGXn4592g5v/s4s/osOs3m4S7ch4cY/wmswgaOfxlzaMMHOIXPyoryWqkrRojOzkSa2zB2Ke4/DTCU2g==</latexit><latexit sha1_base64="BMNGLV1GIjQKiMie4pR3ONbwDZc=">AAAHqXichZVLb9NAEMenLeBSHm3hgsSlIgpCQoo2AQnEqQUqtWpV5UHalL7kNZvUil+ynaRJlC+AxJUeOIHEAfExuPAFOPQjII5F4sKB8dpRHjspthLvzvz+45nd9S73LDMIGTubmp65dPmKNnt17tr1GzfnFxZvbQduwzdE2XAt169wPRCW6YhyaIaWqHi+0G1uiR1efxH5d5rCD0zXeRW2PXFg6zXHrJqGHqKp1DnMHi2kWIbJa0ltZJNGCpIr7y7OrME+vAEXDGiADQIcCLFtgQ4B3nuQBQYe2g6gizYfW6b0C+jBHGobSAkkdLTW8b+Gvb3E6mA/ihlItYFvsfDno3IJ0uwH+8LO2Xf2lf1kfyfG6soYUS5tfPJYK7yj+bd3Sn/+q7LxGcLxQHWBgiN9cU0hVOGprMXE2jxpiao04vjNzul56Vkx3b3PPrFfWN9Hdsa+YYVO87fxuSCKHzB6Wo63wEj70MSnASkc4y72+rly9HWltZfkM+BHFa7s+3ibmOMxqnXZc6E1McIJ8a4TgmsTXJvgOgTXIbgWwbUIrkpwVYKrE1yd4FYIboXg1gluneA2CG6D4CoEVyG4XYLbJThGcEzheDK/0Vqm5pUn8xr71fnk8kvo+wXht3HVD4h92VcpD3M0R7jYQpGBQgYkGeUWrfBhtm9T6WiU+vywYmDvESpL7hscLeO6YQ+lzCsV5ydUXJJfqz1WycCqKjZHshpoNsdyGlZtJfkb6LNwd9pSiNWxFcWxv6pQRYIqKtRLudukICfpKu5B0d4Zz2b/1IjziHcl2pOL4uIZlh0/sdTGdi6TfZTJFR6nlp8np9ks3IV78ADjP4FlWMPxL2MONXgH7+FUe6gVtIr2OkanpxLNbRi5NOMfSyWPcA==</latexit>

W

b1

x

<latexit sha1_base64="EvynLVwq2k3n8W/w6hPV9wqJLEA=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatUqygdpU5oq8ppNasVfsp2kSZQXQOJKD5xA4oB4DC68AIc+AuJYJC4cGK8dOYknxVbi3Znffzyzu97ljqF7PmNnc/MLly5fURavLl27fuPm8srqrT3PbruaqGi2YbtVrnrC0C1R8XXfEFXHFarJDbHPWy8C/35HuJ5uW6/8niOOTLVp6Q1dU300lfP1bH0lxTJMXmvJRjZqpCC6CvbqwhbU4A3YoEEbTBBggY9tA1Tw8D6ELDBw0HYEA7S52NKlX8AQllDbRkogoaK1hf9N7B1GVgv7QUxPqjV8i4E/F5VrkGY/2Bd2zr6zr+wn+zsz1kDGCHLp4ZOHWuHUl9/eKf/5r8rEpw/HseoCBUf64pp8aMBTWYuOtTnSElSphfE7/dPz8rNSenCffWK/sL6P7Ix9wwqtzm/tc1GUPmD0tBxvgZFq0MGnBikc4wH2Rrly9A2kdRjlE/OTClv2Xbx1zPEY1ars2dCdGeGEeNcJwfUIrkdwfYLrE1yX4LoE1yC4BsG1CK5FcBsEt0Fw2wS3TXA7BLdDcFWCqxLcAcEdEBwjOJbgeDS/wVqm5pVH8xr6k/PJ5Zcw8gvCb+Kqj4ma7CcpB3PUJ7jQQpFegvRIMsgtWOHj7MiWpINRGvHjitg+JFSG3Dc4WqZ14x5KWUhUXJhRcVl+reZUJbE1qdidyCrW7E7lNK7KR/lr6DNwd8oniM2pFcWxv5mgSgRVSlAv5W6TgpykG7gHBXtnOJujUyPMI9yVaE8uiItnWHb6xEo29nKZ7KNMrvg4tf48Os0W4S7cgwcY/wmswxaOfwVzaMI7eA+nykOlqFSV1yE6PxdpbsPEpWj/ABJ+j0U=</latexit>

Wij` ⇠ N (0,1W0<latexit sha1_base64="nnbc5hsCBtXrsdTmvR4Ug0hlEGg=">AAAHsnichZXNbtNAEMenLeBSPtrCBYlLRRTEKdqklUCcWqBSq1ZVmpImpSmV16wbK/6S7SRNorwADwAHDnxIHBCPwYUX4NBHQByLxIUD47UjJ/Gk2Eq8O/P7j2d217vcNQ0/YOxsanrm0uUryuzVuWvXb9ycX1i8te87TU8TZc0xHa/KVV+Yhi3KgRGYoup6QrW4KSq88TT0V1rC8w3Hfh50XHFkqSe2oRuaGqCpVrPUoM71XqX/kh0vZFiOyWsp3cjHjQzEV9FZnNmAGrwCBzRoggUCbAiwbYIKPt6HkAcGLtqOoIc2D1uG9Avowxxqm0gJJFS0NvD/BHuHsdXGfhjTl2oN32Liz0PlEmTZD/aFnbPv7Cv7yf5OjNWTMcJcOvjkkVa4x/Ov7+z9+a/KwmcA9UR1gYIjfXFNAejwSNZiYG2utIRValH8Vvft+d7jUrZ3n31iv7C+j+yMfcMK7dZv7fOuKL3D6Fk53gIj1aCFTw0yOMY97A1y5ejrSWs/zifhRxWO7Ht4G5hjHdWq7DnQnhjhlHjXKcF1CK5DcF2C6xJcm+DaBKcTnE5wDYJrENwawa0R3CbBbRLcFsFtEVyV4KoEd0BwBwTHCI6lOB7Pb7iWqXnl8bxG/vR8cvklDPyC8Fu46hOiJvtpysUcjREuslCknyJ9kgxzC1f4MDuwpelwlAb8sCKx9wmVKfcNjpZx3bCHUhZTFRcnVLwnv1ZrrJLEmlZsj2SVaLbHchpW7cT5a+gzcXfaSRHrYyuKY389RZUIqpSinsndJgMFSeu4B4V7ZzSbg1MjyiPalWhPIYyLZ1h+/MRKN/YLufxyrrC7kll9Ep9ms3AX7sEDjP8QVmEDx7+MObjwBt7DB2VFeaGoihah01Ox5jaMXIr5D0euk2w=</latexit>

N0<latexit sha1_base64="be+5Foc0l/Cr4ohUtAHWc5/6x0A=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatUqygdpU5oq8ppNasVfsp2kSZQXQOJKD5xA4oB4DC68AIc+AuJYJC4cGK8dOYknxVbi3Znffzyzu97ljqF7PmNnc/MLly5fURavLl27fuPm8srqrT3PbruaqGi2YbtVrnrC0C1R8XXfEFXHFarJDbHPWy8C/35HuJ5uW6/8niOOTLVp6Q1dU300lfN1Vl9JsQyT11qykY0aKYiugr26sAU1eAM2aNAGEwRY4GPbABU8vA8hCwwctB3BAG0utnTpFzCEJdS2kRJIqGht4X8Te4eR1cJ+ENOTag3fYuDPReUapNkP9oWds+/sK/vJ/s6MNZAxglx6+OShVjj15bd3yn/+qzLx6cNxrLpAwZG+uCYfGvBU1qJjbY60BFVqYfxO//S8/KyUHtxnn9gvrO8jO2PfsEKr81v7XBSlDxg9LcdbYKQadPCpQQrHeIC9Ua4cfQNpHUb5xPykwpZ9F28dczxGtSp7NnRnRjgh3nVCcD2C6xFcn+D6BNcluC7BNQiuQXAtgmsR3AbBbRDcNsFtE9wOwe0QXJXgqgR3QHAHBMcIjiU4Hs1vsJapeeXRvIb+5Hxy+SWM/ILwm7jqY6Im+0nKwRz1CS60UKSXID2SDHILVvg4O7Il6WCURvy4IrYPCZUh9w2OlmnduIdSFhIVF2ZUXJZfqzlVSWxNKnYnsoo1u1M5javyUf4a+gzcnfIJYnNqRXHsbyaoEkGVEtRLudukICfpBu5Bwd4Zzubo1AjzCHcl2pML4uIZlp0+sZKNvVwm+yiTKz5OrT+PTrNFuAv34AHGfwLrsIXjX8EcmvAO3sOp8lApKlXldYjOz0Wa2zBxKdo/CzuPRA==</latexit>

66 / 81x<latexit sha1_base64="05i1Qsam0Fm9F/5SK3jUPFG0cuo=">AAAHp3ichZXNbtNAEMenLeBSPtrCBYlLRRQEl2gTkECcWqBSq1YlHyRN1VSV12xaK45t2c638gJwBXHgBBIHxGNw4QU49BEQxyJx4cB47chJdlpsJd6d+f3HM7vrXe5aph8wdjIzO3fh4iVt/vLClavXri8uLd+o+E7LM0TZcCzHq3LdF5Zpi3JgBpaoup7Qm9wSu7zxLPTvtoXnm479Mui54qCpH9lm3TT0AE2F7uFSimWYvFbURjZupCC+8s7y3AbU4BU4YEALmiDAhgDbFujg470PWWDgou0ABmjzsGVKv4AhLKC2hZRAQkdrA/+PsLcfW23shzF9qTbwLRb+PFSuQJr9YF/YKfvOvrKf7O+ZsQYyRphLD5880gr3cPH1rdKf/6qa+AzgOFGdo+BIn19TAHV4LGsxsTZXWsIqjSh+u//+tPSkmB7cZZ/YL6zvIzth37BCu/3b+FwQxQ8YPS3HW2CkGrTxaUAKx3iAvVGuHH0DaR3G+ST8pMKRfQ9vE3M8RrUuew50zozQJd7VJbgewfUIrk9wfYLrEFyH4OoEVye4BsE1CG6N4NYIbpPgNglui+C2CK5KcFWC2yO4PYJjBMcUjsfzG65lal55PK+RX51PLr+EkV8Q/iau+oSoyb5KuZijOcFFFor0FdInyTC3cIWPsyObSoejNOLHFYl9SKgsuW9wtEzrxj2UMq9UnD+j4pL8WptTlSRWVbE9kVWi2Z7KaVy1E+dvoM/C3WlHIdanVhTH/rpCFQmqqFDP5W6Tgpyk67gHhXtnNJujUyPKI9qVaE8ujItnWHb6xFIblVwm+yCTKzxMrT6NT7N5uA134B7GfwSrsIHjX8YcBLyBt/BOu6+90CpaNUJnZ2LNTZi4NP0f102Oyw==</latexit>

Summer School 数理物理2021 Lecture3

w /N` )

<latexit sha1_base64="k9z/fC3sUS8igmMXyuUyMOpPBzU=">AAAH2HichZXNbtNAEMenLZBSPprCBYmLRRRUJFQ2AQnEqQUqtWoVpQlpUpoS2WaTmtixZTtJEysSNwRHDnDgBBIHxCtwQyBegEMfAXEsEhcOjNeO8uFJsZV4d+b3H8/srncVS9ccl7HDqemZEydPxWZPz505e+78fHzhwrZjNm2VF1RTN+2SIjtc1xq84GquzkuWzWVD0XlRqd/3/cUWtx3NbDx0OxbfM+RaQ6tqquyiqRJnxYqnPe099spc13tS2dEMySursi5leovsOvZrhlxpSzekTMVHrlXiCbbExCVFG6mwkYDwypoLM2tQhidgggpNMIBDA1xs6yCDg/cupICBhbY98NBmY0sTfg49mENtEymOhIzWOv7XsLcbWhvY92M6Qq3iW3T82aiUIMl+sI/siH1nn9hP9ndiLE/E8HPp4FMJtNyqzL+8lP/zX5WBTxf2B6pjFArSx9fkQhXuiFo0rM0SFr9KNYjf6r45yt/NJb2r7D37hfW9Y4fsC1bYaP1WP2zx3FuMnhTjzTFSGVr4VCGBY+xhr5+rgj5PWHthPgN+VGGKvo23hjnuo1oWPRPaEyMcEO86ILgOwXUIrktwXYJrE1yb4KoEVyW4OsHVCW6F4FYIbp3g1glug+A2CK5EcCWC2yG4HYJjBMcinBLOr7+WqXlVwnkN/NH5VMSX0Pdzwm/gqh8QZdGPUhbmqI1wgYUinQjpkKSfm7/Ch9m+LUr7o9TnhxUDe49Q6WLfUNAyrhv2UMpspOLshIrz4ms1xioZWKOKzZGsBprNsZyGVZkwfxV9Ou5OmQixOraiFOyvRqgcQeUi1AOx2yQgLegq7kH+3hnMZv/UCPIIdiXak/bj4hmWGj+xoo3t9FLq5lJ661Zi+V54ms3CZbgCixj/NizDGo5/AXN4DZ/hK3yLPYo9iz2PvQjQ6alQcxFGrtirfwavoUY=</latexit>

1<latexit sha1_base64="Gbam75vABvGioZzco2NfSg9FmmE=">AAAHqXichZVLb9NAEMenLeBSHm3hgsSlIgpCQoo2AQnEqQUqtWpV5UHalKatvGaTWvFLtvNWvgASV3rgBBIHxMfgwhfg0I+AOBaJCwfGa0dO4kmxlXh35vcfz+yud7lj6J7P2NnM7Nyly1eU+asL167fuLm4tHxr17ObribKmm3YboWrnjB0S5R93TdExXGFanJD7PHGi8C/1xKup9vWK7/riENTrVt6TddUH02lzlH2eCnFMkxeK8lGNmqkILry9vLcBlThDdigQRNMEGCBj20DVPDwPoAsMHDQdgh9tLnY0qVfwAAWUNtESiChorWB/3XsHURWC/tBTE+qNXyLgT8XlSuQZj/YF3bOvrOv7Cf7OzVWX8YIcunik4da4Rwvvr1T+vNflYlPH05i1QUKjvTFNflQg6eyFh1rc6QlqFIL47d6p+elZ8V0/z77xH5hfR/ZGfuGFVqt39rngih+wOhpOd4CI1WhhU8NUjjGfewNc+Xo60vrIMon5scVtuy7eOuY4wmqVdmzoT01Qod4V4fgugTXJbgewfUIrk1wbYKrEVyN4BoE1yC4NYJbI7hNgtskuC2C2yK4CsFVCG6f4PYJjhEcS3A8mt9gLVPzyqN5Df3J+eTySxj6BeE3cdXHRFX2k5SDOepjXGihSC9BeiQZ5Bas8FF2aEvSwSgN+VFFbB8QKkPuGxwtk7pRD6XMJyrOT6m4JL9Wc6KS2JpUbI9lFWu2J3IaVe1E+WvoM3B32kkQ6xMrimN/PUEVCaqYoF7K3SYFOUnXcA8K9s5wNoenRphHuCvRnlwQF8+w7OSJlWzs5jLZR5lc4XFq9Xl0ms3DXbgHDzD+E1iFDRz/MuZQh3fwHk6Vh0pBqSivQ3R2JtLchrFL0f4BPJuPbg==</latexit>

!

結合重み

<latexit sha1_base64="QXvz4mzIdmqYGLWxMRKRtVXlQuE=">AAAHsnichZXNbtNAEMenLeBSPtrCBYlLRRTEKdqklUCcWqBSq1ZVmpImpSmV16wbK/6S7SRNorwADwAHDnxIHBCPwYUX4NBHQByLxIUD47UjJ/Gk2Eq8O/P7j2d217vcNQ0/YOxsanrm0uUryuzVuWvXb9ycX1i8te87TU8TZc0xHa/KVV+Yhi3KgRGYoup6QrW4KSq88TT0V1rC8w3Hfh50XHFkqSe2oRuaGqCpVrPUoM71XqX/Mn+8kGE5Jq+ldCMfNzIQX0VncWYDavAKHNCgCRYIsCHAtgkq+HgfQh4YuGg7gh7aPGwZ0i+gD3OobSIlkFDR2sD/E+wdxlYb+2FMX6o1fIuJPw+VS5BlP9gXds6+s6/sJ/s7MVZPxghz6eCTR1rhHs+/vrP3578qC58B1BPVBQqO9MU1BaDDI1mLgbW50hJWqUXxW92353uPS9neffaJ/cL6PrIz9g0rtFu/tc+7ovQOo2fleAuMVIMWPjXI4Bj3sDfIlaOvJ639OJ+EH1U4su/hbWCOdVSrsudAe2KEU+JdpwTXIbgOwXUJrktwbYJrE5xOcDrBNQiuQXBrBLdGcJsEt0lwWwS3RXBVgqsS3AHBHRAcIziW4ng8v+FapuaVx/Ma+dPzyeWXMPALwm/hqk+ImuynKRdzNEa4yEKRfor0STLMLVzhw+zAlqbDURrww4rE3idUptw3OFrGdcMeSllMVVycUPGe/FqtsUoSa1qxPZJVotkey2lYtRPnr6HPxN1pJ0Wsj60ojv31FFUiqFKKeiZ3mwwUJK3jHhTundFsDk6NKI9oV6I9hTAunmH58RMr3dgv5PLLucLuSmb1SXyazcJduAcPMP5DWIUNHP8y5uDCG3gPH5QV5YWiKlqETk/FmtswcinmP07xk20=</latexit>

N1

0
Wjk
xk

k=1

z1

N2

N
K0
X

<latexit sha1_base64="pnBwoIFlxOZzvBwgvbEjhepf8Hc=">AAAHuHichZXNbtNAEMenLeBSPprCBYlLIAriVG0CEqinFqjUqlWVJqRN1ZTgNevUimNbtpM0ifICvAAHuIBUJMRjcOEFOPQREMciceHAeO3ISTwpthLvzvz+45nd9S53TMPzGTubmZ27dPmKMn914dr1GzcXU0u39jy75WqirNmm7Va46gnTsETZN3xTVBxXqE1uin3eeB7499vC9Qzbeul3HXHUVOuWoRua6qOplkpVTaH7quvaneq9NH/FaqkMW2bySicbuaiRgegq2EtzG1CFN2CDBi1oggALfGyboIKH9yHkgIGDtiPoo83FliH9AgawgNoWUgIJFa0N/K9j7zCyWtgPYnpSreFbTPy5qExDlv1gX9g5+86+sp/s79RYfRkjyKWLTx5qhVNbfHun9Oe/qiY+fTiOVRcoONIX1+SDDk9lLQbW5khLUKUWxm/33p2XVorZ/gP2if3C+j6yM/YNK7Tav7XTXVF8j9GzcrwFRqpCG58aZHCM+9gb5srR15fWQZRPzI8rbNl38TYwx2NUq7JnQ2dqhBPiXScE1yW4LsH1CK5HcB2C6xCcTnA6wTUIrkFwawS3RnCbBLdJcFsEt0VwFYKrENwBwR0QHCM4luB4NL/BWqbmlUfzGvqT88nllzD0C8LfxFUfE1XZT1IO5miMcaGFIr0E6ZFkkFuwwkfZoS1JB6M05EcVsX1AqEy5b3C0TOpGPZSykKi4MKXikvxamxOVxNakYnssq1izPZHTqGonyl9Dn4m7006CWJ9YURz76wmqSFDFBPVC7jYZyEtaxz0o2DvD2RyeGmEe4a5Ee/JBXDzDcpMnVrKxl1/OPVrO7z7OrD6LTrN5uAv34SHGfwKrsIHjX8Yc2vABTuGzsqK8VuqKEaKzM5HmNoxdivsPBe2U2Q==</latexit>

b

0

バイアス項

b`i ⇠ N (0,<latexit sha1_base64="YenEqkRjQZTRC/KLFgp+uDfhZ/o=">AAAHynichZXNbtNAEMenLZBSPprCBYlL1CioSKjaBCQQpxao1KpRlCakTdW0kddsUit2bNlO0sTKjQPiBThwAokDQuIKdy68AIc+AuJYJC4cGK8d5cOTYivx7szvP57ZXe9yS9ccl7HTmdm5CxcvxeYvL1y5eu36Ynzpxq5jtmxVlFRTN+0yVxyha01RcjVXF2XLForBdbHHG099/15b2I5mNp+7XUscGkq9qdU0VXHRVI0v86OK0PWqlqg4mpHwKqqiJ3L9FXYP+3VDqfK71XiSrTJ5JaKNdNhIQnjlzaW5TajACzBBhRYYIKAJLrZ1UMDB+wDSwMBC2yF4aLOxpUm/gD4soLaFlEBCQWsD/+vYOwitTez7MR2pVvEtOv5sVCYgxX6wj+yMfWef2E/2d2osT8bwc+nikwdaYVUXX98q/vmvysCnC8dD1TkKjvT5NblQg0eyFg1rs6TFr1IN4rd7b86Kjwsp7w57z35hfe/YKfuGFTbbv9UPO6LwFqOn5HgLjFSBNj5VSOIYe9gb5MrR50lrP8xnyI8rTNm38dYwx2NUK7JnQmdqhBPiXScE1yW4LsH1CK5HcB2C6xBcjeBqBNcguAbBrRPcOsFtEdwWwW0T3DbBlQmuTHD7BLdPcIzgWITj4fz6a5maVx7Oa+CPzieXX8LALwi/gat+SFRkP0pZmKM2xgUWinQipEOSfm7+Ch9lB7Yo7Y/SgB9VDO19QqXLfYOjZVI36qGU+UjF+SkVF+XXakxUMrRGFdmxrIaa7EROo6pcmL+KPh13p1yE2JhYURz7GxGqQFCFCPVM7jZJyEi6hnuQv3cGszk4NYI8gl2J9mT8uHiGpSdPrGhjN7Oavr+a2XmQXHsSnmbzcBuWYQXjP4Q12MTxL2EOr+AzfIGvsWzMjnVjXoDOzoSamzB2xV7+A8oIm6E=</latexit>

と仮定する

b)

NNGPカーネルとニューラルネット (2)N1X

zi1 (x) = b1i +<latexit sha1_base64="be+5Foc0l/Cr4ohUtAHWc5/6x0A=">AAAHqXichZXNbtNAEMenLeBSPtrCBYlLRRSEhBRtAhKIUwtUatUqygdpU5oq8ppNasVfsp2kSZQXQOJKD5xA4oB4DC68AIc+AuJYJC4cGK8dOYknxVbi3Znffzyzu97ljqF7PmNnc/MLly5fURavLl27fuPm8srqrT3PbruaqGi2YbtVrnrC0C1R8XXfEFXHFarJDbHPWy8C/35HuJ5uW6/8niOOTLVp6Q1dU300lfN1Vl9JsQyT11qykY0aKYiugr26sAU1eAM2aNAGEwRY4GPbABU8vA8hCwwctB3BAG0utnTpFzCEJdS2kRJIqGht4X8Te4eR1cJ+ENOTag3fYuDPReUapNkP9oWds+/sK/vJ/s6MNZAxglx6+OShVjj15bd3yn/+qzLx6cNxrLpAwZG+uCYfGvBU1qJjbY60BFVqYfxO//S8/KyUHtxnn9gvrO8jO2PfsEKr81v7XBSlDxg9LcdbYKQadPCpQQrHeIC9Ua4cfQNpHUb5xPykwpZ9F28dczxGtSp7NnRnRjgh3nVCcD2C6xFcn+D6BNcluC7BNQiuQXAtgmsR3AbBbRDcNsFtE9wOwe0QXJXgqgR3QHAHBMcIjiU4Hs1vsJapeeXRvIb+5Hxy+SWM/ILwm7jqY6Im+0nKwRz1CS60UKSXID2SDHILVvg4O7Il6WCURvy4IrYPCZUh9w2OlmnduIdSFhIVF2ZUXJZfqzlVSWxNKnYnsoo1u1M5javyUf4a+gzcnfIJYnNqRXHsbyaoEkGVEtRLudukICfpBu5Bwd4Zzubo1AjzCHcl2pML4uIZlp0+sZKNvVwm+yiTKz5OrT+PTrNFuAv34AHGfwLrsIXjX8EcmvAO3sOp8lApKlXldYjOz0Wa2zBxKdo/CzuPRA==</latexit>

Wij1 x1j (x), x1j (x) =

b0j +

j=1

<latexit sha1_base64="LMYTwKSxLTup9cVMeObg85dJ7Rc=">AAAIIXichZXNbtNAEMcnlJISoB9wQeJiERW1AlWbgARCqtQClVq1qtqUtqma1orNpnFix5btpEks9wF4AQ6cAHFAPAYXuCEkDhUHzohjkbhwYLx2yIenxVbi3Znffzyzu95VLF1zXMaOE+eGzg9fSI5cTF26fGV0bHzi6pZj1m2Vb6qmbtp5pehwXavxTVdzdZ63bF40FJ1vK9XHgX+7wW1HM2tP3ZbF94ziQU0raWrRRZM83mrvZ2RtqjktzUpK0JRuSwWnbsheZTbj73urcsaXttHhaRW/ic8Ksneko6NUp4PCglXWpILOS+5UStlncuVfkGoQZBkDMAxY9aWmXE0VbO2g7E7L42k2w8QlxRuZqJGG6FozJ4YWoQDPwAQV6mAAhxq42NahCA7eu5ABBhba9sBDm40tTfg5+JBCbR0pjkQRrVX8P8DebmStYT+I6Qi1im/R8WejUoJJ9pW9YyfsI3vPfrA/p8byRIwglxY+lVDLLXns+fWN3/9VGfh0odxVnaFQkD67JhdK8EDUomFtlrAEVaph/Eb7xcnGw9ykd4u9Zj+xvlfsmH3ACmuNX+rbdZ57idEnxXhzjFSABj5VSOMYe9jr5KqgzxNWP8qny/crTNG38dYwxzKqi6JnwuGpEZrEu5oE1yK4FsG1Ca5NcIcEd0hwJYIrEVyV4KoEN09w8wS3RHBLBLdMcMsElye4PMHtENwOwTGCYzFOieY3WMvUvCrRvIb++Hwq4kvo+DnhN3DVd4mC6McpC3PU+rjQQpFOjHRIMsgtWOG9bMcWp4NR6vC9iq7dJ1S62DcUtAzqej2Uci1W8dopFW+Ir9UYqKRrjStW+rLqalYGcupVrUb5q+jTcXdajRELAytKwf5CjMoRVC5GPRG7TRqygi7hHhTsneFsdk6NMI9wV6I92SAunmGZwRMr3tjKzmTuzmTX76XnHkWn2QjcgJswhfHvwxws4vhvYg7fE8OJ0cRY8k3yU/Jz8kuInktEmmvQdyW//QWPvLeb</latexit>

NK0
X

k=1

0
Wjk
xk

!

`

`

l 重み Wij とバイアス bi は独立なので、出力は独立な<latexit sha1_base64="RTLnOWpg/EhMklPO2JK2XL9+7Vk=">AAAHrnichZXNbtNAEMenLeBSPtrCBYlLRRTEKdoEJBCnFqjUqlWVJiRN1YTI624aK/6S7SRNorwARy4cygUkDojH4MILcOgjII5F4sKB8dqRk3hSbCXenfn9xzO7613uGLrnM3Y+N79w5eo1ZfH60o2bt24vr6zeKXt229VESbMN261w1ROGbomSr/uGqDiuUE1uiAPeehn4DzrC9XTbeu33HFEz1RNLb+ia6qOpwt9UhWHU9fpKimWYvNaSjWzUSEF05e3VhS2owjHYoEEbTBBggY9tA1Tw8D6CLDBw0FaDAdpcbOnSL2AIS6htIyWQUNHawv8T7B1FVgv7QUxPqjV8i4E/F5VrkGY/2Bd2wb6zr+wn+zsz1kDGCHLp4ZOHWuHUl9/eK/75r8rEpw/NWHWJgiN9eU0+NOCZrEXH2hxpCarUwvid/vuL4vNCevCQfWK/sL6P7Jx9wwqtzm/t874onGH0tBxvgZGq0MGnBikc4wH2Rrly9A2kdRjlE/OTClv2Xbx1zLGJalX2bOjOjHBKvOuU4HoE1yO4PsH1Ca5LcF2CaxBcg+BaBNciuA2C2yC4bYLbJrgdgtshuArBVQjukOAOCY4RHEtwPJrfYC1T88qjeQ39yfnk8ksY+QXhN3HVx0RV9pOUgznqE1xooUgvQXokGeQWrPBxdmRL0sEojfhxRWwfEipD7hscLdO6cQ+lzCcqzs+ouCi/VnOqktiaVOxOZBVrdqdyGlftRflr6DNwd9pLEJtTK4pjfzNBFQiqkKBeyd0mBTlJN3APCvbOcDZHp0aYR7gr0Z5cEBfPsOz0iZVslHOZ7ONMbv9Jav1FdJotwn14AI8w/lNYhy0c/5I8qd7BGXxQmFJWako9ROfnIs1dmLiU5j9fH5G6</latexit><latexit sha1_base64="QZeFkeb+TOw6EdD5Qdv1djkx3bs=">AAAHsXichZXNbtNAEMenLeBSPtrCBYlLRRTEKdqkSCBOLVCpVasqTUiTKCnBazatiWNbtpM0sfIC3BEHJBBIHBCPwYUX4NBHQByLxIUD47UjJ/G02Eq8O/P7j2d217vcNnTXY+xkZnbuwsVLyvzlhStXr11fXFq+se9aHUcTJc0yLKfCVVcYuilKnu4ZomI7Qm1zQ5R560ngL3eF4+qW+czr2+KgrR6aelPXVA9NtfLzujCMhq+/GjaWUizD5LWSbGSjRgqiK28tz21CHV6CBRp0oA0CTPCwbYAKLt41yAIDG20H4KPNwZYu/QKGsIDaDlICCRWtLfw/xF4tsprYD2K6Uq3hWwz8OahcgTT7wb6wU/adfWU/2d8zY/kyRpBLH5881Aq7sfj6VvHPf1VtfHpwFKvOUXCkz6/JgyY8lLXoWJstLUGVWhi/O3h7WnxUSPt32Sf2C+v7yE7YN6zQ7P7WPu+JwjuMnpbjLTBSHbr41CCFY+xjb5QrR58vrcMon5ifVFiy7+CtY45HqFZlz4LemRGOiXcdE1yf4PoENyC4AcH1CK5HcE2CaxJci+BaBLdOcOsEt0VwWwS3TXDbBFchuArBVQmuSnCM4FiC49H8BmuZmlcezWvoT84nl1/CyC8IfxtXfUzUZT9J2ZijPsGFFop0E6RLkkFuwQofZ0e2JB2M0ogfV8T2IaEy5L7B0TKtG/dQynyi4vwZFRfl19qeqiS2JhU7E1nFmp2pnMZVu1H+GvoM3J12E8TG1Iri2N9IUAWCKiSop3K3SUFO0k3cg4K9M5zN0akR5hHuSrQnF8TFMyw7fWIlG/u5THY1k9u7n1p7HJ1m83Ab7sA9jP8A1mATx78kz5w38B4+KKtKVXmh8BCdnYk0N2HiUlr/AKT6ky8=</latexit>

確率変数の和で、中心極限定理から正規分布に従う
1
1
1→ 結合分布 p(z1 (x), z2 (x), · · · , zN2 (x))
も多変量正規分布に従う…ガウス過程<latexit sha1_base64="SAPDs7BXEVQdJa5ckIkKTIutV50=">AAAH0HichZW/b9NAFMdfW8Cl/GgKCxJLRRSUSqg6ByQQUwtUatUqShPSpmpKZLuX1ooTW7bzWxFi7cDKwAQSA2JlhYWFf4ChfwJiLBILA89nR/lxL8VW4rv3Pt/n9+7Od7pjmZ7P2OnU9MyFi5eU2ctzV65euz4fW7ix49l11+B5w7Zst6BrHrfMGs/7pm/xguNyrapbfFevPA38uw3ueqZde+63HX5Q1Y5qZtk0NB9NpVjSSXZeqCU12Vq6FzRSQaNoHNq+J/rddCnVQ9tSKRZny0xci3JDjRpxiK6MvTCzDkU4BBsMqEMVONTAx7YFGnh474MKDBy0HUAXbS62TOHn0IM51NaR4khoaK3g/xH29iNrDftBTE+oDXyLhT8XlYuQYD/YR3bGvrNP7Cf7OzFWV8QIcmnjUw+13CnNn9zK/fmvqopPH44HqnMUOtLn1+RDGR6JWkyszRGWoEojjN/ovDnLPc4munfZe/YL63vHTtk3rLDW+G182ObZtxg9IcabY6QiNPBpQBzHuIu9fq46+rrC2ovyGfCjClv0XbxNzPEY1Zro2dCcGKFFvKtFcG2CaxNch+A6BNckuCbBlQmuTHAVgqsQ3CrBrRLcBsFtENwmwW0SXIHgCgS3R3B7BMcIjkmcHs1vsJapedWjeQ398nzq4kvo+znhr+KqHxBF0ZcpB3M0R7jQQpGeRHokGeQWrPBhtm+T6WCU+vywYmDvESpL7Bs6WsZ1wx5KmZEqzkyoOCe+1upYJQOrrNgayWqg2RrLaViVjvI30Gfh7pSWiLWxFaVjf02isgSVlahnYreJQ0rQZdyDgr0znM3+qRHmEe5KtCcVxMUzTB0/seTGTmpZvb+c2n4QX3kSnWazcBvuQBLjP4QVWMfxz2MOr+EzfIGvSlZpKS+VVyE6PRVpbsLIpZz8Ay6NnJU=</latexit>– 平均は上式から明らかに0– 共分散はK 1 (x, x0 ) ⌘ E [zi1 (x)zi1 (x0 )]=<latexit sha1_base64="5icDyexRZernUEjoOYXqCCw41x0=">AAAIOnichZXPTxNREMenoBbrD0AvJiamsQFKJOS1mmhMTEAkgUAIFIESFpp96ytsuttdu9vf6T/g0YsHT5p4MP4L3rz4D3jg6o1whEQPHpx9u7U/doq7affNzOc7O/Pe7ltuG7rjMnYcGRq+dPlKdORq7Nr1GzdHx8ZvbTtWuaSJLc0yrFKWq44w9KLYcnXXEFm7JFSTG2KHFxa8+E5FlBzdKr5067bYN9XDop7XNdVFV27s7cpBKlmbqU1NxycV8bqsV2KKqbpHnDcXW8rMXuMgldOTtengOjW9H1eU2OSzuOLoh6aa4wfp+IO2UUWjR10L1LV/6lh8sHTBLyQ3lmCzTB7x8CAVDBIQHOvW+PASKPAKLNCgDCYIKIKLYwNUcPDcgxQwsNG3D030lXCky7iAFsRQW0ZKIKGit4D/h2jtBd4i2l5OR6o1vIuBvxIq4zDBfrDP7Ix9Z1/YCfszMFdT5vBqqeOV+1ph50bf3Nn89V+ViVcXjjqqCxQc6Yt7ciEPT2QvOvZmS4/XpebnrzTenW0+zUw0J9lHdor9fWDH7Bt2WKyca582ROY9Zp+Q8y0wkwIVvGqQwDluotWulWOsKb2toJ4O36uwpF3CU8caj1CtSsuC6sAMNeJeNYKrE1yd4BoE1yC4KsFVCS5PcHmCKxBcgeDmCW6e4JYJbpngVghuheCyBJcluF2C2yU4RnAsxPFgfb1nmVpXHqyrHw+vJ5dvQjsuiLiJT32HUKQdpmysUe/hfA9FOiHSIUmvNu8J72bbvjDtzVKb71Z0/C1CZch9g6OnX9cdoZTroY7XB3S8Kd9Ws6+TjjesWO2pqqNZ7aupW7UW1K9hzMDdaS1ELPY9URztxRCVIahMiHohd5sEpCWdxz3I2zv91Wx/Nfw6/F2JjqS9vPgNS/V/scKD7fRs6uFseuNRYu558DUbgbtwH5KY/zHMwRLO/xbW8DtyLzIVSUa/Rn9GT6KnPjoUCTS3oeeInv8Fo9K/sA==</latexit>

67 / 81

2
b +

2
1
1 0
E
[x
(x)x
w
i
i (x )] =Summer School 数理物理2021 Lecture3

2
b +

2
0
C(x,
x
)
w

NNGPカーネルとニューラルネット (3)` 1l ` 1 層目の出力 zj<latexit sha1_base64="vqdzNAQOmaqadAvlqEJ7xT5gIv4=">AAAHsHichZXNbtNAEMenLeBSPtrCBYlLSxTEhWgTkECcWqBSq1ZVmpA2pa4i22xaE3/JdpImUV6AMxIHBBJIHBCPwYUX4NBHQByLxIUD47UjJ9lpayvx7szvP57ZXe/qnmUGIWPHE5NTFy5eUqYvz1y5eu367Nz8je3AbfoGrxiu5fpVXQu4ZTq8EpqhxauezzVbt/iO3ngW+Xda3A9M13kRdjy+b2sHjlk3DS1E00uVW5a6eF9dzNfmMizHxLUgN/JJIwPJVXTnp1ZBhVfgggFNsIGDAyG2LdAgwHsP8sDAQ9s+9NDmY8sUfg59mEFtEymOhIbWBv4fYG8vsTrYj2IGQm3gWyz8+ahcgCz7yb6yE/aDfWO/2L9TY/VEjCiXDj71WMu92uybW+W/56psfIZwmKrOUOhIn11TCHV4LGoxsTZPWKIqjTh+q/vupPyklO3dZZ/Zb6zvEztm37FCp/XH+LLFS+8xelaMN8dIKrTwaUAGx7iHvUGuOvp6wtpP8kn5UYUr+j7eJuZ4iGpN9FxonxrhiHjXEcF1CK5DcF2C6xJcm+DaBFcnuDrBNQiuQXDLBLdMcGsEt0Zw6wS3TnBVgqsS3C7B7RIcIzgmcXoyv9FapuZVT+Y19svzqYsvYeDnhN/GVZ8SqujLlIc5miNcbKHIQCIDkoxyi1b4MDuwyXQ0SgN+WJHa+4TKEvuGjpZx3bCHUhaliounVFwWX6s9VklqlRUbI1mlmo2xnIZVm0n+Bvos3J02JWJlbEXp2F+RqBJBlSTqudhtMlAQdB33oGjvjGdzcGrEecS7Eu0pRHHxDMuPn1hyY7uQyz/IFbYeZpaeJqfZNNyGO3AP4z+CJVjF8a9gDg68hQ/wUSkoVaWmaDE6OZFobsLIpbz+D1y3kZ4=</latexit>

がガウス過程だと仮定すると<latexit sha1_base64="SoxEhCxzgAAaMSMN6eFb2Lc4u2Q=">AAAHsnichZXNbtNAEMenLeBSPtrCBYlLRRTEhWgTKoE4tUClVq2qNCFtSlMi22xaE3/JdpImVl6AB4ADBz4kDojH4MILcOgjII5F4sKB8dqRk+y0tZV4d+b3H8/srnc11zT8gLHjicmpCxcvKdOXZ65cvXZ9dm7+xrbvtDydV3THdLyqpvrcNGxeCYzA5FXX46qlmXxHaz6N/Dtt7vmGYz8Pui7ft9QD22gYuhqgqdZ7Gda4ad7P9+uv63MZlmPiWpAb+aSRgeQqOvNTq1CDV+CADi2wgIMNAbZNUMHHew/ywMBF2z6EaPOwZQg/hz7MoLaFFEdCRWsT/w+wt5dYbexHMX2h1vEtJv48VC5Alv1kX9kJ+8G+sV/s36mxQhEjyqWLTy3Wcrc+++ZW+e+5KgufARymqjMUGtJn1xRAAx6JWgyszRWWqEo9jt/uvTspPy5lw7vsM/uN9X1ix+w7Vmi3/+hftnjpPUbPivHmGKkGbXzqkMExDrE3yFVDXyis/SSflB9VOKLv4W1gjoeoVkXPgc6pEY6Idx0RXJfgugTXI7gewXUIrkNwDYJrEFyT4JoEt0xwywS3RnBrBLdOcOsEVyW4KsHtEtwuwTGCYxKnJfMbrWVqXrVkXmO/PJ+a+BIGfk74LVz1KVETfZlyMUdjhIstFOlLpE+SUW7RCh9mBzaZjkZpwA8rUnufUJli39DQMq4b9lDKolRx8ZSKy+JrtcYqSa2yYmMkq1SzMZbTsGozyV9Hn4m706ZErIytKA37KxJVIqiSRD0Tu00GCoJu4B4U7Z3xbA5OjTiPeFeiPYUoLp5h+fETS25sF3L5B7nC1mJm6Ulymk3DbbgD9zD+Q1iCVRz/Cubgwlv4AB+VReWFoip6jE5OJJqbMHIp5n+DCJNR</latexit>zi` (x) = b`i +NX̀

Wij` x`j (x), x`j (x) = (zj` 1 (x))

j=1<latexit sha1_base64="D+0iXPO1Z5TDOh3tf0nEKC4ghVE=">AAAIF3ichZXfT9NQFMfP+OFw/gD0xcSXxgUDEcjdNNGYkIBKAoEQGAIjFJa13kFZuzZtN7Y15Q/wHzDGJ018MP4RPpgYXn3ggT/B+IiJD/rg6W2X/egB22y955zP9/Sce9tbxdI1x2XsLNHXPzB4JTl0NXXt+o2bwyOjtzYds2qrfEM1ddPOK0WH61qFb7iaq/O8ZfOioeh8Syk/D+JbNW47mll56TYsvmsU9ytaSVOLLroKI2ZzT+a6XtDG6xPSjKRElvRAkp2qUfAOZzL+nrdSCNx+aisMe9qhL09K9dA6ROmkdHyc6rAlzCVbB9p4c88LnFMZX/gnCiNpNs3EIcUHmWiQhuhYNUf7F0CGV2CCClUwgEMFXBzrUAQHzx3IAAMLfbvgoc/GkSbiHHxIobaKFEeiiN4y/u+jtRN5K2gHOR2hVvEuOv5sVEowxk7ZJ3bOTthn9oP9vTCXJ3IEtTTwqoRabhWGX99Z//1flYFXFw7aqksUCtKX9+RCCZ6IXjTszRKeoEs1zF9rvjlff5ob8+6zD+wn9veenbGv2GGl9kv9uMZz7zD7mJhvjplkqOFVhTTOsYdWq1YFY57w+lE9bb5bYQrbxlPDGg9QXRSWCUcXZqgT96oTXIPgGgTXJLgmwR0R3BHBlQiuRHBlgisT3BzBzRHcIsEtEtwSwS0RXJ7g8gS3TXDbBMcIjsU4JVrf4Fmm1lWJ1jWMx9dTEW9CK86JuIFPfZuQhR2nLKxR6+JCD0U6MdIhyaC24AnvZFu+OB3MUovvVLT9PqHSxb6hoKdX1xmhlKuxjlcv6HhdvK1GTydtb1yx3FVVW7PcU1OnaiWqX8WYjrvTSoyY73miFLTnY1SOoHIx6oXYbdKQFXQJ96Bg7wxXs/XVCOsIdyU6kg3y4jcs0/vFig82s9OZh9PZtUfp2WfR12wI7sI9GMf8j2EWFnD+N7CGU/iTGEgMJt8mvyS/JU9CtC8RaW5D15H8/g92B7a7</latexit>

l 平均は同様に0, 共分散はK ` (x, x0 ) ⌘ E [zi` (x)zi` (x0 )]=<latexit sha1_base64="dRe8k7P6/VXC7507Sd2oVEolzP4=">AAAIenichZXNbtNAEMcnLdAQPtrCBYlL2qhtoqbVJlSiQkJqgYpWqao0pTRVnES22SRW7DjYzqeVF+AFOHACiQPiMbhwRuLQR0Aci8SlB8ZrR07ibbGVeHfm9x/P7K53paaqmBYhZ6Gp6WvXb8yEb0Zu3b5zd3Zu/t5rU28ZMj2WdVU38pJoUlVp0GNLsVSabxpU1CSVnkj1547/pE0NU9Ebr6xekxY1sdpQKoosWmgqz/3IlASqqvFusruSiCwL9G1LaUcFTbRqkmTvDIRkoc+IshLvJvzmSqIYFYTI8tOoYCpVTSxLpXR0ddjplNJCMuJHKdv9ku1IhYU1YSE1KCsOqUVtwdCiL7ODOElmxoDEICKotGIVhGZNiXviNUeIWSQ4xhW0Gkq1ZhUj5bkYWSfsigYbKa8RA+/K6vPTuyDAG9BBhhZoQKEBFrZVEMHEuwApINBEWxFstBnYUpifwgAiqG0hRZEQ0VrH/yr2Cp61gX0npsnUMr5FxZ+ByigskZ/kCzkn38lX8otcXBrLZjGcXHr4lFwtbZZn3z04+vtflYZPC2q+6gqFhPTVNVlQgU1Wi4K1NZnFqVJ247f778+PnuSW7GXyifzG+j6SM/INK2y0/8ifD2nuA0ZfYuNNMZIAbXzKEMMxtrE3zFVCn82sAy8fnx9X6Kxv4K1gjjVUi6ynQ+fSCF3Ou7ocrsfhehyuz+H6HK7D4TocrsLhKhyuzuHqHG6bw21zuD0Ot8fhMhwuw+HyHC7P4U453CmHIxyOBDjJm19nLfPmVfLm1fUH51NiX8LQTzl+DVe9TwisH6SamKMyxrkWHmkGSJNLOrk5K3yUHdqCtDNKQ35U4dsHHJXK9g0JLZO6UQ9PmQ1UnL2k4iP2tWoTlfjWoGJ/LCtfsz+R06jqwMtfRp+Ku9NBgNiZWFES9ncCVI5D5QLUC7bbxCDN6AruQc7e6c7m8NRw83B3Jb4n7cTFMyw1eWIFG6/T66lH6+nDjdjWM+80C8NDWIQ4xn8MW7CL438McmgzVApVQ7WZi/BiOBFeddGpkKe5D2NXeOMf9ezVvA==</latexit>

2
b +

2
w Ezi` 1 ⇠GP(0,K ` 1 )⇥(zi` 1 (x))(zi` 1 (x0 ))– この期待値は、(1) ガウス過程回帰 (2) 数値的近似で順に計算するか、(3) ReLUなど特定のφについては、解析的に求められる.

68 / 81

Summer School 数理物理2021 Lecture3⇤

NNGPカーネルとニューラルネット
l φがReLUのとき (Cho&Saul 2009, Lee+ 2017)K ` (x, x0 ) =

2
b +

`
1
✓x,x
0 = cos<latexit sha1_base64="OtJcMBUNis/jw/ks9eudqB2RHqY=">AAAI9HichZVLb9NAEMcnPIN5lMIFiYtF1NIIWm0CCISExKtSUVDVB32gbhPZZtNYtWNju2lay1+AL8ABLoA4ICQOfAUufAEkOMINcQSJCwfGa5uk8bTYSrw78/uPZ3bXu7prmX7A2JfCvv0HDh46XDyiHD12/MTQyeFTi76z4RliwXAsx1vWNV9YZlssBGZgiWXXE5qtW2JJX78T+5c6wvNNp/0g2HLFqq2ttc2maWgBmhrDhVqtzoVljXUvds+XldEbKvfNNVtr6PWqekHlTU8zwtS0Wa9GYZW7ZqRw/7EXhLV6GGvHK1EsL/d3z8fhIpXzlXE2cUV0V5VRlbd8VzNEeFnYEQ9MW/gKt0QzGMNwZpsHLRFo/0I0wjijCHNQxvCV6rhKA2WVG45P+xTumWutoKxwrqREDGTeuFjU1kNUqFkmsuD+MYnCrNjMVs6aSZG91zROltgEk5eab1TSRgnSa8YZ3j8FHB6BAwZsgA0C2hBg2wINfLxXoAIMXLStQog2D1um9AuIQEHtBlICCQ2t6/i/hr2V1NrGfhzTl2oD32Lhz0OlCiPsE3vDfrKP7C37zv7sGiuUMeJctvCpJ1rhNoaenJn//V+Vjc8AWj3VHgod6b1rCqAJ12QtJtbmSktcpZHE72w//Tl/fW4kHGUv2Q+s7wX7wj5ghe3OL+P1rJh7htFH5HgLjMShg08DSjjGIfayXHX0hdIapfn0+J0KR/Y9vE3MsYVqTfYc2Nw1Qpd4V5fgtghui+C2CW6b4DYJbpPgmgTXJLh1glsnuFsEd4vg7hHcPYKrEVyN4JYJbpngHhLcQ4JjBMdynJ7Ob7yWqXnV03lN/Pn51OWXkPkF4bdx1fcILvt5ysUczR1cYqFIP0f6JBnnFq/wfjaz5el4lDK+X9GzR4TKkvuGjpZBXb+HUs7kKp7ZpeJ5+bXaA5X0rHnF/R1Z9TT3B3LqV02n+Rvos3B3ms4RkwMrSsf+ZI6aI6i5HHVX7jYlqEq6iXtQvHcms5mdGkkeya5Ee6pxXDzDKoMnVr6xWJ2oXJqozl4u3bydnmZFOAvnYAzjX4WbMIXjvwBG4X3hc+Fr4VuxU3xefFV8naD7CqnmNOy4iu/+Am4oApU=</latexit>

69 / 81

2 qw

K ` 1 (x, x)K ` 1 (x0 , x0 )

2⇡ ⇣
` 1
⇥ sin ✓x,x
0 + (⇡K ` (x, x0 )

` 1
` 1
✓x,x
)
cos
✓
0
x,x0
!

p
K ` (x, x)K ` (x0 , x0 )Summer School 数理物理2021 Lecture3⌘

NN-iWMMの生成モデル
l ∞混合ガウス分布∞ 混合ガウスモデル

で潜在変数
を生成
l

とカーネル関数のパラメータ b , w
からNNGPカーネル
L
行列 KNNを生成<latexit sha1_base64="0nTLzTnzfKiLkMSsbAJxvm/LH/A=">AAAHunichZXNbtNAEMenLeBSPpqWCxKXiCiIA4o2AQmEOLRApVatqjQhbaqmirzuJrXiL9nOt/ICvAAHJCSQECAegwsvwKGPgDgWiQsHxmtH+fCk2Eq8O/P7j2d217vcMXTPZ+xsbn7h0uUryuLVpWvXb9xcTqys7nt209VESbMN2y1z1ROGbomSr/uGKDuuUE1uiAPeeBH4D1rC9XTbeuV3HXFsqnVLr+ma6qOpmliteHrdVKv8QTJqtauJFMsweSXjjWzUSEF05e2VhU2owAnYoEETTBBggY9tA1Tw8D6CLDBw0HYMfbS52NKlX8AAllDbREogoaK1gf917B1FVgv7QUxPqjV8i4E/F5VJSLMf7As7Z9/ZV/aT/Z0Zqy9jBLl08clDrXCqy69vF//8V2Xi04fTkeoCBUf64pp8qMETWYuOtTnSElSphfFbvTfnxaeFdP8e+8B+YX3v2Rn7hhVard/axz1ReIvR03K8BUaqQAufGqRwjPvYG+bK0deX1kGUz4ifVNiy7+KtY46nqFZlz4b2zAgd4l0dgusSXJfgegTXI7g2wbUJrkZwNYJrEFyD4NYJbp3gtghui+C2CW6b4MoEVya4Q4I7JDhGcCzG8Wh+g7VMzSuP5jX0x+eTyy9h6BeE38RVPyIqsh+nHMxRn+BCC0V6MdIjySC3YIWPs0NbnA5GaciPK0b2AaEy5L7B0TKtG/dQynys4vyMiovyazWnKhlZ44qdiaxGmp2pnMZVu1H+GvoM3J12Y8TG1Iri2N+IUQWCKsSol3K3SUFO0jXcg4K9M5zN4akR5hHuSrQnF8TFMyw7fWLFG/u5TPZhJrf3KLX2PDrNFuEO3IX7GP8xrMEmjn8Jc+jAO/gEn5VnCld0pRGi83OR5hZMXIr/D5pGles=</latexit>

NNGPカーネル<latexit sha1_base64="vsKCCkC3krRXUGjXYPDIRWN2O0k=">AAAHsnichZXNbtNAEMenLeBSPtrCBYlLRBTEKdqklUCcWqBSq0RRmpA2pSmRbTatFX/JdpImVl6AB4ADBz4kDojH4MILcOgjII5F4sKB8dqRk3hSbCXenfn9xzO7613F1jXXY+xsbn7h0uUr0uLVpWvXb9xcXlm9tedaHUflNdXSLaeuyC7XNZPXPM3Ted12uGwoOt9X2k8D/36XO65mmc+9vs2PDPnY1FqaKntoahReFpt+wzFSpdKwuZJmWSauVLKRixppiK6ytbqwDQ14BRao0AEDOJjgYVsHGVy8DyEHDGy0HYGPNgdbmvBzGMISajtIcSRktLbx/xh7h5HVxH4Q0xVqFd+i489BZQoy7Af7ws7Zd/aV/WR/Z8byRYwglz4+lVDL7eby6zvVP/9VGfj04CRWXaBQkL64Jg9a8EjUomFttrAEVaph/O7g7Xn1cSXj32ef2C+s7yM7Y9+wQrP7W/28yyvvMHpGjDfHSA3o4lOFNI6xj71Rrgr6fGEdRvnE/KTCEn0Hbw1zPEG1LHoW9GZGOCXedUpwfYLrE9yA4AYE1yO4HsG1CK5FcG2CaxPcJsFtEtwOwe0QXIHgCgRXJ7g6wR0Q3AHBMYJjCU6J5jdYy9S8KtG8hv7kfCriSxj5OeE3cNXHREP0k5SNOWoTXGihSDdBuiQZ5Bas8HF2ZEvSwSiN+HFFbB8SKl3sGwpapnXjHkpZTlRcnlFxVXytxlQlsTWpKE5kFWuKUzmNq0pR/ir6dNydSglia2pFKdjfSlAVgqokqGdit0lDXtAt3IOCvTOczdGpEeYR7kq0Jx/ExTMsN31iJRt7+WxuLZvfXU9vPIlOs0W4C/fgAcZ/CBuwjeNfwxxseAPv4YO0Lr2QZEkN0fm5SHMbJi5J/wdQ5ZME</latexit>

l 観測値 Y を次元d

L 層 ∞ ユニットNN

ごとにNNGPから生成
クラスター数、ユニット数の指定が不要な教師なしNN70 / 81

Summer School 数理物理2021 Lecture3

t-SNEとの統合
l 復習：

l 前提の違う2つのモデルをいかに組み合わせるか?

71 / 81

Summer School 数理物理2021 Lecture3

t-SNEとの統合: RegBayesl 事後分布 p(θ|Y) : 次の最適化問題の解と一致(A. Zellner, 1988)
Θ：パラメータPl RegBayes：事後分布への制約を考慮したベイズモデ

ルを定式化 (J. Zhu+, 2014)– 制約を踏まえた最適分布
θ, Y の同時分布に採用
72 / 81

Summer School 数理物理2021 Lecture3

NPDVの定式化
l NPDVの同時分布 (q*(θ)とNN-iWMM, t-SNEの対応)

l 分布の具体形

NN-iWMM

t-SNE

NN-iWMM

X, V 間の t-SNEのコスト関数– 可視化表現 V、NNGPカーネルのは非確率的なパラメータ– λ は超パラメータ ※ 本研究では λ=ND で固定
73 / 81

Summer School 数理物理2021 Lecture3

NPDVの学習
l 目的関数：変分分布 Q を使った尤度の下界– Re-parametrization trickを使ってから正規乱数を生成し、(*)をモンテカルロ近似– 近似した後、勾配法ベースで学習

74 / 81

Summer School 数理物理2021 Lecture3

実験設定
l テキストデータ: 20newsgroups (英語)、Livedoor

ニュースコーパス (日本語)– 20newsgroups: InternetのUSENETから取って来た20個のニュースグループのテキスト、
18000記事– Livedoor: Livedoorニュースのテキスト、9カテゴリ、
7300記事l tf.idfで前処理、1,000次元に圧縮してから2次元に可視化
l t-SNE (Maaten+ 2009), Deep K-means (Phard+ 2020)と比較

75 / 81

Summer School 数理物理2021 Lecture3

実験結果 (20newsgroups:英語)lk 近傍法分類精度（太字：最高精度、*：次点の精度）

他手法よりも明確にクラスターを識別可能 ＆ 最高精度を記録
76 / 81

Summer School 数理物理2021 Lecture3

実験結果 (Livedoor:日本語)l77 / 81k 近傍法分類精度（太字：最高精度、*：次点の精度）

Summer School 数理物理2021 Lecture3

文の教師なし可視化 (t-SNE)
l 観測データをt-SNEで直接クラスタリングする→ノイズに弱い、意味を考慮できない

NPDVでは料理クラスタになる文が無関係なクラスタに所属
78 / 81

Summer School 数理物理2021 Lecture3

文の教師なし可視化 (NPDV)
l 学習には教師データは不要だが、∞-GMMを学習しているため、内部的にクラスタリングを行っている
NPDVで得られたクラスター
潜在空間でのクラスター、可視化表現を同時に推定

データ：
京大コーパス(毎日新聞1995年)から5000文○ 意味的類似性のあるクラスターを確認
79 / 81

Summer School 数理物理2021 Lecture3

まとめl 離散的なノンパラメトリックベイズ法である階層

ディリクレ過程と、連続的なガウス過程の両方を
自然言語に適用した講演者の研究を3つ紹介した
l ガウス過程を使うことで、解析的でありながらニューラルネットと等価な表現力を持つ– 多数の重みを学習する必要がない– 局所解や学習率等のハイパーパラメータ不要
l ディリクレ過程の使用により、内在的なカテゴリをその数とともに自動推定することができ、科学的な解釈に繋がる

80 / 81

Summer School 数理物理2021 Lecture3

持橋講演分のまとめ
l ノンパラメトリックな確率過程を用いて、自然言語を数学的に見通しよく解析するための基礎や講演者の研究について紹介しましたl 自然言語のモデル化の問題であり、数学的解析は追い付いていない部分がある– 階層的TSSBは、本来は自己相似過程の一種– 潜在空間で文や文書がどう埋め込まれているかは、研究最前線の課題
l 言語に限らず、離散データ一般に適用できる統計理論(実装も重要)81 / 81

Summer School 数理物理2021 Lecture3