科研費シンポジウム
統計学, 機械学習の数理とその応用
講演予稿集

日時: 2017 年 9 月 21 日(木)〜22 日(金)場所: はこだて未来大学
基盤研究（Ａ）15H01678「大規模複雑データの理論と方法論の総合的研究」
研究代表者: 青嶋誠(筑波大学)
開催責任者: 竹之内高志(はこだて未来大学)

統計学, 機械学習の数理とその応用
会場: はこだて未来大学

プログラム
9/21(木)
14:00-14:05 オープニング
14:05-15:35 (30 分 3)・矢田 和善・青嶋 誠 (筑波大学)
スパース性に基づくノイズ掃き出し法について
・広津千尋 (明星大学、連携研究センター)
A unifying approach to the shape and change-point hypotheses- All in one algorithm of p-value, power and confidence region・張 元宗, 篠崎信雄 (目白大学, 慶應大学)
事前情報に基づく P 次元ポアソン分布の母平均の同時推定とその応用
15:55-16:55(30 分 2)

・江口真透 (統計数理研究所)
一般化平均によるモデルと推定
・林 賢一 (慶応義塾大学)
擬似線形関数を用いたクラスタワイズ回帰モデル
17:15-18:15
・小森(30 分 2)理 (福井大学)

準線形モデルによるポアソン点過程の拡張とその応用
・Stephen Wu (統計数理研究所)
From Small Data to Big Data: Integrating Machine Learning, PhysicalModel and Uncertainty Quantification for Efficient Polymer Design
19:30- 懇親会

9/22(金)10:00-11:00(30 分 2)

・川島 孝行， 藤澤洋徳 (総合研究大学院大学， 統計数理研究所)
一般化線形回帰のロバスト化およびスパース化
・富田 裕章 (総合研究大学院大学)
多重代入法を用いたバイアス補正推定量に関する考察
11:20-11:50(30 分 2)

・川喜田雅則・藤澤洋徳 (九州大学, 統計数理研究所)
最適な半教師付き学習・熊谷 亘, 金森敬文 (理化学研究所 革新知能統合研究センター, 名古屋大学)
パラメータ転移学習における汎化誤差の評価
12:20-12:25 クロージング

スパース性に基づくノイズ掃き出し法について
矢田
青嶋

和善
誠

(筑波大数理物質)(筑波大数理物質)

1. はじめにゲノム科学，情報工学，金融工学などの現代科学の 1 つの特徴は，データがもつ次元数の膨大さにある．こういった高次元データの第一の特徴は，次元数が標本数を遥かに超えることである．第二の特徴は，高次元データは豊富な情報を有するものの，それが巨大なノイズに埋もれ見つけ難いことである．これらの理由から，通常の多変量解
析法では高次元データの推測に精度を保証することができず，間違った解析結果を導くことさえある．そのため，高次元データの解析には，新しい理論と方法論が必要になる．Yata and Aoshima [2] は，高次元小標本における PCA の性質を研究し，PCA が一致性をもつための標本数 n の次元数 d に関するオーダー条件を導き，高次元小標本において PCA が不適解を起こすことを示した．この問題を解決する策として，Yata andAoshima [3] は，高次元小標本データ空間の幾何学的表現を研究し，それに基づいて “ノ
イズ掃き出し法”とよばれる方法論を考案した．一方で，Yata and Aoshima [4] は，高次元大標本も含む一般的な高次元データに対して，power spiked モデルと呼ばれる固有値モデルを考案し，高次元データに対する新しい PCA を構築した．最近，Aoshima
and Yata [1] は，ノイズ掃き出し法による固有ベクトルの推定量を用いることで，新たな高次元二標本検定法を考案した．
本講演では，スパース性を基づくノイズ掃き出し法ついて論じた．閾値を用いて，固有ベクトルの推定量を補正することで，緩い仮定のもとその一致性を与える新たな方法論を提案した．

2. 高次元固有ベクトルの一致性

共分散行列に d 次の半正定値行列 Σ をもつ母集団を考える．母集団から n (≥ 3) 個のd 次データベクトル x1 , ..., xn を無作為に抽出する．Σ の固有値を λ1 ≥ · · · ≥ λd (≥ 0) とし，適当な直交行列 H = [h1 , ..., hd ] で Σ を Σ = HΛH T , Λ = diag(λ1 , ..., λd ) と分解!dTする．標本共分散行列 S のスペクトル分解を S = i=1 λ̂i ĥi ĥi とする．最近，Yata andAoshima [4] は，power spiked モデルとよばれる固有値モデルを考案し，高次元データ!m!pに対する新しい PCA を研究した．いま，Σ(1) = j=1 λj hj hTj ，Σ(2) = j=m+1 λj hj hTjとおき，Σ = Σ(1) + Σ(2) という分解を考える．そのとき，次の条件を満たすようなλ1 ≥ · · · ≥ λd を power spiked モデルと定義する．
mλm に対して，limd→∞ tr(Σk(2))/λkmm = 0 なる (有界な) ある自然数 km が存在する. (1)いま，δj = λ−1j tr(Σ(2) )/(n − 1)，j = 1, ..., k とおく．power spiked モデル (1) のもと，次の定理を得る．
定理 1 ([4]).
成り立つ．

各 j = 1, ..., m について，適当な正則条件のもと，d, n → ∞ のとき次がλ̂j= 1 + δj + op (1) and hTj ĥj = (1 + δj )−1/2 + op (1).
λj定理 1 より，適当な正則条件と (C-i) のもと次を得る．

||ĥj − hj ||2 = 2{1 − (1 + δj )−1/2 } + op (1).

(2)

ここで，|| · || はユークリッドノルムを表す．一方で，Yata and Aoshima [3] は，高次元
小標本データ空間の幾何学的表現を研究し，それに基づいて “ノイズ掃き出し法”とよばれる方法論を考案し，次のような固有値の推定量を提案した．
!
tr(S) − ji=1 λ̂i(j = 1, ..., n − 2).
(3)λ̃j = λ̂j −n−1−jさらに，Σ の固有ベクトルについて，ノイズ掃き出し法による推定を考える．推定量(3) に基づいて，Σ の固有ベクトル hj を h̃j = (λ̂j /λ̃j )1/2 ĥj で推定する．本講演では，h̃j を補正した．いま，h̃1 = (h̃1 , ..., h̃d )T とおき，h̃1 , ..., h̃d を絶対値の大きい順に並べ替えたものを h̃(1) , ..., h̃(d) とおく．すなわち，|h̃(1) | ≥ · · · ≥ |h̃(d) | となる．||h̃1 ||2 > 1 で!k−1!kあることに注意すれば， s=1 h̃2(s) < 1， s=1 h̃2(s) ≥ 1 となる k が一意に定まる．そのとき，h̃1 を次のようにスパース化する．

h́1 = (h́1 , ..., h́d )T .
ただし，

h́s =
とする．そのとき，"h̃s
0(|h̃s | ≥ |h̃(k) |)(|h̃s | < |h̃(k) |),(s = 1, ..., d)||h́1 − h1 ||2 = op (1)なる一致性を高次元のもと示した．同様に，hj (j ≥ 2) についてもスパース化し，その精度を理論的かつ数値的に検証した．

参考文献[1] Aoshima, M. and Yata, K. (2017). Two-sample tests for high-dimension, strongly spiked
eigenvalue models, Statistica Sinica, in press (arXiv:1602.02491).
[2] Yata, K. and Aoshima, M. (2009). PCA consistency for non-Gaussian data in high dimension, low sample size context, Communications in Statistics. Theory and Methods,
Special Issue: Honoring Zacks, S. (ed. Mukhopadhyay, N.), 38, 2634-2652.
[3] Yata, K. and Aoshima, M. (2012). Eﬀective PCA for high-dimension, low-sample-size datawith noise reduction via geometric representations, Journal of Multivariate Analysis, 105,
193-215.
[4] Yata, K. and Aoshima, M. (2013). PCA consistency for the power spiked model in highdimensional settings, Journal of Multivariate Analysis, 122, 334-354.

A unifying approach to the shape and change-point hypotheses-All in one program of p -value, power and confidence region        Ç e ĝ ® ¯   ċ ¦ Ü á 4 @ 6 A Ğ                     
1Ġ  Ą ÕĐjĀ¸ğĉ, gj±î0-[ğĢ*Ï_çfăNĂ

*ė&făğ

.-Ġ.+Áþ]<8>-ğfăv(isotonic regression)*

Û+.-. Bartholomew (1959, a, b)*((ÚÿÙW.ğ.

-Ã§âđ-³čô®Ġ+_çM+ğ

*_ç³IGÆ

, āåi"]

ĄüĖğ[, Ģtě, B÷ÝÒ<8>ğ+ 2 VĎðKJQÕtě#wėP. Bªğøóä19?A7

Hirotsu (1982).B÷_çNĂ

-ºXĜ0{,, p³č0¡-. .*-ğfă, [, ĢNĂ.., fè, 2 ď, 3 ďéàq{Þs.-. µßþÊc³Ä0Õ-ªÆĄ-.

ĄÙBąªÆ-, ¾ě’unifying’Bª,.+Ï_çĢNĂfă~ìāĐ
.+Óp0ć&-.

cÌ<8>£ĒS-. /, fă, [,.., ÂcÌ, 3?A9cÌ, ±Ì<8>-(Hirotsu

and Marumo, 2002). R!, ÂcÌ<8>0ûÄ$făÄfăÄYy,, ĈÂcÌÄBÁS§îëlû.-. m½

A9cÌ, ±Ì<8>'Þ.-. ³ÄìāĐ

,



3?

.+cÌ<8>

- efficient score º'D-. /, ·ìāIÖ-È.FÜá
._çNĂcÌtě0ílÙH\·, ’unifying’ +.-. ÕRğdùrd×¿uíl¿¼êcĀ¸.-Ġ}l, fă~bUn0­

nÔ°Ì0¤É§]«`QÕôØ|op'ć&
kĕ.ğ

º\-m°, ~bUÕC´ñ-. +, Áþ]ē+ğ¢BąªÆ-p', µªÆ’unifying’ -.

2Ġ § Ó < 8 >x𝛬 Ðã:15@æ^ 𝑦𝑖~ 𝑒𝛬𝑖𝛬𝑖𝑦𝑖𝑦𝑖  𝑖  𝑎, 0-. ÍNĂ𝐻𝑚 𝛬𝛬𝑎 0făNĂ 𝐻𝑚 𝛬𝛬𝑎, º-³Äº

ÀD+

.-,
max acc. 𝑡 𝑡𝑡𝑎  𝑡𝑘 𝑡𝑘𝑌𝑘 𝑎𝑘𝑎𝑘𝛬 𝛬𝑌𝑘𝑘𝑘 Bªğ.

ÀÂcÌ<8>           ĝ1Ğ𝑎, 𝑌𝑘 𝑦 𝑦𝑘  𝛬 𝑌𝑎𝑎.


ĝ2Ğ





-NĂ



𝜃𝑖

𝛬𝑖 𝜃𝑖 𝑘,



𝜃𝑖

𝛬𝑖 𝜃 𝑖 𝑘𝑀𝑚𝑘        



𝑎 

𝐻, for all unknown 𝑘              ĝ3Ğ- efficient score º'D-Ġğ 𝐾  +  ğ Â   c Ì < 8 > (2)    
𝐻𝐾 𝑘 𝐾 0ğ
ě'öp

,ğ(

ú

-ãNĂ  𝐻𝐾 𝑘𝐾 º-t,³ÄºìāĐĝ1ĞºjĆ+.-.

cÌ--ÍNĂ´Õ-ĠcÌTĚęz.+ºº\ağ*"TĚęzāå0 1 9?2=;¾ě ’all in one program’ p- (Hirotsu and Tsuruta, 2017).

+ğÍNĂĝ3Ğ- p-value āå©¥¹-.

ğ𝑡.+āå

Worsley (1986)Bõ-𝑡𝑘𝑡𝑘0Dm°¶OM

c²©0ú. Ĕ, 𝑡𝑘 0D𝑡𝑘¶OM-, 9?2=;Zāå-

]

ý-

Bąª3?A9cÌğ±Ì<8>
Æ

§ÓÙ¼Ċ

-Ġ}lğLāå]Ē-Ëğ.0¬ÛČ-. ¬Û- Worsley ª

one program’ ï&+.ğ%, 3?A9cÌ, ±Ì<8>
»&fè-,ğÓĄCğÕCöp-tě’all in\·. µÆ\·-. ³ğéàq{)ğġVĎðKJQÕtě'ôÎúğ^Eªğ-

.

ª

.

BªôÎĘ0-äğ½ÏÅ-(Hirotsu, 2017)Ġ

hò¨Ñ
Bartholomew, D. J. (1959a). A test of homogeneity for ordered alternatives. Biometrika
46, 36-48.
Bartholomew, D. J. (1959b). A test of homogeneity for ordered alternatives .

Biometrika 46, 328-335.
Hirotsu C. (1982). Use of cumulative efficient scores for testing ordered alternatives indiscrete models. Biometrika 69, 567-577.
Hirotsu, C. and Marumo, K. (2002). Change point analysis as a method for isotonic
inference. Scand. J. Statist. 29, 125–138.
Hirotsu, C. (2017). Advanced analysis of variance. Wiley Series in Probability andStatistics.
Hirotsu, C, and Tsuruta, H. (2017).

An algorithm for a new method of change-point

analysis in the independent Poisson sequence. Biometrical Letters 54, 1-24, 2017.
Worsley, K. J. (1986). Confidence regions and tests for a change-point in a sequence of
exponential family of random variables. Biometrika 73, 91–104.

ࣄલ৘ใʹ‫ ͮ͘ج‬p ‫ݸ‬ͷϙΞιϯ฼ฏ‫ۉ‬ͷॖখਪఆྔͱͦͷԠ༻໨നେֶɹுɹ‫ݩ‬फ, ɹɹ‫ܚ‬Ԡٛक़େֶɹࣰ࡚ɹ৴༤

͸͡Ίʹ

1

ɹ X1 , · · · , Xp Λ‫ʹ͍ޓ‬ಠཱʹϙΞιϯ෼෍ Po (λi ), i = 1, · · · , p(≥ 2), ʹ͕ͨ͠͏֬཰ม਺ͱ͢Δɻඪ४Խ̎৐‫ࠩޡ‬ଛࣦؔ਺ʢnormalized squared error lossʣL(λ̂, λ) =

p
!
i=1

2
λ−1
i (λ̂i − λi )(1.1)Λ‫ج‬४ͱͨ͠ͱ͖ɺ฼ฏ‫ ۉ‬λ = (λ1 , · · · , λp )′ ͷಉ࣌ਪఆ໰୊ʹରͯ͠ɺClevension & Zidek (1975) ͸‫ʹ఺ݪ‬ॖখ͢
Δਪఆྔ
"
#
ϕ(Z)
CZ
λ̂i (X) = 1 −Xi ,i = 1, . . . , p,Z +p−1$pΛఏҊͨ͠ɻ͜͜ͰɺZ = i=1 Xi Ͱ͋Δɻϕ(Z) ͕ඇ‫ݮ‬গؔ਺ɺ0 ≤ ϕ(Z) ≤ 2(p − 1) ͳΒ͹͜ͷਪఆྔ͕ෆภਪఆ
ྔ X = (X1 , · · · , Xp )′ Λվྑ͢Δ͜ͱΛࣔͨ͠ɻ͔͠͠ɺ͍͔ͭ͘ͷ λi ͕େ͖ͳ஋Ͱ͋Δ৔߹ɺ‫ʹ఺ݪ‬ॖখ͢Δਪఆྔ͸େ͖ͳվྑΛ༩͑Δͱ͸‫͍ͳ͑ݴ‬ɻGhosh, Hwang & Tsui(1983) ͓Αͼ Tsui (1984,1986) ͸ɺࢦఆͨ͠ඇෛͷ
੔਺఺͋Δ͍͸ॱং౷‫ʹྔܭ‬ॖখ͢ΔਪఆྔΛఏҊͨ͠ɻ͔͠͠ɺ྆࿦จͰఏҊ͞Εͨਪఆྔ͸ෳࡶͰɺվળͷ༨஍
͕͋Δɻ͜͜Ͱɺࣄલ৘ใʹ‫ͮ͘ج‬ϙΞιϯ฼ฏ‫ۉ‬ͷॖখਪఆཧ࿦Λ౷߹͠ɺ͋Δਖ਼ͳ஋·ͨ‫؍‬ଌ஋ͷ࠷খ஋ʹॖখ
͢ΔΑ͏ͳਪఆྔͷΫϥεΛ࠶ߏங͢Δɻ͞ΒʹɺϙΞιϯ฼ฏ‫ ʹۉ‬simple tree order ੍໿͕͋Δ৔߹ʹɺ isotonic
regression ਪఆྔΛվྑ͢Δํ๏ΛఏҊ͢Δɻ·ͨɺmultiplicative Poisson models Ͱͷ฼ฏ‫ۉ‬ͷಉ࣌ਪఆ໰୊΋औΓ্͛ɺॱং౷‫΁ྔܭ‬ͷॖখਪఆྔΛఏҊ͢Δɻ

ࣄલ৘ใʹ‫ͮ͘ج‬ॖখ

2͜ͷઅͰ͸ɺࢦఆ͞Εͨඇෛͳ஋‫ॱͼٴ‬ং౷‫΁ྔܭ‬ͷॖখΛ࿦͡Δɻ࣍અͰ̍ͭͷԠ༻ྫͱͯ͠ɺ฼ฏ‫ ʹۉ‬simple
tree order ੍໿৚͕݅͋Δ৔߹ɺisotonic regression ਪఆྔΛॖখ͢Δಉ࣌ਪఆྔΛఏҊ͢Δɻ

2.1

ඇෛͳ஋΁ͷॖখai ≥ 0, i = 1, · · · , p ͱ͠ɺ෦෼ू߹ C = {(x1 , . . . , xp )|xi ≥ ai , i = 1, . . . , p} ͱͦͷΠϯδέʔλؔ਺Λ IC ͱ͢Δɻai
ʹॖখ͢ΔਪఆྔΛͭ͗ͷΑ͏ʹߟ͑Δɻλ̂i (X) = Xi − ϕ(ZC )(Xi − ai )IC ,ZC + di = 1, . . . , p.

$p
͜͜ͰɺZC = i=1 (Xi − ai ) Ͱ͋Γɺd > 0 Ͱ͋Δɻ෦෼ू߹ C Ͱ (1.1) ͷଛࣦؔ਺ͷԼͰɺX ͱ λ̂(X) = (λ̂1 (X), . . . , λ̂p (X)) ͱͷฏ‫ۉ‬ଛࣦͷࠩΛධՁ͢Δ͜ͱͰɺλ̂(X) ͕ X Λվྑ͢ΔͨΊͷे෼৚݅Λͭ͗ͷఆཧͰ༩͑Δɻ
ఆཧ 2.1. p ≥ 2 ͱ͢Δɻଛࣦؔ਺ (1.1) ͷԼͰɺλ̂(X) ͕ X Λվྑ͢ΔͨΊͷे෼৚݅͸ ϕ(·) ͸ඇ‫ݮ‬গؔ਺Ͱɺ
0 ≤ ϕ(·) ≤ 2(p − 1), d ≥ sup ϕ(·)/2 Ͱ͋Δɻ࣍ʹɺ͋Δ k ≥ 2 ʹରͯ͠ɺ෦෼ू߹ Ck = {(x1 , . . . , xp )|xi ≥ ai , i = 1, . . . , k, xj < aj , j = k + 1, . . . , p} ͱ͢Δɻ͜ͷΑ͏ͳ 2p − p − 1 ‫ݸ‬ͷ‫ʹ͍ޓ‬ഉ൓ͳ෦෼ू߹ͷͦΕͧΕͰ ai ʹॖখ͢ΔΑ͏ͳਪఆྔΛߟ͑Δɻͭ·ΓɺX ∈ Ckͷͱ͖ɺ⎧(Xi − ai )⎨Xi − ϕk (ZCk ),i = 1, . . . , k,λ̂i (X) =ZCk + dk⎩Xi ,i = k + 1, . . . p,$kΛߟ͑Δɻ͜͜ͰɺZCk = i=1 (Xi − ai ) Ͱ͋Γɺ dk > 0 Ͱ͋Δɻ֤෦෼ू߹Ͱ X ͱ λ̂(X) ͱͷฏ‫ۉ‬ଛࣦͷࠩΛධՁ͢Δ͜ͱͰɺλ̂(X) ͕ X Λվྑ͢ΔͨΊͷे෼৚݅Λͭ͗ͷఆཧͰ༩͑Δɻ
ఆཧ 2.2 ɹଛࣦؔ਺ (1.1) ͷԼͰɺλ̂(X) ͕ X Λվྑ͢ΔͨΊͷे෼৚݅͸ ϕk (·) ͸ඇ‫ݮ‬গؔ਺Ͱɺ0 ≤ ϕk (·) ≤2(k − 1), dk ≥ sup ϕk (·)/2 Ͱ͋Δɻ

2.2

ॱং౷‫΁ྔܭ‬ͷॖখp ≥ 3 ͱ͠ɺ࠷খ஋ X(1) = min{X1 , . . . , Xp } ʹॖখ͢ΔΑ͏ͳਪఆྔΛλ̂i (X) = Xi − ϕ(W )Λߟ͑Δɻ͜͜ͰɺW =$p

k=1 (Xk − X(1) ) Ͱ͋ΔɻXi − X(1),W +di = 1, . . . , pఆཧ 2.3 ଛࣦؔ਺ (1.1) ͷԼͰɺλ̂(X) ͕ X Λվྑ͢ΔͨΊͷे෼৚݅͸ ϕ(·) ͸ඇ‫ݮ‬গؔ਺Ͱɺ0 ≤ ϕ(·) ≤ 2(p−2), d ≥sup ϕ(·)/2 Ͱ͋Δɻ͜ͷे෼৚݅͸ඪຊۭؒΛ p ‫ݸ‬ͷ‫ʹ͍ޓ‬ഉ൓ͳ෦෼ू߹ʹ੾Γ෼͚ɺ֤ू߹Ͱͷฏ‫ۉ‬ଛࣦͷࠩΛධՁ͢Δ͜ͱͰ
ࣔ͞ΕΔɻ

3

Ԡ༻ྫʵ฼ฏ‫੍ʹۉ‬໿৚͕݅͋Δ৔߹ͷ isotonic regression ਪఆྔͷվྑ:

ྫ 3.1. Xi ∼ Po (λi ), i = 0, 1, . . . , p ʹै͍ɺ฼਺ʹ simple tree order ੍໿৚݅ɺλ0 ≤ λi , i = 1, . . . , p ͕͋Δ৔߹ɺλ ͷ isotonic regression ਪఆྔ͸࣍ͷΑ͏ʹ༩͑ΒΕΔɻ
!
Xi ,for i ∈ S cstλ̂i (X) =AX (S),
for i ∈ S,
͜͜ͰɺҰൠੑΛࣦ͏͜ͱͳ͘ɺS = {0, 1, . . . , k}, S c = {k + 1, . . . , p} Ͱ͋Δͱ͠ɺAX (S) =AX (S), i ∈ S c Ͱ͋Δɻp − k ≥ 2 ͷͱ͖ɺλ̂sti (X) Λ࣍ͷΑ͏ʹॖখ͢Δɻ
⎧Xi − AX (S)⎨Xi − ϕp−k (WS c ),for i ∈ S c
λ̂m
(X)
=
c
W
+
d
S
p−k
i
⎩
AX (S),
for i ∈ S,
͜͜ͰɺWS c ="i∈S Xi /(k +1), Xi ≥"p

i=k+1 (Xi − AX (S)) Ͱ͋Δɻ

ఆཧ 3.1 ଛࣦؔ਺ (1.1) ͷԼͰɺλ̂m (X) ͕ λ̂st (X) Λվྑ͢ΔͨΊͷे෼৚݅͸ ϕp−k (·) ͸ඇ‫ݮ‬গؔ਺Ͱ,dp−k ≥sup ϕp−k (·)/2 Ͱ͋Δɻ·ͨɺ฼਺ʹ࣍ͷΑ͏ͳ੍໿৚͕݅༩͑ΒΕΔ৔߹ʹ΋Ԡ༻Ͱ͖Δɻ


















ᅗ





ᅗ

ྫ 3.2. ਤ 1 ʹࣔ͞ΕΔΑ͏ʹɺ฼਺ʹԼ‫ه‬ͷΑ͏ͳ੍໿৚݅

λ1 ≤ λ3 ,λ 3 ≤ λ4 ,λ3 ≤ λ 5 ,λ 2 ≤ λ5͕༩͑ΒΕΔͱ͢Δɻλi ͷ isotonic regression ਪఆྔΛ Xi∗ , i = 1, . . . , 5 ͱ͢Δͱ X4∗ = X4 ɺX5∗ = X5 ʹͳΔͨΊͷඞཁे෼৚݅͸ X4 ≥ X3∗ ɺX5 ≥ max(X2∗ , X3∗ ) Ͱ͋Δɻ͜ͷ৔߹ʹ‫ͯͬݶ‬ɺ(X4 , X5 ) Λ (X3∗ , max(X2∗ , X3∗ )) ʹॖখ͢Δɻྫ 3.3. ਤ 2 ʹࣔ͞ΕΔΑ͏ʹɺ฼਺ʹԼ‫ه‬ͷΑ͏ͳ੍໿৚݅

λ1 ≤ λ3 ,λ 3 ≤ λ5 ,λ 3 ≤ λ6 ,λ2 ≤ λ 4 ,λ 2 ≤ λ6 ,λ 4 ≤ λ7͕༩͑ΒΕΔͱ͠ɺλi ͷ isotonic regression ਪఆྔΛ Xi∗ , i = 1, . . . , 7 ͱ͢Δɻͦͷͱ͖ɺX5∗ = X5 , X6∗ = X6 ͔
ͭ X7∗ = X7 , ͷ৔߹ (X5 , X6 , X7 ) Λ (X3∗ , max(X2∗ , X3∗ ), X4∗ ) ΁ॖখ͢ΔɻX5∗ = X5 , X6∗ = X6 ͔ͭ X7∗ > X7 ͷ৔߹ɻ(X5 , X6 ) Λ (X3∗ , max(X2∗ , X3∗ )) ʹॖখ͢Δɻಉ༷ʹɺ (X5∗ = X5 , X6∗ > X6 , X7∗ = X7 ) ‫( ͼٴ‬X5∗ > X5 , X6∗ =
X6 , X7∗ = X7 ) ͷ৔߹ʹ΋ॖখ͢Δ͜ͱ͕Ͱ͖Δɻ

4

multiplicative Poisson models Ͱͷ฼ฏ‫ۉ‬ͷಉ࣌ਪఆ໰୊΁ͷԠ༻

multiplicative Poisson models Ͱͷ฼ฏ‫ۉ‬ͷಉ࣌ਪఆ໰୊Λߟ͑Δͱ͖ɺ্‫Ͱه‬ड़΂ͨࣄલ৘ใʹ‫ͮ͘ج‬ॖখʹؔ͢Δཧ࿦͸࠷໬ਪఆྔͷॱং౷‫΁ྔܭ‬ͷॖখʹ΋Ԡ༻͢Δ͜ͱ͕ग़དྷΔɻҰൠԽฏ‫ʹۉ‬ΑΔϞσϧͱਪఆߐ‫ޱ‬ɹਅಁ
౷‫਺ܭ‬ཧ‫ॴڀݚ‬Let φ be a nonnegative, strictly increasing, convex function defined on R. We discuss generalized geodesic
connecting probability density functions p(x) and q(x) as!
"
pφ (x, π) = φ πφ−1 (q(x)) + (1 − π)φ−1 (p(x)) − κφ (π)(1)
for π of (0, 1). A typical example of φ is given by an exponential function. Thus,!
"
pexp (x, π) = exp π log q(x) + (1 − π) log p(x) − κexp (π) .

is nothing but the e-geodesic, and κexp (π) is called the cumulant function. We observe that$#∂#
κexp (π)#= {log p(x) − log q(x)}p(x)dx,∂π
π=0
which is the KL divergence between p and q. We find for a general φ that$#∂#κφ (π)#= {φ−1 (p(x)) − φ−1 (q(x))}Ξ(p(x))dx, say Dφ (p, q),
∂π
π=0
where

Ξ(p(x)) = %φ′ (φ(p(x)))φ′ (φ(p(y)))dy

Hence, we call Dφ (p, q) the generalized KL divergence confirming to satisfy the first axiom of a distancefunction, that is,Dφ (p, q) ≥ 0 with equality if and only if p = q(2)

In summary, we the geodesic {pφ (x, π) : π ∈ [0, 1]} connecting density function p and q as given in (1)naturally associates with the divergence Dφ (p, q). In this discussion we can consider the dual geodesic
defined byp∗φ (x, π) = (1 − π)Ξ(p(x)) + πΞ(q(x)).
If φ = exp, thenp∗exp (x, π) = (1 − π)p(x) + πq(x)which is nothing but the m-geodesic. In accordance, {pφ (x, π) : π ∈ [0, 1]} is a generalization of e-geodesic;{p∗φ (x, π) : π ∈ [0, 1]} is that of m-geodesic. In this line, we observe a generalization of Pythagoras identity
associated with orthogonal e-geodesic and m-geodesic.

If we adopt a parametric mode p(x, θ) for given data {xi }ni=1 , then we discuss a loss function given bynLφ (θ) = −1 ! −1 −1φ (Ξ (p(xi , θ))).
n i=1

and the estimator for the parameter θ is proposed by θ̂φ = argminθ Lφ (θ). In accordance with the formalation
we confirm the asymptotic consistency for θ̂φ as follows. Let p(x, θ0 ) be the underlying density function of
the data. Then the expected loss function is given by"Lφ (θ, θ0 ) = − φ−1 (Ξ−1 (p(x, θ)))p(x, θ0 ).
Then we observe thatLφ (θ0 , θ0 ) − Lφ (θ, θ0 ) = Dφ (Ξ−1 (p(·, θ0 )), Ξ−1 (p(·, θ))),which is nonnegative with equality if and only if θ = θ0 because of (2). Hence we conclude the asymptotic
consistency of θ̂φ for θ0 noting Lφ (θ) is almost surely converges to L(θ, θ0 ). The proof is essentially equivalent
to that for the asymptotic consistency of the MLE, cf. Wald (1949). In fact, if φ = exp, then Ξ equals theidentity function and Lexp (θ) is the negative log-likelihood function, so that θ̂φ is the MLE of θ0 .
We discuss to explore this view for the model and estimation methods in a context of generalized regression
analysis and clustering analysis, cf. Rose (1998), Notsu et al. (2016) and Omae et al. (2017). A typicalexample of φ is considered in a class of cumulative distribution functions including exponential and Parate
distributions.

References[1] Rose, Kenneth. ”Deterministic annealing for clustering, compression, classification, regression, and related optimization problems.” Proceedings of the IEEE 86.11 (1998): 2210-2239.
[2] Notsu, Akifumi, and Shinto Eguchi. ”Robust clustering method in the presence of scattered observations.”
Neural computation 28.6 (2016): 1141-1162.
[3] Omae, Katsuhiro, Osamu Komori, and Shinto Eguchi. ”Quasi-linear score for capturing heterogeneous
structure in biomarkers.” BMC bioinformatics 18.1 (2017): 308.
[4] Wald, A. (1949). Note on the consistency of the maximum likelihood estimate. Annals of Mathematical
Statistics 20, 595-601.

ٖࣅઢ‫਺ؔܗ‬Λ༻͍ͨΫϥελϫΠζճ‫ؼ‬Ϟσϧ‫ܚ‬ጯٛक़େֶ ཧ޻ֶ෦ ྛ ‫ݡ‬Ұ

1

എ‫ܠ‬

Ϗοάσʔλͷ࣌୅ͱ͍ΘΕΔࡢࠓɼ3 ͭͷ V ͱͯ͠‫͛ڍ‬ΒΕΔʮྔʢvolumeʣɼ଎౓ʢvelocityʣɼଟ༷ੑʢvarietyʣʯ͸౷‫ܭ‬Պֶʹଟ͘ͷ՝୊Λಥ͖෇͚͍ͯ
ΔʢLaney, 2001ʣɽ͜ͷྲྀΕͷதͰɼσʔλͷ‫ن‬໛΍ෳࡶੑ͸૿͠ɼҟ࣭ͳूஂΛෳ਺‫ؚݸ‬ΉΑ͏ͳσʔλ͕ಘΒΕΔ͜ͱ͕ଟ͘ͳͬͨɽ͍ͬ͡͞ɼҟ࣭ͳूஂΛ‫ؚ‬Ήσʔλ͸ੜ໋Պֶ΍ೝ஌ՊֶɼϚʔέςΟϯάɾαΠΤϯεͳͲɼॾछͷ෼໺ʹ͓͍ͯ‫͞࡯؍‬Ε͍ͯΔʢO’Driscoll et al., 2012; Suk et al., 2014ͳͲʣɽ͜ͷΑ͏ͳ৔߹ɼൺֱతখ‫ن‬໛ͳ౷੍͞Εͨσʔλʹ͓͍ͯԾఆ͢Δ
Α͏ͳɼ୯Ұͷ֬཰෼෍ʹΑͬͯσʔλͷഎ‫ʹܠ‬જΉߏ଄Λղ͖໌͔͢ࢼΈʹ͸‫ݶ‬ք͕͋Δɽ͍·ɼσʔλ {(xi , yi ); i = 1, . . . , n} ͕‫؍‬ଌ͞Ε͍ͯΔͱ͢Δɽxi ͸ d ‫ݸ‬ͷઆ໌ม਺Λ‫ؚ‬ΉϕΫτϧɼyi ͸ඃઆ໌ม਺Ͱ͋Δɽ͜ͷͱ͖ɼyi Λؔ਺ µ(xi )ʹΑΓ༧ଌɾઆ໌͢Δ໰୊Λߟ͑Δɽ·ͨɼσʔλʹ͸ K ‫ݸ‬ͷҟ࣭ͳ෦෼ू
ஂ͕‫·ؚ‬Εɼ֤‫ݸ‬ମ͸ͦΕΒͷ͏ͪͷ͍ͣΕ͔ʹଐ͢Δ΋ͷͱ͢Δɽ
͜ͷΑ͏ͳઃఆԼͰͷճ‫ؼ‬໰୊ʹ͍ͭͯɼDesarbo and Cron (1988) ͸ҎԼͷΑ͏ͳ֬཰ϞσϧΛಋೖͨ͠ɽ
#%K!
$ i )2
1(yi − β ⊤
kx
yi ∼
pk "
exp −
.
(1)
2σk2
2πσk2
k=1
&K
⊤
$ i = (1, x⊤
͜͜Ͱɼxi ) ɼp1 , . . . , pK ͸
k=1 pk = 1ɼpk ≥ 0 (k = 1, . . . , K) ΛΈͨ͢ະ஌ύϥϝʔλͰ͋Δɽ͜ΕͱྨࣅͷϞσϧͱͯ͠ɼࠞ߹ΤΩεύʔτ
ϞσϧʢMoE; mixture of expertsʣ͕‫͛ڍ‬ΒΕΔʢJacobs et al., 1991ʣɽMoEϞσϧ͸ɼ৚݅෇֬཰ʢີ౓ʣؔ਺ f (yi |xi ) Λ࣍ͷΑ͏ʹද‫͢ݱ‬Δɽ

f (yi |xi ; β, γ) =

K
!
k=1

πk (xi ; γ)fk (yi |xi ; β k ).

(2)

$i)
exp(γ ⊤
kx
͜͜Ͱɼβ = (β 1 , . . . , β K )⊤ ɼγ = (γ 1 , . . . , γ K )⊤ ɼπk (xi ; γ) = &K
ɼ
⊤$
exp(γ
x
)
i
ℓ
ℓ=1
fk (yi |xi ; β k ) ͸ύϥϝʔλ β k Λ΋ͭɼୈ k ෦෼ूஂʹ͓͚Δ৚݅෇֬཰ʢີ
౓ʣؔ਺Ͱ͋Δɽ

2

ٖࣅઢ‫਺ؔܗ‬Λ༻͍ͨճ‫਺ؔؼ‬ຊ‫Ͱڀݚ‬͸ɼҰൠԽઢ‫ܗ‬Ϟσϧ E [Y |x] = g −1 (µ(x; θ)) ʹର͠ʢg ͸ϦϯΫؔ਺ɼx ∈ Rd ɼθ ͸ύϥϝʔλϕΫτϧʣɼҎԼͷΑ͏ͳճ‫਺ؔؼ‬ΛఏҊ͢Δɽ
' K
(
!
1
µ(x; θ) = log
pk (x; µ) exp (τ µk (x; β k )) .
(3)
τ
k=1

1

͜͜Ͱɼτ ͸࣮਺஋ΛͱΔϋΠύʔύϥϝʔλɼpk (x; µ) ͸ύϥϝʔλ µ Λ!Kd΋ͪɼ೚ҙͷ x ∈ R ʹର͠ k=1 pk (x; µ) = 1 ΛΈͨ͢ඇෛͷؔ਺ɼθ =
⊤
⊤ ⊤
(β ⊤
(3) ͸ɼK ‫ݸ‬ͷճ‫਺ؔؼ‬Λ‫ڱ‬ٛ୯ௐ૿Ճͳ
1 , . . . , β K , µ ) Ͱ͋Δɽճ‫਺ؔؼ‬
"!
#
K
−1
ؔ਺ φ(·) Λ༻͍ͨ݁߹ φp(x;µ)φ(µ(x;β))ͱ‫ݟ‬Δ͜ͱ͕Ͱ͖Δɽk
k=1 k͜ͷΑ͏ͳ φ ʹΑΔ݁߹Λٖࣅઢ‫ͱ߹݁ܗ‬ΑͼʢOmae et al., 2017ʣɼຊߘͰ
͸ٖࣅઢ‫߹݁ܗ‬Λ༻͍ͨճ‫ؼ‬Ϟσϧ (3) ΛΫϥελϫΠζٖࣅઢ‫ܗ‬ʢclusterwise quasi-linear; CWQLʣճ‫ؼ‬ϞσϧͱΑͿ͜ͱʹ͢Δɽؔ਺ (3) ͸ɼφ(z) =exp(τ z) ͷ৔߹ͷٖࣅઢ‫͋Ͱ߹݁ܗ‬Δɽ࣮਺ τ ͸ɼ֤ճ‫ ਺ؔؼ‬µk ͷॏΈΛௐ੔͢ΔϋΠύʔύϥϝʔλͰ͋Γɼτ → 0 ͷͱ͖ʹ୯७ͳॏΈ෇͖ฏ‫ۉ‬!Kµ(x; θ) =k=1 pk (x; µ) exp (τ µk (x; β k )) ͱͳΔɽ·ͨɼτ → ∞ ͷͱ͖͸µ(x; θ) = max (µ1 (x; β 1 ), . . . , µK (x; β K )) ͱͳΓɼ༗‫ݶ‬஋ͷ τ ͸ฏ‫ۉ‬஋ͱ࠷େ஋ʢτ → −∞ Ͱ࠷খ஋ʣͷؒΛௐ੔͢ΔύϥϝʔλͱղऍͰ͖Δɽ⊤ ⊤ຊ‫Ͱڀݚ‬͸ɼ֤Ϋϥελ k ʹ͓͚Δճ‫਺ؔؼ‬Λ µk (x; β k ) = β ⊤k (1, x ) ͱ͠ɼؔ਺ pk (x; µk ) ʹ͍ͭͯҎԼͷΑ͏ͳ‫ܗ‬Λߟ͑Δɽ

exp (−ω||x − µk ||2 )pk (x; µ) = !K2ℓ=1 exp (−ω||x − µℓ || )

(4)

⊤ ⊤
͜͜Ͱɼµ = (µ⊤1 , . . . , µK ) ɼω ͸ਖ਼ͷϋΠύʔύϥϝʔλͰ͋Δɽ͜ͷؔ
% K
&
n
$
1$
਺ (4) ͸ɼࣗ༝ΤωϧΪʔ −log
exp(−ω||xi − µk ||2 ) ʹ͍ͭͯͷ
ω i=1
k=1

࠷େԽ໰୊ͷղͱͯ͠ಘΒΕΔʢRose et al., 1990ʣɽࣗ༝ΤωϧΪʔ΋·ͨ φ(z) = exp(ωz) ʹΑΔٖࣅઢ‫߹݁ܗ‬ͷ‫Ͱࣜܗ‬ද͞Ε͍ͯΔ͜ͱʹ஫໨͢Δ
ͱɼ(3) ʹ͓͚Δ τ ͷ‫ࣅྨͱݶۃ‬ͷ݁Ռ͕‫͞࡯؍‬ΕΔɽ͢ͳΘͪɼω → 0 ͷͱ1
͖ pk (x; µ) =ͱͳΓɼ͢΂ͯͷճ‫͕਺ؔؼ‬౳͘͠ॏΈ͚ͮΒΕΔɽ·ͨɼK
ω → ∞ ͷͱ͖͸ k-means ๏ͱಉ༷ͷ݁Ռ͕ಘΒΕΔɽ0 < ω < ∞ ͷͱ͖͸ɼ
pk ͸Ϋϥελ k ΁ͷॴଐͷ౓߹͍Λࣔ͢Α͏ͳؔ਺ͱղऍͰ͖ɼΫϥελؒͰ෼ࢄ͕౳͍͠৔߹ͷࠞ߹ਖ਼‫ن‬෼෍Ϟσϧͱଊ͑Δ͜ͱ͕Ͱ͖ΔɽҎ্ΑΓɼ‫ط‬ଘͷճ‫ؼ‬ϞσϧΛ֦ுͨ͠΋ͷͰ͋Δ͜ͱ͕Θ͔Δɽ
ਪఆ্ͷ໰୊΍ɼ਺஋࣮‫ݧ‬ͷ݁ՌͳͲ͸ɼ౰೔ใࠂ͢Δɽ

References[ 1 ] DeSarbo, W.S., Cron, W.L. (1988). Journal of Classification, 5, 249–282.
[ 2 ] Jacobs, R.A., et al. (1991). Neural Computation, 3, 79–87.
[ 3 ] Laney, D. (2001). http://blogs.gartner.com/doug-laney/files/2012/01/ad9493D-Data-Management-Controlling-Data-Volume-Velocity-and-Variety.pdf[ 4 ] O’Driscoll, D.M., et al. (2012). Sleep, 35, 1269–1275.
[ 5 ] Omae, K., et al. (2017). BMC Bioinformatics, 18, 308.
[ 6 ] Rose, K., et al. (1990). Physical Review Letters, 65, 945–948.
[ 7 ] Suk, H., et al. (2014). PLoS ONE, 9 (2): e87056.

2

Extension of Poisson point process based on quasi-linear
modeling
Osamu Komori1 , Yusuke Saigusa2 , Shinto Eguchi3
12
3

University of Fukui

Yokohama City University

The Institute of Statistical Mathematics
Abstract

The investigation to clarify the relationship between habitat distribution of somespecies and the environmental variables is important for its conservation and and management purpose. To do this, the maximum entropy method (Maxent) or Poisson point
process (PPP) is widely employed using the presence-only data. In this paper, we propose an extension of PPP based on the quasi-linear modeling to improve the estimation
accuracy. The eﬀect of sampling bias is also considered in our model. Some simulation
studies and real data analysis are conducted to show its practical utility.

1

Poisson point process based on quasi-linear modeling

For a study area A there are m presence locations {s1 , . . . , sm }. The log-likelihood of Poisson
point process (PPP) based on a quasi-linear modeling is defined as"m!
L(β, δ) =log(λτ (si )) −λτ (s)ds,
(1)
s∈A

i=1

where

#

$
%
1
λτ (s) = exp
log π exp(τ β ⊤ x(s)) + (1 − π) exp(τ δ ⊤ z(s))
τ

&

(2)

and π is a mixing proportion; x(s) = (1, x1 (s), . . . , xp (s))⊤ and z(s) = (1, z1 (s), . . . , zq (s))⊤
denote environmental and sampling-bias variables at a location s, respectively. When the
shape parameter τ converges to 0, then we have$%lim λτ (s) = exp πβ ⊤ x(s) + (1 − π)δ ⊤ z(s) ,
(3)
τ →0

1

which is nothing but the intensity of the original PPP. Clearly, we have the intensity of superposed PPP when τ = 1(Streit, 2010) as
λ1 (s) = π exp(β ⊤ x(s)) + (1 − π) exp(δ ⊤ z(s)).

(4)

Moreover, we have1
π exp(−β x(s)) + (1 − π) exp(−δ ⊤ z(s))!
"
lim λτ (s) = exp max{β ⊤ x(s), δ ⊤ z(s)}τ →∞!
"lim λτ (s) = exp min{β ⊤ x(s), δ ⊤ z(s)} .
λ−1 (s) =

⊤

τ →−∞

(5)
(6)
(7)

In general, the quasi-linear modeling in (2) is formulated by Kolmogorov-Nagumo average(Eguchi & Komori, 2015). See (Omae et al., 2017) for the application to the classification
problems.
The study area A is split into n grid cells, resulting in the approximated log-likelihood(Renner & Warton, 2013)
n
%
&
$
#
Lτ (β, δ) =wi yi log(λτ (si ; )) − λτ (si ) ,(8)
i=1

where {sm+1 , . . . , sn } are the centers of the grid cells containing no presence location, yi = I(i ∈{1, . . . , m})/wi with a quadrature weight wi , and I(·) is the indicator function. The weight wi
is a grid cell area divided by the number of locations {s1 , . . . , sn } contained in the cell. If there
is no duplication of locations in each cell, wi = |A|/n, where |A| is the area of A.
Here we haven$∂ #Lτ (β, δ) =wi q(si ){yi − λτ (si )}x(si ) = 0
(9)
∂β
i=1
where

n
$
∂ #
Lτ (β, δ) =wi (1 − q(si )){yi − λτ (si )}z(si ) = 0.
∂δi=1q(s) =

π exp(τ β ⊤ x(s)).
π exp(τ β ⊤ x(s)) + (1 − π) exp(τ δ ⊤ z(s))

(10)

(11)

If τ β ⊤ x(si ) has a large positive value, which is often the case where the impact of environmental variables is strong, then q(si ) becomes nearly 1. On the other hand, if τ δ ⊤ z(s) has alarge positive value, which is often the case where the impact of bias sampling is strong, thenq(si ) becomes nearly zero. Hence we expect that q(si ) can adjust the balance of the impactsof the two variables. This is an extension of the asymmetric logistic regression model (Komori
et al., 2016), where the weight function has an important role in adjusting the imbalancenessof the sample sizes of the two populations.
2





, Sep.21-22, 2017,

From Small Data to Big Data: Integrating Machine Learning, Physical Model and Uncertainty
Quantification for Efficient Polymer Design Wu*, Yukiko Kondo**, Isao Kuwajima##, Guillaume Lambard*,##, Kenta Hongo#,##,%,
Junko Morikawa**, Yibin Xu##, Ryo Yoshida*,##* The Institute of Statistical Mathematics, Tokyo, 190-8562, Japan** Tokyo Institute of Technology, Tokyo, 152-8550, Japan#
Japan Advanced Institute of Science and Technology, Ishikawa, 923-1292, Japan##
National Institute for Materials Science, Tsukuba, 305-0047, Japan%
PRESTO, JST, Kawaguchi, Saitama, 332-0012, Japan
Speaker at the conferenc
1Introduction

this talk, we will demonstrate a specific implementation of

In the past, data could not be efficiently collected, stored

such an idea using the PolyInfo database [8] for searching highand accessed, and calculation required tremendous amount ofthermal conductivity polymers. Our implementation aims at

human power. Especially for material science, data collection

providing an efficient end-to-end material design process thatcomes from experiments that usually take a long period ofincorporates Bayesian inversion, machine learning models,time, and consume a large amount of financial and humanand experimental design concepts.

resources. To design materials for a specific purpose, scientists

2Methods

had to plot their research map step-by-step carefully to reduce

Thermal conductivity of polymers has continuously

the probability of significant failures. This traditional approach,attracted many attentions for a long period of time [9-12]. Yet,which we refer as the small data approach, has led to many

we are far from understanding the underlying mechanism

successes to understand the physics of material properties atenough for general design purpose. For that, we adopt a rather

the level of continuum models. However, the problem

ambitious vision outlined in [13] that is to perform inverse

becomes much harder at the molecular level due to thematerial design with generation of new molecules. Althoughexponential increase of the complexity. With the advancement

we would like to implement the idea directly using theof experimental and computational technologies, it is time for

PolyInfo database [10], one of the largest databases ofmaterial scientists to move from the small data approach to apolymers in the world, the data availability of thermal

big data approach. Instead of manually hypothesizing andconductivity appears to be significantly less than othertesting different prediction models for new material discovery,

material properties: among the over a thousand of data points,

machine learning offers a promising tool for big data analysis

only less than a hundred of unique homopolymers data is

to create effective empirical models. Such models can, then, beavailable with large variance. Given the expensive cost of

integrated into a data-driven material design framework.

obtaining new experimental data, we propose an experimental

Similar successes have been seen in various fields, such as

design schedule that integrates existing knowledge andimage recognition [1,2], natural language processing [3,4],

machine learning techniques, in order to increase thegames [5], financial trading [6], and so on. The necessary costprobability of new material discovery while we are moving

for such an approach is the large amount of informative data.

toward the final goal of fully data-driven inverse design. We

Here, large and informative are very subjective concepts

achieve that by observing a correlation between the glass

depending on the targeted problem. For example, while 250

transition temperature and the thermal conductivity based ondata points seem to be enough for training a machine learning

the empirical equation in [14]. As a result, we can exploit themodel to identify liquid crystallinity of five-ring bent-coremachine learning model using the rich data of glass transition

molecules [7], one would not expect that the same amount ofand melting temperature in PolyInfo using iqspr, and then, we

data would be enough to build a model for predicting therank the potential of each candidate by screening through thethermal conductivity of any amorphous homopolymers. In

newly generated molecules using the van Krevelen grouporder to build more general data-driven models for material

contribution method [15], a empirical model that is generaldesign, we aim at constructing a bridge that bring us from theenough for our purpose here. We impose uncertainty

existing small data approaches to a big data approach by better

quantification for the molecules generated from iqspr as well

exploitation of the existing machine learning techniques. In

as the screening results in order to have a more intuitive

ranking at the end.

the polymer experts that we have consulted. A key problem

3Results

with the current approach is the lack of consideration forsynthesis ability, which is particular important when we are in

In the iqspr package [13], we set the target region to be

the process of creating more data. To tackle this issue, we plan

320C-800C for glass transition temperature and 430C-800C

to implement a machine learning classifier to filter out

for melting temperature. Default values are used for the

candidates without liquid crystallinity, which is an important

regression model along with a train data set that includes allproperty for the ease of synthesis in practice. We are preparing

the homopolymers that have the glass transition temperature

to experimentally test the proposed candidates in the nearvalues recorded. For the prior model used for generating

future. The new results will be used for verification and“homopolymer-like” molecules, we used all of the 14,424

feedback into our database to help us improving our prediction

homopolymers available in the PolyInfo database for training.

models. 

Figure 1 shows four snapshots of the generated molecules.

REFERENCES

After that, all generated molecules are screened through the[1]

M.S.Bartlett, G.Littlewort, C.Lainscsek, I.Fasel, J.Movellan, “Machine learning

van Krevelen group contribution method to pick out

methods for fully automatic recognition of facial expressions and facial actions”,

candidates with a high probability of achieving high thermal

Proc. IEEE Int’l Conf. Systems, Man and Cybernetics, 1(2004), 592-597.

conductivity, as described in Section 2.

Target[2]

I.Goodfellow, J.Pouget-Abadie, M.Mirza, B.Xu, D.Warde-Farley, S.Ozair,
A.Courville, Y.Bengio, “Generative adversarial nets”, In NIPS (2014),

Target

2672-2680.

[3]

Z.Solan, D.Horn, E.Ruppin, S.Edelman,“Unsupervised learning of natural languages”, Proc. of The National Academyof Sciences of the USA, 102(2005), 33:11629-11634.

[4]

TargetP.M.Nadkarni, L.Ohno-Machado, W.W.Chapman,“Natural language processing: an introduction”, Journal of the American

TargetMedical Informatics Association, 18(2011), 5:544-551.

[5]

D.Silver et. al, “Mastering the game of Go with deep neural networks and tree
search”, Nature, 529(2016), 484-489.

[6]

D.Enke, S.Thawornwong, “The use of data mining and neural networks forforecasting stock market returns”, Expert Systems with Applications, 29(2005),
4:927-940.

[7]

J.Antanasijević,

D.Antanasijević, V.Pocajt, N.Trišovića, K.Fodor-Csorbac, “A

Figure  Evolution of homopolymer design of high glass

QSPR study on the liquid crystallinity of five-ring bent-core molecules usingtransition and melting temperature using iqspr. Orange dots

decision trees, MARS and artificial neural networks”, RSC Adv., 6(2016),

denote the training data in PolyInfo, and blue circles denote
generated homopolymers with radius proportional to the18452-18464.

[8]

prediction uncertainty.

PolyInfo, http://polymer.nims.go.jp/index_en.html, accessed on August 16,
2017.

[9]



4Conclusion

D.R.Anderson, “Thermal Conductivity of Polymers”, Chem.
Rev., 66(1966), 6:677-690.

In many scientific applications, we may have alreadyaccumulated a relatively large data set, but not yet large
enough for direct machine learning application due to thecomplexity of the problem of interest. We demonstrated aroadmap leading us from the traditional small data approach to
the new generation big data approach. We focused on the[10] C.L.Choy, “Thermal conductivity of polymers”, Polymer, 18(1977),
10:984-1004.

[11] E.Algaer, “Thermal Conductivity of Polymer Materials - Reverse
Nonequilibrium Molecular Dynamics Simulations”, Ph.D. Thesis, Technische
Universität, Darmstadt (2010).

[12] G.H.Kim, D.Lee,

A.Shanker, L.Shao, M.S.Kwon, D.Gidley, J.Kim,

optimal use of resource during the process, that is to increase

K.P.Pipe, “High thermal conductivity in amorphous polymer blends by

the probability of new material discovery by integrating

engineered interchain interactions”, Nature Materials, 14(2015), 295–300.

machineuncertainty[13] H.Ikebata, K.Hongo, T.Isomura, R.Maezono, R.Yoshida, “Bayesian molecularquantification techniques. We observed that the resulting

learning,

physical

model

and

design with a chemical language model”, Journal of computer-aided molecularmolecules from our iqspr simulation exhibit similar patterns,

design, 31(2017), 4:379-391.

demonstrating the capability of a machine learning toautomatically cluster chemical structures relevant to a target
material property. However, the resulting candidates after
screening the group contribution model were not intuitive to[14] J.Bicerano, “Prediction of Polymer Properties”, Chap.14, CRC Press (2002),
503-512.

[15] D.W.van Krevelen, “Properties of Polymers”, 3rd Ed., Elsevier Science,
Amsterdam (1990).

ҰൠԽઢ‫ܗ‬ճ‫ؼ‬ͷϩόετԽ͓ΑͼεύʔεԽ૯߹‫ڀݚ‬େֶӃେֶɹ઒ౡɹ޹ߦ
౷‫਺ܭ‬ཧ‫ॴڀݚ‬ɹ౻ᖒɹ༸ಙ
1. ͸͡Ίʹɹ KL μΠόʔδΣϯεʹ‫ͮ͘ج‬ճ‫ؼ‬͸ɼ࣍ͷΑ͏ʹఆٛͰ͖ΔɼDKL (g(y|x), f (y|x; θ); g(x)) =DKL (g(y|x), f (y|x; θ)g(x)dx. ͨͩ͠ɼg(y|x) ͓Αͼ g(x) ͸ɼσʔλΛੜ੒͢Δ෼෍Ͱɼf (y|x; θ) ͸ύϥϝτϦοΫϞσϧͰ͋Δɽg(y|x) ͓Αͼ g(x) ΛɼͦΕͧΕ‫ ਺ؔ౓ີݧܦ‬ḡ(y|x) ͱ ḡ(x) Ͱஔ͖‫͑׵‬Δͱɼ
࠷໬ਪఆʹ‫ͮ͘ج‬ճ‫ʹؼ‬Ұக͢Δɽ·ͨɼద੾ͳύϥϝτϦοΫϞσϧΛબ୒͢Δ͜ͱͰɼઢ‫ܗ‬ɼϩδεςΟοΫ͓ΑͼϙΞιϯճ‫͍ͨͬͱؼ‬ओཁͳճ‫ؼ‬Ϟσϧͷଟ͘ΛҰൠԽઢ‫ܗ‬ճ‫ؼ‬ͷ࿮૊ΈͰଊ͑Δ͜ͱ͕Ͱ͖Δ (Nelder and Wedderburn 1972)ɽ͔͠͠ɼKL μΠόʔδΣϯε͸ɼ֎Ε஋ʹऑ͍ɽͦ͜Ͱɼզʑ͸ɼϩόετͳμΠόʔδΣϯεͱͯ͠஌ΒΕ͍ͯΔɼΨϯϚɾμΠόʔδΣϯε (Fujisawa and Eguchi 2008)ʹ‫ͮ͘ج‬ճ‫ؼ‬Λߟ͑Δɽ·ͨɼ͜Εʹεύʔεਖ਼ଇԽΛ૊Έ߹ΘͤΔ͜ͱͰɼϩόετ͔ͭεύʔεͳҰൠԽઢ‫ܗ‬ճ‫ؼ‬Λୡ੒͢Δɽ!

2. ΨϯϚɾμΠόʔδΣϯεʹ‫ͮ͘ج‬ϩόετ͔ͭεύʔεͳճ‫ؼ‬ɹΨϯϚɾμΠόʔδΣϯεʹ‫ج‬ͮ͘ճ‫ؼ‬͸ɼ࣍ͷΑ͏ʹఆٛͰ͖Δɼ1
arg min dγ (g(y|x), f (y|x; θ); g(x)) = − Eg(x,y) [f (y|x; θ)γ /γ
θ"#f (y|x; θ)1+γ dy

γ
$ 1+γ

].

͜Εʹɼεύʔεਖ਼ଇԽΛՃ͑ͨ΋ͷ͸ҎԼͰද͞ΕΔɽ

1
arg min dγ (g(y|x), f (y|x; θ); g(x)) = − Eg(x,y) [f (y|x; θ)γ /γ
θ"#f (y|x; θ)

1+γ

dy

γ
$ 1+γ

] + λP (θ).

(1)͜Εͷ‫ݧܦ‬ਪఆΛߟ͑Δɽ

arg min − log
θ

%

n

1&
f (yi |xi ; θ)γ
n i=1

' "#

f (y|xi ; θ)

1+γ

dy

γ (
$ 1+γ

+ λP (θ).

(2)

P (θ) ͱͯ͠͸ɼL1 ਖ਼ଇԽ (Tibshirani 1996)ɼelastic net (Zou and Hastie 2005)ɼGroup lasso (Yuan and
Lin 2006) ͱ͍༷ͬͨʑͳਖ਼ଇԽΛߟ͑Δ͜ͱ͕ՄೳͰ͋ΔɽҎԼʹɼ۩ମతʹɼઢ‫ܗ‬ճ‫ؼ‬ɾϩδεςΟο
Ϋճ‫ؼ‬ɾϙΞιϯճ‫ͦؼ‬ΕͧΕʹ͍ͭͯͷৄࡉΛड़΂Δɽ
2.1. ઢ‫ܗ‬ճ‫ؼ‬ɹ (2) ͷ࠷దԽ͸ɼ௨ৗͷํ๏Ͱ͸ɼਪఆΞϧΰϦζϜͷಋग़ࣗମ͕ࠔ೉ͱͳΔɽͦ͜
ͰɼMM ΞϧΰϦζϜ (Hunter and Lange 2004) ͷߟ͑ํΛ༻͍ɼҎԼͰද͞ΕΔ࠷దԽ͕༰қͳิॿؔ਺͕ಘΒΕͨɽ
hM M (θ|θ(m) ) =(m)n
1
1 & (m) (yi − β0 − xTi β)2log σ 2 +α
+ λ||β||1 ,2(1 + γ)2 i=1 i
σ2

(m) γ

)
i ;θ
= !nf (yfi |x(m) )γ Ͱ͋Δɽ͜ΕʹΑΓɼεύʔεਖ਼ଇԽΛՃ͑ͨ৔߹Ͱ΋ɼ໨తؔ਺஋Λ୯ௐ
l=1 (yl |xl ;θʹ‫ݮ‬গͤ͞ΔਪఆΞϧΰϦζϜ ͷಋग़Λߦͬͨɽ਺஋࣮‫͓ݧ‬Αͼ࣮σʔλʢϚΠΫϩΞϨΠσʔλʣͷղੳʹ͓͍ͯ΋ɼൺֱख๏ (Alfons et al. 2013 and Khan et al. 2007) ʹ͘Β΂ͯඇৗʹେ͖ͳվળΛ‫ͨͤݟ‬(Kawashima and Fujisawa 2017).

ͨͩ͠ɼαi2.2. ϩδεςΟοΫճ‫ؼ‬ɹ Kanamori and Fujisawa (2015) Ͱ͸ɼճ‫ؼ‬໰୊ʹ͓͍ͯɼ֎Ε஋ͷׂ߹͕આ໌ม਺ x ʹґଘ͢Δ৔߹͸ɼҐஔɾई౓෼෍ʹ‫ݶ‬ఆ ͨ͠ͱ͖ͷΈɼϩόετੑΛಘΔ͜ͱ͕ՄೳͰ͋ͬ
ͨɼ͔͠͠ɼ(2) ʹ‫ͮ͘ج‬ճ‫Ͱؼ‬͸ɼͦͷΑ͏ͳԾఆ͸ඞཁͳ͘ɼϩδεςΟοΫճ‫ؼ‬ͷ৔߹Ͱ΋ɼϩόετੑΛಘΔ͜ͱ͕ՄೳͰ͋Δɽ·ͨɼਪఆΞϧΰϦζϜͷಋग़͸ɼઢ‫ܗ‬ճ‫ؼ‬ͷ৔߹ͱಉ͡Α͏ʹɼMM ΞϧΰϦζϜͷҰྫͰ͋ΔɼBohning and Lindsay (1988) ͷํ๏Λ༻͍ͯɼҎԼͰද͞ΕΔิॿؔ਺͕ಘΒΕͨɽhM M (θ|θ(m) ) =
"

#2
n "
γLth !
1 (m)(m)
β0 + xTi β (m) +τ̃i − β0 − xTi β + λ||β||1 ,2n i=1
Lth
γ
# 1+γ
"

#

$
%
(1+γ)2 (1+3γ)2y−͓Αͼ Lth >max 4+8γ , 4+8γ −γ
i
T
(m)
exp{
exp{(1+γ)x̃i θ}+1}+1Ͱ͋Δɽ͜ΕʹΑΓɼϩδεςΟοΫճ‫ؼ‬ͷ৔߹Ͱ΋ɼઢ‫ܗ‬ճ‫ͱؼ‬ಉ͡Α͏ʹ ໨తؔ਺஋Λ୯ௐʹ‫ݮ‬গͤ͞Δਪ
ఆΞϧΰϦζϜ ͷಋग़Λߦͬͨɽ·ͨɼΞϧΰϦζϜͷऩଋੑʹؔͯ͠΋ɼMiral (2013) ͷ݁ՌΛ༻͍Δ͜ͱͰɼେҬతऩଋੑ ͕ಘΒΕͨɽ(m)ͨͩ͠ɼτ̃i=(m)
exp{yi (1+γ)x̃T
}
i θ
(m)
(1+γ)x̃Ti θ(m)exp{(1+γ)x̃T}
i θ

2.3. ϙΞιϯճ‫ؼ‬ɹϙΞιϯճ‫ؼ‬ͷ৔߹Ͱ΋ɼKanamori and Fujisawa (2015) ͷԾఆ͸ඞཁͳ͘ɼ֎
Ε஋ͷׂ߹͕આ໌ม਺ʹґଘ͢Δ৔߹Ͱ΋ɼϩόετੑΛಘΔ͜ͱ͕Մೳ Ͱ͋Δɽ͔͠͠ͳ͕ΒɼϙΞι&'( γϯճ‫ؼ‬ͷ৔߹͸ɼ f (y|xi ; θ)1+γ dy 1+γ ͷ߲Ͱɼ௒‫ز‬Կ‫਺ڃ‬ͷ‫͕ࢉܭ‬ඞཁͱͳΓɼ(2) ʹ‫ݶͮ͘ج‬Γɼ௨ৗͷํ๏Ͱ͸ޮ཰తͳਪఆΞϧΰϦζϜͷಋग़͕ࠔ೉ ͱͳΔɽͦ͜Ͱɼ‫ݧܦ‬ਪఆΛ‫ܦ‬༝͢Δ͜ͱͳ͘ɼ௚
઀ɼ(1) Λ࠷খԽ͢Δɼ֬཰త࠷దԽͷ࿮૊ΈΛ༻͍ͯɼ͜ͷ໰୊Λࠀ෰ ͨ͠ɽಛʹɼ֬཰త࠷దԽͷख๏ͷதͰ΋ɼඇತ͔ͭ‫׈‬Β͔Ͱͳ͍໨తؔ਺Λର৅ͱͨ͠ɼRandomized Stochastic Projected Gradient(Ghadimi et al. 2016) Λ༻͍Δ͜ͱͰɼେҬతऩଋੑΛ࣋ͭɼΦϯϥΠϯਪఆΞϧΰϦζϜ Λಋग़ͨ͠ɽ
*
)
+
mt
*
1 !
f (yt,i |xt,i ; θ)γ
1
*
(t+1)
θ
= arg min −
∇θ &'
,
θ
+ λP (θ) +∥θ − θ(t) ∥22 ,
*
γ
( 1+γ
*
mt
2η
1+γ
t
θ
f (y|x ; θ)
dy
i=1

t,i

θ=θ (t)ͨͩ͠ɼmt ͸֤൓෮ t ʹ͓͚ΔɼϛχόονͷαΠζΛද͢ɽ·ͨɼϙΞιϯճ‫ݶʹؼ‬Βͣɼ্Ͱड़΂ͨճ‫ʹؼ‬΋ΦϯϥΠϯਪఆΞϧΰϦζϜΛద༻͢Δ͜ͱ͸ՄೳͰ͋Δɽ͜ΕʹΑΓɼϩόετ͔ͭεύʔεͳҰൠԽઢ‫ܗ‬ճ‫ؼ‬Λୡ੒͢Δɽ

ଟॏ୅ೖ๏Λ༻͍ͨόΠΞεิਖ਼ਪఆྔʹؔ͢Δߟ࡯૯߹‫ڀݚ‬େֶӃେֶɹ෋ాɹ༟ষ
౷‫਺ܭ‬ཧ‫ॴڀݚ‬ɹ౻ᖒɹ༸ಙɹɹ౷‫਺ܭ‬ཧ‫ॴڀݚ‬ɹҳ‫ݟ‬ɹণ೭ܽଌ஋Λ‫ؚ‬Ήσʔλʹର͢Δղੳ๏ͷ 1 ͭͱͯ͠ Rubin (1987) ͸ ଟॏ୅ೖ๏ʢMultiple Imputationɿ

MIʣΛఏএͨ͠ɽMI Ͱ͸ɼܽଌ஋Λ‫ؚ‬Μͩσʔλʹରͯ͠ɼ୅ೖ๏ΛಠཱʹԿ౓΋ߦ͍ɼͦͷ౓ʹਪఆྔΛ‫͠ࢉܭ‬ɼͦΕΒΛ౷߹ͯ͠࠷ऴతͳਪఆྔʢ౷߹ਪఆྔʣΛಘΔɽMI ʹ͸ɼ‫؍‬ଌ͞ΕͨσʔλΛ͢΂ͯར༻ͨ͠ਪఆ͕ՄೳͰ͋Δ͜ͱɼิ‫׬‬Λߦͬͨσʔλʹର͢Δղੳ͸௨ৗͷσʔλʹର͢Δղੳख๏Λར༻ՄೳͰ͋Δ͜ͱɼ‫܁‬Γฦ͠ิ‫׬‬Λߦ͏͜ͱͰਪఆ஋ͷ෼ࢄ΋ධՁՄೳͰ͋Δ͜ͱɼͱ͍͏ར఺͕͋Δ൓໘ɼద੾ʹଟॏ୅ೖ๏ΛߦΘͳ͍ͱ౷߹ਪఆྔ͕ଥ౰ͳ΋ͷͰͳ͘ͳΔ͜ͱ͕஌ΒΕ͍ͯΔɽ
‫ݹ‬యతʹ͸ଟॏ୅ೖ๏͕ద੾ʢproperʣͳ৔߹ʹ͸ɼ౷߹ਪఆྔ͸ҰகੑΛ࣋ͭ͜ͱ͕ࣔ͞Ε͍ͯΔ(Rubin (1987)). ద੾ͳϞσϧͷԼͰɼϕΠζཧ࿦ʹ‫ޙࣄͮ͘ج‬෼෍Λ༻͍ͯ୅ೖ஋Λੜ੒ͨ͠৔߹ʹ͸ଟॏ୅ೖ๏͕ద੾Ͱ͋Δ͜ͱ͕஌ΒΕ͍ͯΔɽ͔͠͠ͳ͕ΒɼҰൠͷଟมྔ෼෍Ͱ͸ਖ਼֬ͳࣄ‫ޙ‬෼෍Λੜ੒
͢Δ͜ͱ͸೉͘͠ɼ·ͨద੾Ͱ͋Δ͔Λ֬ೝ͢Δ͜ͱ΋ࠔ೉Ͱ͋Δ (van Buuren (2012)). ·ͨɼଟॏ୅ೖ
๏͕ద੾Ͱͳͯ͘΋୅ೖϞσϧ͕ਖ਼͘͠ಉఆ͞Ε͍ͯΕ͹౷߹ਪఆྔ͸ҰகੑΛ࣋ͭ͜ͱ͕஌ΒΕ͍ͯΔ(Little (1992)) ͕ɼಉఆ͕‫͍ͨͯͬޡ‬৔߹ʹ͸౷߹ਪఆྔ͕ਅͷ஋ʹ֬཰ऩଋ͢Δ͜ͱ͸อূ͞ΕͣɼόΠΞε͕ൃੜ͢Δ͜ͱ͕͋Δ (Clayton et al. (1998), Robins and Wang (2000)ʣɽ
ຊൃදͰ͸ɼճ‫ؼ‬Ϟσϧͷਪఆʹ͓͍ͯɼઆ໌ม਺͕ܽଌ͍ͯ͠Δঢ়‫گ‬ԼͰɼଟॏ୅ೖ๏Λར༻ͨ࣌͠ʹܽଌ஋Λิ‫͢׬‬ΔϞσϧʹ‫ޡ‬Γ͕͋ͬͨͱͯ͠΋ɼิ‫׬‬Ϟσϧͷ‫ޡ‬ΓʹΑΔόΠΞεΛิਖ਼ͯ͠Ұகੑͷ͋
ΔਪఆྔΛಋग़͢Δํ๏ʢTomita et al. (2017)ʣʹ͍ͭͯड़΂Δɽ۩ମతʹॻ͖Լ͢ͱҎԼͷ௨ΓͰ͋Δɽ

Y ΛԠ౴ม਺ͱ͠ɼX, Z Λઆ໌ม਺ͱ͢Δɽ͜͜ͰɼX ͸ܽଌ͕ൃੜ͠͏Δม਺ͱ͠ɼZ ʹ͸ܽଌ͕ൃੜ͠ͳ͍΋ͷͱͨ͠ɽR Λ‫؍‬ଌࢦඪͱ͠ɼR = 1 ͷͱ͖ X ͸‫؍‬ଌ͞ΕɼR = 0 ͷͱ͖ X ͸‫؍‬ଌ͞Εͳ͍
ͱ͢Δɽਪఆྔ θ̂ Λɼ⎡θ̂ = arg max ⎣θ

#i:Ri =1⎤m# 1 #log f (Yi |Xi , Zi ; θ) +w(Xij , Yi , Zi ) log f (Yi |Xij , Zi ; θ)⎦m j=1
i:Ri =0(1)

ͱఆٛ͢Δɽ͜͜Ͱɼw(x, y, z) ͸ॏΈ෇͚ؔ਺w(x, y, z) =g(x|y, z)h(x|y, z)Ͱ͋Δɽͨͩ͠ɼg(x|y, z) ͸ Y = y, Z = z ͷ΋ͱͰͷ X ͷਅͷ৚݅෇͖෼෍Λ͋ΒΘ͢ɽҰఆͷ৚݅ͷ΋ͱͰɼm, n −→ ∞ ͱ͢Δͱɼਪఆྔ θ̂ ͸฼਺ͱͷҰகੑΛ΋ͭɽw ͷ‫͍༻Ͱࢉܭ‬Δ X ͷਅͷ৚݅෇͖෼෍ g(x|y, z) ͸Ұൠʹະ஌Ͱ͋ΔͨΊɼ࣮ࡍͷղੳͰ͸৚݅෇͖ີ

౓ਪఆʹΑͬͯਪఆͨ͠෼෍ ĝ(x|y, z) ʹஔ͖‫͑׵‬Δɽ৚݅෇͖ີ౓ਪఆ๏ͱͯ͠ɼΧʔωϧີ౓ਪఆʹ‫ج‬ͮ͘ख๏Ͱ͋Δ࠷খೋ৐৚݅෇͖ີ౓ਪఆ๏ʢLeast Square Conditional Density EstimationɼLSCDEʣ

(Sugiyama et al. (2010)) Λར༻͢Δɽ
ղੳखॱΛҎԼʹࣔ͢ɽ

Step 1.
1ܽଌͨ͠ X ʹର͠ɼԿΒ͔ͷิ‫׬‬Ϟσϧ h(x|y, z) ΛԾఆ͠ɼิ‫׬‬஋Λಠཱʹ m ճൃੜͤ͞Δɽ

Step 2.
LSCDE ʹΑͬͯ ĝ(x|y, z) Λਪఆ͠ɼॏΈؔ਺w(x, y, z) =ĝ(x|y, z)h(x|y, z)jΛิ‫ͨ͠׬‬ඪຊ (Xi , Yi , Zi ) ͢΂ͯʹ͍ͭͯ‫͢ࢉܭ‬Δɽ

Step 3.
ࣜ (1) ʹ‫͍ͯͮج‬ਪఆྔ θ̂ Λ‫ٻ‬ΊΔɽ͜ͷख๏ΛόΠΞεิਖ਼ଟॏ୅ೖ๏ʢBias corrected multiple imputationʣͱশ͢Δɽ
ͳ͓ɼόΠΞεิਖ਼ଟॏ୅ೖ๏ʹ͍ͭͯ਺஋࣮‫ݧ‬΍࣮σʔλղੳʹΑͬͯධՁΛߦ͍ͬͯΔ͕ɼຊใࠂॻ
Ͱ͸লུ͢Δɽࠓ‫ޙ‬ͷ՝୊ͱͯ͠͸ɼΑΓෳࡶͳҰൠͷܽଌ໰୊ʹରͯ͠ఏҊख๏ͷ֦ுΛߟ͑Δ͜ͱɼ৴པ۠ؒͳͲΛ
ߏங͢Δ্Ͱඞཁͳਪఆྔͷ෼ࢄΛ༩͑Δ͜ͱ͕ߟ͑ΒΕΔɽँࣙຊ‫ڀݚ‬ͷҰ෦͸ࠃཱ‫ڀݚ‬։ൃ๏ਓ೔ຊҩྍ‫ڀݚ‬։ൃ‫( ߏػ‬AMED) ͷྟচ‫ڀݚ‬ɾ࣏‫ݧ‬ਪਐ‫ۀࣄڀݚ‬(171k0201061h0002) ͷࢧԉʹΑͬͯߦΘΕͨɽࢀߟจ‫ݙ‬
van Buuren, S. (2012) Flexible Imputation of Missing Data (Chapman & Hall/CRC Interdisciplinary
Statistics): Chapman and Hall/CRC.
Clayton, D., Spiegelhalter, D., Dunn, G., and Pickles, A. (1998) “Analysis of longitudinal binary
data from multiphase sampling,” Journal of the Royal Statistical Society: Series B (Statistical
Methodology), Vol. 60, No. 1, pp. 71–87.
Little, R. J. A. (1992) “Regression With Missing X’s: A Review,” Journal of the American Statistical
Association, Vol. 87, No. 420, pp. 1227-1237.
Robins, J. M. and Wang, N. (2000) “Inference for Imputation Estimators,” Biometrika, Vol. 87, No.
1, pp. 113–124.
Rubin, D. B. (1987) Multiple Imputation for Nonresponse in Surveys: Wiley, pp.258.
Sugiyama, M., Takeuchi, I., Suzuki, T., Kanamori, T., Hachiya, H., and Okanohara, D. (2010) “Leastsquares conditional density estimation,” IEICE Transactions on Information and Systems, Vol. 93,
No. 3, pp. 583–594.
Tomita, H., Fujisawa, H., and Masayuki, H. (2017) “A bias-adjusted estimator in multiple imputation
for missing data,” in submission.

2࠷దͳ൒‫ֶ͖෇ࢣڭ‬श઒‫ాت‬խଇʢ‫۝‬भେֶʣɾ౻ᖒ༸ಙʢ౷‫਺ܭ‬ཧ‫ॴڀݚ‬ʣ

1

͸͡Ίʹ

൒‫ֶ͖෇ࢣڭ‬शʹ͓͍ͯɼ‫͠ͳࢣڭ‬σʔλͷ༗ޮར༻͕‫͔ͭز‬ఏҊ͞Ε͍ͯΔɽKawakita (unpublished) ͸ɼີ౓ൺΛ্ख͘ߏ੒͢Δ͜ͱͰɼ‫ֶ͖෇ࢣڭ‬शΛվྑ͢Δख๏ΛఏҊͨ͠ɽͦΕΛDRESS (Density-Ratio Estimation based on Semi Supervised learning) ͱ໊෇͚ͨɽKawakita and
Takeuchi (2014) ͸ɼີ౓ൺΛΑΓ্ख͘ਪఆ͢Δख๏ΛఏҊͨ͠ɽ͜ΕΒͷྑ͞͸ɼฏ‫ۉ‬ೋ৐‫ࠩޡ‬ʹ‫͍ͨͮج‬ϐλΰϦΞϯؔ܎ʹΑͬͯɼઆ໌͢Δ͜ͱ͕Ͱ͖ΔɽKawakita and Kanamori (2013) ͸ີ౓ൺΛϊϯύϥϝτϦοΫʹਪఆ͢Δ͜ͱΛఏҊͨ͠ɽຊ‫Ͱڀݚ‬͸ɼ࠷దੑͷ‫͔఺؍‬Βɼ‫͠ͳࢣڭ‬σʔλͷ༗ޮར༻Λߟ͑ΔɽಘΒΕͨ݁Ռ͔Βࣗવʹɼ‫͠ͳࢣڭ‬σʔλͷ৽͍͠༗ޮར༻๏͕‫͔ͭݟ‬Δɽ

2൒‫ֶ͖෇ࢣڭ‬श‫͖෇ࢣڭ‬σʔλ͸ҎԼͰ͋Δɿ
!

x1
y1

"

,...,

!

x1
y1

"

∼ gx,y (x, y).

‫͠ͳࢣڭ‬σʔλ͸ҎԼͰ͋Δɿ

xn+1 , . . . , xN ∼ gx (x).

൒‫͖෇ࢣڭ‬σʔλ͸্‫ه‬Λ߹Θͤͨ΋ͷͰ͋Δɽ
‫ֶ͖෇ࢣڭ‬शͷྫ͸ҎԼͰ͋Δɿ#%n$β̂0 = arg solve β :uβ (xi , yi ; β) = 0 .
i=1͜͜Ͱ uβ ͷ୅දྫ͸ઢ‫ܗ‬ճ‫ؼ‬Λҙਤͨ͠ uβ (x, y; β) = y − β T x Ͱ͋Δɽ൒‫ֶ͖෇ࢣڭ‬श͸൒‫෇ࢣڭ‬͖σʔλΛར༻ֶͨ͠शͰ͋Δɽຊ‫Ͱڀݚ‬͸ɼ‫͠ͳࢣڭ‬σʔλΛ্ख͘ར༻ͯ͠ɼਪఆྔ β̂0 Λվྑ͢Δख๏ΛఏҊ͢Δɽಛʹ࠷దੑͷ‫͔఺؍‬Βߟ࡯Λߦ͏ɽ

3൒‫ֶ͖෇ࢣڭ‬शͷఏҊྫ
Kawakita (unpublished) ͸ີ౓ൺΛར༻ͯ࣍͠ͷ DRESS ΛఏҊͨ͠ɽ#%n$gx (xi ; η̂ ′ )β̂K = arg solve β :uβ (xi , yi ; β) = 0 ,βg (x ; η̂)
i=1 x i
#
%
#
%
n
N
$
$
′
′
′
η̂ = arg solve η :sη (xi ; η) = 0 ,η̂ = arg solve η :sη (xi ; η ) = 0 .
i=1

i=n+1

͜͜Ͱɼgx (x; η) ͸ gx (x) Λਪ͠ྔΔͨΊͷύϥϝτϦοΫϞσϧͰ͋Γɼsη (x; η) = (∂/∂η) log gx (x; η)Ͱ͋ΔɽԾʹਅͷ෼෍ gx (x) ͕෼͔͍ͬͯͨͱ͠Α͏ɽີ౓ൺͷ෼ࢠ͕ gx (xi ; η̂ ′ ) Ͱ͸ͳ͘ gx (x) Ͱ͋ͬͨͱ͠Α͏ɽ༗‫ݶ‬ඪຊ͔ΒಘΒΕΔີ౓ਪఆ gx (x; η̂) ͸ɼgx (x) ͱͣΕ͍ͯΔɽͦͷͣΕΛີ౓ൺͰิਖ਼͢Δ͜ͱͰશମͷਪఆͷͣΕΛิਖ਼͠Α͏ͱ͍͏ߟ͑Ͱ͋Δɽ͜ͷߟ͑͸‫཰֬ٯ‬ॏΈ๏ʹ௨͡Δߟ͑Ͱ͋Δɽ࣮ࡍʹɼద౰ͳ৚݅ͷԼͰ͸ɼ઴ۙ෼ࢄͷߴ࣍ͷ෦෼Ͱɼβ̂K ͸ β̂0 Λվྑͯ͠
͍Δ͜ͱ͕ɼূ໌Ͱ͖Δɽ͜ͷ֦ுʹؔͯ͠͸ɼKawakita and Takeuchi (2014) ΍ Kawakita and Kanamori (2013) ͰఏҊ͞Ε͍ͯΔ͕ɼলུ͢Δɽ

1

4

࠷దͳ൒‫ֶ͖෇ࢣڭ‬श൒‫͖෇ࢣڭ‬σʔλ͔Β࡞ΒΕΔਪఆྔ β̂ ͸ʮ઴ۙઢ‫ܗ‬ల։ʯΛ΋ͪʮਖ਼ଇʯͰ͋Δͱ͢Δɽ઴ۙઢ‫ܗ‬ల։Λ΋ͭͱ͸ɼ͕࣍੒Γཱͭ͜ͱͰ͋Δɿ

√

n
N
!
1 !
1
n(β̂ − β0 ) = √φ1 (xi , yi ) + √φ2 (xi ) + op (1).
n i=1N − n i=n+1"#
ͨͩ͠ β0 = arg solve β : Egx,y [uβ (x, y; β)] = 0 ͱ͢Δɽ·ͨɼ֩ؔ਺ φ1 (x, y) ͱ φ2 (x) ͸ɼద౰ͳੑ࣭Λຬͨ͢ͱ͢Δɽਖ਼ଇͰ͋Δͱ͸ɼηϛύϥϝτϦοΫཧ࿦ͷҙຯͰਖ਼ଇͰ͋Δͱ͍͏ҙຯͰ
͋Δ (Tsiatis, 2006)ɽ௒༗ޮਪఆྔΛഉআ͢ΔͨΊͷੑ࣭Ͱ͋Δ͕ɼ͜͜Ͱ͸ৄࡉ͸লུ͢Δɽ͜
ͷΑ͏ͳԾఆͷԼͰ࣍ͷఆཧ͕੒Γཱͭɽͳ͓ɼূ໌ͳͲͷৄࡉ͸ɼKawakita and Fujisawa (2017)Λࢀর͞Ε͍ͨɽఆཧ 1β̂ ͸઴ۙઢ‫ܗ‬ల։Λ΋ͪਖ਼ଇͰ͋Δͱ͢Δɽͦͷͱ͖֩ؔ਺͸࣍ͷੑ࣭Λຬͨ͢ɿ1−1
φ1 (x, y) = Jββuβ (x, y; β0 ) − √ φ2 (x).
r

ͨͩ͠ɼr = (N − n)/nɼJββ = Egx,y [−(∂uβ /∂β)(x, y; β0 )] ͱ͢Δɽఆཧ 2β̂ ͸઴ۙઢ‫ܗ‬ల։Λ΋ͪਖ਼ଇͰ͋Δͱ͢Δɽβ̂ ͷ઴ۙ෼ࢄΛ࠷খʹ͢Δͷ͸࣍ͷͱ͖Ͱ͋Δɿ√r −1
φ2 (x) =J Eg [uβ (X, Y ; β0 )|X = x].
1 + r ββ y|x͜ͷఆཧ͔Βɼ࠷దͳਪఆྔΛ࡞ΔͨΊʹ͸ɼ৚݅෇ີ౓ؔ਺ g(y|x) ʹؔ͢Δ‫ظ‬଴஋ΛಘΔඞཁ
͕͋Δɽ͔͠͠ɼ͜Ε͸ɼ൒‫͖෇ࢣڭ‬σʔλ͔Β͸ਪఆ͠ʹ͍͘ɽͦͷͨΊɼ‫࣮ݱ‬తʹ͸ɼਪఆ͠΍͍͢ΫϥεͷதͰɼ্‫࠷ʹه‬΋͍ۙ֩ؔ਺ΛಘΔ͜ͱΛߟ͍͑ͨɽͳ͓ɼҎԼͷఆཧͰ͸ɼؔ਺
ψ(x) ͸ద౰ͳ‫ج‬ఈؔ਺ΛΠϝʔδ͍ͯ͠Δɽ
ʢͨͱ͑͹ B εϓϥΠϯ΍Ψ΢γΞϯΧʔωϧ͕୅දతͰ͋Δɽʣ
ఆཧ 3 φ2 (x) = Bψ(x) ͱ੍‫͞ݶ‬Ε͍ͯΔͱ͢Δɽ͜ͷΫϥεͷதͰ઴ۙ෼ࢄΛ࠷খʹ͢Δ β̂ Ͱ͸ҎԼ͕੒Γཱͭɿ
√
$
%
$
%−1
r −1
φ2 (x) =Jββ Egx,y uβ (x, y, ; β0 )ψ(x)T Egx ψ(x)ψ(x)T
ψ(x).
1+r

্‫ه‬ͷఆཧͷதʹ͋Δ φ2 (x) ͸൒‫͖෇ࢣڭ‬σʔλ͔ΒਪఆՄೳͰ͋ΔɽͳͥͳΒ͹ɼ‫ظ‬଴஋͸͢΂ͯपล෼෍΍ಉ࣌෼෍ͷԼͷͨΊɼ௨ৗͷ‫ݧܦ‬ਪఆ͕Մೳ͔ͩΒͰ͋ΔɽͦͷΑ͏ͳਪఆྔ β̂ Λ༩͑Δਪ
ఆํఔࣜ͸ɼࣗવʹಋग़Ͱ͖ΔɽͦͷҰ͕ͭ࣍ʹ‫͛ڍ‬Δ nDRESS II Ͱ͋Δɿ w(x; θ) = exp{θT ψ(x)},&'n!
β̂KF = arg solve β :w(xi ; θ̂)uβ (xi , yi ; β) = 0 ,
i=1

&

'
n
N
1!
1 !
θ̂ = arg solve θ :ψ(xi )w(xi ; θ) −ψ(xi ) = 0 .
n i=1
N i=1

ͳ͓ɼఆཧ 3 ͷੑ࣭Λ΋ͭਪఆྔ β̂ Λ༩͑Δਪఆํఔࣜ͸ɼͦͷଞʹ΋৭ʑͱ࡞Δ͜ͱ͕Ͱ͖ΔɽͦΕʹ͍ͭͯ͸ Kawakita and Fujisawa (2017) Λࢀর͞Ε͍ͨɽ

2ύϥϝʔλసҠֶशʹ͓͚Δ൚Խ‫ࠩޡ‬ͷධՁ‫۽‬୩࿱ ∗ ɼۚ৿‫ܟ‬จ †,∗ཧԽֶ‫৽ֵ ॴڀݚ‬஌ೳ౷߹‫ڀݚ‬ηϯλʔ ∗໊‫ݹ‬԰େֶ େֶӃ৘ใֶ‫ڀݚ‬Պ †ैདྷͷ‫ػ‬ցֶशͰ͸, σʔλ͸୯Ұͷ෼෍͔ΒಠཱಉҰʹൃੜ͢ΔͱԾఆ͞Ε͍ͯΔɽ͔͠͠, ͜ͷԾఆ͸࣮ࡍͷԠ༻Ͱ͸ඞͣ͠΋੒ཱ͠ͳ͍ɽͦͷͨΊ, ҟͳΔ෼෍͔Βൃੜͨ͠αϯϓϧΛѻ͏͜ͱ͕Ͱ͖Δํ๏Λൃలͤ͞Δ͜ͱ͕ॏཁͰ͋Δͱߟ͑ΒΕΔɽ͜ͷͱ͖ɼసҠֶश͸͜ΕΒͷঢ়‫ʹگ‬ରॲ͢ΔͨΊͷҰൠతͳํ๏Λఏ‫͢ڙ‬ΔɽసҠֶशͰ͸య‫ܕ‬తʹ, ໨తͷλεΫʹؔ࿈͢Δগ਺ͷαϯϓϧʹՃ͑ͯ, ଞͷυϝΠϯ͔Βൃੜͨ͠๛෋ͳαϯϓϧ͕ར༻Մ
ೳͰ͋Δ͜ͱ͕૝ఆ͞Ε͍ͯΔɽͦͷͱ͖, సҠֶशͷ໨త͸ɼଞͷυϝΠϯͷσʔλ͔Β༗༻ͳ஌ࣝΛநग़͠ɼͦΕΛ༻͍ͯ໨తͷλεΫʹର͢ΔΞϧΰϦζϜͷੑೳΛ޲্ͤ͞Δ͜ͱ
Ͱ͋ΔɽసҠ͞ΕΔ஌ࣝͷछྨʹԠͯ͡, సҠֶशͷ໰୊Λղܾ͢ΔͨΊͷΞϓϩʔν͸, ΠϯελϯεసҠ, ಛ௃సҠ, ύϥϝʔλసҠͳͲʹ෼ྨ͢Δ͜ͱ͕Ͱ͖Δɼຊ‫Ͱڀݚ‬͸, ͋ΔछͷύϥϝτϦοΫϞσϧ͕૝ఆ͞ΕɼసҠ͞Εͨ஌ࣝ͸ύϥϝʔλʹΤϯίʔυ͞ΕΔΑ͏ͳύϥϝʔλసҠΞϓϩʔνʹ͍ͭͯߟ࡯͢Δɽຊ‫ڀݚ‬ͷ໨త͸, ύϥϝʔλసҠΞϓϩʔνʹ‫ج‬ͮ͘ΞϧΰϦζϜʹରͯ͠ɼཧ࿦తղੳΛߦ͏͜ͱͰ͋ΔɽҎԼͰ͸ɼύϥϝʔλసҠΞϓϩʔνʹ͍ͭͯ໰୊ઃఆΛड़΂Δɽ͸͡Ίʹɼ‫͔ͭز‬ͷ‫ه‬๏Λ؆୯ʹಋೖ͢Δɽ·ͣɼX ͱ Y ͸ͦΕͧΕɼαϯϓϧۭؒͱϥϕϧۭؒͰ͋Δͱ͢Δɽϥϕϧ෇͖αϯϓϧͷۭؒ Z := X × Y ͱɼͦͷ্ͷ݁߹෼෍ P (x, y) ͷ૊ΈΛྖҬͱ‫Ϳݺ‬ɽͦͷͱ͖ɼαϯϓϧۭؒ X ͱ X ্ͷपล෼෍ P (x) ͷ૊ΛυϝΠϯͱ‫ͼݺ‬ɼ·ͨɼϥϕϧू߹ Y ͱ ৚݅෇͖෼෍ P (y|x) ͷ૊ΈΛλεΫͱ‫Ϳݺ‬ɽ͞Βʹɼ H = {h : X → Y} ΛԾઆۭؒͱ͠ɼℓ : Y × Y → R≥0 Λଛࣦؔ਺ͱ͢Δɽͦͷͱ͖, ‫ظ‬଴ϦεΫͱ‫ݧܦ‬ϦεΫΛ! n (h) := 1 "n ℓ(yj , h(xj )) ʹΑͬͯఆΊΔɽసҠֶशͷઃ
R(h) := E(x,y)∼P [ℓ(y, h(x))] ͱ R
j=1nఆʹ͓͍ͯɼ໨ඪυϝΠϯͱ‫ݺ‬͹ΕΔɼ‫ڵ‬ຯͷ͋ΔυϝΠϯ͔Βੜ੒͞ΕΔαϯϓϧʹՃ͑ɼ‫ݩ‬υϝΠϯͱ‫ݺ‬͹ΕΔɼผͷυϝΠϯ͔Βੜ੒͞ΕΔαϯϓϧ͕ར༻ՄೳͰ͋ΔͱԾఆ͞ΕΔɽ
ຊ‫Ͱڀݚ‬͸ɼ໨ඪυϝΠϯͱ‫ݩ‬υϝΠϯΛ۠ผ͢ΔͨΊʹɼPT ΍ RS ͷΑ͏ʹɼT ΋͘͠͸S Ͱද͞ΕΔఴࣈΛ෇͢͜ͱͱ͢Δɽ
ҎԼɼ໨ඪυϝΠϯͰ͸ YT ⊂ R ͱԾఆ͠, ໨ඪυϝΠϯͷύϥϝτϦοΫͳಛ௃ࣸ૾ ψθ :
XT → Rm Λ༻͍ͯɼԾઆ hT ,θ,w : XT → YT ͕࣍ͷΑ͏ʹද͞ΕΔͱԾఆ͢Δ:

hT ,θ,w (x) := ⟨w, ψθ (x)⟩.
1(1)

͜͜Ͱɼύϥϝʔλ͸ θ ∈ Θ ͱ w ∈ WT ͱ͠ɼΘ ͸ϊϧϜ ∥ · ∥ ͕෇ਵͨ͠ϊϧϜۭؒͷ෦! T (hT ,θ,w ) Λ෼ू߹ɼWT ͸ Rm ͷ෦෼ू߹ͱ͢ΔɽҎԼͰ͸, ୯७ʹ RT (hT ,θ,w ) ͓Αͼ R! T (θ, w) ͷΑ͏ʹ‫ه‬ड़͢Δ. ‫ݩ‬υϝΠϯʹ͸, αϯϓϧ෼෍ PS,θ,w ΍ԾઆRT (θ, w) ͓Αͼ R
hS,θ,w ͳͲͷύϥϝτϦοΫϞσϧ͕ଘࡏ͢Δͱ͠ɼύϥϝʔλۭؒͷҰ෦ Θ ͸‫ݩ‬υϝΠϯͱ໨ඪυϝΠϯͱ‫ڞ‬༗͞Ε͍ͯΔͱԾఆ͢Δɽͦͷͱ͖, θS∗ ∈ Θ ͱ wS∗ ∈ WS ͸‫ྖݩ‬Ҭʹ͓͍ͯԿΒ͔ͷࢦඪʹؔͯ͠༗ޮͳύϥϝʔλͰ͋Δ͢Δɽ͔͠͠ͳ͕Β, ຊ‫Ͱڀݚ‬͸ θS∗ ∈ Θ ͱ
wS∗ ∈ WS ʹରͯ͠໌ࣔతͳԾఆ͸՝͞ͳ͍ɽ
࣍ʹɼຊ‫Ͱڀݚ‬ѻ͏ύϥϝʔλసҠΞϧΰϦζϜʹ͍ͭͯઆ໌͢Δɽ‫ݩ‬υϝΠϯͱ໨ඪυϝΠϯͰͦΕͧΕ N ‫ ͱݸ‬n ‫ݸ‬ͷαϯϓϧΛ࢖༻Ͱ͖Δͱ͢ΔɽύϥϝʔλసҠΞϧΰϦζϜ͸ɼ·ͣ N ‫ݸ‬ͷαϯϓϧΛ࢖༻ͯ͠ɼθS∗ ͷਪఆ஋ θ!N ∈ Θ Λग़ྗ͢Δɽ࣍ʹΞϧΰϦζϜ͸໨ඪυϝΠϯͷύϥϝʔλwT∗:= argmin RT (θS∗ , w)w∈WTʹର͠ɼn ‫ݸ‬ͷαϯϓϧΛ༻͍ͯਪఆ஋

!T ,n (θ!N , w) + ρr(w)! N,n := argmin Rww∈WTΛग़ྗ͢Δɽ͜͜Ͱ r(w) ͸ ∥ · ∥2 ʹؔͯ͠ 1-‫ڧ‬ತͳਖ਼ଇԽ߲ͱ͠ɼρ ͸ਖ਼ͷ࣮਺ͱ͢Δ. ‫ݩ‬υϝΠϯ͕ԿΒ͔ͷҙຯͰ໨ඪυϝΠϯʹؔ܎͍ͯ͠Δ৔߹ɼ‫ݩ‬υϝΠϯͰͷ༗ޮͳύϥϝʔλθS∗ ΋໨ඪλεΫʹͱͬͯ༗༻Ͱ͋Δͱ‫ظ‬଴͞ΕΔɽຊൃදͰ͸ RT (θS∗ , wT∗ ) Λ༧ଌੑೳͷ‫ج‬४஋ͱͯ͠࠾༻͢Δɽ͜ͷͱ͖͍͔ͭ͘ͷٕज़తԾఆͱͱ΋ʹɼຊ‫ʹͨ৽͍͓ͯʹڀݚ‬ಋೖ͞Εͨ‫҆ॴہ‬ఆੑͱసҠֶशՄೳੑͱ͍͏৚݅Λ༻͍Δ͜ͱͰɼҎԼͷֶशό΢ϯυΛಋग़͢Δ
͜ͱ͕Ͱ͖Δ (‫҆ॴہ‬ఆੑ͓ΑͼసҠֶशՄೳੑͷఆٛʹؔͯ͠͸࿦จ [1] ΛࢀরͤΑ)ɽ
ఆཧ 1 (ֶशό΢ϯυ). ύϥϝτϦοΫಛ௃ࣸ૾ ψθ ͕‫҆ॴہ‬ఆͰ͋Δͱ͢Δɽ·ͨɼ‫ݩ‬υϝΠϯͰֶश͞Εͨ θS∗ ∈ Θ ͷਪఆྔ θ!N ͸֬཰ 1 − δ̄ ͰύϥϝʔλసҠֶशՄೳੑΛຬͨ͢ͱ͢
Δɽਖ਼ଇԽύϥϝʔλ ρ Λద੾ʹઃఆ͢Δͱ͖, ҎԼͷෆ౳͕ࣜ֬཰ 1 − (δ + 2δ̄) Ͱ઴ۙతʹ੒Γཱͭ :$%%2/7 &"#∗∗−1/3 % !
∗%!
! N,n − RT (θS , wT ) ≤ c max n
RT θ N , w, %θ N − θ S %͜͜Ͱ c ͸ N ͓Αͼ n ʹΑΒͳ͍ఆ਺Ͱ͋Δɽຊ‫ؔʹڀݚ‬࿈͢Δଞͷཧ࿦త݁Ռ͓Αͼ਺஋࣮‫ݧ‬ͷ݁Ռ͸౰೔ใࠂ͢Δɽࢀߟจ‫ݙ‬[1] W. Kumagai, Learning Bound for Parameter Transfer Learning, NIPS, (2016).

2