ᶵᲔ▱ぬ㻒䝻䝪䝔䜱䜽䝇䜾䝹䞊䝥୰㒊኱Ꮫ㻌㻌ᕤᏛ◊✲⛉㻌᝟ሗᕤᏛᑓᨷ

㻞㻜㻞㻝ᖺᗘ䚷ᒣୗ◊✲ᐊ䚷ಟኈㄽᩥⓎ⾲䚷䜰䝤䝇䝖䝷䜽䝖㻰㼑㼑㼜㻌㻸㼑㼍㼞㼚㼕㼚㼓䚷䚷㻾㼛㼎㼛㼠㼕㼏㼟䚷䚷⏕άᨭ᥼䝻䝪䝑䝖䛻䜘䜛≀య㓄⨨䝍䝇䜽䛻䛚䛡䜛௚≀య䛸䛾⾪✺ணἙྜ㻌➉ᙪ䛻㛵䛩䜛◊✲

㻰㼑㼑㼜㻌㻸㼑㼍㼞㼚㼕㼚㼓䚷䚷㻻㼎㼖㼑㼏㼠㻌㻰㼑㼠㼑㼏㼠㼕㼛㼚㻳㼞㼕㼐㻙㼣㼕㼟㼑㻙㼍㼠㼠㼑㼚㼠㼕㼛㼚䜢ᑟධ䛧䛯≀య᳨ฟ䛸どぬⓗㄝ᫂䛻㛵䛩䜛◊✲ᮌᮧ㻌⛅ᩯ㻰㼑㼑㼜㻌㻸㼑㼍㼞㼚㼕㼚㼓䚷䚷㻴㼡㼙㼍㼚㻌㻼㼛㼟㼑㻌㻱㼟㼠㼕㼙㼍㼠㼕㼛㼚㼀㼞㼍㼚㼟㼒㼛㼞㼙㼑㼞䛻䜘䜛୰㛫≉ᚩ䜢⪃៖䛧䛯ேయጼໃ᥎ᐃ䛻㛵䛩䜛◊✲ᑠᯇ㻌ᝆᩯ㻰㼑㼑㼜㻌㻸㼑㼍㼞㼚㼕㼚㼓䚷䚷㻿㼑㼙㼍㼚㼠㼕㼏㻌㻿㼑㼓㼙㼑㼚㼠㼍㼠㼕㼛㼚䚷䚷㻼㼛㼕㼚㼠㻌㻯㼘㼛㼡㼐㻿㼑㼘㼒㻙㻭㼠㼠㼑㼚㼠㼕㼛㼚䜢⏝䛔䛯Ⅼ⩌䝕䞊䝍䛛䜙䛾䝉䝬䞁䝔䜱䝑䜽䝉䜾䝯䞁䝔䞊䝅䝵䞁䛻㛵䛩䜛◊✲㕥ᮌ㻌㈗኱
㻰㼑㼑㼜㻌㻸㼑㼍㼞㼚㼕㼚㼓䚷䚷㻿㼑㼙㼍㼚㼠㼕㼏㻌㻿㼑㼓㼙㼑㼚㼠㼍㼠㼕㼛㼚䝉䝬䞁䝔䜱䝑䜽䝉䜾䝯䞁䝔䞊䝅䝵䞁䛻䛚䛡䜛䝬䝹䝏䝗䝯䜲䞁Ꮫ⩦䛻㛵䛩䜛◊✲ṇᮌ㻌⩧኱㻰㼑㼑㼜㻌㻸㼑㼍㼞㼚㼕㼚㼓䚷䚷㻾㼑㼕㼚㼒㼛㼞㼏㼑㼙㼑㼚㼠㻌㻸㼑㼍㼞㼚㼕㼚㼓䚷䚷㻹㼡㼘㼠㼕㻙㼍㼓㼑㼚㼠䝬䝹䝏䜶䞊䝆䜵䞁䝖ᙉ໬Ꮫ⩦䜢⏝䛔䛯䝕䝑䝗䝻䝑䜽ၥ㢟䛾ゎỴ䛻㛵䛩䜛◊✲஬⸨㻌ᙉᚿ

URL: http://mprg.jpੜ‫ࢧ׆‬ԉϩϘοτʹΑΔ෺ମ഑ஔλεΫʹ͓͚Δଞ෺ମͱͷিಥ༧ଌʹؔ͢Δ‫ڀݚ‬ࢦಋ‫ڭ‬तɿࢁԼ ོٛ

TP20010 Տ߹ ஛඙
1. ͸͡Ίʹ
ߴྸऀ΍਎ମো֐ऀͷੜ‫׆‬Λࢧԉ͢Δհ‫ऀޢ‬ͷਓखෆ଍͕େ͖ͳࣾձ໰୊ʹͳ͍ͬͯΔɽͦͷͨΊɼՈఉͰՈࣄ΍հ‫Ͳͳޢ‬ͷੜ‫ࢧ׆‬ԉΛߦ͏ੜ‫ࢧ׆‬ԉϩϘοτ [1] ͷಋೖ͕‫ظ‬଴͞Ε͍ͯΔɽੜ‫ࢧ׆‬ԉʹ͓͍ͯɼίοϓ౳ͷ೔༻඼ΛӡͿ෺ମӡൖ͸ॏཁͳ‫ج‬ຊతλεΫͰ͋Δɽ෺ମӡൖλεΫʹ͸ɼ೺࣋ͨ͠෺ମΛࢦఆ͞Εͨ৔ॴʹ഑ஔ͢ΔλεΫ΋
‫·ؚ‬ΕΔɽੜ‫Ͱڥ؀׆‬͸༷ʑͳ෺ମ͕ཚࡶʹஔ͔Ε͍ͯΔ͜ͱ͕ଟ͍ͨΊɼࣄલʹଞͷ෺ମͱিಥ͕ൃੜ͢Δ͔Λ༧ଌ͠ɼিಥΛ๷͙ඞཁ͕͋Δɽ෺ମ഑ஔλεΫʹ͓͚Δি
ಥ༧ଌ๏ͱͯ͠ PonNet [2] ͕ఏҊ͞Ε͍ͯΔɽPonNet ͸೺࣋ͨ͠෺ମΛɼཚࡶʹ෺ମ͕ஔ͔ΕͨΤϦΞ಺ʹ഑ஔ͢Δࡍʹিಥ͕ൃੜ͢Δ֬཰Λ༧ଌ͢Δɽೖྗ৘ใ͸ϩϘοτ͕ࡱӨͨ͠ RGB-D ը૾ͱ೺࣋ͨ͠෺ମͷϞσϧ৘ใͰ
͋ΔɽϞσϧ৘ใ͸ɼ‫ط‬஌ͷ෺ମσʔλϕʔεΛࢀরͯ͠ಘΔͨΊɼະ஌෺ମΛ೺࣋ͨ͠৔߹ʹਖ਼͘͠িಥ༧ଌ͕Ͱ
͖ͳ͍. ͞ΒʹɼϞσϧ৘ใͷΈͰ͸‫ࡏݱ‬ͷ೺࣋࢟੎ΛߟྀͰ͖ͣɼ೺࣋࢟੎ͷมԽʹΑΓൃੜ͢ΔিಥΛ༧ଌͰ͖ͳ͍ɽͦ͜Ͱຊ‫Ͱڀݚ‬͸ɼϋϯυΧϝϥΛ༻͍ͯΞϐΞϥϯε
ϕʔεͰ೺࣋ͨ͠෺ମΛଊ͑Δিಥ༧ଌ๏ΛఏҊ͢ΔɽΞϐΞϥϯεϕʔεͰ͸Ϟσϧ৘ใ͕ෆཁͰɼ೺࣋ͨ͠෺ମ
ͷ‫ܗ‬ঢ়΍େ͖͞ɼ೺࣋࢟੎౳ͷಛ௃Λଊ͑Δ͜ͱ͕Ͱ͖Δɽ

2. ෺ମ഑ஔλεΫͷিಥ༧ଌ෺ମ഑ஔλεΫͷিಥ༧ଌख๏Ͱ͋Δ PonNet ͸ɼRGB
ͱ Depth ͷ 2 ͭͷϞʔμϧͷಛ௃Λଊ͑ΔͨΊʹɼAttention Branch Network (ABN)[3] ΛϚϧνϞʔμϧʹ֦ு
͍ͯ͠ΔɽABN Λ༻͍Δ͜ͱͰɼಛ௃ϚοϓͷॏཁͳྖҬΛ‫ڧ‬ௐͭͭ͠ɼΞςϯγϣϯϚοϓͷՄࢹԽ͕ՄೳͰ͋
Δɽਓͱੜ‫ࢧ׆‬ԉϩϘοτͷ‫ڞ‬ಉੜ‫͍͓ͯʹ׆‬͸৴པੑ͕
ॏཁͰ͋ΔɽPonNet ͸ϩϘοτͷ஫ࢹྖҬΛՄࢹԽͯ͠
൑அࠜ‫ڌ‬Λ໌Β͔ʹ͢Δ͜ͱͰɼ৴པੑͷ޲্Λਤ͍ͬͯΔɽ·ͨɼੜ‫ڥ؀׆‬ͷΑ͏ͳཚࡶʹ෺ମ͕ஔ͔Ε͍ͯΔঢ়‫گ‬Լʹͯ෺ମ഑ஔΛߦ͏ͱিಥ͕ൃੜ͠ɼϩϘοτͷഁଛ΍‫ނ‬োͷϦεΫ͕ߴ͍ɽͦ͜ͰɼPonNet Ͱ͸ϦΞϧͳඳըͱ࿈࠯తͳ෺ཧ࡞༻ΛߟྀͰ͖ΔγϛϡϨʔλΛ༻͍ͯɼিಥ༧ଌͷͨΊͷσʔληοτ BILA-12K Λ࡞੒͠ɼֶश
ʹ༻͍͍ͯΔɽ

3. ఏҊख๏
෺ମͷϞσϧ৘ใΛ༻͍Δিಥ༧ଌ๏Ͱ͸ɼະ஌෺ମʹରԠͰ͖ͣ, ·ͨɼϦΞϧλΠϜͳ೺࣋࢟੎৘ใΛߟྀͰ͖ͳ͍ɽ೺࣋ͨ͠෺ମΛϋϯυΧϝϥͰࡱӨ͢Δ͜ͱͰɼΞϐΞϥϯεϕʔεͰ෺ମͷ‫ܗ‬ঢ়΍େ͖͞ɼ೺࣋࢟੎ʹؔ͢Δಛ௃Λଊ͑Δ͜ͱ͕Ͱ͖Δɽͦ͜Ͱɼຊ‫Ͱڀݚ‬͸ɼϋϯυΧϝϥʹΑͬͯ೺࣋෺ମͷΞϐΞϥϯεΛଊ͑ΔϚϧνϞʔμϧ ABN ΛఏҊ͢ΔɽఏҊख๏ͷྲྀΕΛਤ 1 ʹࣔ͢ɽ
シミュレータでデータセット

Featureextractor 1
RGB画

Attention
BranchFeatureextractor 1

入

𝑒

𝐽
𝑭
Featureextractor 2

Attention
機構

𝑥

Depth

Attention
Branch( 線マップ)Featureextractor 1
ハンド画：データの れ： 測確
：Attention Branch Network (ABN)：Balancing

𝑭
Featureextractor 2

Attention
機構

𝛼 𝑭

𝑒

𝑭

Perception
block

𝐽

正解
ラベル𝑦

𝛼 ：softmax(𝑒 )𝐽Attention
機構

𝑭
Featureextractor 2

Attention
Branch𝐽𝑒：𝑽 tanh(𝑾 𝑭 + 𝒃 )𝑽, 𝑾 , 𝒃 ：学習可能な重み
J ：クロスエントロピー誤差
𝑘 ∈ rgb, deptℎ, hand

ਤ 2: ϚϧνϞʔμϧ ABN ͷߏ଄Έ෇͚͢Δ Balancingɼ൑ఆΛߦ͏ Perception block Ͱߏ੒͞ΕΔɽϕʔεωοτϫʔΫʹ ResNet-18 Λ࢖༻͠ɼୈ 3ϨΠϠʔͰ෼ׂͯ͠ Feature extractor 1,2 ʹ෼͚Δɽ·ͣɼ֤ը૾ xk Λ Feature extractor 1 ʹೖྗͯ͠ಘΒΕͨಛ௃ϚοϓΛ Attention ‫ ͱߏػ‬Attention Branch ʹೖྗ͢Δɽ
Attention Branch ͷग़ྗ͸Ϋϥε༧ଌ֬཰ PkAB (xk ) ͱΞςϯγϣϯϚοϓͰ͋ΔɽAttention ‫ʹߏػ‬ΑΓΞςϯγϣ
ϯϚοϓͰॏΈ෇͚ͨ͠ಛ௃ϚοϓΛ Feature extractor
2 ʹೖྗ͠ɼ256 ࣍‫ݩ‬ͷಛ௃ϕΫτϧ Fk ΛಘΔɽ͜ΕΛɼ
Blancing ʹΑΓॏΈ෇͚ͯ͠ F ! ΛಘΔɽF ! Λ Perception
block ʹೖྗ͠ɼ࠷ऴతͳ༧ଌ P P B (x) ΛಘΔɽ࠷ऴతͳ
ଛࣦؔ਺ J ͸ࣜ (1) ʹΑΓ‫ٻ‬ΊΔɽAB
AB
AB
J = J P B + Jrgb
+ Jdepth+ Jhand(1)͜ͷͱ͖ɼ֤ J ͸, ֤ P (x) ͱਖ਼ղϥϕϧ ŷ ͷΫϩεΤϯ
τϩϐʔ‫ٻͰࠩޡ‬ΊΔɽ

3.2 σʔληοτ࡞੒ຊ‫Ͱڀݚ‬͸ɼ೺࣋෺ମͷΞϐΞϥϯεΛ‫ؚ‬Ήিಥ༧ଌσʔληοτΛɼPonNet Ͱߏங͞ΕͨγϛϡϨʔλΛ༻͍ͯ࡞੒͢Δɽੜ‫ࢧ׆‬ԉϩϘοτ͸ɼHuman Support Robot(HSR) Λର৅ͱ͢Δɽσʔληοτͷ࡞੒खॱΛ Step1ʙ
6 Ͱࣔ͢ɽ

Step 1 എ‫ܠ‬ɼর໌‫ͱڥ؀‬Ո۩Λબ୒ɽ
Step 2 28 छྨͷো֐෺ͷީิ෺ମ͔ΒϥϯμϜʹબ୒͠ɼ഑ஔΤϦΞ্ʹམԼͤ͞ো֐෺ͱͯ͠ઃஔɽ
Step 3 ഑ஔΤϦΞͷঢ়ଶΛ RGB-D ը૾ͱͯ͠ HSR ͷϔουΧϝϥΛ༻͍ͯࡱӨɽ
Step 4 HSR ͕഑ஔ͢Δ෺ମΛ೺࣋ʢ೺࣋࢟੎͸ਤ 3(a)ʣɽ
Step 5 ೺࣋ͨ͠෺ମΛϋϯυΧϝϥΛ༻͍ͯࡱӨ (ਤ 3(b))ɽ
Step 6 ೺࣋෺ମΛ഑ஔΤϦΞʹ഑ஔ͠ɼিಥ৘ใΛ‫ه‬࿥ɽ

(1)

ネットワーク

ラベル

り

RGB

or



し
Depth
(

域の可 化

マップ化）

ハンド

マルチモーダルABNりしਤ 1: ఏҊख๏ͷྲྀΕຊ‫Ͱڀݚ‬͸ɼੜ‫ڥ؀׆‬Λ࠶‫ͨ͠ݱ‬γϛϡϨʔλΛ༻͍ͯɼ೺࣋࢟੎ͷมԽ΍ະ஌෺ମΛ‫ؚ‬Ήঢ়‫Ͱگ‬ͷσʔληοτΛ࡞੒͢Δɽ͜ͷσʔληοτΛ༻͍ͯωοτϫʔΫΛֶश͠ɼিಥ༧ଌͱ஫ࢹྖҬͷՄࢹԽΛߦ͏ɽ

3.1 ωοτϫʔΫߏ଄ຊख๏ͷωοτϫʔΫߏ଄Λਤ 2 ʹࣔ͢ɽ֤Ϟʔμϧk ∈ {rgb, depth, hand} ʹରԠ͢Δ 3 ͭͷ ABNɼಛ௃ʹॏ

(2)
90°
(a) ೺࣋࢟੎(b) ϋϯυΧϝϥը૾ਤ 3: ೺࣋࢟੎ͷछྨিಥ༗Γͷ൑ఆ͸ো֐෺ಉ࢜ͷিಥɼϩϘοτΞʔϜͱো֐෺ͷিಥɼ഑ஔ͢Δ෺ମͱো֐෺ͷিಥɼো֐෺ͱ‫ڥ؀‬ͷিಥͷͦΕͧΕͷ৚݅Ͱɼᮢ஋ 0.1m/s Ҏ্ͷিಥͱ͢Δɽ͜ΕΛඞཁͳαϯϓϧ਺͕ಘΒΕΔ·Ͱ‫܁‬Γฦ͠ੜ੒ද 1: σʔληοτͷিಥϥϕϧ಺༁ [%]
NDC

DC

O-O

T-O

A-O

O-D

P1

P2

69.67

30.33

3.15

21.11

14.41

10.46

23.19

76.06

NDC ͸িಥແ͠Ͱ͋ΓɼDC ͸িಥ༗ΓͰ͋ΔɽDC ͸ɼԼ‫ه‬ͷিಥ৚݅ͷ͍ͣΕ͔ 1 ͭҎ্͕ൃੜͨ͠৔߹Ͱ͋ΔɽO-O ͸഑ஔΤϦΞ಺ͷো֐෺ಉ࢜ͷিಥΛද͢ɽT-O ͸഑ஔͨ͠෺ମͱো֐෺ͷিಥΛද͢ɽA-O ͸ো֐෺ͱϩϘοτΞʔϜͷিಥΛද͢ɽO-D ͸ো֐෺ͱ഑ஔΤϦΞ͓Αͼͦͷपล‫΁ڥ؀‬ͷিಥΛද͢ɽP1ɼP2 ͸ DC ϥϕϧͰ೺࣋࢟੎ 1ɼ೺࣋࢟੎ 2 ͷ৔߹Ͱ͋Δɽ೺࣋࢟੎ 2 ͷিಥ༗Γϥϕϧ͸೺࣋࢟੎ 1 ͷ 3.2 ഒఔ౓͋Γɼ೺࣋࢟੎ 2 ͸িಥ͕ൃੜ͠΍͍͢͜ͱ͕Θ͔Δɽ͜ͷΑ͏ʹɼ഑ஔΤϦΞͷঢ়ଶ͚ͩͰͳ͘ɼ೺࣋࢟੎΋ߟྀ͢Δ͜ͱ͕ॏཁͰ͋Δɽ

4. ධՁ࣮‫ݧ‬ຊষͰ͸ɼఏҊख๏ͷ༗ޮੑΛ‫͢ূݕ‬ΔͨΊͷධՁ࣮‫ݧ‬Λߦ͏ɽධՁ࣮‫Ͱݧ‬͸෺ମ഑ஔλεΫʹ͓͚Δিಥ༧ଌͱ
ͯ͠ NDC or DC ͷ 2 Ϋϥε෼ྨΛߦ͍ɼ֤ख๏ͱͷਫ਼౓ൺֱΛߦ͏ɽ
ຊ࣮‫ʹݧ‬͸ɼલઅͰ࡞੒ͨ͠σʔληοτΛ༻͍ɼֶश
ʹ 42,100 ճɼ‫ ʹূݕ‬5,300 ճɼධՁʹ 5,300 ճ෼ͷαϯϓϧΛ࢖༻͢ΔɽϕʔεϞσϧʹ͸ ResNet-18 Λ࢖༻͢Δɽֶशճ਺͸ 10 ΤϙοΫɼ࠷దԽؔ਺ʹ Adam Λ࢖༻͠ɼॳ‫ֶظ‬श཰Λ 10−5 ͱ͢Δɽ·ͨɼֶश݁Ռʹ͸ɼֶ֤शΤϙοΫʹ͓͍ͯɼ࠷΋‫ূݕ‬ਫ਼౓͕ߴ͍ϞσϧͷධՁ݁ՌΛ༻͍ɼͦΕͧΕ 5 ճֶशͨ݁͠Ռͷฏ‫ۉ‬Λ༻͍Δɽ

4.2 ‫ط‬஌෺ମʹର͢Δিಥ༧ଌͷఆྔతධՁ
֤Ϟσϧͷিಥ༧ଌͷਖ਼ղ཰ͷൺֱΛද 2 ʹࣔ͢ɽ

ද 2: ೺࣋࢟੎ͷมԽ༗Γͷিಥ༧ଌͷਖ਼ղ཰
Ponnet[2]
ఏҊख๏

Ϟσϧ৘ใ
!
-

ϋϯυը૾
-ɹ
-ɹ
!

ਖ਼ղ཰ [%]
84.49 ± 0.22
84.25 ± 0.30
87.10 ± 0.33ఏҊख๏͸ϋϯυΧϝϥͷը૾Λར༻͠ɼਫ਼౓͕޲্͠
ͨɽ͜ͷ͜ͱ͔ΒɼϋϯυΧϝϥͷը૾Λར༻͢Δ͜ͱͰɼ෺ମͷ೺࣋࢟੎΍େ͖͞ͷಛ௃Λଊ͍͑ͯΔ͜ͱ͕Θ͔Δɽ
ྫ͑͹ɼ഑ஔΤϦΞͷো֐෺ͷ഑ஔ͕ಉ͡Ͱ͋Δ͕ɼ෺ମͷ೺࣋࢟੎͕ҧ͏৔߹ʹɼϋϯυΧϝϥʹΑͬͯ೺࣋࢟੎
͕ಘΒΕɼద੾ͳ༧ଌ͕ՄೳʹͳΔɽ

4.3 ΞςϯγϣϯϚοϓͷՄࢹԽʹΑΔఆੑతධՁ
࣍ʹɼਖ਼ղϥϕϧͱ༧ଌ݁Ռ͕ DC ͷ৔߹ͷਖ਼ղྫͷΞςϯγϣϯϚοϓͷՄࢹԽ݁ՌΛਤ 4 ʹࣔ͢ɽRGB

4.4 Insertion ʹΑΔΞςϯγϣϯϚοϓͷධՁΞςϯγϣϯϚοϓͷ༗ޮੑͷ‫ূݕ‬ͷͨΊʹ InsertionΛߦͬͨ݁ՌΛਤ 5 ࣔ͢ɽ
100
80
60
40

PonNet

20
0
0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Insertionਤ 5: Insertion ʹΑΔΞςϯγϣϯϚοϓͷධՁ

4.1 ࣮‫ݧ‬৚݅ख๏ը૾ͱ Depth ը૾ͰҟͳΔಛ௃Λֶश͢Δ͜ͱͰɼਫ਼౓
͕޲্ͨ͠ͱߟ͑ΒΕΔɽ࣍ʹɼϋϯυը૾ͷΞςϯγϣ
ϯϚοϓͷՄࢹԽ݁ՌͰ͸ɼ೺࣋ͨ͠෺ମΛ஫ࢹ͍ͯ͠Δ
͜ͱ͕Θ͔Δɽ͜ΕʹΑͬͯɼ෺ମͷछྨ΍େ͖͞ͱিಥ
ͷൃੜ͠΍͢͞ͷؔ܎ͳͲͷಛ௃Λֶश͍ͯ͠Δͱߟ͑ΒΕΔɽ·ͨɼಛఆͷ෺ମͷΈʹΞςϯγϣϯ͕ൃੜ͢Δ܏
޲͋ͬͨɽ͜Ε͸ɼ೺࣋࢟੎ 1 ͷঢ়ଶ΍େ͖͕͞খ͍͞෺
ମͳͲɼিಥͷ‫ݥة‬ੑ͕௿͍෺ମͱͦ͏Ͱͳ͍෺ମΛ۠ผ
͍ͯ͠Δͱߟ͑ΒΕΔɽ

[%]͠ɼσʔηοτΛߏங͢Δɽ࣮ࡍʹ࡞੒ͨ͠σʔληοτͷিಥϥϕϧͷ಺༁Λද 1 ʹࣔ͢ɽ

Depth

Hand

Insertion ͸ೖྗը૾ʹର͢ΔΞςϯγϣϯϚοϓͷ֤ϐΫηϧͷ஋Λᮢ஋ҎԼΛ 0 ʹ͢ΔॲཧΛߦ͍ɼೖྗը૾ΛΞςϯγϣϯϚοϓͰϚεΫͨ͠‫ʹޙ‬, ࠶౓ωοτϫʔΫʹೖྗͯ͠ਪ࿦Λߦ͏࣮‫͋Ͱݧ‬Δɽᮢ஋Λঃʑʹ૿Ճͤͨ͞
ࡍͷ܏޲͔ΒɼΞςϯγϣϯ͕ೖྗը૾ͷॏཁͳ෦෼ʹൃੜ
͍ͯ͠Δ͔‫͢ূݕ‬Δ͜ͱ͕Ͱ͖ΔɽఏҊख๏͸ɼInsertionͷׂ߹͕௿͍৔߹ʹਫ਼౓͕௿Լ͠ɼInsertion ͷׂ߹͕ 1 ͷͱ͖ʹਫ਼౓͕࠷΋ߴ͍͜ͱ͔ΒɼΞςϯγϣϯͷ஋͕ߴ͍෦෼͕ɼೖྗը૾ͷॏཁͳ෦෼Λଊ͍͑ͯΔ͜ͱ͕Θ͔Δɽ

4.5 ະ஌෺ମʹର͢Δিಥ༧ଌͷఆྔతධՁ
ֶशʹ࢖༻͍ͯ͠ͳ͍ະ஌෺ମʹର͢Δিಥ༧ଌΛද 3ʹࣔ͢ɽະ஌෺ମͰ΋ɼPonNet ͱൺֱͯ͠ఏҊख๏Ͱ͸ਫ਼౓͕޲্͓ͯ͠Γɼϋϯυը૾ʹΑΓ෺ମͷಛ௃Λଊ͑
͍ͯΔ͜ͱ͕Θ͔Δɽද 3: ະ஌෺ମʹର͢Δিಥ༧ଌͷਖ਼ղ཰ख๏
PonNet
ఏҊख๏

Ϟσϧ৘ใ
-

ϋϯυը૾
!

ਖ਼ղ཰ [%]
72.23
74.06

5. ͓ΘΓʹ
ຊ‫Ͱڀݚ‬͸ɼϋϯυΧϝϥʹΑͬͯΞϐΞϥϯεϕʔεͰ೺࣋෺ମΛଊ͑ΔϚϧνϞʔμϧ ABN ΛఏҊͨ͠ɽ·ͨɼ೺࣋෺ମͷΞϐΞϥϯεΛ‫ؚ‬Ή෺ମ഑ஔλεΫͷিಥ༧ଌͷͨΊͷσʔληοτΛ࡞੒ͨ͠ɽ࡞੒ͨ͠σʔληοτͰֶशͨ݁͠ՌɼϋϯυΧϝϥΛ௥Ճͨ͠ωοτϫʔΫͰͷিಥ༧ଌͷਫ਼౓͕޲্ͨ͠ɽࠓ‫ޙ‬͸഑ஔલͷը૾͚ͩͰͳ͘഑ஔ‫ޙ‬ͷը૾Λར༻͢ΔͳͲɼ༷ʑͳ৚݅Ͱͷিಥ༧ଌͱ৴པੑͷ޲্Λ໨ࢦ͢ɽࢀߟจ‫ݙ‬[1] T. Yamamoto. et al.: “Human Support Robot (HSR)”ɼ
ACMɼ2018.
[2] A. Magassouba. et al.: “Predicting and attending todamaging collisions for placing everyday objects inphoto-realistic simulations”ɼARɼ2021.
[3] H. Fukui. et al: “Attention Branch Network: Learning ofAttention Mechanism for Visual Explanation”ɼCVPRɼ
2019.

‫੷ۀڀݚ‬
ਤ 4: ΞςϯγϣϯϚοϓͷՄࢹԽ݁Ռ
ਤ 4 ͷ RGB ը૾Ͱ͸ɼো֐෺͕ଘࡏ͠ͳ͍৔ॴʹ஫ࢹྖҬ͕ൃੜ͓ͯ͠Γɼฏ໘Λ஫ࢹ͍ͯ͠Δ͜ͱ͕Θ͔Δɽ
·ͨɼDepth ը૾Ͱ͸ো֐෺ʹ஫ࢹྖҬ͕ൃੜ͓ͯ͠Γɼো֐෺Λ஫ࢹ͍ͯ͠Δ͜ͱ͕Θ͔Δɽ͜ͷ͜ͱ͔ΒɼRGB[1] Տ߹஛඙ɼฏ઒ཌྷɼࢁԼོٛ, ౻٢߂࿱, ਿӜ޸໌,“഑ஔ෺ମΛߟྀͨ͠ Attention ‫ߏػ‬ϚϧνϞʔμϧωοτϫʔΫʹΑΔিಥ༧ଌ”, ೔ຊϩϘοτֶձֶज़ߨԋձ, 2021.
[2] Տ߹஛඙ɼฏ઒ཌྷɼࢁԼོٛ, ౻٢߂࿱,“೺࣋ಈ࡞݁ՌΛ༻͍ͨ௥ՃֶशʹΑΔ෺ମ೺࣋ͷߴਫ਼౓Խ”, ి‫ؾ‬ɾిࢠɾ৘ใؔ
܎ֶձ౦ւࢧ෦࿈߹େձ, 2020.

Grid-wise-attention Λ༻͍ͨ෺ମ‫ݕ‬ग़ͷࢹ֮తઆ໌ੑʹؔ͢Δ‫ڀݚ‬ࢦಋ‫ڭ‬तɿࢁԼོٛ

TP20011 ໦ଜळే1. ͸͡Ίʹ
෺ମ‫ݕ‬ग़͸ը૾্ͷ෺ମҐஔͱͦͷ໊শΛಛఆ͢Δٕज़
Ͱ͋Δɽࣗಈӡసʹ͓͚Δ҆શ֬อʹ͸าߦऀ΍ं྆ͷ‫ݕ‬ग़͕͔ܽͤͳ͍ɽಛʹɼϨϕϧ 5 ͷ‫׬‬શࣗಈӡసͷ࣮‫ʹݱ‬͸ɼ‫ݕ‬ग़ਫ਼౓ͷ޲্͚ͩͰͳ͘ɼαʔϏεΛ‫ڗ‬ड͢Δਓ͕
৴པͰ͖ΔΑ͏ʹ‫ݕ‬ग़݁Ռͷ൑அࠜ‫ڌ‬Λࣔ͢ඞཁ͕͋Δɽຊ‫Ͱڀݚ‬͸ɼาߦऀΛ‫ݕ‬ग़ର৅ͱ͠ɼͦͷ൑அࠜ‫ڌ‬ΛΞςϯγϣϯϚοϓͱͯ͠ग़ྗ͢Δख๏ΛఏҊ͢ΔɽఏҊख๏͸ɼը૾ΛάϦουঢ়ʹ෼ׂ͠ɼ֤άϦουʹର͢ΔΞ
ςϯγϣϯϚοϓΛऔಘ͢Δɽ·ͨɼఏҊख๏Ͱ͸ɼऔಘͨ͠ΞςϯγϣϯϚοϓΛಛ௃ϚοϓʹॏΈ෇͚ͯ͠ҐஔਪఆΛߦ͏͜ͱͰɼ‫ݕ‬ग़ਫ਼౓ͷ޲্Λ࣮‫͢ݱ‬Δɽ

2. ؔ࿈‫ڀݚ‬෺ମ‫ݕ‬ग़ख๏͸ɼ෺ମͷҐஔਪఆͱΫϥε෼ྨΛಉ࣌ʹ
ߦ͏ one-stage ͱɼ෺ମͷҐஔਪఆΛͨ͠‫ޙ‬ɼ֤Ґஔʹରͯ͠Ϋϥε෼ྨΛߦ͏ two-stage ͕͋Δɽone-stage ͷख๏͸ɼߴ͍ਫ਼౓Λୡ੒ͭͭ͠ߴ଎Խ͞Ε͓ͯΓɼ෺ମ‫ݕ‬ग़ͷ୅දతͳΞϓϩʔνͱͳ͍ͬͯΔɽone-stage ‫ݕ‬ग़ख๏ͷ
1 ͭͰ͋Δ Center and Scale Prediction(CSP)[1] ͸ɼาߦऀͷத৺ͱεέʔϧΛਪఆͯ͠าߦऀΛ‫ݕ‬ग़͢Δɽ͜ΕʹΑΓɼখ͍͞෺ମ΍ΦΫϧʔδϣϯʹ‫ݕͳ݈ؤ‬ग़͕Մೳ
Ͱ͋Δɽ͔͠͠ɼCSP Λ‫ؚ‬Ήैདྷͷ෺ମ‫ݕ‬ग़ख๏͸ɼωοτϫʔΫͷதؒ৘ใΛՄࢹԽͨ͠ࡍʹෳ਺ͷ෺ମΛಉ࣌ʹ‫ݕ‬ग़͢ΔͨΊɼͲͷ෺ମͷ൑அࠜ‫ڌ‬Λࣔͨ͠΋ͷͰ͋Δ͔
ෆ໌Ͱ͋Δͱ͍͏໰୊͕͋Δɽ

3. ఏҊख๏
ຊ‫Ͱڀݚ‬͸ɼը૾ΛάϦουঢ়ʹ෼ׂ֤ͨ͠άϦουʹ
ର͢ΔΞςϯγϣϯϚοϓΛऔಘ͢Δ Grid-wise-attentionʹΑΔ෺ମ‫ݕ‬ग़ΛఏҊ͠ɼΞςϯγϣϯϚοϓ͔Β෺ମ‫ݕ‬ग़ͷ൑அࠜ‫ڌ‬Λࣔ͢ɽ

3.1. ωοτϫʔΫߏ଄ఏҊख๏ͷωοτϫʔΫߏ଄Λਤ 1 ʹࣔ͢ɽຊख๏ͷωοτϫʔΫߏ଄͸ɼFeature extractorɼ֤άϦουʹର͢Δ
Ϋϥε֬཰Λ‫ٻ‬ΊΔ Classification branch ͱ Grid-wiseattention ʹΑΔॏΈ͚ͮʹΑ֤ͬͯάϦουʹର͢Δ෺ମͷҐஔΛ‫ٻ‬ΊΔ Localization branch ͔Βߏ੒͞ΕΔɽ

ͯߦ͍ɼ֤ Query ʹର͢ΔΞςϯγϣϯϚοϓΛऔಘ͢Δɽऔಘͨ͠ΞςϯγϣϯϚοϓΛϕʔεωοτϫʔΫ͔Βऔಘͨ͠ಛ௃ϚοϓʹॏΈ෇͚͢Δɽ͜ΕʹΑΓɼ஫ࢹྖҬΛߟྀͨ͠෺ମ‫ݕ‬ग़͕Ͱ͖Δɽͦͯ͠ɼLocalization
branch ʹ͓͍ͯɼॏΈ෇͚͞Εͨಛ௃Ϛοϓ͔Β֤άϦουʹରͯ͠ͷ෺ମҐஔΛग़ྗ͢Δɽ1 ͭͷ෺ମʹରͯ͠ෳ਺ͷ‫ݕ‬ग़݁ՌΛग़ྗ͢Δ͜ͱΛ๷͙ͨΊʹɼඇ࠷େ஋཈੍
Λ༻͍Δɽ

3.2. ֶशํ๏
ఏҊख๏͸ɼग़ྗ֤ͨ͠άϦουʹରͯ͠ͷΫϥε֬཰ͱ෺ମҐஔ͔Β‫ٻ‬Ίͨ‫ݕ‬ग़݁Ռʹର͢ΔΫϥε֬཰ͷଛࣦ
Lcls ͱਪఆҐஔͱਖ਼ղҐஔͷଛࣦ Lbox Λ༻ֶ͍ͯश͢Δɽ
Ϋϥε෼ྨʹ͸ Focal loss Λ࢖༻͢ΔɽFocal loss ͸ɼߴ͍໬౓Ͱೝࣝͨ͠Ϋϥεʹରͯ͠ɼখ͞ͳॏΈΛଛࣦʹ৐ࢉ͢Δɽ͜ΕʹΑΓɼೝ͕ࣝࠔ೉ͳΫϥεΛॏࢹ͢ΔΑ͏ͳֶश͕Ͱ͖ΔɽLcls ɼLbox ΛͦΕͧΕࣜ (2)ɼࣜ (3) ʹࣔ͢ɽ
1 !
Lcls = c−(1 − p̂i )γ log(p̂i )(2)
N i

Lbox =1 !
I(gi = 1)lr (bi , ti )
Nr i

(3)

͜͜Ͱɼγ Λ؆୯ͳΫϥεʹର͢ΔॏΈΛௐ੔͢ΔύϥϝʔλͰຊ‫Ͱڀݚ‬͸ γ = 2 ͱ͢ΔɽNkc Λ‫ݕ‬ग़݁Ռ਺ɼgi
Λ i ൪໨ͷ‫ݕ‬ग़݁Ռʹର͢Δਖ਼ղΫϥεɼlr Λ smooth L1
Lossɼbi Λ i ൪໨ͷ‫ݕ‬ग़݁Ռͷ෺ମҐஔɼti Λ i ൪໨ͷ‫ݕ‬ग़݁Ռʹର͢Δਖ਼ղҐஔɼ I(·) Λਖ਼ղͱఆٛ͞ΕͨΞϯΧʔͷΈʹ੍‫͢ݶ‬Δࢦඪؔ਺ͱ͢Δɽp̂i ͸ɼi ൪໨ͷ‫ݕ‬ग़݁ՌͷΫϥε֬཰ pi Λ༻͍ͯࣜ (4) ͷ৚݅Ͱ‫ٻ‬ΊΒΕΔɽ"pi
if gi = 1p̂i =(4)
1 − pi otherwise

3.3. ෺ମ‫ݕ‬ग़ͷ஫ࢹྖҬͷऔಘ
Grid-wise-attention ‫ߏػ‬͸Ґஔ (i, j) ͷάϦουʹର͢ΔΞςϯγϣϯϚοϓ Attentionij Λऔಘ͢Δɽ͜ΕΛಛ௃ϚοϓͷશͯͷҐஔʹରͯ͠‫ٻ‬Ίɼ̎࣍‫ʹݩ‬഑ஔ͢Δ͜
ͱͰΞςϯγϣϯϚοϓΛ‫ٻ‬ΊΔɽຊख๏ͷ෺ମ‫ݕ‬ग़͸ɼ֤άϦουʹରͯ͠ͷΫϥε֬཰ͱ෺ମҐஔΛ‫ͨ͏ߦʹج‬ΊɼάϦουຖʹ‫ݕ‬ग़݁ՌΛग़ྗ͢ΔɽͦͷͨΊɼᮢ஋ॲཧ΍ඇ࠷େ஋཈੍Λద༻ͨ͠‫ʹޙ‬෺ମΛ‫ؚ‬ΉάϦουʹରͯ͠ͷΞςϯγϣϯϚοϓΛՄࢹԽ͢Δ͜ͱͰɼ‫ݕ‬ग़ର৅ຖͷ൑அࠜ‫ڌ‬Λऔಘ͢Δ͜ͱ͕Ͱ͖Δɽ

3.4. Ϛϧνεέʔϧߏ଄ͷಋೖ

ਤ 1 : ఏҊख๏ͷωοτϫʔΫߏ଄͸͡Ίʹը૾ΛϕʔεωοτϫʔΫͰ͋Δ ResNet-50 ʹೖྗ͠ɼಛ௃ϚοϓΛऔಘ͢Δɽ࣍ʹɼಛ௃ϚοϓΛ W ×HͷάϦουঢ়ʹ෼ׂ͠ɼClassification branch ͷ৞ΈࠐΈ૚ͱγάϞΠυؔ਺ʹΑΓάϦουຖʹ෺ମͷΫϥε֬཰Λग़ྗ͢Δɽ͜ͷͱ͖ͷಛ௃ϚοϓΛ Grid-wise-attention‫͍͓ͯʹߏػ‬ɼW × H ‫ݸ‬෼ͷ QueryɼϕʔεωοτϫʔΫ͔Βऔಘͨ͠ಛ௃ϚοϓΛ Key ͱͯ͠ɼQuery ͱ Keyͷ಺ੵʹΑͬͯΞςϯγϣϯϚοϓΛ W × H ‫ݸ‬෼औಘ
͢ΔɽҐஔ (i, j) ͷάϦουʹର͢ΔΞςϯγϣϯϚοϓ
Attentionij Λࣜ (1) ʹࣔ͢ɽ

Attentionij = softmax(Qij K T )(1)औಘ͞ΕΔ Attentionij ͸ɼTransformer[2] ͱಉ༷ʹ Queryͱ Key ͷྨࣅ౓Λද͢ɽ͜ͷॲཧΛશͯͷ Query ʹ͍ͭਤ 2 : Ϛϧνεέʔϧߏ଄ຊख๏͸ɼਤ 2 ʹࣔ͢Α͏ʹҟͳΔεέʔϧͷάϦου
Λ༻͍ͯଟॏղ૾౓Խ͢ΔɽϕʔεωοτϫʔΫ͔ΒҟͳΔεέʔϧͷಛ௃ϚοϓΛऔಘ͠ɼͦΕͧΕΛ Classification
branch ͱ Localization branch ʹೖྗͯ͠Ξςϯγϣϯ
Ϛοϓͷऔಘͱ෺ମͷ‫ݕ‬ग़Λߦ͏ɽ͜ΕʹΑΓɼߴղ૾౓
ͷಛ௃Ϛοϓ͔Βখ͞ͳ෺ମΛ‫ݕ‬ग़ɼ௿ղ૾౓ͷಛ௃Ϛοϓ͔Βେ͖ͳ෺ମͷ‫ݕ‬ग़Λ͢Δ͜ͱ͕ՄೳͱͳΔɽਤ 4 : ΞςϯγϣϯϚοϓՄࢹԽ݁Ռ

4. ධՁ࣮‫ݧ‬ຊ࣮‫Ͱݧ‬͸ɼఏҊख๏ʹΑΔΞςϯγϣϯϚοϓΛՄࢹ
Խ͢Δ͜ͱͰɼาߦऀ‫ݕ‬ग़ͷ൑அࠜ‫ڌ‬Λࣔ͢ɽ·ͨɼఏҊख๏ͷ‫ݕ‬ग़ਫ਼౓͔Βɼ஫ࢹྖҬΛߟྀͨ͠าߦऀ‫ݕ‬ग़ͷ༗ޮੑΛௐࠪ͢Δɽ

4.1. ࣮‫֓ݧ‬ཁֶशɼධՁʹ͸ɼֶश༻ʹ 2,975 ຕɼධՁ༻ʹ 500 ຕͷը૾Λ‫ؚ‬Ή CityPersons σʔληοτΛ༻͍ΔɽఏҊख๏͸ֶशճ਺Λ 150 ΤϙοΫɼ࠷దԽख๏Λ Adamɼॳ‫ظ‬ֶश཰Λ 1.0 × 10−5 ͱ͢Δɽൺֱ࣮‫Ͱݧ‬͸ैདྷͷ෺ମ‫ݕ‬ग़ख๏Ͱ͋Δ CSP Λ༻͍Δɽ
ఆྔతͳධՁࢦඪʹ͸ log-average miss rate Λ༻͍Δɽ
log-average miss rate ͸ 1 ը૾͋ͨΓͷ‫ݕޡ‬ग़཰ (FPPI)Λ 0.01 ͔Β 100 ͷൣғ಺͔Βର਺εέʔϧͰ౳ִؒʹऔಘ͠ɼ֤஍఺ͷ FPPI ʹର͢Δະ‫ݕ‬ग़཰ (Miss Rate) ͷฏ‫͔ۉ‬Β‫ٻ‬ΊΔɽlog-average miss rate ͕௿͍΄Ͳߴਫ਼౓Ͱ
͋Δ͜ͱΛࣔ͢ɽਖ਼ղͰ͋Δาߦऀͷߴ͕͞ 250 pixel Ҏ্ ͷ΋ͷΛ Largeɼ250 pixel ະຬ͔ͭ 50 pixel Ҏ্ͷ΋ͷΛ Middleɼ50 pixel ະຬͷ΋ͷΛ Small ͢Δɽ·ͨɼΞςϯγϣϯϚοϓΛՄࢹԽ͠ɼ‫͘ڧ‬஫ࢹ͢ΔՕॴΛ‫ʹج‬൑அࠜ‫ڌ‬Λਪଌ͢ΔɽҟͳΔεέʔϧͷಛ௃Ϛοϓ͔Βऔಘ͢ΔΞςϯγϣϯϚοϓΛೖྗಛ௃Ϛοϓͷղ૾౓͕௿͍ॱʹ Scale1ɼ2ɼ3 ͱͯ͠ɼεέʔϧຖͷΞςϯγϣϯϚοϓΛൺֱ͢Δɽ

4.2. ‫ݕ‬ग़ਫ਼౓ͷൺֱैདྷͷาߦऀ‫ݕ‬ग़ख๏ͱఏҊख๏ͷൺֱ݁ՌΛද 1 ʹࣔ͢ɽఏҊख๏͸ MiddleɼSmall Ͱ͸ͦΕͧΕ CSP ʹൺ΂ 0.5 ϙΠϯτɼ4.9 ϙΠϯτ૿Ճ͍ͯ͠Δ͕ɼLarge Ͱ͸
12.4 ϙΠϯτ௿Լ͓ͯ͠ΓɼશମͰ͸ 7.1 ϙΠϯτ௿Լ͠ɼਫ਼౓͕޲্͍ͯ͠Δɽ
ද 1 : ‫ݕ‬ग़ਫ਼౓ͷൺֱʢlog-average miss rateʣ
All
Large
MiddleSmall

CSP[1] ɹ
ఏҊख๏

20.2
13.1

23.2
10.8

5.8
6.3

15.1
20.0

·ͨɼDET ΧʔϒΛਤ 3 ʹࣔ͢ɽఏҊख๏͸ɼCSP ΑΓ DET Χʔϒ͕‫͍ͨۙʹ఺ݪ‬ΊɼߴੑೳͰ͋Δͱ͍͑Δɽ‫ݕ‬ग़ͨ͠าߦऀͷΞςϯγϣϯϚοϓΛਤ 4 ʹࣔ͢ɽਤͷΞςϯγϣϯϚοϓ͸ന࿮ͷάϦουʹରͯ͠ͷΞςϯγϣϯϚοϓͰ͋Γɼ੨৭ʹ͍ۙ΄Ͳऑ͘ɼԫ৭ʹ͍ۙ΄Ͳ‫͘ڧ‬൓Ԡ͍ͯ͠Δ͜ͱΛࣔ͢ɽ ਤ 4(a) ͔Β (d) ͷΞςϯγϣϯϚοϓ͔Βɼาߦऀͱਪଌ͞ΕΔର৅ͷମ෇ۙΛ‫͘ڧ‬஫ࢹ͍ͯ͠Δ͜ͱ͕Θ͔Δɽ·ͨɼਤ 4(a)ɼ(b) Ͱ͸ɼ֤εέʔϧຖʹରԠ͢Δେ͖͞ͷาߦऀʹରͯ͠‫͘ڧ‬஫ࢹ͓ͯ͠Γɼਤ 4(c)ɼ(d) ͷ Scale3 Ͱ͸ɼۙลͷาߦऀʹରͯ͠എ‫ܠ‬ΑΓ΋ऑ͘஫ࢹ͍ͯ͠Δɽ͜ͷ͜ͱ͔Βɼຊख๏͸ରԠ͠ͳ͍େ͖͞าߦऀͷ‫ݕޡ‬ग़Λ๷͗ͭͭɼาߦऀͷମʹ஫໨ͯ͠‫ݕ‬ग़͍ͯ͠Δ͜ͱ͕Θ͔ΔɽҰํͰɼਤ 4(d)ͷ Scale1ɼ2 ͷΞςϯγϣϯϚοϓ͸ɼಉ͡าߦऀʹରͯ͠‫͘ڧ‬஫ࢹ͍ͯ͠ΔͨΊɼ‫ݕ‬ग़݁Ռ͕ॏෳͯ͠͠·͍ͬͯ
ΔɽScale ͕ҟͳΔ͜ͱͰɼඇ࠷େ஋཈੍Ͱ࡟আ͖͠Εͣ‫ݕޡ‬ग़ͱͳͬͨɽͦͷͨΊɼఏҊख๏͕ CSP ΑΓ Middleʹ͓͍ͯ௿͍ਫ਼౓ͱͳͬͨɽ·ͨɼਤ 4(e) ͷԕํͷখ͍͞αΠζͷาߦऀʹର͢ΔΞςϯγϣϯϚοϓ͸ɼখ͞ͳ෺
ମͷ‫ݕ‬ग़͕Ͱ͖Δ Scale3 ʹ͓͍ͯ΋าߦऀʹରͯ͠஫ࢹ
Ͱ͖͍ͯͳ͍ɽ͜ͷ͜ͱ͔ΒɼఏҊख๏͕ CSP ΑΓ Smallʹ͓͍ͯਫ਼౓͕௿͍ͷ͸ɼҰ෦ͷখ͞ͳ෺ମʹରͯ͠Ξςϯγϣϯ͕௿͍ͨΊͰ͋Δͱߟ͑Δɽ

5. ͓ΘΓʹ
ຊ‫Ͱڀݚ‬͸ɼάϦουঢ়ʹ෼ׂ֤ͨ͠஍఺ʹର͢ΔΞςϯγϣϯϚοϓΛऔಘ͢Δ Grid-wise-attention ʹΑΔาߦऀ‫ݕ‬ग़ख๏ΛఏҊͨ͠ɽैདྷͷาߦऀ‫ݕ‬ग़ख๏ͱͷൺֱ࣮‫Ͱݧ‬͸ɼఏҊख๏͕શମͰ 7.1 ϙΠϯτਫ਼౓͕޲্ͨ͠
͜ͱ͔Βɼ஫ࢹྖҬΛߟྀͨ͠ॏΈ෇͚ʹΑͬͯߴਫ਼౓ͳ
าߦऀ‫ݕ‬ग़͕Ͱ͖Δ͜ͱΛࣔͨ͠ɽ·ͨɼ஫ࢹྖҬ͔Βาߦऀ‫ݕ‬ग़ͷ൑அࠜ‫ڌ‬Λࣔͨ͠ɽάϦουʹର͢ΔΞςϯγϣ
ϯϚοϓ͔Βຊख๏ʹΑΔาߦऀ‫ݕ‬ग़͸ɼରԠ͢Δେ͖͞ͷาߦऀͷମ͕൑அࠜ‫ͳͱڌ‬Δ͜ͱ͕Θ͔Γɼ‫ݕޡ‬ग़΍ະ‫ݕ‬ग़͕ൃੜͨ͠ཁҼΛࣔ͢͜ͱ͕Ͱ͖ͨɽࠓ‫ޙ‬͸ɼ֤άϦουʹҐஔͷ৘ใΛ༩͑Δ͜ͱͰΑΓର৅෺ମΛ஫ࢹ͢ΔΞ
ςϯγϣϯϚοϓͷ֫ಘͱෳ਺ΫϥεΛ‫ݕ‬ग़͢Δࡍͷ൑அࠜ‫ڌ‬ͷՄࢹԽΛߦ͏ɽࢀߟจ‫ݙ‬[1] W. Liu, et al., “Center and Scale Prediction: A
Box-free Approach for Pedestrian and Face Detection”,CVPR, 2019.
[2] A. Vaswani, et al., “Attention is all you need”,NIPS,
2017.

‫੷࣮ڀݚ‬[1] ໦ଜळే, ฏ઒ཌྷ, ࢁԼོٛ, ౻٢߂࿱, “Replution LossΛ༻͍ͨ Single Stage Headless ߏ଄ʹΑΔԕําߦऀ‫ݕ‬ग़”, ը૾ηϯγϯάٕज़‫ڀݚ‬ձ, 2020.

ਤ 3 : DET Χʔϒ

[2] ໦ଜळే, ௕಺३थ, ฏ઒ཌྷ, ࢁԼོٛ, ౻٢߂࿱, “Pixelwise-Attention ʹΑΔ Point ‫ܕ‬าߦऀ‫ݕ‬ग़ͷ൑அࠜ‫ڌ‬ͷՄࢹԽ”, ࣗಈंٕज़ձֶज़େձ, 2021.

Transformer ʹΑΔதؒಛ௃Λߟྀͨ͠ਓମ࢟੎ਪఆʹؔ͢Δ‫ڀݚ‬ࢦಋ‫ڭ‬तɿࢁԼོٛ

TP20012 খদ༔ే1. ͸͡Ίʹ
ਓମͷ࢟੎ਪఆ͸ɼ2 ࣍‫ݩ‬ը૾্ͷਓମͷؔઅҐஔΛਪఆ͢Δ໰୊Ͱ͋ΓɼϞʔγϣϯΩϟϓνϟ΍ಈ࡞ೝࣝ౳ʹ༻͍ΒΕΔɽ͜Ε·Ͱʹਓͷ࢟੎มԽʹରԠͨ͠ख๏͕ଟ਺
ఏҊ͞Ε͍ͯΔ΋ͷͷɼ͋Δ৚݅ԼͰ͸ؔઅҐஔΛਖ਼͘͠ଊ͑ΒΕͳ͍͜ͱ͕͋Δɽྫ͑͹ɼҰ෦ͷؔઅʹରͯ͠ΦΫϧʔδϣϯ͕ൃੜ͢Δγʔϯ΍ɼର৅ͷؔઅͱपғͷഎ‫͢ࣅྨ͕ܠ‬Δγʔϯ͕‫͛ڍ‬ΒΕΔɽͦͷ‫ݪ‬Ҽͱͯ͠ɼର৅
ͷؔઅ΍ؔ࿈͢Δ෦Ґ౳ͷಛ௃৘ใΛωοτϫʔΫ͕ਖ਼֬ʹ೺ѲͰ͖͍ͯͳ͍͔ΒͰ͋Δɽຊ‫Ͱڀݚ‬͸ɼTransformer͕ਓମͷؔઅʹର͢Δେ‫ہ‬తͳؔ܎ੑΛଊ͑΍͍͢ಛੑʹ
ண໨͠ɼTransformer ʹΑΔؔઅʹର͢Δଟ༷ͳؔ܎ੑΛଊ͑ͨதؒಛ௃Λߟྀͨ͠ਓମͷ 2 ࣍‫࢟ݩ‬੎ਪఆΛఏҊ͢Δɽ͜ΕΛϚϧνεέʔϧϞσϧͱͯ͠ߏங͢Δ͜ͱͰؔઅʹର͢Δ‫ॴہ‬తͳؔ܎ੑ͔Βؔઅಉ࢜ͷେ‫ہ‬తͳؔ
܎ੑ·ͰΛಉ࣌ʹߟྀͰ͖Δɽ·ͨɼ͋Δؔઅͱͦͷؔઅʹؔ࿈͢Δ෦Ґʹରͯ͠ΑΓண໨ͤ͞ΔͨΊʹ Attention
Convolution (Att-conv.) ΛఏҊ͢Δɽ

2. ਓମͷ 2 ࣍‫࢟ݩ‬੎ਪఆͷैདྷख๏2 ࣍‫࢟ݩ‬੎ਪఆ͸ Convolutional Pose Machines[1] ͷొ৔Ҏ߱ɼ֤ؔઅͷҐஔΛώʔτϚοϓͱͯ͠ग़ྗ͢Δख๏͕Ұൠతͱͳ͍ͬͯΔɽ͜ΕΒैདྷͷख๏͸ɼෳ਺ͷ৞ΈࠐΈ૚Λੵ૚͢Δ͜ͱͰɼؔઅपล͚ͩͰͳ͘ଞͷؔઅͱͷؔ܎ੑΛଊ͑Δ͜ͱ͕ग़དྷΔΑ͏ʹͳͬͨɽ·ͨɼTransformer Λ༻͍ͨ࢟੎ਪఆख๏΋ଟ͘ఏҊ͞Ε͍ͯΔɽ
TransPose[2] ͸৞ΈࠐΈॲཧͰը૾ͷಛ௃Λଊ͑ͨ‫ޙ‬ɼ࿈ଓͨ͠ Transformer Encoder ʹॱ࣍ೖྗ͢Δ͜ͱͰਓମ
ͷ 2 ࣍‫࢟ݩ‬੎ਪఆΛߦ͏ɽ͜ΕʹΑΓɼपลͷؔઅ͚ͩͰ
ͳ͘ɼΑΓ཭Εͨؔઅͱͷେ‫ہ‬తͳؔ܎ੑΛଊ͑Δ͜ͱ͕
Ͱ͖͍ͯΔɽ

ͱಉ༷ͷ࣍‫߹ʹ਺ݩ‬Θͤͨ‫ޙ‬ɼUpsampling ͢Δɽ4 ૚ͷ
Att-conv. Transformer Encoder ͷ֤ಛ௃ϚοϓΛࣜ (1)ͷΑ͏ʹ࿈݁͢Δɽ

F = F1 ⊕ F2 ⊕ F 3 ⊕ F4(1)

͜ΕʹΑΓ‫ॴہ‬తͳؔ܎ͱେ‫ہ‬తͳؔ܎ͷ྆ऀΛߟྀͨ͠ಛ௃ϚοϓͱͳΔɽͦͷ‫ޙ‬ɼ‫ٯ‬৞ΈࠐΈ૚ͱ৞ΈࠐΈ૚ʹΑΔॲཧΛࢪ֤ͯؔ͠અͷώʔτϚοϓΛग़ྗ͢Δɽ

3.2 Att-conv. Transformer Encoder
Att-conv. Transformer Encoder ͷߏ଄Λਤ 2 ʹࣔ͢ɽ
Att-conv. Transformer Encoder ͸ɼPosition Embedding ͷલʹɼ৞ΈࠐΈ૚Ͱಛ௃ϚοϓΛ Key ͱ Value ʹม‫͢׵‬Δ Att-conv. Λಋೖͨ͠ Transformer Encoder Ͱ͋Δɽ͜ΕʹΑΓɼEncoder ʹೖྗͨ͠ಛ௃Ϛοϓ͔ΒΑΓॏཁͳྖҬͷಛ௃Λଊ͑ͨಛ௃ϚοϓΛ֫ಘͰ͖Δɽ
Att-conv. ʹΑΔॲཧͷ‫ޙ‬ɼ2D Sine Position EmbeddingʹΑΔҐஔ৘ใΛ Query ͱ Key ʹຒΊࠐΈɼͦΕΒΛ
Multi-Head Attention ʹೖྗ͢ΔɽQuery ͱ Key ͷ಺ੵΛ Softmax ʹΑΓਖ਼‫ن‬Խ͢Δ͜ͱͰɼ஫ࢹྖҬͰ͋ΔΞ
ςϯγϣϯϚοϓΛ֫ಘ͢Δɽ͜ͷΞςϯγϣϯϚοϓͱ
Value ͷ಺ੵ F ! Λ‫ٻ‬ΊɼMulti-Head Attention ͷग़ྗͱ͢Δɽ

3. ఏҊख๏
࢟੎ਪఆʹ͓͍ͯɼΦΫϧʔδϣϯ΍എ‫ͱܠ‬ͷྨࣅੑʹରԠ͢ΔͨΊʹ͸ؔઅͷಛ௃Λଊ͑Δ͚ͩͰͳ͘ɼؔઅ
ؒͷؔ܎ੑ΍͞ΒʹͦΕΒʹؔ࿈͢Δ෦Ґͱͷؔ܎ੑΛ
ଊ͑Δ͜ͱ͕ॏཁͰ͋Δɽͦ͜ͰɼTransformer ͕ਓମͷؔઅʹର͢Δେ‫ہ‬తͳؔ܎ੑΛଊ͑΍͍͢ಛੑʹண໨͠ɼTransformer ͷதؒಛ௃Λར༻ͨ͠ਓମͷ 2 ࣍‫࢟ݩ‬੎ਪఆΛఏҊ͢Δɽຊख๏Ͱ͸ɼೖྗαΠζ͕ҟͳΔ Transformer
Encoder ͔Βग़ྗ͞Εͨதؒಛ௃Λू໿͢Δ͜ͱͰɼؔઅ
ʹର͢Δ‫ॴہ‬తͳؔ܎͔Βؔઅಉ࢜ͷେ‫ہ‬తͳؔ܎ͷ྆ऀΛߟྀͨ͠ਪఆ͕Ͱ͖Δɽ·ͨɼର৅ͷؔઅͱͦͷؔ
࿈͢Δ෦Ґʹରͯ͠ΑΓண໨͢Δಛ௃Λ֫ಘ͢ΔͨΊʹ
Attention Convolution (Att-conv.) Λಋೖͨ͠ Att-conv.
Transformer Encoder ΛఏҊ͢Δɽਤ 2: Att-conv. Transformer Encoder ͷߏ଄

3.3 ֶशํ๏
ఏҊख๏Ͱ͸ɼωοτϫʔΫ࠷‫ޙ‬ͷ৞ΈࠐΈ૚Ͱ֫ಘ͠
ͨਪఆώʔτϚοϓͱਖ਼ղώʔτϚοϓؒͷ‫ࠩޡ‬Λ L2 LossʹΑΓࢉग़͢ΔɽL2 Loss Λࣜ (2) ʹࣔ͢ɽ͜͜Ͱɼŷi ͸ਪఆώʔτϚοϓͰ͋Γɼyi ͸ਖ਼ղώʔτϚοϓͰ͋Δɽ
n

3.1 ωοτϫʔΫߏ଄ͱதؒಛ௃ͷར༻ఏҊख๏ͷωοτϫʔΫߏ଄Λਤ 1 ʹࣔ͢ɽఏҊख๏͸ɼࣄલֶशࡁΈͷ ResNet Ͱߏ੒͞ΕͨόοΫϘʔϯʹը૾Λೖྗͯ͠ɼਓମʹର͢Δಛ௃ϚοϓΛ‫ٻ‬ΊΔɽͦͯ͠ɼಛ௃ϚοϓΛฏୱԽͨ͠‫ޙ‬ɼAtt-conv. Transformer
Encoder 1ɼ2 ʹͯਓମͷ‫ॴہ‬తͳಛ௃Λଊ͑Δɽͦͷ‫ޙ‬ɼOverlapping Patched Embedding[3] ʹΑΓ৞ΈࠐΈ૚ʹͯಛ௃ϚοϓΛॖখ͠ɼAtt-conv. Transformer Encoder
3 ʹೖྗ͢Δɽ͜͜Ͱ͸ɼؔઅʹର͢Δେ‫ہ‬తͳؔ܎ੑΛ
ଊ͑Δɽ͜ͷॲཧΛ 2 ճ‫܁‬Γฦ͠ɼॖখͨ͠ಛ௃ϚοϓͷαΠζΛ֦େ͢ΔͨΊʹɼ1 ͭͷ৞ΈࠐΈ૚ʹΑͬͯ࣍‫਺ݩ‬Λॖখલͷ 2 ૚ͷ Att-conv. Transformer EncoderL=1!
(ŷi − yi )2
n i=1

(2)

ఏҊख๏͸ΤϙοΫ਺Λ 260ɼόοναΠζΛ 32 ͱͯ͠
ֶश͢Δɽ࠷దԽख๏ʹ͸ɼAdam Λ༻͍Δɽ·ͨɼॳ‫ظ‬ͷֶश཰͸ 0.0001 Ͱ͋Γɼ100ɼ150ɼ200ɼ250 ΤϙοΫͰ‫ݮ‬ਰͤ͞ɼ࠷ऴతʹֶश཰ 0.00001 Ͱֶश͢Δɽ

4. ධՁ࣮‫ݧ‬ఏҊख๏ͷ༗ޮੑΛ֬ೝ͢ΔͨΊʹɼTransformer Λ༻͍ͨ࢟੎ਪఆͷैདྷख๏Ͱ͋Δ TransPose ͱਫ਼౓ൺֱΛߦ͏ɽσʔληοτʹ͸ɼMS COCO σʔληοτΛ༻͍Δɽ
ධՁํ๏͸ɼਪఆ࢟੎ͱਖ਼ղ࢟੎ͷద߹཰Ͱ͋Δ Average
Precision (AP) Λ༻͍Δɽ

4.1 ैདྷख๏ͱͷਪఆਫ਼౓ͷൺֱਤ 1: ఏҊख๏ͷωοτϫʔΫߏ଄ैདྷख๏ͱͷධՁൺֱͰ͸ɼAP ͷଞʹωοτϫʔΫͷύϥϝʔλ਺ͱ‫͋Ͱྔࢉܭ‬Δ FLOPs (Floating point
operations) Λ༻͍ͯൺֱΛߦ͏ɽैདྷख๏ͱͷධՁൺֱ݁ՌΛද 1 ʹࣔ͢ɽఏҊख๏͸ɼTransformer Λ΋ͱʹͨ͠ TransPose Λϕʔεʹ͍ͯ͠ΔͨΊɼCNN Λ༻͍ͨैਤ 3: ֤ Encoder ͷΞςϯγϣϯϚοϓͷՄࢹԽྫਤ 4: ਪఆώʔτϚοϓͷՄࢹԽྫද 1: ैདྷख๏ͱͷධՁൺֱද 2: Att-conv. ͱதؒಛ௃ͷར༻ʹΑΔਪఆਫ਼౓

Method# Params

GFLOPsAP

Hourglass

25.1M

14.3

66.9

CPN

27.0M

6.2

68.6

SimpleBaseline

68.6M

15.7

72.0

HRNet-W32

28.5M

7.1

73.4

HRNet-W48

63.6M

14.6

75.1

TransPose R-A4

13.4M

8.8

74.2

ఏҊख๏

24.0M

11.2

75.5

དྷख๏ͱൺ΂ɼύϥϝʔλ਺͕গͳ͍ɽ·ͨɼैདྷख๏ͱ
ൺ΂ɼAP ͕޲্͍ͯ͠Δ͜ͱ͕֬ೝͰ͖Δɽ͔͠͠ͳ͕
ΒɼεέʔϧͷॖখͳͲͷͨΊʹ৞ΈࠐΈ૚Λ௥Ճ͍ͯ͠
ΔͨΊɼTransPose ͱൺֱ͢Δͱύϥϝʔλ਺͕૿Ճͯ͠
͍Δɽ·ͨɼ‫͋Ͱྔࢉܭ‬Δ FLOPs ΋ಉ༷ʹ૿Ճ͍ͯ͠Δ
͜ͱ͕֬ೝͰ͖Δɽ

4.2 ΞςϯγϣϯϚοϓ͓ΑͼώʔτϚοϓͷՄࢹԽ

Att-conv.

AP

AP50

AP75

74.2

92.5

81.5

!

74.5
75.2

92.5
92.5

81.5
82.6

!

75.5

92.6

82.7

தؒಛ௃

!
!

AP ͕ 0.3 pt ޲্ͨ͠ɽ·ͨɼதؒಛ௃Λ࿈݁ͯ͠ར༻͠
ͨωοτϫʔΫͰ͸ɼAP ͕ 1.0 pt ޲্ͨ͠ɽ͜ͷ͜ͱ͔Βɼதؒಛ௃Λར༻͢Δํ͕ Att-conv. Λಋೖ͢Δ͜ͱΑΓ΋Ө‫͕ڹ‬େ͖͍͜ͱ͕ߟ͑ΒΕΔɽ·ͨɼ྆ํͷख๏Λ૊Έ߹ΘͤͨωοτϫʔΫͰ͸ AP ͕ 1.3 pt ޲্͢ΔͨΊɼ྆ํͷߏ଄Λಋೖ͢Δ͜ͱ͕࠷΋༗ޮͰ͋Δͱ֬ೝͰ͖Δɽ·ͨɼதؒಛ௃Λ࿈݁ͯ͠ར༻ͨ͠ωοτϫʔΫͱ྆ํͷख๏Λ૊Έ߹ΘͤͨωοτϫʔΫ͸ɼAP50 Ͱ͸ਫ਼౓ͷมԽ͸‫ࠩޡ‬ఔ౓͕ͩɼAP75 ͓Αͼ AP ʹ͓͍ͯਫ਼౓
͕޲্͍ͯ͠ΔͨΊɼਖ਼ղؔઅҐஔʹΑΓ͍ۙͮͨਪఆ͕Ͱ͖͍ͯΔͱߟ͑ΒΕΔɽ

֤ Encoder ͷΞςϯγϣϯϚοϓͷՄࢹԽྫΛਤ 3 ʹࣔ͢ɽ1 ૚໨ͱ 2 ૚໨ͷΞςϯγϣϯϚοϓͰ͸ɼTransPose͕ਓମͷपลશମʹରͯ͠Ξςϯγϣϯ͕ൃੜ͍ͯ͠Δͷʹର͠ɼఏҊख๏Ͱ͸ΑΓਓମͷ෦Ґ΍ؔઅʹΞςϯγϣ
ϯ͕ൃੜ͍ͯ͠Δ͜ͱ͕֬ೝͰ͖ΔɽఏҊख๏ͷ 3 ૚໨ͷΞςϯγϣϯϚοϓͰ͸ɼਓମͷؔઅʹΞςϯγϣϯ͕ൃ
ੜ͍ͯ͠Δ͜ͱ͕֬ೝͰ͖Δɽ4 ૚໨ͷ TransPose ͷΞςϯγϣϯϚοϓͰ͸ɼ෦Ґ΍ؔઅͷ·ΘΓʹ‫ॴہ‬తʹΞςϯγϣϯ͕ൃੜ͍ͯ͠ΔɽҰํɼఏҊख๏ͷΞςϯγϣϯ
Ϛοϓ͸ɼؔઅҐஔͷपลͷଞʹɼਓ෺͕།Ұ஍໘ʹ઀͠
͍ͯΔࠨ଍ʹΞςϯγϣϯ͕ൃੜ͍ͯ͠Δ͜ͱ͕֬ೝͰ͖
Δɽ͜ΕΒͷ͜ͱ͔ΒఏҊख๏Ͱ͸ɼਓମͷؔઅʹର͢Δ
ਪఆʹ͓͍ͯॏཁͳ෦෼Λը૾த͔Βଊ͑Δ͜ͱ͕Ͱ͖ͨ
ͱ͍͑ΔɽਪఆώʔτϚοϓͷՄࢹԽྫΛਤ 4 ʹࣔ͢ɽ੺࿮ͰғΜͩࠨ଍टͷਪఆ݁Ռʹ஫໨͢ΔͱɼTransPose Ͱ͸ɼෳ਺ՕॴʹϐʔΫ͕ൃੜ͍ͯ͠ΔɽͦΕʹର͠ɼఏҊख๏Ͱ͸ϐʔΫ͕ҰέॴͰ͋Δ͜ͱ͕֬ೝͰ͖ɼఏҊख๏ʹΑΔվ
ળΛఆੑతʹ֬ೝͰ͖Δɽ

5. ͓ΘΓʹ

4.3 Att-conv. ͱதؒಛ௃ͷ༗ແʹΑΔਪఆਫ਼౓‫੷ۀڀݚ‬

Att-conv. Transformer Encoder ͷಋೖͱதؒಛ௃Λར༻ͨ͠ωοτϫʔΫߏ଄͕ɼ࢟੎ਪఆʹ͓͍ͯ༗ޮͰ͋
Δ͔Λ࣮‫ʹݧ‬ΑΓௐࠪ͢ΔɽAtt-conv. ͱதؒಛ௃ͷར༻ํ๏ʹΑΔਪఆਫ਼౓Λද 2 ʹࣔ͢ɽAtt-conv. Λಋೖͨ͠ωοτϫʔΫ͸ɼಋೖ͍ͯ͠ͳ͍ωοτϫʔΫͱൺֱͯ͠

[1] খদ༔ే ౳, “2 ࣍‫Ͱۭؒݩ‬ͷԞߦ͖ϕΫτϧ৔Λߟྀͨ͠ਓ
෺ͷ 3 ࣍‫࢟ݩ‬੎ਪఆ”, ୈ 27 ճ ըηϯγϯάγϯϙδ΢Ϝ,2021.
(ଞ 1 ݅)ຊ‫Ͱڀݚ‬͸ɼTransformer ʹΑΔதؒಛ௃Λߟྀͨ͠ਓମͷ 2 ࣍‫࢟ݩ‬੎ਪఆΛఏҊͨ͠ɽ·ͨɼAtt-conv. ΛTransformer Encoder ʹಋೖ͢Δ͜ͱͰɼର৅ͷؔઅͱͦͷؔ࿈͢Δ෦Ґʹରͯ͠ΑΓண໨͢Δಛ௃Λ֫ಘ͢Δ͜ͱ
ΛՄೳͱͨ͠ɽධՁ࣮‫Ͱݧ‬͸ɼఏҊख๏͸ैདྷख๏Λ௒͑
Δਪఆਫ਼౓Λୡ੒͠ɼ·ͨɼಋೖ֤ͨ͠ߏ଄ʹ͓͍ͯ΋ಋೖલʹൺ΂ਫ਼౓͕޲্͓ͯ͠ΓɼఏҊख๏ͷ༗ޮੑΛ֬ೝͨ͠ɽࠓ‫ޙ‬ͷ՝୊ͱͯ͠ɼதؒಛ௃ʹର͢Δଛࣦͷద༻ͳͲ͕‫͛ڍ‬ΒΕΔɽ

ࢀߟจ‫ݙ‬[1] S. Wei, et al., “Convolutional Pose Machines”, CVPR,
2016.
[2] S. Yan, et al., “Transpose: Towards explainable humanpose estimation by transformer”, arXiv, 2020.
[3] W. wang, et al., “Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions”, ICCV, 2021.

Self-Attention Λ༻͍ͨ఺‫܈‬σʔλ͔ΒͷηϚϯςΟοΫηάϝϯςʔγϣϯʹؔ͢Δ‫ڀݚ‬ࢦಋ‫ڭ‬तɿࢁԼོٛ
TP20015 ླ໦‫و‬େ
1. ͸͡Ίʹ
શํҐ LiDAR ͔Βऔಘͨ͠ 3 ࣍‫܈఺ݩ‬σʔλʹରͯ͠ηϚϯςΟοΫηάϝϯςʔγϣϯΛߦ͏͜ͱͰɼಓ࿏౳ͷ੩త෺ମɼं྆΍ਓ౳ͷಈత෺ମΛ೺ѲͰ͖Δɽ͜ΕʹΑΓɼࣗಈӡసͰඞཁͳं྆पลͷશํҐʹ౉Δ‫ڥ؀‬Λೝࣝ͢Δ͜ͱ͕Ͱ͖Δɽ͜ͷͱ͖ɼશํҐ LiDAR ͔Βऔಘͨ͠
3 ࣍‫܈఺ݩ‬σʔλͷ֤఺ʹରͯ͠ॲཧΛࢪ͢ͱ๲େͳ‫ྔࢉܭ‬͕ඞཁͱͳΔɽͦͷͨΊɼ3 ࣍‫܈఺ݩ‬σʔλΛٙࣅը૾ʹม‫͠׵‬ɼηϚϯςΟοΫηάϝϯςʔγϣϯΛߦ͏͜ͱͰߴ଎
Խͨ͠ख๏͕ఏҊ͞Ε͍ͯΔɽैདྷख๏͸ 3 ࣍‫܈఺ݩ‬σʔλΛٙࣅը૾΁ม‫͢׵‬Δࡍʹଟ͘ͷ఺‫܈‬σʔλ͕ܽམ͢Δ
͜ͱ͔Βɼখ෺ମ (ըૉ਺͕গͳ͍ਓ΍ඪࣝΫϥε) ͷࣝผਫ਼౓͕௿Լ͢Δ܏޲ʹ͋Δɽຊ‫Ͱڀݚ‬͸ɼScan-UnfoldingʹΑΔϓϩδΣΫγϣϯख๏͓Αͼ Self-Attention Blockʹ‫ ͍ͨͮج‬1Dimentional Self-Attention Block(1D-SAB)Λಋೖͨ͠ηϚϯςΟοΫηάϝϯςʔγϣϯΛఏҊ͢ΔɽఏҊख๏ʹΑΓɼٖࣅը૾ม‫࣌׵‬ͷ఺‫܈‬σʔλͷܽམΛ཈੍্ͨ͠Ͱɼ఺‫ؒ܈‬ͷؔ܎ੑΛߟྀ͢Δ͜ͱͰߴੑೳԽ͕
‫ظ‬଴Ͱ͖Δɽ

2. ؔ࿈‫ڀݚ‬
3 ࣍‫܈఺ݩ‬σʔλΛٖࣅը૾ʹม‫ͯ͠׵‬ηϚϯςΟοΫη
άϝϯςʔγϣϯΛߦ͏ख๏ͱͯ͠ɼSalsaNext[1] ͕͋ΔɽSalsaNext ͸ɼSalsaNet ΛϕʔεʹɼContext Moduleɼ
Pixel-Shuffle Layer Λಋೖ͠ɼߴ͍ࣝผਫ਼౓ͱϦΞϧλΠ
ϜੑΛ࣮‫͍ͯ͠ݱ‬ΔɽContext Module ͸ɼάϩʔόϧͳ৘ใΛऔಘ͢Δ͜ͱͰɼࣝผਫ਼౓ͷ޲্ʹߩ‫͍ͯ͠ݙ‬Δɽ
·ͨɼPixel-Shuffle Layer ͸ɼग़ྗ͞Εͨಛ௃ϚοϓΛฒͼସ͑ͯΞοϓαϯϓϦϯάΛ͢ΔͨΊॏΈΛ࣋ͨͣɼ‫ܭ‬ࢉίετΛ࡟‫͖Ͱݮ‬Δɽ͞ΒʹɼSalsaNext Ͱ͸৞ΈࠐΈॲཧʹ Dilated Convolution Λ༻͍Δ͜ͱͰɼReceptive
field Λ֦େ͢Δࡍͷύϥϝʔλ਺ͷ૿ՃΛ཈͑ɼϦΞϧλΠϜੑΛ֫ಘ͍ͯ͠ΔɽSalsaNext ͸ 3 ࣍‫܈఺ݩ‬σʔλΛٙࣅը૾ʹม‫͢׵‬Δࡍʹɼଟ͘ͷσʔλ͕ܽམ͢Δ͜ͱ͔
Βɼখ෺ମͷࣝผਫ਼౓͕௿͍ͱ͍͏໰୊఺͕͋Δɽ

3. ఏҊख๏
શํҐ LiDAR ͔Βऔಘͨ͠఺‫܈‬σʔλ͸ɼԣํ޲ʹີͰ
͋Δɽ͜ͷಛੑʹΑΓɼม‫ޙ׵‬ͷٖࣅը૾΋ɼԣํ޲ʹର͢Δ఺‫܈‬σʔλͷܽམ͕গͳ͍ɽͦ͜Ͱɼຊ‫Ͱڀݚ‬͸ɼখ෺ମ΁ͷࣝผਫ਼౓Λ޲্ͤ͞ΔͨΊɼԣํ޲ʹͷΈண໨ͨ͠ॲཧ
Λߦ͏ 1Dimentional Self-Attention Block(1D-SAB) Λಋೖͨ͠ 1D-Salsa Self-Attention Network(1D-SalsaSAN)
ΛఏҊ͢Δɽ

1D wave data

projection(Scan-Unfolding)

3D point cloud2D pseudo image
2048×64×5

1Dimensional Self-Attention Block (1D-SAB)

Skip Connection⨁ Element-wise Additioncclogits2048×64×20

k1d1
dropout

Block Vdropoutc

2048×64×32

512×32×64

256×16×128

Block IV

k3d1+bn
k3d2+bn
k2d2+bn

64×4×256

128×8×256

dropout

Block III
pixel-shuffle

64×4×256

128×8×256c ⨁Block II
pooling

k1d1

k3d1+bn
k3d2+bn
k2d2+bn

Block I

dropout

512×32×64

256×16×128

2048×64×5

Concatenation

ਤ 1: ॲཧͷྲྀΕsegmentation result(a) ैདྷख๏(b) Scan-Unfoldingਤ 2: 3 ࣍‫܈఺ݩ‬σʔλͷٖࣅը૾΁ͷม‫ྫ׵‬⨀

Pointwise Conv.
1×1×#

1×1×#′

Bn/ReLU⨁： 近傍点1： 注⽬点

Self Attention Map： 近傍点2

1×1×#′! " #! ：マッピング関数

,&
1×1×#

$, &： 学習可能な関数

1×1×#

⼊⼒

⨀

Pointwise Conv.
#
%
1×%×#

1×1×#

1×1×#′! " #1×%×#′

Pointwise Conv.
Bn/ReLU,&
1×1×#" # ：! " − $ "Generated SAM

1×1×#

⨁

特徴マップ出⼒SAM
1×%×#

スキップ接続

ਤ 3: 1D-SAB ͷߏ଄

3.1. ॲཧͷྲྀΕ
ਤ 1 ʹॲཧͷྲྀΕΛࣔ͢ɽ·ͣɼ3 ࣍‫܈఺ݩ‬σʔλΛϓϩδΣΫγϣϯʹΑΓٙࣅը૾΁ม‫͢׵‬Δɽ࣍ʹɼٙࣅը૾΁ม‫ͨ͠׵‬σʔλΛ 1D-SAB ʹೖྗ͠ɼ఺‫ؒ܈‬ͷؔ܎ੑΛߟྀͨ͠ॲཧΛߦ͏ɽೖྗνϟϯωϧ͸ɼ(x, y, z) ࠲ඪɼ൓ࣹ‫౓ڧ‬ɼ‫཭ڑ‬஋ͷ 5 νϟϯωϧͰ͋Δɽͦͯ͠ɼ1D-SAB
͔Βग़ྗͨ͠ಛ௃ϚοϓΛɼContext Module ΛऔΓআ͍
ͨ SalsaNext ʹೖྗ͠ɼ৞ΈࠐΈॲཧ౳Λߦ͏ɽͦͷ‫ޙ‬ɼ
softmax ؔ਺ʹͯΫϥε֬཰Λࢉग़͠ɼηάϝϯςʔγϣϯΛߦ͏ɽ

3.2. લॲཧɿ3 ࣍‫܈఺ݩ‬σʔλͷٖࣅը૾΁ͷม‫׵‬ఏҊख๏Ͱ͸ɼϓϩδΣΫγϣϯख๏ʹ Scan-Unfolding[2]Λ༻͍Δɽਤ 2 ʹɼैདྷख๏ͱ Scan-Unfolding ʹΑΔม‫ྫ׵‬Λࣔ͢ɽਤ 2(a) தͷന࿮ʹࣔ͢Α͏ʹɼैདྷख๏Ͱ͸
3 ࣍‫܈఺ݩ‬σʔλΛٖࣅը૾΁ม‫͢׵‬Δࡍɼଟ͘ͷσʔλ͕ܽམ͢ΔɽScan-Unfolding Ͱ͸ɼ·ͣ 3 ࣍‫܈఺ݩ‬σʔλͷ֤఺ͷٖࣅը૾্ʹ͓͚Δԣํ޲ͷ࠲ඪͷΈΛࢉग़͢Δɽ࣍ʹɼԣํ޲ͷ֤ྡ઀఺ؒͷ෯Λࢉग़͠ɼҰఆ෯Ҏ্ͷ৔߹ɼϨʔβ ID ͕มΘͬͨͱΈͳ͠ɼॎํ޲ͷ࠲ඪΛԼ΁Ҡಈ͢Δɽ͜ͷΑ͏ʹϓϩδΣΫγϣϯ͢Δ͜ͱͰɼٙࣅը૾্ͷ఺ͷॏͳΓΛ௿‫͠ݮ‬ɼܽམΛ཈੍Ͱ͖Δɽ

3.3.1D Self-Attention Block (1D-SAB)ਤ 3 ʹ 1D-SAB ͷߏ଄Λࣔ͢ɽਤ 1 ͷ 2d pseudo imageதʹ੺͍೾ઢͰࣔ͢Α͏ʹɼٙࣅը૾ΛϨʔβ ID ͝ͱʹ
1 ࣍‫ݩ‬೾‫ܗ‬σʔλͱΈͳ͠ɼ1D-SAB ʹೖྗ͢ΔɽೖྗαΠζ͸ɼ1 × w(ٖࣅը૾ͷԣ෯) × c(νϟϯωϧ਺ɿ5) Ͱ͋Δɽೖྗͨ͠σʔλ͸ 1 ఺ͣͭॲཧ͠ɼରԠ͢Δ఺ʹର͢
Δ Self-Attention Λࢉग़͢Δɽਤ 3 ͷ੺৭ͷ஋Λॲཧͷ஫໨఺ͱͨ͠ͱ͖ɼ྘৭͕ۙ๣఺ 1ɼ੨৭͕ۙ๣఺ 2 ͱͳΔɽ֤ۙ๣఺ʹରͯ͠ઢ‫ܗ‬ม‫׵‬ͷͨΊɼPointwise ConvolutionॲཧΛߦ͏ɽ·ͨɼॏΈܾఆͷͨΊʹ஫໨఺ͱۙ๣఺ΛֶशՄೳͳؔ਺ ϕ, ψ ΁ೖྗ͢Δɽͦͯ͠ɼϕ, ψ Λ༻͍ͯؔ܎ؔ਺ δ Λ‫ٻ‬ΊΔɽؔ܎ؔ਺ δ ͷఆٛࣜΛࣜ (1) ʹࣔ͢ɽ͜͜Ͱɼx ͕஫໨఺ɼxt ͕ۙ๣఺Λࣔ͢ɽδ(ϕ(x), ψ(xt )) = ϕ(x) − ψ(xt )(1)ͦͷ‫ޙ‬ɼϚοϐϯάؔ਺ γ ʹΑͬͯνϟϯωϧ਺Λ 1 ͭ໨ͷग़ྗͱ߹ΘͤΔɽͦͯ͠ɼ্ड़ͷಛ௃ྔͱͷཁૉੵΛࢉ
ग़͢Δɽ͜ͷॲཧΛۙ๣఺෼ߦ͍ɼͦΕΒΛ૯࿨͢Δ͜ͱ
Ͱ Self-Attention Map(SAM) Λੜ੒͢Δɽੜ੒ͨ͠ SAM
͸ɼPointwise Convolution ॲཧʹΑΓೖྗνϟϯωϧ਺ͱಉ͡νϟϯωϧ਺ʹ͢Δɽ͜ͷग़ྗʹɼεΩοϓ‫ͱߏػ‬ͯ͠ೖྗσʔλΛՃࢉ͠ɼ࠷ऴग़ྗͱ͢Δɽ1D-SAB Λ༻͍Δ͜ͱͰɼ఺‫ؒ܈‬ͷॏཁͳҐஔʹେ͖ͳॏΈΛ༩͑ɼ఺‫ؒ܈‬ͷؔ܎ੑΛߟྀ͢Δ͜ͱ͕Ͱ͖Δɽ

(b) SalsaNext(a) Ground truth(c) Proposed

traffic sign

mean-IoU

17.3

1.2

84.9

28.4

54.7

4.6

61.5

29.2

59.6

25.5

54.7

11.2

36.3

30.8

20.2

24.3

2.9

88.5

42.4

65.5

18.7

73.8

41.0

68.5

36.9

58.9

12.9

41.0

39.6

RangeNet++

91.4

25.7

34.4

25.7

23.0

38.3

38.8

4.8

91.8

65.0

75.2

27.8

87.4

58.6

80.5

55.1

64.6

47.9

55.9

52.5

SalsaNext[1]

93.2

51.9

39.3

31.7

29.3

60.3

57.8

8.9

91.7

61.3

75.7

29.0

89.1

61.8

83.2

64.1

67.6

53.8

61.4

58.5

Proposed

93.2

52.2

39.8

41.4

28.8

62.1

63.6

23.3

91.2

60.0

75.1

28.5

88.1

60.0

80.8

63.6

64.6

52.9

63.1

59.6

ͱͷਫ਼౓Λൺֱ͢Δͱɼ19 Ϋϥεத 8 Ϋϥεͷ IoU ͕࠷΋ߴ͘ͳͬͨɽಛʹɼbicyclist ΍ motorcyclistɼtraffic signͱ͍ͬͨখ෺ମͷਫ਼౓͕େ͖͘޲্ͨ͠ɽ
ఆੑతධՁ: ਤ 4 ʹɼSalsaNext ͱఏҊख๏ʹΑΔηά
ϝϯςʔγϣϯ݁ՌྫΛࣔ͢ɽਤதͷന࿮ͷ෦෼͸ traffic
sign Ϋϥεͷਖ਼ղྫ͓Αͼࣝผ݁ՌͰ͋ΔɽSalsaNext ͸ɼtraffic sign ΛҰ෦Λ fence ͱ‫ࣝޡ‬ผͨ͠ɽҰํɼఏҊख๏͸ traffic sign ͱਖ਼ࣝ͘͠ผͰ͖ͨɽ

truck

65

RangeNet++

55

mIoU [%]

Proposed

SalsaNext

60

50

45

SqueezeSegV2

40

35SqueezeSeg

30
40

50

60

70

80

90

100

pole

sidewalk

16.5

15.9

terrainparking

4.8

14.5

trunk

road

4.1

22.6

vegetation

motorcyclist

5.1

21.0

fence

bicyclist

18.1

82.7

building

person68.3

SqueezeSegV2othe-vehicle

bicycleSqueezeSeg

motorcycle

Approachcarother-groundਤ 4: ηάϝϯςʔγϣϯ݁Ռྫද 1: ैདྷख๏ͱఏҊख๏ͷਫ਼౓ [%]

110

120

130

processing speed [Hz]

ਤ 5: ֤ख๏ʹ͓͚Δਫ਼౓ͱॲཧ଎౓ͷؔ܎άϥϑ

4. ධՁ࣮‫ݧ‬ධՁ࣮‫ʹݧ‬ΑΓɼఏҊख๏ͷ༗ޮੑΛ‫͢ূݕ‬Δɽ

4.1. ࣮‫֓ݧ‬ཁຊ࣮‫Ͱݧ‬͸ɼैདྷख๏ͱͷਫ਼౓ൺֱ͓Αͼॲཧ଎౓ͷൺֱΛߦ͏ɽൺֱ͢Δैདྷख๏͸ɼఏҊख๏ͱಉ༷ʹٙ
ࣅը૾ϕʔεͷख๏Ͱ͋Δ SqueezeSegɼSqueezesegV2ɼ
RangeNet++ɼSalsaNext Ͱ͋Δɽ1D-SAB ͷ༗ޮੑͷ‫ݕ‬ূͷͨΊɼSalsaNext ͷϓϩδΣΫγϣϯʹ͸ Scan-UnfoldingΛ༻͍Δɽֶशઃఆͱͯ͠ɼֶशճ਺Λ 300 epochɼόοναΠζΛ 24 ͱ͢Δɽଛࣦؔ਺ʹ͸ΫϩεΤϯτϩϐʔ‫ࠩޡ‬ɼ࠷దԽख๏ʹ͸ MomentumSGD Λ࢖༻͠ɼॳ‫ֶظ‬श཰͸ 0.01 ͱ͢Δɽֶश࣌͸ 1 epoch ͝ͱʹ 0.01 ‫ݮ‬ਰͤ͞ΔɽධՁࢦඪʹ͸ɼ֤఺‫ʹ܈‬ରͯ͠ηάϝϯςʔγϣϯͨ݁͠Ռͱ֤఺‫܈‬ͷਖ਼ղϥϕϧͷॏͳΓ཰ (IoU) Λ༻͍Δɽ

4.2. σʔληοτσʔληοτʹ͸ɼશͯͷ఺‫܈‬σʔλʹରͯ͠Ξϊςʔγϣϯ͕ࢪ͞Ε࣮ͨ‫ڥ؀‬σʔληοτͰ͋Δ SemanticKITTI[3]Λ༻͍ΔɽSemanticKITTI ͸ɼ22 γʔϯ 43,000 ϑϨʔϜͰߏ੒͞ΕΔɽ͜ͷ͏ͪɼγʔϯ 00 ͔Βγʔϯ 10 ·Ͱͷ 23,201 ϑϨʔϜΛֶश༻ɼγʔϯ 11 ͔Βγʔϯ 21 ·Ͱͷ 20,351 ϑϨʔϜΛධՁ༻ͱ͢Δɽֶश༻γʔϯͷ͏ͪɼγʔϯ 08 ͷ 4,071 ϑϨʔϜΛ‫༻ূݕ‬σʔλͱֶͯ͠शΛߦ͏ɽຊ‫Ͱڀݚ‬ͷࣝผର৅͸ 19 ΫϥεͰ͋Δɽ

4.3. ࣮‫݁ݧ‬Ռ
ද 1 ʹैདྷख๏ͱఏҊख๏ͷਫ਼౓ɼਤ 4 ʹ SalsaNext ͱఏҊख๏ͷηάϝϯςʔγϣϯ݁ՌྫΛࣔ͢ɽ
1D-SAB ͷ༗ޮੑͷධՁ: ද 1 ͔ΒɼఏҊख๏͸ SalsaNext ͱൺֱͯ͠ mIoU ͕ 1.1pt ޲্ͨ͜͠ͱ͕Θ͔Δɽ
·ͨɼΫϥε͝ͱͷਫ਼౓Λൺֱ͢Δͱɼ19 Ϋϥεத 7 Ϋϥεͷ IoU ͕޲্ͨ͠ɽதͰ΋ɼখ෺ମͰ͋Δ bicycleɼmotorcycleɼpersonɼbicyclistɼmotorcyclistɼtraffic sign ͷ
IoU ͕޲্ͨ͠ɽಛʹɼmotorcyclist ͷ IoU ͸ 14.4pt ͱ࠷େͱͳΔਫ਼౓޲্Λ֬ೝͨ͠ɽ
ैདྷख๏ͱͷਫ਼౓ൺֱ: ද 1 ͔ΒɼఏҊख๏͸ैདྷख๏ͱ
ൺֱͯ͠ mIoU ͕࠷΋ߴ͍͜ͱ͕Θ͔Δɽ·ͨɼΫϥε͝

4.4. ॲཧ଎౓ͷൺֱ
ਤ 5 ʹɼ֤ख๏ʹ͓͚Δਫ਼౓ͱॲཧ଎౓ͷؔ܎άϥϑΛࣔ͢ɽॲཧ଎౓ͷ‫ܭ‬ଌ͸ɼNVIDIA Quadro RTX A6000
Λ༻͍ͯߦͬͨɽਤ 5 ʹࣔ͢Α͏ʹɼఏҊख๏ͷॲཧ଎౓͸
77.6Hz ͱͳΓɼSalsaNext ΑΓ 18.0Hz ߴ଎Ͱ͋ͬͨɽ͜
Ε͸ɼSalsaNext ͔ΒऔΓআ͍ͨ Context Module ΑΓ΋ɼ
1D-SAB ͷ‫ॲࢉܭ‬ཧ͕গͳ͍͔ΒͰ͋ΔɽશํҐ LiDAR͸௨ৗ 5Hzʙ20Hz Ͱճస͠ͳ͕ΒσʔλΛऔಘ͢ΔͨΊɼఏҊख๏͸ϦΞϧλΠϜੑΛ֬อͰ͖ͨɽ·ͨɼॲཧ଎౓
͕࠷΋ߴ଎ͳ SqueezeSeg ͸ɼද 1 ͔Β mIoU ͕࠷΋௿͍ɽ
SqueezeSegV2 ΋ಉ༷ʹɼॲཧ଎౓͸ఏҊख๏ΑΓߴ଎ͩ
͕ɼmIoU ͸ఏҊख๏ΑΓ 20.0pt ௿͍ɽࣗಈӡసλεΫͰ͸ɼਫ਼౓ͱॲཧ଎౓͸‫ʹڞ‬ॏཁͳࢦඪͰ͋ΔͨΊɼ͜ͷ྆໘͔Βߟ͑ΔͱఏҊख๏͸ηϚϯςΟοΫηάϝϯςʔγϣϯʹ࠷΋༗ޮͰ͋Δͱ‫͑ݴ‬Δɽ

5. ͓ΘΓʹ
ຊ‫Ͱڀݚ‬͸ɼখ෺ମ΁ͷਫ਼౓޲্Λ໨తͱͨ͠ 1D-SalsaSANʹΑΔηϚϯςΟοΫηάϝϯςʔγϣϯΛఏҊͨ͠ɽධՁ
࣮‫͔ݧ‬Βɼ1D-SAB ʹΑΓશମతͳηάϝϯςʔγϣϯਫ਼౓͕޲্͠ɼಛʹখ෺ମͷਫ਼౓޲্ʹߩ‫͢ݙ‬Δ͜ͱ͕֬ೝ
Ͱ͖ͨɽ·ͨɼॲཧ଎౓͸ SalsaNext ΑΓ޲্͠ɼࣗಈӡసʹඞཁͳϦΞϧλΠϜੑ΋े෼֬อ͞Ε͍ͯΔ͜ͱΛࣔͨ͠ɽࠓ‫ޙ‬ͷ՝୊ʹ͸ɼσʔλΦʔάϝϯςʔγϣϯ౳ʹ
ΑΔ͞ΒͳΔࣝผਫ਼౓ͷ޲্΍ଞͷσʔληοτΛ༻͍ͨධՁ࣮‫ʹݧ‬ΑΔఏҊख๏ͷ൚Խੑͷ֬ೝͳͲ͕‫͛ڍ‬ΒΕΔɽ

ࢀߟจ‫ݙ‬[1] C.Tiago, et al., “SalsaNext: Fast, Uncertainty-aware
Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving”, arXiv, 2020.
[2] T.Larissa, et al., “Scan-based Semantic Segmentationof LiDAR Point Clouds: An Experimental Study”, IV,
2020.
[3] B.Jens, et al., “SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences”, ICCV,2019.

‫੷ۀڀݚ‬[1] T.Suzuki, et al., “1D Self-Attention Network for Point
Cloud Semantic Segmentation using Omnidirectional
LiDAR”, ACPR, 2021.

ʢଞ 2 ݅ʣ

ηϚϯςΟοΫηάϝϯςʔγϣϯʹ͓͚ΔϚϧνυϝΠϯֶशʹؔ͢Δ‫ڀݚ‬ࢦಋ‫ڭ‬तɿࢁԼོٛ

TP20017 ਖ਼໦ᠳେ1. ͸͡Ίʹ

3. ఏҊख๏
ຊ‫Ͱڀݚ‬͸ɼσʔληοτΛ࠶ߏங͢Δ͜ͱͳ͘ෳ਺ͷυϝΠϯΛಉ࣌ʹֶश͢ΔͨΊʹɼυϝΠϯ‫ݻ‬༗ͷग़ྗ͕ՄೳͳϚϧνϔουߏ଄Λಋೖͨ͠ηάϝϯςʔγϣϯख๏ΛఏҊ͢ΔɽఏҊख๏ͷωοτϫʔΫߏ଄Λਤ 1 ʹࣔ͢ɽ

3.1. ‫ڞ‬༗ωοτϫʔΫ‫ڞ‬༗ωοτϫʔΫ͸ɼ୯ҰͷϞσϧͰෳ਺υϝΠϯ৘ใΛ‫ڞ‬༗͢Δ෦෼Ͱ͋Δɽ͜ͷ‫ڞ‬༗ωοτϫʔΫʹ͸ɼResNet101
Λ Backbone ʹͨ͠ DeepLab v3+[1] Λ༻͍Δɽ·ͨɼ
ResNet101 ͷ Residual Block ʹ Domain Attention (DA)
module[3] Λద༻͢ΔɽDA module ͸ɼυϝΠϯʹର͢Δ
Attention ʹΑΓɼ‫ڞ‬༗ͨ͠υϝΠϯ৘ใ͔Βೖྗʹରͯ͠ॏཁͳಛ௃ද‫ݱ‬Λ֫ಘ͢Δ‫͋Ͱߏػ‬ΔɽDA module ͷߏ଄Λਤ 2 ʹࣔ͢ɽDA module ͸ɼSE Adapter ͱ Domain
Assignment ʹΑͬͯߏ੒͞ΕΔɽSE Adapter ͸ɼෳ਺ͷ
SE module Ͱߏ੒͞Ε͍ͯΔɽͦΕͧΕͷग़ྗΛ࿈݁͢Δ
͜ͱͰશυϝΠϯͷද‫ۭؒݱ‬Λ‫ܗ‬੒Ͱ͖ΔɽDomain Assignment ͸ɼGAP ͱશ݁߹૚ͱιϑτϚοΫε૚Ͱߏ੒͞Ε͓ͯΓɼυϝΠϯʹదԠͨ͠ॏΈΛ֫ಘ͢ΔɽGAP ͷ‫ޙ‬ͷશ݁߹૚͔Βͷग़ྗ͸ɼSE Adapter ͷ SE module ͷ਺ͱಉ͡ʹͳΔɽ֫ಘͨ͠ॏΈ͸ SE Adapter ͷग़ྗͱ৐
ࢉ͠ɼSigmoid ؔ਺Ͱࢉग़͢Δɽ͜ΕʹΑΓɼSE Adapter͔ΒυϝΠϯʹదԠͨ͠ॏΈϕΫτϧ͕֫ಘͰ͖Δɽ

3.2. υϝΠϯʹରԠͨ͠Ϛϧνϔουߏ଄ҰൠతͳηάϝϯςʔγϣϯωοτϫʔΫ͸ɼ͋Β͔͡Ίఆٛͨ͠Ϋϥεʹର͢Δग़ྗΛߦ͏γϯάϧϔουߏ଄
Ͱ͋ΔͨΊɼΫϥε਺͕ҟͳΔσʔληοτΛಉ࣌ʹֶशͰ͖ͳ͍ɽͦ͜ͰɼҟͳΔΦϒδΣΫτΫϥεΛ࣋ͭσʔληοτ΋ಉ࣌ʹֶश͢ΔͨΊϚϧνϔουߏ଄ΛఏҊ͢Δɽ‫ڞ‬༗ωοτϫʔΫͰ֫ಘͨ͠ಛ௃ϚοϓΛσʔληοτ‫ݻ‬༗ͷग़ྗϔουʹೖྗ͢ΔɽͦͷͨΊσʔληοτ‫ݻ‬༗ͷύϥϝʔλ͸ɼग़ྗϔουͷΈʹͳΔɽ͜ΕʹΑΓɼಛ௃நग़‫ث‬͸‫ڞ‬༗ͱͳΔͨΊɼσʔληοτ૿ՃʹΑΔύ

3×3conv.

1×1conv.
1×1conv.

3×3conv.

concatHead 2
3×3conv.

%#!"
Cityscapes

ASPP

ResNet101+
DA module

Cityscapes!#

BDD
1×1conv.

BDD

1×1conv.

共有ネットワーク

Synscapes

3×3conv.

Head &
3×3conv.

%$!$
Synscapes

マルチヘッド構造

ਤ 1: ఏҊख๏ͷωοτϫʔΫߏ଄
Residual

Residual Block

2. ηϚϯςΟοΫηάϝϯςʔγϣϯηϚϯςΟοΫηάϝϯςʔγϣϯͷ୅දతͳख๏ͱ͠
ͯɼDeepLab v3+[1] ͕͋ΔɽDeepLab v3+͸ɼEncoderDecoder ߏ଄ͱɼAtrous Spatial Pyramid Pooling (ASPP)
Λ࠾༻͍ͯ͠ΔɽASPP ͸ɼҟͳΔ Dilation ͷ৞ΈࠐΈॲཧΛฒྻͰߦ͍ɼಛ௃ϚοϓΛ౷߹͢Δ͜ͱͰɼҟͳΔ
Receptive field Λߟྀ͢ΔॲཧͰ͋ΔɽैདྷͷηϚϯςΟο
Ϋηάϝϯςʔγϣϯ͸ɼະֶशͷυϝΠϯʹରͯ͠ೝࣝ
ਫ਼౓͕௿Լ͢Δ໰୊͕͋ΔɽMSeg[2] ͸ɼσʔληοτ͝ͱͰ෇༩͞Ε͍ͯΔϥϕϧΛ‫ڞ‬༗ϥϕϧʹม‫ֶͯ͠׵‬श͢
Δ͜ͱͰϚϧνυϝΠϯֶशΛ࣮‫͍ͯ͠ݱ‬Δɽ͔͠͠ɼ‫ڞ‬༗ϥϕϧͷ࠶ఆٛʹ͸ίετ͕͔͔Δ͜ͱ΍ɼҰ෦ͷΫϥε͸࡟আ͞ΕΔ໰୊͕͋Δɽ

3×3conv.

Head 1%"

Residual BlockηϚϯςΟοΫηάϝϯςʔγϣϯ͸ɼϐΫηϧϨϕϧ
ͰͷΫϥεࣝผΛߦ͏͜ͱͰɼΦϒδΣΫτͷछྨ͚ͩͰ
ͳ͘‫ܗ‬ঢ়΋ೝࣝ͢Δ͜ͱ͕Ͱ͖ΔɽֶशͱҟͳΔγʔϯͳͲυϝΠϯͷมԽ͕ੜ͡Δͱೝࣝਫ਼౓͸ஶ͘͠௿Լ͢Δͨ
Ίɼର৅ͱͳΔυϝΠϯ͝ͱʹϞσϧΛֶश͢Δඞཁ͕͋ΔɽͦͷͨΊɼ༷ʑͳ஍ҬͰӡ༻͞ΕΔࣗಈӡసγεςϜʹ༻͍Δ৔߹ɼ஍Ҭ͝ͱʹͦΕͧΕͷσʔλͰֶशͨ͠Ϟσϧ͕ඞཁͱͳΓɼϝϞϦίετ͕ߴ͘ͳΔɽͦ͜Ͱຊ‫ݚ‬‫Ͱڀ‬͸ɼ୯ҰͷϞσϧͰෳ਺ͷυϝΠϯσʔλΛֶश͢ΔͨΊʹɼϚϧνϔουߏ଄Λಋೖͨ͠ηϚϯςΟοΫηά
ϝϯςʔγϣϯΛఏҊ͢Δɽ֤ϔου͸ɼσʔληοτ‫ݻ‬༗ͷΫϥεΛग़ྗ͢Δɽ͜ΕʹΑΓɼΦϒδΣΫτΫϥε
͕ҟͳΔσʔληοτΛಉ࣌ʹֶश͢Δ͜ͱ͕ՄೳͱͳΔɽఏҊख๏͸ɼϕʔεϞσϧͱൺ΂ͯ 1.28 ഒͷύϥϝʔλ૿ՃͰෳ਺ͷυϝΠϯʹରԠՄೳͳϞσϧΛ࣮‫͖Ͱݱ‬Δ͜
ͱΛ֬ೝͨ͠ɽ

C×H×W

1×1 Conv.
C×1×1

3×3 Conv.

GAP

1×1 Conv.

C
×1
r

FC

FC

FC

DA module

C
×1
r

ReLU

ReLU

ReLU

⨁

C×1

FC

FC

FC

C×N

Concat

C×1×1

sigmoid

1×1 Conv.

GAP

C×1

FC

N×1

softmax

N×1⨂

3×3 Conv.
1×1 Conv.

⨂

DA module

Scale

⨁

⨁

ਤ 2: Domain Attention module ͷߏ଄ϥϝʔλ਺ͷ૿ՃΛ཈͑Δ͜ͱ͕Մೳͱ͢Δɽ

3.3. ଛࣦؔ਺
ҰൠతͳϚϧνυϝΠϯֶश [3] Ͱ͸ɼҟͳΔυϝΠϯ
Λ࣋ͭσʔλΛॱ൪ʹೖྗͯ͠ɼ֤ϔουͰଛࣦΛࢉग़͠ɼஞ࣍‫఻ٯ‬೻Λߦ͏ɽ͜ͷͱ͖ɼσʔληοτຖʹύϥϝʔλ͕ߋ৽͞ΕΔͨΊɼ‫఻ٯ‬೻ͤ͞Δॱ൪ʹΑͬͯಛఆͷσʔληοτʹόΠΞε͕͔͔ΔՄೳੑ͕͋Δɽͦ͜Ͱɼશͯ
ͷυϝΠϯσʔλΛೖྗͯ͠ɼ֤ϔουͰग़ྗͨ͠ଛࣦΛ߹‫͔ͯ͠ܭ‬Β‫఻ٯ‬೻͢Δ Mix Loss Λ࢖༻͢Δɽx Λ֤υϝΠϯͷೖྗαϯϓϧɼLCE ΛΫϩεΤϯτϩϐʔଛࣦؔ਺ͱͯ͠ɼN छྨͷυϝΠϯΛֶश͢Δࡍͷɼଛࣦؔ਺ L͸ࣜ (1) ͷΑ͏ʹ‫ٻ‬ΊΔ͜ͱ͕Ͱ͖Δɽ

L=

N
!

LCE (xn )

(1)

n=1

શͯͷυϝΠϯͷଛࣦ͔Β‫఻ٯ‬೻͢Δ͜ͱͰɼυϝΠϯ‫ݻ‬༗ͷภΓΛ๷͙͜ͱ͕Ͱ͖Δɽ

4. ධՁ࣮‫ݧ‬ຊষͰ͸ɼෳ਺ͷσʔληοτΛ༻͍ͯఏҊख๏ͷ༗ޮੑΛࣔͨ͢Ίʹɼෳ਺ͷσʔληοτΛ༻͍ͯධՁ࣮‫ݧ‬Λߦ͏ɽ

4.1. ࣮‫֓ݧ‬ཁ࣮‫ʹݧ‬͸ɼಉҰͷΦϒδΣΫτΫϥεͰߏ੒͞Ε͍ͯΔ
CityscapesɼBDDɼSynscapes ͷ૊Έ߹ΘͤͱɼҟͳΔΦ
ϒδΣΫτΫϥεͰߏ੒͞Ε͍ͯΔ CityscapesɼA2D2ɼ
IDD ͷ૊Έ߹ΘͤΛ༻͍Δɽֶशճ਺͸ 100 epochɼධՁࢦඪʹ͸ mIoU Λ༻͍Δɽ֤σʔληοτʹ‫·ؚ‬ΕΔσʔλ਺͸ҟͳΔͨΊɼ1epoch ͋ͨΓʹ༻͍Δσʔλ਺Λֶशຕ਺͕࠷΋ଟ͍σʔληοτʹ߹ΘͤΔ͜ͱͰσʔλ਺Λௐ੔͢Δɽ࠷దԽʹ͸ɼϞʔϝϯλϜΛ 0.9ɼॏΈ‫ݮ‬ਰ
Λ 0.0001 ʹઃఆͨ͠ SGD Λ༻͍Δɽֶश࣌ͷϛχόο
ν͸ɼಉ͡σʔληοτͷσʔλͷΈͰߏ੒͢Δɽೖྗը૾γϯάϧυϝΠϯ

Ground truthఏҊख๏ (w/o DAmodule)ఏҊख๏ (w/ DAmodule)ਤ 3: ηϚϯςΟοΫηάϝϯςʔγϣϯ݁Ռͷൺֱද 1: ಉҰΫϥε਺ͷσʔληοτͰͷਫ਼౓ͱύϥϝʔλ਺ [%]Cityscapes
77.57
75.55
77.92
78.49

γϯάϧυϝΠϯ
γϯάϧϔουఏҊख๏ (w/o DA module)ఏҊख๏ (w/ DA module)

BDD
61.55
63.14
62.51
62.63

Synscapes
91.55
86.91
90.14
90.18

Mean
76.89
75.34
76.86
77.10

Params
178.02M
59.34M
61.94M
76.37M࡟‫[ ཰ݮ‬%]

66.67
65.21
57.10

road

sidewalkbuilding

wall

fence

pole

light

sign

veg.

terrainsky

person

rider

car

truck

bus

train

mbike

bike

ද 2: Cityscapes ͰͷΫϥεผਫ਼౓ [%]mIoUγϯάϧυϝΠϯ

98.04

84.36

92.31

51.50

59.37

61.20

69.01

77.39

92.49

65.66

95.03

81.48

60.62

95.07

79.73

88.71

77.98

67.48

76.38

77.57

γϯάϧϔου

97.14

79.30

91.70

56.22

58.36

60.90

69.33

76.57

92.21

60.13

93.98

79.26

53.34

95.00

83.01

86.72

65.51

61.97

74.83

75.55

ఏҊख๏ʢw/o DA moduleʣ 97.77

82.27

92.08

53.58

60.83

61.65

70.09

78.77

92.48

61.49

94.35

81.45

60.46

95.30

81.78

90.16

81.18

67.45

77.29

77.92

ఏҊख๏ʢw/ DA moduleʣ

83.29

92.50

54.94

62.88

62.59

69.90

78.45

92.56

62.72

94.36

81.67

62.04

95.10

80.65

91.14

82.81

68.61

77.32

78.49

97.87

4.2. ಉҰΫϥεΛ࣋ͭσʔληοτͰͷֶशຊ࣮‫Ͱݧ‬͸ɼCityscapesɼBDDɼSynscapes Λର৅σʔληοτͱͯ͠༻͍Δɽද 1 ʹೝࣝਫ਼౓ͱύϥϝʔλ਺ͷൺֱɼද 2 ʹ Cityscapes ͰͷΫϥεผਫ਼౓ɼਤ 3 ʹ֤ख๏ͷηϚϯςΟοΫηάϝϯςʔγϣϯ݁ՌΛࣔ͢ɽγϯάϧϔουߏ଄ͱͷൺֱ Ϛϧνϔουߏ଄ͷ༗ޮੑΛ֬ೝ͢ΔͨΊʹɼγϯάϧϔουߏ଄ͱͷਫ਼౓ൺֱΛߦ͏ɽද 1 ΑΓɼϚϧνυϝΠϯֶशΛߦ͏ͱ͖ɼϚϧνϔουߏ଄Ͱͳ͍৔߹ɼBDD Ͱ͸ߴ͍ਫ਼౓Λୡ੒͕ͨ͠ɼ
Cityscapes ͱ Synscapes Ͱ͸ਫ਼౓͕௿Լͨ͜͠ͱ͕Θ͔Δɽ͜ΕΒͷ͜ͱ͔Βɼ1 ͭͷσʔληοτʹภͬͨϞσϧͱ͍͑ΔɽҰํɼఏҊख๏Ͱ͸ɼશͯͷσʔληοτʹରͯ͠࠷΋ߴ͍ਫ਼౓ͱͳΓɼόϥϯεΑֶ͘श͞Ε͍ͯΔ
͜ͱ͕Θ͔ΔɽDA module ద༻ʹΑΔൺֱ ද 1 ΑΓɼϚϧνϔου
ߏ଄ͷωοτϫʔΫʹɼDA module Λద༻͢Δ͜ͱͰɼશ
ͯͷσʔληοτͰ DA module ͳ͠ͷϚϧνϔουߏ଄ωοτϫʔΫΑΓ΋ೝࣝਫ਼౓͕޲্ͨ͠ɽ·ͨɼCityscapesͱ BDD Ͱ͸ɼγϯάϧυϝΠϯͰֶशͨ͠Ϟσϧͱൺֱͯ͠ೝࣝਫ਼౓͕޲্ͨ͠ɽද 2 ΑΓɼΫϥε͝ͱͷਫ਼౓Λൺֱ͢Δͱɼ19 Ϋϥεத 16 Ϋϥεͷ IoU ͕޲্ͨ͠ɽroad΍ sky ͱ͍ͬͨେ͖ͳ෺ମ͚ͩͰͳ͘ɼpole ΍ person ͱ͍ͬͨখ෺ମͷਫ਼౓΋޲্ͨ͠ɽ·ͨɼਤ 3 ͷఆੑධՁΛ
ൺֱ͢ΔͱɼDA module Λద༻͢Δ͜ͱͰ building ͱ‫ޡ‬ೝ͍ࣝͯ͠Δ pole ΫϥεΛਖ਼͘͠ೝࣝͰ͖͍ͯΔɽύϥϝʔλ਺ͷൺֱ σʔληοτ୯ମͰֶशͨ͠γϯάϧυϝΠϯͷϞσϧ͸ɼύϥϝʔλ਺͕σʔληοτͷ
਺͚ͩ૿Ճ͢Δɽද 1 ΑΓɼఏҊख๏Ͱ͸‫ڞ‬༗ͷωοτϫʔΫΛ࢖༻͢Δ͜ͱͰɼγϯάϧυϝΠϯͷϞσϧΛෳ਺༻ҙ͢Δ৔߹ΑΓ΋ύϥϝʔλ਺Λ 57.10% ࡟‫ͨ͠ݮ‬ɽ͜ΕΒͷ݁ՌΑΓɼఏҊख๏ʹΑΓϕʔεͷਫ਼౓Ҏ্ɼ·ͨ͸ಉ౳ͷਫ਼౓Λ‫ͳ͔ۇ‬ύϥϝʔλ਺ͷ૿ՃͰୡ੒͢Δ
͜ͱ͕Ͱ͖ͨɽ

4.3. ҟͳΔΫϥεΛ࣋ͭσʔληοτͰͷֶश࣍ʹɼҟͳΔΫϥεΛ࣋ͭσʔληοτΛಉ࣌ʹֶशՄೳ͔࣮‫͢ݧ‬Δɽ࣮‫ʹݧ‬͸ɼ19 Ϋϥεͷ Cityscapesɼ18 Ϋ

ද 3: ҟͳΔΫϥε਺ͷσʔληοτͰͷਫ਼౓ [%]γϯάϧυϝΠϯ
ఏҊख๏

Cityscapes
77.57
77.84

A2D2
78.08
77.74

IDD
67.80
68.27

ϥεͷ A2D2ɼ26 Ϋϥεͷ IDD Λ༻͍Δɽ·ͨɼϞσϧ͸ 4.2 અͷ࣮‫ߴͰݧ‬ਫ਼౓Ͱ͋ͬͨɼDA module Λద༻ͨ͠ϞσϧΛ࢖༻͢ΔɽγϯάϧυϝΠϯͱͷਫ਼౓ൺֱΛද
3 ʹࣔ͢ɽද 3 ΑΓɼγϯάϧυϝΠϯͷਫ਼౓ͱ΄΅ಉ౳ͷਫ਼౓Ͱ͋Δ͜ͱΛ֬ೝͨ͠ɽ͜ͷ݁ՌΑΓɼҟͳΔΫϥεΛ࣋ͭσʔληοτͰ΋ಉ࣌ʹֶशՄೳͰ͋Δ͜ͱΛ֬ೝͨ͠ɽ

5. ͓ΘΓʹ
ຊ‫Ͱڀݚ‬͸ɼҟͳΔυϝΠϯΛಉ࣌ʹֶश͢ΔϚϧνϔουߏ଄ͷηϚϯςΟοΫηάϝϯςʔγϣϯख๏ΛఏҊͨ͠ɽσʔληοτ͝ͱʹग़ྗϔουΛ༻ҙ͢ΔϚϧνϔουߏ଄ͷಋೖʹΑΓɼҟͳΔΫϥεΛ࣋ͭσʔληοτʹରͯ͠΋୯ҰͷϞσϧͰֶश͢Δ͜ͱΛՄೳͱͨ͠ɽ
ࠓ‫ޙ‬͸ɼଞͷϕʔεωοτϫʔΫ΁ͷద༻ʹΑΓ൚༻ੑΛ֬ೝ͢Δɽࢀߟจ‫ݙ‬[1] L.-C.Chen, et al., “Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation”,
ECCV, 2018.
[2] J.Lambert, et al., “MSeg: A Composite Dataset forMulti-Domain Semantic Segmentation”, CVPR, 2020.
[3] X.Wang, et al., “Towards Universal Object Detection by
Domain Attention”, CVPR, 2019.

‫੷ۀڀݚ‬[1] S.Masaki, et al., “Multi-Domain Semantic-Segmentation
using Multi-Head Model”, ITSC, 2021.
(ଞ 3 ݅)ϚϧνΤʔδΣϯτ‫ڧ‬ԽֶशΛ༻͍ͨσουϩοΫ໰୊ͷղܾʹؔ͢Δ‫ڀݚ‬ࢦಋ‫ڭ‬तɿࢁԼོٛ

TP20014 ‫ޒ‬౻‫ࢤڧ‬
1. ͸͡Ίʹ
‫ڧ‬ԽֶशΛෳ਺ͷΤʔδΣϯτ͕ଘࡏ͢Δ࣮໰୊ʹద༻
͢ΔࡍɼΤʔδΣϯτͷ૬‫༻࡞ޓ‬Λߟྀ͢Δඞཁ͕͋Δɽྫ
ͱͯ͠ෳ਺ͷं͕྆ߦ͖ަ͏৔໘ͰͷࣗಈӡసλεΫͳͲ͕‫͛ڍ‬ΒΕΔɽ͜ͷΑ͏ͳλεΫʹ͸ɼϚϧνΤʔδΣϯτ‫ڧ‬Խֶश (Multi Agent Reinforcement LearningɿMARL)ͱ‫ݺ‬͹ΕΔ‫ऺ܈‬ͷߦಈΛֶश͢Δख๏͕༻͍ΒΕΔɽֶशͷࡍʹɼෳ਺ͷΤʔδΣϯτ͕ࣗ‫ݾ‬རӹͷΈΛ༏ઌ͢Δͱɼ
σουϩοΫ͕ൃੜ͢ΔλεΫͰ͸ΤʔδΣϯτಉ͕࢜িಥ͠ɼֶशͷఀ଺΍‫ॴہ‬ղ΁ͷؕΓ͕‫͜ى‬Δɽຊ‫Ͱڀݚ‬͸ɼ
MARL ʹਂ૚ֶशΛಋೖ͢ΔࡍʹɼωοτϫʔΫΛ୯ҰϞσϧͱͯ͠ΤʔδΣϯτ͝ͱʹϒϥϯνΛ෼͚ͯಉֶ࣌श
͢Δख๏ΛఏҊ͢Δɽࣗಈӡసʹ͓͍ͯσουϩοΫ͕ൃ
ੜ͢Δ‫ڥ؀‬Λର৅ͱ͠ɼఏҊख๏ͷ༗ޮੑΛ֬ೝ͢Δɽ͞
ΒʹɼMask-Attention ‫ߏػ‬Λಋೖ͠ɼֶशʹΑͬͯ֫ಘͨ͠ߦಈʹ͓͚Δ൑அࠜ‫ڌ‬ΛՄࢹԽ͠ɼͲͷΑ͏ʹɼσουϩοΫΛճආͨ͠ͷ͔Λ֬ೝ͢Δɽ

δΣϯτຖͷ Actor-Critic ʹ Mask-Attention ‫ߏػ‬Λ૊ΈࠐΉ͜ͱʹΑΓɼಘΒΕͨํࡦʹର͢Δࢹ֮తͳઆ໌ΛՄ
ೳͱ͢Δɽ

3.1. Ϛϧνϒϥϯνߏ଄ఏҊख๏ͷωοτϫʔΫߏ଄Λਤ 1 ʹࣔ͢ɽϚϧνϒϥ
ϯνߏ଄͸ɼFeature extractor Λ‫ڞ‬༗͠ɼߦಈΛܾఆ͢
Δ Actor-Cirtic ΛΤʔδΣϯτ͝ͱʹϒϥϯν෼͚͢Δߏ
଄Ͱ͋Δɽ֤ Actor-Critic ͸ಠཱ͓ͯ͠ΓɼͦΕͧΕׂΓ౰ͯΒΕͨΤʔδΣϯτͷߦಈΛग़ྗ͢Δɽ‫ʹ಺ڥ؀‬ଘࡏ
͢ΔֶशΛߦ͏ΤʔδΣϯτͷ਺͚ͩ Actor-Cirtic Λ༻ҙ͠ɼఏҊख๏Λ༻͍ͯશମͷ੍‫ޚ‬ɼֶशΛಉ࣌ʹߦ͏ɽఏҊख๏ͷωοτϫʔΫ͸ं྆෇ۙͷௗᛌը૾ɼ଎౓ͱ‫཰ۂ‬Λೖྗͱ͢ΔɽఏҊख๏Λ༻͍Δ͜ͱͰɼෳ਺ͷࢹ఺Λߟྀ͢Δ͜ͱ͕ՄೳͱͳΓɼϚϧνΤʔδΣϯτ‫͚͓ʹڥ؀‬Δ૬खΛߟྀͨ͠࠷దͳߦಈΛ֫ಘ͢Δ͜ͱ͕Ͱ͖ΔɽҎԼʹఏҊख๏ͷֶशͷྲྀΕΛࣔ͢ɽ

2.Mask-Attention A3C

1. ΤʔδΣϯτͷ਺͚ͩϒϥϯν෼͚ͨ͠ϞσϧΛ࡞੒

Mask Attention A3C(Mask A3C)[2] ͸ɼਂ૚‫ڧ‬Խֶशͷ୅දతͳख๏Ͱ͋Δ Asynchronous Advantage ActorCritic(A3C)[1] ʹ Attention ‫ߏػ‬Λಋೖ͠ɼΤʔδΣϯτͷ
൑அࠜ‫ڌ‬ͷࢹ֮తઆ໌ΛՄೳͱ͢Δख๏Ͱ͋ΔɽMaskA3C
͸ɼํࡦΛग़ྗ͢Δ Policy branch ͱঢ়ଶՁ஋Λग़ྗ͢Δ
Value branch ͦΕͧΕʹ Mask-Attention ‫ߏػ‬Λಋೖ͢
ΔɽMask-Attention ‫ߏػ‬͸ɼதؒ૚ͷಛ௃Ϛοϓ F (st )ʹରͯ͠ɼΞςϯγϣϯϚοϓ M (st ) Λ༻͍ͨϚεΫॲཧΛߦ͏‫͋Ͱߏػ‬ΓɼMask-Attention ‫ߏػ‬Λಋೖֶͯ͠
श͢Δ͜ͱʹΑΓɼਪ࿦࣌ʹɼ൑அࠜ‫ڌ‬ͷࢹ֮తઆ໌ͱͳ
ΔΞςϯγϣϯϚοϓ͕֫ಘͰ͖ΔɽΞςϯγϣϯϚοϓ
ΛՄࢹԽ͢Δ͜ͱʹΑΓɼωοτϫʔΫͷ஫ࢹྖҬ͕֬ೝ
Ͱ͖Δɽಛ௃Ϛοϓʹର͢ΔΞςϯγϣϯϚοϓΛ༻͍ͨϚεΫॲཧΛࣜ (1) ʹࣔ͢ɽMask-Attention ͸ɼFeature
extractor ͔Βͷಛ௃Ϛοϓʹର͠ɼ1 × 1× νϟωϧ਺ͷ৞ΈࠐΈ૚ͱ sigmoid ؔ਺ʹΑΓ֫ಘ͍ͯ͠Δɽ

2. ֤ΤʔδΣϯτͷঢ়ଶΛରԠ͢Δϒϥϯν΁ೖྗF ! (st ) = F (st ) · M (st )3. ఏҊख๏

(1)

Actor-Critic
Mask-Attention

SigmoidConv.

agent 1 policy branch

FC

Relu

Conv.

ConvLSTM!"#$%&

agent 1 value branch

FC

Relu

Conv.

Maxpool
+Relu

Conv.

Maxpool+Relu

Conv.

Maxpool+Relu

Conv.

'(#)*

Actor-Critic
Mask-Attention

curvature

Sigmoid

Conv.

speed

agent 1 policy branch

5. ରԠ͢Δϒϥϯν‫ͼٴ‬৞ΈࠐΈ૚ʹ‫ࠩޡ‬Λ఻೻

3.2.Mask-Attention ͷಋೖఏҊख๏͸ɼਤ 1 ʹࣔ͢Α͏ʹɼΤʔδΣϯτ͝ͱͷ
Actor-Critic ʹ Mask-Attention ‫ߏػ‬Λಋೖ͢ΔɽMaskAttention ‫ߏػ‬͸ɼதؒ૚ͷಛ௃Ϛοϓʹରͯ͠ɼAttentionΛ༻͍ͨϚεΫॲཧΛߦ͏‫͋Ͱߏػ‬ΓɼֶशʹΑͬͯΞςϯγϣϯϚοϓ͕֫ಘՄೳͱͳΔɽΞςϯγϣϯϚοϓΛՄࢹԽ͢Δ͜ͱʹΑΓɼΤʔδΣϯτͷ஫ࢹྖҬΛදࣔ͢
Δ͜ͱ͕Ͱ͖Δɽຊ‫Ͱڀݚ‬͸ɼΤʔδΣϯτͷߦಈʹର͢
Δ஫ࢹྖҬΛ֬ೝ͢ΔͨΊɼPolicy ϒϥϯνͷΈʹ MaskAttention ‫ߏػ‬Λಋೖ͢Δɽ·ͨɼೖྗը૾ʹର͢Δۭ࣌ؒ
৘ใ͕ܽམ͠ͳ͍Α͏֤ Actor-Critic ʹ͸ɼConvLSTMΛ༻͍ΔɽσουϩοΫ͕ੜ͡Δ 4 γʔϯΛର৅ʹͨ͠ධՁ࣮‫ݧ‬Λߦ͏ɽ

4.1. ࣮‫ڥ؀ݧ‬ຊ‫Ͱڀݚ‬͸ɼࣗಈӡస‫͍͓ͯʹڥ؀‬ɼσουϩοΫ͕ੜ
͡Δ 4 γʔϯ (ਤ 2) Λର৅ͱ͢Δɽγʔϯ 1ɼ3ɼ4 ͸ද 1ɼ
γʔϯ 2 ͸ද 2 ʹࣔ͢ใुϧʔϧʹ‫ֶ͍ͯͮج‬शΛߦ͏ɽ͢΂ͯͷ‫ڥ؀‬͸ɼ૸ߦϨʔϯΛਖ਼͘͠૸ߦ͠ɼଞंʹিಥ
͢ΔͱࣦഊͱͳΔ‫͍ͯͬͳͱڥ؀‬Δɽ͢΂ͯͷ‫͍͓ʹڥ؀‬ͯɼ੍‫͢ޚ‬ΔΤʔδΣϯτ਺Λ 2 ͱ͢Δɽߦಈ͸଎౓૿‫ݮ‬‫ݮ૿཰ۂͼٴ‬มԽͳ͠ͷ૊Έ߹ΘͤͰ߹‫ ܭ‬9 ௨ΓͰ͋Γɼ࠷େ଎౓͕ 3m/sɼ࠷௿଎౓͕-1m/sɼ࠷େ‫ ͕཰ۂ‬0.25ɼ࠷௿‫͕཰ۂ‬-0.25 ͱ͢Δɽ4.2. ࣮‫֓ݧ‬ཁ֤γʔϯͷ‫ֶ͍͓ͯʹڥ؀‬शΛߦ͍ྦྷੵใुͷൺֱΛߦ͏ɽൺֱख๏͸ɼ௨ৗͷ‫ڧ‬ԽֶशͰ͋Δ A3CɼMARL ʹ
͓͚Δैདྷख๏Ͱ͋Δ MADDPGɼఏҊख๏ͷ‫ ܭ‬3 ͭͰ͋ΔɽͦΕͧΕͷख๏͝ͱʹɼγʔϯ 1 Ͱ͸ɼ2.0 × 107 εςοϓɼγʔϯ 2 Ͱ͸ɼ2.0 × 107 εςοϓɼγʔϯ 3 Ͱ

FC

Relu

Conv.

ConvLSTM

ද 1: ใुϧʔϧ 1!"#$%&

agent 1 value branch

FC

Relu

Conv.

ਤ 1: Ϟσϧશମͷߏ଄

4. ΤʔδΣϯτ͝ͱʹଛࣦؔ਺Λ‫ࢉܭ‬

4. ධՁ࣮‫ݧ‬ϚϧνΤʔδΣϯτ‫ڧ‬Խֶशʹ͸ɼҟͳΔΤʔδΣϯτ͕ͦΕͧΕͷࣗ‫ݾ‬རӹΛ‫ٻ‬Ίͯ͠·͏͜ͱʹΑΔσουϩο
Ϋ໰୊͕͋Δɽಠཱͨ͠ωοτϫʔΫΛෳ਺༻ֶ͍ͯशΛߦ͏৔߹ɼଞΤʔδΣϯτͷֶशΛ൓ө͢Δ‫͍ͨͳ͕ߏػ‬Ίࣗ‫ݾ‬ͷֶशͷΈΛߟྀͨ͠ߦಈʹΑΓɼσουϩοΫͷ
ճආߦಈ֫ಘ͕஗Εͯ͠·͏ɽͦ͜Ͱɼ୯ҰωοτϫʔΫͰΤʔδΣϯτ͝ͱʹ Actor-Critic Λϒϥϯν෼͚ͯ͠ಉ࣌
ʹֶशΛߦ͏Ϛϧνϒϥϯνߏ଄ΛఏҊ͢Δɽ·ͨɼΤʔ

Feature extractor

3. ΤʔδΣϯτຖʹใुΛ஝ੵ'(#)*ද 2: ใुϧʔϧ 2

ใु஋

ใु஋

ΰʔϧใु

+10

ΰʔϧใु

+10

தؒ஍఺ʹۙͮ͘

+2

ं྆ͷલਐ

+0.5
+1.5

িಥϖφϧςΟ

-10ࠨଆ௨ߦ૸ߦκʔϯ཭୤ϖφϧςΟ

-10িಥϖφϧςΟ

-10ंؒ‫ʹ཭ڑ‬ΑΔϖφϧςΟ

-3଎౓มԽʹΑΔϖφϧςΟ

-0.5

ंؒ‫ʹ཭ڑ‬ΑΔϖφϧςΟ

-3

制御⾞両

シーン1

道路

その他⾞両

障害物＆道路外

ゴール

中間地点

シーン3

シーン2

シーン4

ਤ 2: ֤γʔϯຖͷֶश‫ڥ؀‬ɿยଆҰंઢͰɼ‫ݟ‬௨͠ͷѱ͍ಓ࿏ʹ࿏றं͕྆ఀं͍ͯ͠Δγʔϯ (γʔϯ 1)ɼ
૸ߦंઢʹো֐෺͕͋Γɼର޲ंΛආ͚ͯ૸ߦ͢Δγʔϯ (γʔϯ 2)ɼ‫װ‬ઢಓ࿏ʹ߹ྲྀ͢Δγʔϯ (γʔϯ 3)ɼ
‫͍ڱ‬ग़ೖΓ‫Ͱޱ‬ͷγʔϯ (γʔϯ 4)
ද 3: ֤γʔϯʹ͓͚Δ֫ಘͨ͠ྦྷੵใुʹΑΔൺֱ (100 Τϐιʔυͷฏ‫)ۉ‬
γʔϯ 1

γʔϯ 2

γʔϯ 3

γʔϯ 4

ΤʔδΣϯτ 1ΤʔδΣϯτ 2ΤʔδΣϯτ 1ΤʔδΣϯτ 2ΤʔδΣϯτ 1ΤʔδΣϯτ 2ΤʔδΣϯτ 1ΤʔδΣϯτ 2

ఏҊख๏

961.5

580.2

997.0

994.4

785.0

585.3

773.4

589.6

̖̘̏

324.7

275.3

76.1

84.8

195.0

140.8

134.8

137.1

MADDPG

170.0

165.1

109.1

120.0

109.5

126.7

154.4

134.0

ਤ 4: ΞςϯγϣϯϚοϓͷՄࢹԽ݁Ռ

ਤ 3: ߦಈͷՄࢹԽ݁Ռɿ໼ҹ͸֤ं྆ͷਐߦํ޲͸ɼ5.0 × 106 εςοϓɼγʔϯ 4 Ͱ͸ɼ5.0 × 107 εςοϓֶशΛߦ͏ɽ͢΂ͯͷ‫͍͓ͯʹڥ؀‬ɼΤϐιʔυͷऴྃ৚݅͸ΤʔδΣϯτ͕ಓ࿏֎ɼো֐෺‫ͼٴ‬ଞं྆ʹিಥͨ͠৔߹ͱ 1.0 × 105 εςοϓ‫ܦ‬աͰऴྃͱ͢Δɽ

4.3. ྦྷੵใुʹΑΔൺֱ֤γʔϯ 100 ΤϐιʔυධՁΛߦͬͨ৔߹ͷฏ‫ੵྦྷۉ‬ใुΛද 3 ʹࣔ͢ɽද 3 ͔Βɼશͯͷγʔϯʹ͓͍ͯఏҊख๏Λ༻͍ͨΤʔδΣϯτ͕࠷΋ߴ͍ใुΛಘΔ͜ͱ͕Ͱ͖
͓ͯΓɼA3C ‫ ͼٴ‬MADDPG ͱ 300 ఺Ҏ্ͷείΞ͕ࠩ
͋Δɽ͜Ε͸ɼఏҊख๏ͷΈσουϩοΫΛճආ͢Δ͜ͱ
ͰɼใुΛ૿΍͢͜ͱʹ੒ޭ͍ͯ͠ΔͨΊྦྷੵใु͕େ͖
͘޲্͍ͯ͠Δɽ‫ʹٯ‬ɼA3C ‫ ͼٴ‬MADDPG Ͱ͸ɼσουϩοΫঢ়ଶΛճආͰ͖ͣɼ‫ॴہ‬ղ΁ؕͬͯ͠·͍ͬͯΔͨΊ (ྦྷੵ) ใु͕௿Լ͍ͯ͠Δɽ͜ΕΒͷ͜ͱ͔ΒఏҊख๏ͷ༗ޮੑ͕֬ೝͰ͖Δɽ

4.4. ఏҊख๏ͷϞσϧΛ༻͍ͨఆੑతධՁ
γʔϯ 1 ʹ͓͚ΔσουϩοΫൃੜ࣌ͷΤʔδΣϯτͷ
ߦಈͱΤʔδΣϯτ͝ͱͷΞςϯγϣϯϚοϓΛਤ 3ɼ4ʹࣔ͢ɽ͜͜Ͱ੺ं͕྆ΤʔδΣϯτ 1 Ͱ͋Γɼ੨ं͕྆
ΤʔδΣϯτ 2 Ͱ͋Δɽਤ 3 ͔ΒɼΤʔδΣϯτ 2 ͕ t+1
͔Β t+3 ·Ͱ଴‫͠ػ‬ɼΤʔδΣϯτ 1 ͕௨Γա͗Δͱಈ
͖ग़͍ͯ͠Δɽ͜ΕΑΓɼσουϩοΫΛճආ͢ΔͨΊʹΤʔδΣϯτ 2 ͸ΤʔδΣϯτ 1 ͕௨Γա͗ΔͷΛ଴ͭͱ͍͏૬खΛߟྀͯ͠σουϩοΫΛճආ͢ΔߦಈΛ֫ಘͰ
͖͍ͯΔͱߟ͑ΒΕΔɽ·ͨɼਤ 4 ͔ΒɼσουϩοΫ͕ൃੜͨ͠৔໘ʹ͓͍ͯɼΤʔδΣϯτ 1 ͸पғͷಓͱઌͷಓΛ஫ࢹ͠ɼΤʔδΣϯτ 2 ͸ɼࣗ਎ͱଞं྆Λ஫ࢹͯ͠

͍Δ͜ͱ͕֬ೝͰ͖Δɽ͜ͷ͜ͱ͔ΒɼΤʔδΣϯτ 1 ͸૬खΛ‫ͣͤʹؾ‬ɼઌʹಓΛਐΉΤʔδΣϯτͰ͋ΓɼΤʔδΣϯτ 2 ͸૬खͱࣗ෼Λ஫ࢹ͠ɼ҆શӡస͢ΔΤʔδΣϯτͰ͋Δͱߟ͑ΒΕΔɽ͜ΕΒͷ͜ͱ͔ΒɼఏҊख๏Λ༻͍Δ͜ͱͰɼσουϩοΫ͕ൃੜͨ͠৔߹ɼ૬खΛߟྀͯ͠σουϩοΫʹΑΔিಥΛճආ͢ΔΤʔδΣϯτΛ֫
ಘ͢Δ͜ͱ͕ՄೳͱͳΔɽ·ͨɼMARL ʹ͓͍ͯɼ૬खͱ
ҟͳΔࢹ఺Λ࣋ͪɼҟͳΔߦಈΛߦ͏͜ͱͰɼσουϩο
Ϋ໰୊Λղܾ͍ͯ͠Δͱ͍͏͜ͱ͕֬ೝͰ͖ͨɽ

5. ͓ΘΓʹ
ຊ‫Ͱڀݚ‬͸ɼଞΤʔδΣϯτͷֶशΛߟྀ͢ΔϚϧνΤʔδΣϯτ‫ڧ‬Խֶशͷख๏Λ༻͍Δ͜ͱͰɼσουϩο
Ϋ͕ൃੜ͢Δ‫͚͓ʹڥ؀‬Δ໰୊Λղܾͨ͠ɽࣗಈӡసʹ͓͍ͯσουϩοΫ͕ൃੜ͢Δ‫Ͱڥ؀‬ఏҊख๏ͷ༗ޮੑΛࣔ
ͨ͠ɽ·ͨɼMask-Attention Λ༻͍ͯΤʔδΣϯτຖͷํࡦΛ֬ೝͨ͠ͱ͜Ζɼ૬खΛߟྀͯ͠σουϩοΫΛճ
ආ͢ΔߦಈΛ֫ಘ͍ͯ͠Δ͜ͱ΋֬ೝͰ͖ͨɽࠓ‫ޙ‬ͷ༧ఆͱͯ͠͸ɼΑΓଟ༷ͳ‫͚͓ʹڥ؀‬Δ࠷దͳߦಈͷ֫ಘ΍ɼΑΓ૬खΛߟྀ͠ɼ‫ڠ‬ௐߦಈΛߦ͏Ϟσϧͷ࣮‫ڍ͕Ͳͳݱ‬͛ΒΕΔɽࢀߟจ‫ݙ‬[1] V. Mnih , et al., “Asynchronous methods for deep reinforcement learning”, ICML, 2016.
[2] H. Itaya, et al., “Visual Explanation using Attention
Mechanism in Actor-Critic-based Deep Reinforcement
Learning”, IJCNN, 2021.

‫੷ۀڀݚ‬[1] ‫ޒ‬౻‫ ࢤڧ‬౳, “ΠϯλϥΫγϣϯΛߟྀͨ͠Ϛϧνϒϥϯν
ωοτϫʔΫʹΑΔਂ૚‫ڧ‬Խֶश”, ਓ޻஌ೳֶձશࠃେձ,2020.