Ϋϥε৘ใΛߟྀͨ͠ελΠϧม‫ʹ׵‬ΑΔςετλΠϜ֦ுͷߴਫ਼౓Խʹؔ͢Δ‫ڀݚ‬ࢦಋ‫ڭ‬तɿ౻٢ ߂࿱TR190002 ࠓࢬ ߤ1. ͸͡Ίʹਂ૚ֶशʹΑΔελΠϧม‫׵‬͸ɼೖྗը૾͕࣋ͭελΠϧΛҟͳΔελΠϧ΁ม‫͢׵‬ΔΞϓϩʔνͰ͋ΔɽελΠϧม‫׵‬Λ༻͍ͯҟͳΔυϝΠϯͷը૾ʹม‫͢׵‬Δ͜ͱͰɼ෺ମೝࣝͷυϝΠϯదԠ͕ՄೳͱͳΔɽελΠϧม‫׵‬Λ༻͍ͨυϝΠϯదԠͷ୅දతͳख๏Ͱ͋Δ CyCADA[3] ͸ɼը૾ͷΫϥε͕Ұ؏ͨ͠ม‫׵‬͸ՄೳͰ͋Δ͕ɼೖྗը૾ʹରͯ͠ੜ੒Ͱ͖Δม‫׵‬ը૾͸ 1 ຕͰ͋Δɽຊ‫Ͱڀݚ‬͸ɼελΠϧม‫׵‬ͷࡍʹΫϥε৘ใΛߟྀͯ͠ෳ਺ͷม‫׵‬ը૾Λੜ੒Ͱ͖ΔελΠϧม‫׵‬ख๏ΛఏҊ͢Δɽ͞Βʹɼը૾ͷೝࣝ࣌ʹσʔλ֦ுΛߦ͏ςετλΠϜ֦ுʹఏҊख๏Λ༻͍Δ͜ͱͰߴਫ਼౓Խ͕‫ظ‬଴Ͱ͖ΔɽఏҊख๏͸ɼΫϥε৘ใΛߟྀ͠ͳ͕Β༷ʑͳελΠϧม‫͕׵‬ՄೳͱͳΓɼߴ͍ΞϯαϯϒϧޮՌΛ࣮‫͖Ͱݱ‬Δɽ

Mapping network

Step1: 潜在変数を用いたスタイル変換Step2: 画像の真贋判定

Discriminator
R/F

Generator
Encoder

R/F

Decoder

Step4: 各画像のクラス識別

Class

Style encoder

Step3: 画像の再構成

SematicClassifier consistency
ClassDecoder

Encoder

Cycleconsistency

2. ैདྷ‫ڀݚ‬2.1 ελΠϧม‫׵‬ελΠϧม‫׵‬͸ɼҟͳΔελΠϧΛ࣋ͭը૾ؒʹ͓͍ͯɼҰํ޲΋͘͠͸૒ํ޲ʹը૾ม‫͢׵‬ΔΞϓϩʔνͰ͋ΔɽελΠϧม‫׵‬ͷ୅දతͳख๏ͱͯ͠ɼCycleGAN[1] ͕͋ΔɽCycleGAN ͸ɼม‫׵‬ઌͷϖΞը૾Λඞཁͱ͠ͳ͍ม‫׵‬ख๏Ͱ͋Γɼ2 ͭͷελΠϧؒͰ૒ํ޲ʹը૾ม‫͖Ͱ׵‬Δɽ·ͨɼ3 ͭҎ্ͷελΠϧΛม‫͢׵‬Δख๏ͱͯ͠ɼStarGAN
v2[2] ͕͋ΔɽStarGAN v2 ͸ೖྗը૾ͱ͸ҟͳΔը૾ɼ΋͘͠͸Ψ΢ε෼෍ʹ‫ͮ͘ج‬જࡏม਺͔ΒελΠϧίʔυΛੜ੒͠ɼελΠϧίʔυʹຒΊࠐ·ΕͨελΠϧʹͳΔΑ͏ʹೖྗը૾Λม‫͢׵‬Δɽ

2.2 υϝΠϯదԠਤ 1 : ఏҊख๏ͷߏ଄3.2 ‫਺ؔࠩޡ‬‫਺ؔࠩޡ‬͸ɼΫϥεࣝผ‫ث‬Λֶश͢Δ Ltask ͱɼελΠϧม‫͚͓ʹ׵‬ΔΫϥε৘ใͷҰ؏ੑΛߟྀ͢Δ Lsem Λ༻͍Δɽ֤‫਺ؔࠩޡ‬͸ࣜ (1)ɼࣜ (2) ͱఆٛ͢Δɽ

Ltask = CE(C(Genc (xs )), ts )Lsem = CE(C(Genc (G(xs , st )), C(Genc (xs ))+ CE(C(Genc (G(xt , ss )), C(Genc (xt ))

(1)
(2)

υϝΠϯదԠ͸ɼ‫ࢣڭ‬ϥϕϧΛ࣋ͨͳ͍λʔήοτυϝΠϯͱɼ‫ࢣڭ‬ϥϕϧΛ࣋ͭιʔευϝΠϯͷҟͳΔυϝΠϯؒͷΪϟοϓΛܰ‫͢ݮ‬ΔΑ͏ʹֶश͢Δ͜ͱͰɼλʔήοτυϝΠϯͰͷࣝผਫ਼౓Λ޲্ͤ͞Δٕज़Ͱ͋ΔɽελΠϧม‫ʹ׵‬ΑΔυϝΠϯదԠͱͯ͠ Cycle-Consistent
Adversarial Domain Adaptation(CyCADA)[3] ͕ఏҊ͞Ε͍ͯΔɽCyCADA ͸ɼCycleGAN ʹΫϥε৘ใΛߟྀ
͢Δ Semantic consistency Λಋೖ͍ͯ͠Δɽ͜ΕʹΑΓɼιʔευϝΠϯͰࣄલֶशͨࣝ͠ผ‫ث‬Λ༻͍ͯɼม‫׵‬લ‫ޙ‬ͷը૾ʹ͓͚ΔΫϥε৘ใ͕Ұக͢ΔΑ͏ʹֶशͰ͖Δɽ͜͜Ͱɼx ͕ೖྗը૾ɼts ͕ιʔευϝΠϯͷ‫ࢣڭ‬ϥϕϧɼC ͕Ϋϥε෼ྨ‫ث‬ɼG ͕ Generatorɼs ͕ελΠϧίʔυɼCE ͕ΫϩεΤϯτϩϐʔ‫͋Ͱࠩޡ‬Δɽ·ͨɼఴ͑ࣈ͸֤υϝΠϯΛද͓ͯ͠Γɼs ͕ιʔευϝΠϯɼt ͕λʔήοτυϝΠϯΛද͍ͯ͠ΔɽLtask ͸ɼ‫ࢣڭ‬ϥϕϧΛ࣋ͭιʔευϝΠϯͷը૾ͷΈͰ‫ࠩޡ‬Λ‫͢ࢉܭ‬ΔɽLsem ͸ɼΫϥε෼ྨ‫ث‬ΑΓग़ྗ͞ΕΔม‫׵‬લͷΫϥε֬཰ΛԾϥϕϧͱ͠ɼม‫ޙ׵‬ͷΫϥε֬཰ͱͷΫϩεΤϯτϩϐʔ‫ࠩޡ‬Λ‫ܭ‬ࢉ͢Δɽ͜ΕʹΑΓɼը૾ͷม‫׵‬લ‫Ͱޙ‬Ϋϥε৘ใͷҰ؏ੑΛߟֶྀͨ͠श͕ՄೳͱͳΔɽ

3. ఏҊख๏

3.3 ςετλΠϜ֦ுຊ‫Ͱڀݚ‬͸ɼStarGAN v2 Ͱͷը૾ม‫ʹ࣌׵‬ΫϥεࣝผΛߦ͏ิॿλεΫΛ௥Ճ͢Δ͜ͱʹΑΓɼελΠϧม‫࣌׵‬ʹΫϥε৘ใΛߟྀ͢ΔɽิॿλεΫʹ͸ɼCyCADA ͷ
Semantic consistency Λ༻͍Δɽ͜ΕʹΑΓɼຊख๏͸Ϋϥε৘ใΛߟྀ͠ͳ͕Βෳ਺ͷը૾΁ม‫͖Ͱ͕׵‬Δɽຊख๏ΛධՁ࣌ʹσʔλ֦ு͢ΔςετλΠϜ֦ுʹద༻͢Δ͜ͱͰɼߴਫ਼౓ͳࣝผ͕ՄೳͱͳΔɽ

3.1 Ϋϥε৘ใΛߟྀͨ͠ StarGAN v2ςετλΠϜ֦ு͸ɼࣝผ࣌ςεταϯϓϧʹରͯ͠σʔλ֦ுΛߦ͍ɼ֤ը૾ʹର͢ΔΫϥε֬཰Λฏ‫ͯ͠ۉ‬࠷ऴతͳ༧ଌ݁ՌΛܾఆ͢Δɽຊख๏Ͱ͸ɼιʔευϝΠϯͰࣄલֶशͨࣝ͠ผ‫΁ث‬λʔήοτυϝΠϯͷը૾Λೖྗ͢ΔࡍʹɼελΠϧม‫ʹ׵‬ΑΔυϝΠϯదԠΛߦ͏ɽಉ࣌ʹɼજࡏม਺Λෳ਺ճαϯϓϦϯά͠ɼ༷ʑͳελΠϧͷը૾΁ม‫͢׵‬Δɽͦͯ͠ɼैདྷͱಉ༷ʹࣜ (3) Ͱఆٛ͢ΔΞϯαϯϒϧਪ࿦ʹͯ࠷ऴతͳ༧ଌ݁ՌΛܾఆ͢ΔɽNఏҊख๏ͷߏ଄Λਤ 1 ʹࣔ͢ɽఏҊख๏͸ɼιʔεը૾xs ͱλʔήοτը૾ xt ʹରͯ͠ҎԼͷॲཧΛॱʹߦ͏ɽ

Step1 જࡏม਺ z ΑΓੜ੒͞ΕΔελΠϧίʔυ s̃t Λ༻͍ͨελΠϧม‫׵‬ɽ
Step2 ࣮ը૾ xs ͱม‫׵‬ը૾ x̃t ͷਅ‫آ‬൑ఆɽ
Step3 ࣮ը૾ͷελΠϧίʔυ ŝs Λ༻͍ͨը૾ x̃t ͷ࠶ߏ੒ɽ
Step4 ࣮ը૾ xs ͱม‫׵‬ը૾ x̃t ͷΫϥεࣝผɽ
͜͜ͰɼGenerator Λ Encoder-Decoder Ϟσϧͱͨ͠ͱ͖ɼೖྗը૾͕࣋ͭίϯςΩετΛ Encoder Ͱநग़ɼDecoderͰελΠϧ৘ใͷ෇༩Λߦ͏ͨΊɼGenerator ͷதؒಛ௃͸υϝΠϯෆมͷಛ௃ྔͱͳΔɽͦͷͨΊɼGenerator ͷதؒʹΫϥεࣝผ‫ث‬Λ௥Ճ͢Δ͜ͱʹΑΓɼ྆ํͷυϝΠϯʹ͓͍ͯे෼ͳࣝผੑೳ͕‫ظ‬଴Ͱ͖Δɽ·ͨɼ௥Ճͨ͠Ϋϥε෼ྨ‫ث‬͸ը૾ม‫ʹ࣌׵‬͸༻͍ͳ͍ɽ

pi =1 !
σi (f (G(x, F (z n ))))
N n=1(3)͜͜Ͱɼi ͕Ϋϥε൪߸ɼp ͕༧ଌ݁ՌɼN ͕૿෯਺ɼσ͕ Softmax ؔ਺ɼf ͕ࣄલֶश͞Εͨࣝผ‫ث‬ɼF ͕જࡏม਺ΑΓελΠϧίʔυΛੜ੒͢Δ Mapping networkɼz͕જࡏม਺Ͱ͋Δɽ

4. ධՁ࣮‫ݧ‬ఏҊख๏ͷ༗ޮੑΛࣔͨ͢Ίʹɼม‫׵‬ը૾ͷ܏޲ௐࠪͱ
ςετλΠϜ֦ுʹ͓͚Δࣝผੑೳͷൺֱɼ͓Αͼม‫׵‬ը૾ͷ෼෍ௐࠪΛߦ͏ɽධՁ࣮‫͢༻࢖ʹݧ‬Δσʔληοτ͸ɼ਺ࣈೝࣝ༻ͷσʔληοτͰ͋Δ SVHN ͱ SynthDigitsΛ༻͍Δɽ͜͜ͰɼιʔευϝΠϯʹ͸ɼCG ը૾Ͱ͋Δ
SynthDigitsɼλʔήοτυϝΠϯ͸࣮ը૾Ͱ͋Δ SVHNͱ͠ɼΫϥε਺͸ 10 Ͱ͋Δɽ·ͨɼςετλΠϜ֦ுʹ
͓͚Δ૿෯਺͸ 10 ͱ͢Δɽ

ソース
ドメイン
(CG画像)

ターゲット
ドメイン
(実画像)

CycleGANStarGAN v2

提案手法ਤ 2 : ֤ख๏ʹΑΔม‫݁׵‬Ռ(a) ελΠϧม‫׵‬લͷ෼෍(b) ελΠϧม‫ޙ׵‬ͷ෼෍

ਤ 3 : UMAP ʹΑΔ෼෍ͷՄࢹԽ
ද 1 : ςετλΠϜ֦ுʹ͓͚Δࣝผਫ਼౓

4.3 UMAP ʹΑΔม‫׵‬ը૾ͷ෼෍ͷௐࠪ

ResNet-20

CNN(701.4k)(24.3k)-74.8866.54ϕʔεϥΠϯ

177.4876.44(StarGAN v2)

1079.8579.50

֤υϝΠϯͷը૾ͱఏҊख๏Ͱม‫ͨ͠׵‬ը૾Λ UMAPͰ࣍‫ݩ‬ѹॖ͠ɼՄࢹԽͨ݁͠ՌΛਤ 3 ʹࣔ͢ɽ͜͜Ͱɼ੨
࿮͕ιʔευϝΠϯɼ੺࿮͕λʔήοτυϝΠϯΛදͯ͠
͍Δɽ࣍‫ݩ‬ѹॖʹ࢖༻͢Δը૾ຕ਺͸ɼ֤Ϋϥε͔ΒϥϯμϜʹ 25 ຕબ୒͠ɼ֤υϝΠϯ 250 ຕͱ͢Δɽ·ͨɼѹॖ͢Δಛ௃ʹ͸ఆྔతධՁͰ࢖༻ͨ͠ CNN ͷதؒಛ௃Λ
༻͍Δɽਤ 3(a) ͸ελΠϧม‫׵‬Λߦ͍ͬͯͳ͍ͨΊɼυϝΠϯ͝ͱͷ෼෍͕ҟͳ͍ͬͯΔɽҰํɼελΠϧม‫׵‬Λߦͬͨਤ 3(b) ͸֤Ϋϥεͷ෼෍͕΄΅Ұக͍ͯ͠Δɽ

187.7385.775. ͓ΘΓʹ1089.2187.91ຊ‫Ͱڀݚ‬͸ɼΫϥε৘ใΛߟྀͨ͠ελΠϧม‫׵‬ख๏͓ΑͼɼςετλΠϜ֦ுʹ͓͚ΔελΠϧม‫׵‬Λ༻͍ͨσʔλ֦ுΛఏҊͨ͠ɽධՁ࣮‫͍͓ͯʹݧ‬ɼఏҊख๏ʹΑΔελΠϧม‫׵‬͸ɼΫϥε৘ใΛߟྀͨ͠ม‫͍͖ͯͰ͕׵‬Δ͜ͱΛ֬ೝͨ͠ɽ·ͨɼςετλΠϜ֦ுʹΑΔఆྔతධՁͰ͸ɼม‫׵‬ແ͠ͱൺֱͯ͠ɼ14.33pt ͷਫ਼౓޲্Λ֬ೝͨ͠ɽࠓ‫ޙ‬͸ɼηϚϯςΟοΫηάϝϯςʔγϣϯͳͲͷҟͳΔλεΫ΁ͷదԠΛ‫ݕ‬౼͢Δɽࣝผ཰ [%]ม‫׵‬ख๏ม‫׵‬ແ͠ఏҊख๏૿෯਺

4.1 ม‫׵‬ը૾ͷఆੑతධՁఏҊख๏͓Αͼैདྷख๏ (CycleGANɼStarGAN v2) Λ༻͍ͯม‫݁ͨ͠׵‬ՌΛਤ 2 ʹࣔ͢ɽ͜ΕΑΓɼैདྷख๏Ͱ͸Ϋϥε৘ใͷҡ͕࣋ࠔ೉Ͱ͋Γɼม‫͋Ͱݩ׵‬ΔλʔήοτυϝΠϯͷը૾͕௿ը࣭Ͱ͋ͬͨ৔߹͸ͦͷ܏޲͕‫ݦ‬ஶͰ͋ΔɽҰํɼఏҊख๏ʹΑΔม‫׵‬͸ɼΫϥε৘ใΛҡ࣋ͨ͠ঢ়ଶͰҟͳΔελΠϧͷը૾΁ม‫͍͖ͯͰ׵‬Δ͜ͱ͕෼͔Δɽ

4.2 ςετλΠϜ֦ுΛ༻͍ͨఆྔతධՁςετλΠϜ֦ுʹ࢖༻͢ΔࣄલֶशࡁΈࣝผ‫ʹث‬͸ɼ
ResNet-20 ͱɼυϝΠϯؒͷࠩҟΛ‫ٵ‬ऩ͢Δ͜ͱ͕ࠔ೉ͳ
5 ૚ͷ CNN Λ༻͍ͯධՁ࣮‫ݧ‬Λߦ͏ɽςετλΠϜ֦ுʹΑΔࣝผਫ਼౓Λද 1 ʹࣔ͢ɽ͜͜Ͱɼ֤ωοτϫʔΫͷύϥϝʔλ਺͸Ϟσϧ໊ͷԼʹࣔ͢ɽද 1 ΑΓɼResNet-20 ͷ݁Ռʹ͓͍ͯɼఏҊख๏͸ม‫׵‬ແ͠ͱൺֱͯࣝ͠ผਫ਼౓ͷ޲্͕֬ೝͰ͖Δɽ·ͨɼ૿෯਺ 1 Ͱ͸ɼఏҊख๏ʹΑΔࣝผਫ਼౓͸ϕʔεϥΠϯͷ StarGAN v2 ͱൺֱ͠ɼ10.25pt
ͷਫ਼౓޲্Λ֬ೝͨ͠ɽ͞Βʹɼ૿෯਺ 10 Ͱ͸ɼ૿෯਺
1 ͱൺֱͯ͠ 1.48pt ͷਫ਼౓޲্Λ֬ೝͨ͠ɽ͜ͷ܏޲͸
CNN ͷࣝผ݁Ռʹ͓͍ͯ΋ಉ༷Ͱ͋Δɽࢀߟจ‫ݙ‬[1] J.Y.Zhuɼet al.ɼ“Unpaired Image-to-Image Translation
using Cycle-Consistent Adversarial Networks”In ICCVɼ2017ɽ
[2] Y.Choiɼet al.ɼ“StarGAN v2: Diverse Image Synthesis
for Multiple Domains”ɼIn CVPRɼ2020ɽ
[3] J.Hoﬀmanɼet al.ɼ“CyCADA: Cycle Consistent Adversarial Domain Adaptation”ɼIn ICMLɼ2018ɽ‫੷ۀڀݚ‬[1] ࠓࢬߤ ౳ɼ“Attention ‫ߏػ‬Λಋೖͨ͠ CycleGAN ʹΑΔࣝผʹ༗ޮͳελΠϧม‫”׵‬, ը૾ͷೝࣝɾཧղγϯϙδ΢Ϝ,2019.
[2] ࠓࢬߤ ౳ɼ“Generative Adversarial Networks Λ༻͍͔ͨΒ͋͛ϐοΩϯάʹ͓͚Δલॲཧͱͯ͠ͷը૾ม‫”׵‬, Ϗδϣϯٕज़ͷ࣮ར༻ϫʔΫγϣοϓ, 2020.