Vol.2017-SE-196 No.20
2017/7/20

৘ใॲཧֶձ‫ڀݚ‬ใࠂ
IPSJ SIG Technical Reportଟ‫ޠݴ‬ରԠͷͨΊͷࣈ۟ղੳ‫ߏػ‬Λ࣋ͭίʔυΫϩʔϯ‫ݕ‬ग़πʔϧͷ։ൃ੉ଜ ༤Ұ1,a)٢ా ଇ༟2ቐ Ըಬ3

Ҫ্ ࠀ࿠1֓ཁɿࣈ۟୯ҐͷίʔυΫϩʔϯ‫ݕ‬ग़πʔϧͰ͋Δ CCFinder ͸ɼର৅ͷιʔείʔυͷࣈ۟ղੳΛߦͬ
͍ͯΔɽରԠ͢Δ‫ޠݴ‬Λ૿΍͢ʹ͸ͦͷ‫ޠݴ‬ͷࣈ۟ղੳΛ࣮૷͢Δඞཁ͕͋Δ͕ɼଟ͘ͷ‫ʹޠݴ‬ରͯ͠ 1ͭͣͭࣈ۟ղੳΛ࣮૷͢Δͷ͸ɼπʔϧͷ։ൃऀʹ֤‫΁ޠݴ‬ͷཧղ͕‫ٻ‬ΊΒΕɼ·ͨखؒͷ͔͔Δ࡞‫Ͱۀ‬͋Δɽπʔϧͷ࢖༻ऀ͕ɼ৽ͨͳ‫ʹޠݴ‬༰қʹରԠͰ͖Δ࢓૊ΈΛ࡞Δ͜ͱͰɼ։ൃऀͷ࿑ྗΛ‫ݮ‬Β͢͜ͱ͕Ͱ͖Δɽຊ‫Ͱڀݚ‬͸ɼࣈ۟ղੳʹ͓͍ͯ‫ʹޠݴ‬ґଘ͢Δ෦෼Λɼ༰қͳΦϓγϣϯͰมߋՄೳͳࣈ۟
ղੳ‫ߏػ‬ΛఏҊ͢Δɽద༻࣮‫Ͱݧ‬͸ 175 ‫ޠݴ‬ͷιʔείʔυʹରͯ͠ɼίϝϯτʹؔ͢ΔΦϓγϣϯΛઃఆͯ͠πʔϧΛ࣮ߦ͠ɼ90 ˋҎ্ͷ‫Ͱޠݴ‬ίϝϯτআ‫͕ڈ‬Մೳͳ͜ͱΛࣔͨ͠ɽ·ͨɼ༧໿‫ޠ‬ͷϦετΛ
༩࣮͑ͯߦ͢Δ͜ͱͰɼλΠϓ 2 ·ͰͷίʔυΫϩʔϯ͕‫ݕ‬ग़Մೳͳ͜ͱΛࣔͨ͠ɽΩʔϫʔυɿίʔυΫϩʔϯɼࣈ۟ղੳɼNgram

A Clone Detection Tool
with Flexible Multilingual Tokenization
Yuichi Semura1,a)Norihiro Yoshida2

1. ͸͡Ίʹ

Eunjong Choi3

Katsuro Inoue1

͓ͯ͘ඞཁ͕͋Δ͕ɼେ‫ن‬໛ͳιϑτ΢ΣΞʹ͓͍ͯ͸ɼίʔυΫϩʔϯʹ͋ͨΔՕॴ͕๲େͳ਺ʹͳΔ͜ͱʹΑΓɼίʔυΫϩʔϯͱ͸ɼιʔείʔυதʹଘࡏ͢Δ‫ʹ͍ޓ‬ͦͷશͯΛೝ͓ࣝͯ͘͜͠ͱ͸‫࣮ݱ‬తͰͳ͍ɽͦ͜Ͱɼ։

Ұகɼ·ͨ͸ྨࣅͨ͠ίʔυยΛࢦ͢ɽίʔυΫϩʔϯ͸ɼൃऀΛࢧԉ͢ΔͨΊʹίʔυΫϩʔϯ‫ݕ‬ग़πʔϧ͕ར༻͞ओʹ‫ط‬ଘͷίʔυยͷίϐʔΞϯυϖʔετʹΑͬͯੜ੒Ε͍ͯΔ [1]ɽ

͞ΕΔ [2]ɽͦͷଘࡏ͸ɼιϑτ΢ΣΞอकΛࠔ೉ʹͯ͠ίʔυΫϩʔϯ‫ݕ‬ग़πʔϧͷ 1 ͭͰ͋Δ CCFinder ͸ɼ͍Δେ͖ͳཁҼͷ 1 ͭͱͯ͠‫͛ڍ‬ΒΕ͍ͯΔɽ͋Δίʔυ

CɼC++ɼJavaɼCOBOLɼFortran ͷ‫͔ॻͰޠݴ‬Εͨιʔεยʹόά͕‫͔ͨͬͭݟ‬৔߹ɼͦͷίʔυยͷίʔυΫϩʔίʔυͷίʔυΫϩʔϯ‫ݕ‬ग़ʹରԠ͍ͯ͠Δ [4]ɽCCFinderϯʹ΋όά͕‫·ؚ‬ΕΔՄೳੑ͕ߴ͍ɽͦͯ͠ɼ։ൃऀ͸͋Ͱ͸τʔΫϯ୯ҐͷίʔυΫϩʔϯΛ‫ݕ‬ग़͢ΔͨΊͷલΔίʔυยʹόά͕‫͔ͨͬͭݟ‬৔߹ɼҰக·ͨ͸ྨࣅ͢Δ

ॲཧͱͯ͠ɼιʔείʔυΛ‫ޠݴ‬ͷจ๏ʹԊͬͯτʔΫϯՕॴʹରͯ͠ಉҰͷमਖ਼Λߦ͏͔‫ݕ‬౼͢Δඞཁ͕͋Δ [3]ɽ୯Ґʹ෼ׂ͍ͯ͠Δɽ͜ͷॲཧ͸Ұൠతʹࣈ۟ղੳͱ‫ݺ‬͹ͦͷͨΊɼ։ൃऀ͕ίʔυΫϩʔϯʹؔ͢Δ৘ใΛೝࣝ͠

ΕΔ [8]ɽಉ͘͡ CCFinderX ͸ɼCCFinder ͷόʔδϣϯΞοϓͱͯ͠։ൃ͞ΕͨίʔυΫϩʔϯ‫ݕ‬ग़πʔϧͰ͋Γɼ

1
2
3
a)

େࡕେֶ
Osaka University໊‫ݹ‬԰େֶ
Nagoya Universityಸྑઌ୺Պֶٕज़େֶӃେֶ
Nara Institute of Science and Technology
y-semura@ist.osaka-u.ac.jp

c 2017 Information Processing Society of Japanࣈ۟ղੳ෦ͷϢʔβʹΑΔมߋΛՄೳʹ͍ͯ͠Δ [11]ɽ͜
Ε͸ίʔυΫϩʔϯ‫ݕ‬ग़Λߦ͍͍ͨ‫ʹޠݴ‬ରԠ͢Δࣈ۟ղੳ෦ΛϢʔβ͕༻ҙ͢Δ͜ͱͰɼͦͷ‫ޠݴ‬ͷίʔυΫϩʔϯ‫ݕ‬ग़͕ՄೳʹͳΔͱ͍͏͜ͱΛҙຯ͢Δɽ͔͠͠ɼࣈ۟

1

Vol.2017-SE-196 No.20
2017/7/20

৘ใॲཧֶձ‫ڀݚ‬ใࠂ
IPSJ SIG Technical Reportղੳ෦Λ༻ҙ͢Δʹ͸‫ޠݴ‬ͷจ๏Λཧղ͢Δ͜ͱ͕ඞཁͰ

ۭനɼλϒจࣈɼվߦ΍ίϝϯτͳͲΛআ͍ͯҰக͋͢Γɼࣈ۟ղੳͷίʔσΟϯά͸खؒͷ͔͔Δ࡞‫͋Ͱۀ‬Δɽ

ΔίʔυΫϩʔϯɽ

CCFinder Ͱ͸ࣈ۟ղੳʹΑͬͯτʔΫϯΛ෼ׂͨ͠‫ޙ‬

λΠϓ 2 ɹʹಛఆͷτʔΫϯͷม‫׵‬Λߦ͍ͬͯΔɽ͜Ε͸ม਺໊΍ؔ

λΠϓ 1 ͷ৚݅ʹՃ͑ͯɼϦςϥϧɼ‫ܕ‬ɼࣝผࢠΛআ਺໊ͳͲΛશͯ 1 ͭͷτʔΫϯʹஔ͖‫͑׵‬ΔॲཧͰ͋Γɼ͍ͯҰக͢ΔίʔυΫϩʔϯɽ࣮༻తʹҙຯͷ͋ΔίʔυΫϩʔϯ͚ͩΛ‫ݕ‬ग़͢ΔͨΊ

CCFinder ͸ɼࣈ۟୯ҐͷίʔυΫϩʔϯΛ‫ݕ‬ग़͢ΔπʔʹߦΘΕΔɽม਺໊΍ؔ਺໊Ͱ࢖༻Ͱ͖ΔτʔΫϯ͸ɺϓϧͰ͋Δ [4]ɽCCFinder ͷಛ௃ͱͯ͠ҎԼͷΑ͏ͳ΋ͷ͕ϩάϥϛϯά‫ʹޠݴ‬Αͬͯઃఆ͞Ε͍ͯΔͨΊʹɼ‫ʹޠݴ‬‫͛ڍ‬ΒΕΔɽΑͬͯผͷઃఆΛߦΘͳ͚Ε͹ͳΒͳ͍ɽຊ‫Ͱڀݚ‬͸ଟ‫ޠݴ‬ͷτʔΫϯ୯ҐͷίʔυΫϩʔϯ‫ݕ‬ग़Λ໨తͱͯ͠ɼϢʔβʹΑΔ༰қʹࣈ۟ղੳ෦ͷมߋ͕ՄೳʹͳΔΑ͏ͳࣈ۟ղੳ‫ߏػ‬ΛఏҊ͢Δɽࣈ۟ղੳ‫ߏػ‬ͷมߋՕॴΛɼओʹίϝϯτআ‫ڈ‬ͷϧʔϧͱஔ͖‫͕͑׵‬ඞཁͳࣈ۟ʹ‫ݶ‬ఆͨ͠ɽ·ͨίʔυΫϩʔϯ‫ݕ‬ग़෦ͷ࣮૷ʹ͓͍ͯߴ଎ԽΛਤΔͨΊɼίʔυΫϩʔϯ‫ݕ‬ग़πʔϧͰ͋Δ YOCCA ͷΞϧΰϦζϜΛࢀߟʹͨ͠ [5]ɽYOCCA ͸ɼ͋Δ 1 ͭͷϑΝΠϧ

• ม਺໊΍ؔ਺໊ͳͲͷτʔΫϯΛஔ͖‫͑׵‬Δ͜ͱͰɼλΠϓ 2 ·ͰͷίʔυΫϩʔϯΛ‫ݕ‬ग़Ͱ͖Δɽ• ਺ඦສߦ‫ن‬໛ͷγεςϜʹ΋࣮༻࣌ؒͰղੳՄೳͰ
͋Δɽ• ‫ޠݴ‬ґଘ෦෼ΛऔΓସ͑Δ͜ͱͰ͞·͟·ͳ‫ʹޠݴ‬ରԠ͍ͯ͠Δɽ

• ͖͍͠஋Λ༩͑Δ͜ͱͰɼτʔΫϯ਺͕͖͍͠஋ະຬ
ͷίʔυΫϩʔϯΛ‫ݕ‬ग़͠ͳ͍Α͏ʹͰ͖Δɽ
͜͜ͰɼCCFinder ͷॲཧ֓ཁʹ͍ͭͯ‫ه‬ड़͢Δɽ4 ͭͱ਺ඦສߦҎ্ͷ‫ن‬໛Λ࣋ͭ‫ڊ‬େͳιʔείʔυ‫܈‬ͷؒͷͷ Step ʹ෼͚ΒΕɼ֤ Step ͷৄࡉΛઆ໌͢ΔɽίʔυΫϩʔϯΛߴ଎ʹ‫ݕ‬ग़͢ΔπʔϧͰ͋ΔɽYOCCA

Step 1:ࣈ۟ղੳ ɹͰ͸ N-gram ͱ͍͏ख๏Λ࢖༻ͯ͠‫ڊ‬େͳιʔείʔυ‫܈‬ιʔείʔυΛϓϩάϥϛϯά‫ޠݴ‬ͷจ๏ʹԊͬͯ

Λσʔλϕʔεʹอଘͯ͠ɼίʔυΫϩʔϯ‫ݕ‬ग़෦ͷߴ଎τʔΫϯྻʹม‫͢׵‬Δɽͦͷࡍɼۭനͱίϝϯτ͸‫ػ‬ԽΛਤ͍ͬͯΔɽ͔͠͠ɼYOCCA ͸ 1 ͭͷϑΝΠϧͱ‫ڊ‬େͳιʔείʔυ‫܈‬ͷؒͷίʔυΫϩʔϯΛߴ଎ʹ‫ݕ‬ग़ͰೳʹӨ‫͍ͳ͠ڹ‬ͷͰແࢹ͞ΕΔɽ

Step 2:ม‫ॲ׵‬ཧ ɹ͖Δ΋ͷͷɼશͯͷϑΝΠϧؒͷίʔυΫϩʔϯ‫ݕ‬ग़ʹ͸෼ׂ͞ΕͨτʔΫϯྻ͔Β࣮༻తʹҙຯͷ͋Δίʔυద͍ͯ͠ͳ͍ɽࠓճ͸ YOCCA Ͱ࢖༻͞Ε͍ͯͨ N-gramΫϩʔϯ͚ͩΛ‫ݕ‬ग़͢ΔͨΊͷม‫׵‬Λߦ͏ɽ͜Ε͸มΛ༻͍ͨΞϧΰϦζϜΛࢀߟʹɼશͯͷϑΝΠϧؒͷίʔ਺໊΍ؔ਺໊ͳͲΛશͯ 1 ͭͷτʔΫϯʹஔ͖‫͑׵‬Δ

υΫϩʔϯ‫ݕ‬ग़Λߦ͏πʔϧΛ࡞੒ͨ͠ɽॲཧͰ͋Δɽ

ຊ‫Ͱڀݚ‬͸ɼఏҊͨ͠ॊೈ͔ͭ༰қʹมߋՄೳͳࣈ۟ղ

Step 3:Ϋϩʔϯ‫ݕ‬ग़ ɹੳ‫ߏػ‬ͷద༻࣮‫ͯ͠ͱݧ‬ɼଟ‫ޠݴ‬ͷιʔείʔυʹରͯ͠ɼม‫͞׵‬ΕͨτʔΫϯྻΛɼൺֱ͠ίʔυΫϩʔϯΛ‫ݕ‬ίϝϯτআ‫ڈ‬Λߦ͏લ‫ޙ‬ͷιʔείʔυΛൺֱΛߦͬͨɽ

ग़͢Δɽ͜ͷൺֱʹ͸ suﬃx-tree ͱ͍͏໦ߏ଄Λ༻͍͜ͷ݁Ռɼ໿ 92 ˋͷ‫Ͱޠݴ‬ίϝϯτআ‫͕ڈ‬Մೳͩͬͨɽ·ͨΞϧΰϦζϜΛ࠾༻͍ͯ͠ΔɽͨఏҊख๏Λ༻͍ͯ։ൃͨ͠πʔϧΛ࣮ߦ͢Δ͜ͱͰɼશ

Step 4:ग़ྗ੔‫ॲܗ‬ཧ ɹͯͷ‫Ͱޠݴ‬ਖ਼͘͠ίʔυΫϩʔϯ͕‫ݕ‬ग़͞Ε͍ͯΔ͜ͱΛ࠷‫ʹޙ‬ग़ྗ੔‫ॲܗ‬ཧΛߦ͍ɼ‫ݕ‬ग़͞ΕͨΫϩʔϯϖΞࣔͨ͠ɽʹ͍ͭͯɼ΋ͱͷιʔείʔυͰͲͷϑΝΠϧʹଘࡏ

2 ষͰ͸ຊ‫ڀݚ‬ͷഎ‫ͯ͠ͱܠ‬ɼίʔυΫϩʔϯ‫ݕ‬ग़πʔ

͢Δ͔ɼߦ൪߸ͱྻ൪߸͕ग़ྗ͞ΕΔɽ

ϧͰ͋Δ CCFinder ͷઆ໌Λߦ͏ɽ3 ষͰ͸ຊ‫Ͱڀݚ‬ఏҊ

CCFinderX ͸ CCFinder ͷόʔδϣϯΞοϓͰ͋Γɼಉ

͢Δࣈ۟ղੳͷख๏ͱɼN-gram Λ༻͍ͨίʔυΫϩʔϯ༷ʹτʔΫϯ୯ҐͷίʔυΫϩʔϯΛ‫ݕ‬ग़͢ΔπʔϧͰ‫ݕ‬ग़෦ͷΞϧΰϦζϜʹ͍ͭͯઆ໌͢Δɽ4 ষͰ͸։ൃ͠

͋ΔɽCCFinder ͱൺ΂Δͱੑೳ͕޲্͍ͯͯ͠ɼࣈ۟ղ

ͨπʔϧͷ༗༻ੑΛࣔͨ͠ద༻࣮‫ݧ‬Λ·ͱΊɼ5 ষͰ͸·ੳ෦ͷϢʔβʹΑΔมߋ͕Մೳʹͳ͍ͬͯΔ [11]ɽ͜Ε͸ͱΊͱࠓ‫ޙ‬ͷ՝୊ʹ͍ͭͯड़΂ΔɽίʔυΫϩʔϯ‫ݕ‬ग़Λߦ͍͍ͨ‫ʹޠݴ‬ରԠ͢Δࣈ۟ղੳ෦

2. ίʔυΫϩʔϯ‫ݕ‬ग़πʔϧ:CCFinderຊষͰ͸ຊ‫ڀݚ‬ͷഎ‫ͯ͠ͱܠ‬ɼίʔυΫϩʔϯ‫ݕ‬ग़πʔϧͰ͋Δ CCFinder ʹ͍ͭͯͷઆ໌Λߦ͏ɽΛϢʔβ͕༻ҙ͢Δ͜ͱͰɼͦͷ‫ޠݴ‬ͷίʔυΫϩʔϯ‫ݕ‬ग़͕ՄೳʹͳΔͱ͍͏͜ͱΛҙຯ͢Δɽ͔͠͠ɼࣈ۟ղੳ
෦Λ༻ҙ͢Δʹ͸‫ޠݴ‬ͷจ๏Λཧղ͢Δ͜ͱ͕ඞཁͰ͋
Γɼࣈ۟ղੳͷίʔσΟϯά͸खؒͷ͔͔Δ࡞‫͋Ͱۀ‬Δɽ

ίʔυΫϩʔϯʹ͸ɼීวతఆٛ͸ଘࡏ͠ͳ͍ɽຊ࿦จ

ຊ‫ڀݚ‬͸ɼࣈ۟ղੳ෦Λ‫߹ʹޠݴ‬Θͤͯ༰қʹมߋͰ͖Ͱ͸ɼίʔυΫϩʔϯͷఆٛͱͯ͠ҎԼͷ 2 ͭͷλΠϓͷ

ΔΑ͏ʹ։ൃ͠ɼଟ‫ޠݴ‬ͷίʔυΫϩʔϯ‫ݕ‬ग़Λߦ͏͜ͱ෼ྨΛ༻͍Δ [7]ɽΛ໨ඪͱͨ͠ɽ

λΠϓ 1

ɹ

c 2017 Information Processing Society of Japan

2

Vol.2017-SE-196 No.20
2017/7/20

৘ใॲཧֶձ‫ڀݚ‬ใࠂ
IPSJ SIG Technical Reportϧ͸‫Ͱͭز‬΋௥ՃͰ͖Δ͕ɼઌʹ௥Ճͨ͠ϧʔϧΛ༏ઌత䝋䞊䝇䝣䜯䜲䝹

&ŝŶĚĞƌ^t
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000^ƚĞƉϭ

0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Ꮠྃゎᯒ䝁䝯䞁䝖㝖ཤ
䝖䞊䜽䞁ศ๭
㆑ูᏊุูʹॲཧ͢ΔͷͰɼ‫͢߹ڝ‬Δϧʔϧ͕ଘࡏ͢ΔՄೳੑ΋͋Δɽ

䝁䝯䞁䝖䝹䞊䝹

͍ͯ‫͢ه‬ɽྫͷதͷ੺͍จࣈ͕ɼࣈ۟ղੳʹ͓͍ͯແࢹ͞

ண⣙ㄒ

䝖䞊䜽䞁ิ

^ƚĞƉϮ

00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

ኚ᥮ฎ⌮ኚ᥮῭䝖䞊䜽䞁ิ^ƚĞƉϯ

0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

䜽䝻䞊䞁᳨ฟ
䜽䝻䞊䞁᝟ሗ

^ƚĞƉϰ

00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

ฟຊᩚᙧฎ⌮
䜽䝻䞊䞁䝨䜰᝟ሗਤ 1

ҎԼʹɺ5 ͭͷϧʔϧʹ͍ͭͯͷৄࡉΛͦΕͧΕྫΛ༻

CCFinderSW ͷ֓ཁ

Fig. 1 An Overview of CCFinderSW

ΕΔ෦෼Ͱ͋Δɽ
ߦίϝϯτ ɹɹ͋Δ‫͔߸ه‬Βߦ຤·ͰΛίϝϯτͱͯ͠ѻ͏ίϝϯτΛɼߦίϝϯτͱ‫Ϳݺ‬ɽC ‫ޠݴ‬΍ Java Ͱ͸ɼʠ//ʡҎ͕߱ߦίϝϯτͱΈͳ͞ΕΔɽ

v=v+i; //commentෳ਺ߦίϝϯτ

ຊষͰ͸ɼຊ‫Ͱڀݚ‬։ൃΛߦͬͨ༰қʹมߋՄೳͳࣈ۟

ɹɹ͋Δ։࢝‫͔߸ه‬Βऴྃ‫ʹͰ·߸ه‬ग़‫͢ݱ‬ΔจࣈΛίϝϯτͱͯ͠ѻ͏ίϝϯτΛɼෳ਺ߦίϝϯτ·ͨ͸ϒϩοΫίϝϯτͱ‫Ϳݺ‬ɽC ‫ޠݴ‬΍ Java Ͱ͸ɼʠ/*ʡ
ͱʠ*/ʡͰғΘΕͨจࣈ͕ෳ਺ߦίϝϯτͱͯ͠ѻΘΕΔɽɹ

3. ఏҊπʔϧɿCCFinderSW



ߦίϝϯτ



ෳ਺ߦίϝϯτ

v=v+i; ɹ/*comment
printf(”Hello world”); ɹˡ comment */ղੳ‫ߏػ‬Λ࣋ͭίʔυΫϩʔϯ‫ݕ‬ग़πʔϧͰ͋Δ CCFind-

printf(”Hello world”); ɹɹ/*comment*/


erSW ͷॲཧ֓ཁͱɼͦͷ֤εςοϓͷৄࡉΛड़΂Δɽπʔɹෳ਺ߦίϝϯτͷωετΛ‫͕ޠݴ͢ڐ‬ଘࡏ͢Δɽίϧͷ࣮૷͸ Java Ͱߦͬͨɽ

ϝϯτͷωετͱ͸ɼෳ਺ߦίϝϯτͰғΘΕͨதʹɼ

ਤ 1 ͸ ։ ൃ ͠ ͨ CCFinderSW ͷ ॲ ཧ ֓ ཁ Ͱ ͋ Δ ɽ͞Βʹෳ਺ߦίϝϯτͷ‫ه‬ड़͕ଘࡏ͢Δ৔߹Ͱ͋Δɽ

CCFinder ͱಉ͘͡ɼλΠϓ 2 ·Ͱͷࣈ۟୯Ґͷίʔυ۩ମతͳྫΛ༻͍ͯɼωετΛ‫͢ڐ‬৔߹ͷίϝϯτʹ

ΫϩʔϯΛ‫ݕ‬ग़͢ΔɽCCFinder ͷॲཧ֓ཁΛࢀߟʹ࣮૷͍ͭͯઆ໌͢Δɽ

Λߦ͍ͬͯΔͨΊɼ͓͓·͔ͳॲཧͷॱ൪͸ಉ͡Ͱ͋ΔɽมߋՄೳͳࣈ۟ղੳ෦ͷ࣮૷ʹ͓͚ΔओཁͳϙΠϯτͱͯ͠ɼίϝϯτআ‫ͱڈ‬τʔΫϯ෼ׂͷํ๏Λ‫͛ڍ‬Δɽ
πʔϧΛಈ͔͢ࡍͷΦϓγϣϯͷೖྗͱͯ͠ɼίϝϯτϧʔϧͱ༧໿‫ݶʹޠ‬ఆͨ͠ɽίϝϯτϧʔϧ͸ࣈ۟ղੳ෦
ͷίϝϯτআ‫ॲڈ‬ཧͰ࢖༻͠ɼ༧໿‫ޠ‬͸ࣝผࢠ൑ผͰ࢖༻͢Δɽ

3.1 અɼ3.2 અͰࣈ۟ղੳ෦ͷৄࡉʹ͍ͭͯ‫ه‬ड़͢Δɽ3.3
અͰ͸ࣝผࢠ൑ผͱม‫ॲ׵‬ཧʹ͋ͨΔ෦෼Λ‫ه‬ड़͢Δɽ3.4
અͰ͸ Step3 Ͱ࣮ࡍʹ༻͍ΒΕ͍ͯΔख๏ͷৄࡉΛ‫ه‬ड़
͠ɼ3.5 અͰग़ྗ੔‫ॲܗ‬ཧͱΫϩʔϯϖΞ৘ใͷղੳʹ͍ͭͯ‫ه‬ड़͢Δɽ

3.1 ίϝϯτͷআ‫ڈ‬ϓϩάϥϛϯάʹ͓͚Δίϝϯτ͸ɼਓ͕ؒಡΉ͜ͱΛ໨తʹϝϞͱͯ͠ιʔείʔυதʹૠೖ͞ΕΔ஫ऍͷ͜ͱΛࢦ͢ɽͦͷͨΊɼϓϩάϥϜͷ‫ػ‬ೳʹ͸ؔ܎͕ͳ͍ɽλΠϓ 1 ͷίʔυΫϩʔϯͷఆٛΑΓɼίϝϯτΛআ͍ͨίʔυΫϩʔϯΛ‫ݕ‬ग़͢ΔɽωετΛ‫͢ڐ‬৔߹

1 ɹ/*
2 ɹɹ/*
3 ɹɹɹ comment
4 ɹɹ*/
5 ɹɹ comment
6 ɹ*/


ɹ 1 ߦ໨ʹग़‫͢ݱ‬Δ/*ʹରԠ͢Δͷ͸ɼωετΛ‫͞ڐ‬ͳ͍৔߹͸ 4 ߦ໨ͷ*/Ͱ͋Γɼ2 ߦ໨ͷ/*͸ແࢹ͞Ε
͍ͯΔɽͦΕʹରͯ͠ωετΛ‫͢ڐ‬৔߹Ͱɼ1 ߦ໨ʹग़‫͢ݱ‬Δ/*ʹରԠ͢Δͷ͸ɼ6 ߦ໨ͷ*/Ͱ͋Δɽͦͷҧ͍͸ 2 ߦ໨ͷ/*΋ίϝϯτͷ։࢝‫ͱ߸ه‬Έͳ͞Ε͍ͯΔ͔ΒͰ͋Δɽ4 ߦ໨ͷʠ*/ʡʹΑͬͯ 2 ߦ໨͔Β࢝·Δίϝϯτ͸ऴྃ͞Ε͍ͯΔ͕ɼ5 ߦ໨͸ 1 ߦ໨͔Β࢝·Δίϝϯτ͕·ͩ‫ܧ‬ଓ͞Ε͍ͯΔͨΊɼίϝϯτͱΈͳ͞ΕΔɽ
ɹ͜ͷΑ͏ͳωετͷ༗ແ͸‫ޠݴ‬ͷΑͬͯҧ͍͕͋Δ
ͷͰɼෳ਺ߦίϝϯτʹωετΛ‫ڐ‬Մ͢Δ͔Ͳ͏͔ͷຊ‫Ͱڀݚ‬͸ɼίϝϯτͷϧʔϧΛ 5 ͭʹ෼ྨͨ͠ɽϧʔc 2017 Information Processing Society of Japan

3

Vol.2017-SE-196 No.20
2017/7/20

৘ใॲཧֶձ‫ڀݚ‬ใࠂ
IPSJ SIG Technical Reportද 1Φϓγϣϯ΋ଘࡏ͢Δɽ֤ϧʔϧʹඞཁͳཁૉ

Table 1 Necessary elements for each ruleߦશମίϝϯτɹϧʔϧ։࢝‫߸ه‬

ऴྃ‫߸ه‬

ωετͷ༗ແ

ɹ͋Δ։࢝‫͕߸ه‬ઌ಄ʹଘࡏ͢ΔߦશମΛίϝϯτ

ߦίϝϯτ

˓

ʷ

ʷ

ͱͯ͠ѻ͏ίϝϯτΛఆٛ͠ɼຊ‫Ͱڀݚ‬͸͜ΕΛߦશෳ਺ߦίϝϯτ˓˓˓ମίϝϯτͱ‫Ϳݺ‬ɽFortran ͷίϝϯτͰ͸ɼߦ಄ʹ։࢝‫߸ه‬ʠCʡ·ͨ͸ʠ*ʡ͕͋ΔߦશମΛίϝϯτͱߦશମίϝϯτ

˓ʷʷෳ਺ߦશମίϝϯτ

˓˓ʷจࣈɼจࣈྻϦςϥϧ

˓

˓

ʷ

ѻ͍ͬͯΔͨΊɼ͜ͷΑ͏ͳίϝϯτΛແࢹ͢ΔͨΊʹϧʔϧΛઃ͚ͨɽߦίϝϯτͱ͸ҧ͍ɼߦͷઌ಄Ͱif(b==c) value=i ;͸ͳ͍৔ॴʹ‫ݱ͕߸ه‬Εͯ΋ίϝϯτͷ։࢝‫͠ͱ߸ه‬ͯೝࣝ͠ͳ͍ɽɹ

ߦશମίϝϯτ

c

This is a comment*This is a commentෳ਺ߦશମίϝϯτ



if

(

b

=

=

c

)

value

if

(

$

=

=

$

)

$



ɹ

ਤ 2

=

i

;

= $ ;

τʔΫϯ෼ׂͱม‫ॲ׵‬ཧ

Fig. 2 Tokenization and Transformation

ɹ͋Δ։࢝‫͕߸ه‬ઌ಄ʹଘࡏ͢Δߦ͔Βऴྃ‫͕߸ه‬ઌ಄ʹଘࡏ͢Δߦ·ͰΛίϝϯτͯ͠ѻ͏ίϝϯτҎ্ͷϧʔϧΛఆٛͨ͠ͱ͜ΖͰɼ֤ϧʔϧΛ௥Ճ͢Δ

Λఆٛ͠ɼຊ‫Ͱڀݚ‬͸͜ΕΛෳ਺ߦશମίϝϯτͱ‫ݺ‬ࡍʹͲͷ৘ใ͕ඞཁʹͳΔ͔Λද 1 ʹ·ͱΊͨɽඞཁͳΒͿɽRuby ͷίϝϯτͰ͸ɼʠ=beginʡͰ࢝·Δߦ͔˓ɼෆඞཁͳΒʷͰද͍ͯ͠ΔɽΒɼ
ʠ=endʡͰ࢝·Δߦ·ͰΛίϝϯτͱѻ͍ͬͯΔɽɹෳ਺ߦશମίϝϯτ



3.2 τʔΫϯ෼ׂຊ‫Ͱڀݚ‬ఏҊ͢Δख๏Ͱ͸ɼτʔΫϯ෼ׂ͸ίϝϯτআ=begin comment‫ڈ‬ͷ‫ߦʹޙ‬ΘΕΔɽτʔΫϯ෼ׂͰ࢖ΘΕΔϧʔϧ͸ҎԼɹ puts ”Comment 1”ɹ commentͷ௨ΓͰ͋Δɽ൪߸͕খ͍͞ϧʔϧ΄Ͳ༏ઌ͞ΕΔɽ

ɹ puts ”Comment 2”ɹ comment( 1 ) จࣈɼจࣈྻϦςϥϧ͸ 1 τʔΫϯͱ͢Δɽ

=end ɹ comment( 2 ) ۭനͱվߦͷલ‫Ͱޙ‬τʔΫϯΛ෼ׂ͢Δɽ
( 3 ) ‫߸ه‬͸ 1 จࣈͣͭͰ෼ׂ͢Δɽ‫͕߸ه‬ෳ਺จࣈͰ 1 ͭputs ”Hello world”
จࣈϦςϥϧɼจࣈྻϦςϥϧͳͲɹͷҙຯΛද͢৔߹Ͱ΋ɼ1 จࣈͰ 1 τʔΫϯͱ͢Δɽ( 4 ) ͦΕҎ֎ͷ࿈ଓͨ͠ΞϧϑΝϕοτ·ͨ͸਺ࣈͷྻ͸ɹ͜ͷϧʔϧ͸ίϝϯτͷϧʔϧͰ͸ແ͍͕ɼίϝϯ1 τʔΫϯͱ͢Δɽ

τʹ༏ઌ͞ΕΔϧʔϧͱͯ͠ઃ͚͍ͯΔ΋ͷͰ͋Δɽ

Java Ͱ͸ɼγϯάϧΫΥʔςʔγϣϯͰғ·Εͨจ

3.3 ม‫ॲ׵‬ཧࣈΛจࣈϦςϥϧɼμϒϧΫΥʔςʔγϣϯͰғ·Εม‫ॲ׵‬ཧ͸ɼ࣮༻తʹҙຯͷ͋ΔίʔυΫϩʔϯ͚ͩΛͨจࣈΛจࣈྻϦςϥϧͱ͞Ε͍ͯΔɽϦςϥϧͱ͸‫ݕ‬ग़͢ΔͨΊʹߦΘΕΔɽ͜Ε͸ࣈ۟ղੳͰࣝผࢠͱ൑ผ

σʔλͦͷ΋ͷΛࢦ͠ɼ͜ͷதʹॻ͔Εͨจࣈ͸஋ͱ͞Εͨม਺໊΍ؔ਺໊ͳͲΛɼશͯ 1 ͭͷτʔΫϯʹஔ͖ͯ͠ϓϩάϥϜͷதʹଘࡏ͍ͯ͠ΔɽΑͬͯɼ͜ͷத‫͑׵‬ΔॲཧͰ͋Δɽม਺໊΍ؔ਺໊ʹ࢖༻Ͱ͖ͳ͍จࣈྻ

ʹίϝϯτϧʔϧͰ࢖༻͞ΕΔจࣈ͕ग़‫ͯ͠ݱ‬΋ɼί͸ϓϩάϥϛϯά‫ʹޠݴ‬ΑͬͯఆΊΒΕ͍ͯͯɼ͜Ε͸༧ϝϯτͷ։࢝఺ͱऴྃ఺ʹ‫·ؚ‬Εͳ͍ɽҎԼ͸ Java໿‫ݺͱޠ‬͹ΕΔɽif ΍ while ͳͲϓϩάϥϜͷྲྀΕͷ੍‫ޚ‬Ͱ࢖༻͞ΕΔจࣈϦςϥϧɼจࣈྻϦςϥϧͷྫͰ͋ʹ࢖༻͞ΕΔ୯‫ޠ‬͸ɼࢦఆ͞Ε͍ͯΔ͜ͱ͕ଟ͍ɽ

ΔɽμϒϧΫΥʔςʔγϣϯͰғ·Εͨจࣈͷதʹɼ

ਤ 2 ͸͋ΔιʔείʔυதͷҰจ͕τʔΫϯ෼ׂ͞Εɼෳ਺ߦίϝϯτͷ։࢝‫·ؚ͕߸ه‬Ε͍ͯΔ͕ɼෳ਺ߦ

ม਺໊Λʠ$ʡͱ͍͏จࣈʹม‫͢׵‬ΔྲྀΕΛ͍ࣔͯ͠Δɽάίϝϯτͷ։࢝ͱ͸Έͳ͞Εͳ͍ɽɹϦʔϯͷϘοΫε͸ӳࣈྻΛද͍ͯͯ͠ɼϒϧʔͷϘοΫจࣈϦςϥϧɼจࣈྻϦςϥϧ ε͸ɼม‫͞׵‬Εͨม਺໊Λද͍ͯ͠Δɽ

String x = ”ɹ Line Comment start = /*ɹ”;String y = ”ɹ Line Comment end = */ɹ”; ɹ

3.4 N-gram Λ༻͍ͨίʔυΫϩʔϯ‫ݕ‬ग़ΞϧΰϦζϜ



CCFinderSW ͸ɼίʔυΫϩʔϯ‫ݕ‬ग़Λߴ଎ʹ‫ݕ‬ग़͢Δ
ͨΊɼYOCCA ͱಉ༷ʹ N-gram Λ࢖༻͍ͯ͠Δ [5]ɽDe-

c 2017 Information Processing Society of Japan

4

Vol.2017-SE-196 No.20
2017/7/20

৘ใॲཧֶձ‫ڀݚ‬ใࠂ
IPSJ SIG Technical Reportද 24-grams ͷྫTable 2 An Example of 4-gramsॴॱϦετͷྫΛࣔͨ͠ɽࠓճɼઆ໌ͷͨΊʹਤ͍ࣔͯ͠Δϋογϡ஋͸ɼπʔϧʹ࣮૷͞Ε͍ͯΔ΋ͷͱ͸ҧ͍ͬͯΔɽ࣮ࡍͷϋογϡԽ൪߸

4-gram0

if($=͸ɼJava ͷ String Ϋϥεʹ༻ҙ͞Εͨ hashcode ͱ͍͏ϝ

1

($==

ιουΛ༻͍ͯɼString ‫ܕ‬ͷจࣈྻͷϋογϡ஋Λੜ੒͠

2

$==$

͍ͯΔɽ·ͨɼ͜ͷઆ໌Ͱ͸ϋογϡ஋ͷিಥ͸ߟ͑ͳ͍

3

==$)

4

=$)$

5

$)$+

6

)$=$

͜ͷग़‫ݱ‬ՕॴॱϦετͷϋογϡ஋ͷू߹͔Βɼॏෳͷ

7

$=$;

ͳ͍શͯͷϋογϡ஋Λબͼग़͢ɽͦͷબͼग़͞Εͨϋοͱ͢Δɽͭ·Γϋογϡ஋͕ಉ஋ͳΒ͹ɼ΋ͱͷจࣈྻ΋ಉ஋Ͱ͋Δͱߟ͑Δɽγϡ஋ͷ֤ʑʹରԠ͢Δɼग़‫ݱ‬ՕॴΛશͯબͼग़͢ɽϋοbian GNU/Linux ͳͲͷେ‫ن‬໛ͳ FOSSʢFree and Openγϡ஋͔Βग़‫ݱ‬ՕॴͷରԠ͸ɼ1 ର 1 Ͱ͸ͳ͍ͨΊɼଟ਺

Source Softwareʣ͔ΒΫϩʔϯΛ‫ݕ‬ग़͢Δ৔߹ɼ๲େͳ‫ݕ‬

ଘࡏ͢Δ৔߹΋͋Δɽϋογϡ஋ͱग़‫ݱ‬Օॴͷू߹Ληοग़͕࣌ؒඞཁ͔ɼ·ͨ͸Ϧιʔεෆ଍ͷͨΊʹΫϥογϡ

τͰ‫ه‬࿥ͨ͠΋ͷΛɼϢχʔΫϦετͱ‫Ϳݺ‬ɽද 4 ʹɼϢ͢ΔՄೳੑ͕͋ΔɽLivieri Β͸ɼͦͷ໰୊ͷղܾࡦͱ͠

χʔΫϦετͷྫΛࣔͨ͠ɽͯɼ‫ط‬஌ͷιʔείʔυͷू߹ͱ͋Δ 1 ͭͷϑΝΠϧؒͷɼ࣍ʹίʔυΫϩʔϯͷ୳ࡧΛߦ͏ɽ࡞੒ͨ͠ϢχʔΫϦτʔΫϯ୯ҐͷίʔυΫϩʔϯ‫ݕ‬ग़Λߦ͏ͨΊͷ࣌ؒత͔ετͷҰߦ໨ͷɼϋογϡ஋ (1234) ʹରԠͨ͠ग़‫ݱ‬Օॴۭͭؒతʹޮ཰తͳख๏ΛఏҊ͠ɼYOCCA Λ։ൃͨ͠ɽ

ͷू߹Ͱ͋Δɼ{0, 5} ʹ஫໨͢Δɻ͜ͷू߹ͷ֤ཁૉʹରN-gram ͱ͸ɼ͋ΔςΩετͷ૯ମΛલ͔Βॱʹ೚ҙ͠ɼग़‫ݱ‬ՕॴॱϦετͰରԠ͢ΔߦΛΈΔͱಉ͡ϋογϡͷ N ‫ݸ‬ͷจࣈྻ·ͨ͸୯‫ޠ‬ͷ૊Έ߹ΘͤͰ෼ׂͨ͠΋ͷ஋͕ग़‫͍ͯ͠ݱ‬Δͱ͍͏͜ͱ͕Θ͔Δɽͭ·Γɼग़‫ݱ‬ՕॴͰ͋Δ [10]ɽN ͷ஋ʹԠͯ͡ɼ1-(uni)gramɼ2-(bi)gramɼ0 ͱ 5 ʹొ৔͢Δ 4-gram ͔Βੜ੒͞Εͨϋογϡ஋͸ಉ

3-(tri)gramɼͱ‫ݺ‬͹ΕΔɽN-gram Λ࢖༻ͨ͠‫ޠݴ‬Ϟσϧ஋Ͱ͋ΓɼΑͬͯ 4-gram ͕ಉ஋Ͱ͋Δͱߟ͑ΒΕΔɽ‫ݴ‬͸ɼྡΓ߹͏จࣈྻͷग़‫ݱ‬ස౓Λௐࠪ͢Δͷʹ࢖༻͞Εɼ͍‫͑׵‬Ε͹ɼ‫ݩ‬ͷจࣈྻͰ 0 จࣈ໨͔Β 3 จࣈ໨·Ͱͷࣗવ‫ֶޠݴ‬ͷ౷‫ܭ‬తͳ෼໺Ͱར༻͞ΕΔ͜ͱ͕ଟ͘ [9]ɼจ

4 จࣈͱɼ5 จࣈ໨͔Β 8 จࣈ໨·Ͱͷ 4 จࣈ͸ಉ஋Ͱ͋ࣈྻͷ‫͍༻ʹࡧݕ‬ΒΕΔ͜ͱ΋͋Δɽ

Δͱߟ͑ΒΕɼ͜ΕΛίʔυΫϩʔϯͱ͢Δɽ·ͨɼϋο

CCFinderSW ͰɼίʔυΫϩʔϯ‫ݕ‬ग़ʹ༻͍ͨΞϧΰγϡ஋ (2342) ʹରԠͨ͠ग़‫ݱ‬Օॴͷू߹ {1, 6} ͱɼϋοϦζϜʹ͍ͭͯɼઆ໌͢Δɽࣈ۟ղੳͯ͠ੜ·ΕͨτʔΫγϡ஋ (3421) ʹରԠͨ͠ग़‫ݱ‬Օॴͷू߹ {2, 7} Ͱ΋ಉ͡ϯ͔ΒɼN-gram Λ࡞੒͢Δɽ͜ͷ࣌ͷ N ͷ਺ࣈ͸Ϣʔβ͜ͱ͕͍͑ΔͨΊɼ͜ͷจࣈྻ͔Β͸Ϋϩʔϯηοτ͕ 3ʹΑͬͯઃఆ͞Εͨ΋ͷͰ͋Γɼ‫ݕ‬ग़͞ΕΔίʔυΫϩʔ૊ग़‫͍ͯ͠ݱ‬Δͱ͍͑ΔɽτʔΫϯ਺͕͖͍͠஋ͷίʔυϯͷτʔΫϯ਺ͷ͖͍͠஋Ͱ͋Δɽ؆୯ͷͨΊɼࠓճ͸࠷Ϋϩʔϯ͸͜ͷΑ͏ʹͯ͠‫ݕ‬ग़͞ΕΔɽ௿τʔΫϯ਺Λ 4 ͱ͢Δɽ2 Ͱม‫ॲ׵‬ཧ͕ߦΘΕͨτʔΫ͖͍͠஋ΑΓ΋େ͖͍ίʔυΫϩʔϯ΋ଘࡏ͢ΔͨΊɼϯྻͷɼྡ߹͏ 4 τʔΫϯΛ݁߹ͯ͠ 4-gram Λ࡞੒͢Δɽͦͷ୳ࡧΛߦ͏ΞϧΰϦζϜΛઆ໌͢Δɽ͋ΔΫϩʔϯͦͷ݁Ռɼද 2 ͷΑ͏ʹͳΔɽ

ηοτͷཁૉͷͦΕͧΕͷɼ࣍ͷग़‫ݱ‬Օॴͷϋογϡ஋Λ͔͜͜Β͸؆୯ͷͨΊɼ࣮ࡍͷΞϧΰϦζϜͰ͸τʔ‫ݟ‬Δɽྫ͑͹ɼ4 จࣈͷΫϩʔϯηοτ {0, 5} ͷཁૉͦΕΫϯ୯ҐͰ N-gram Λ࡞੒͍ͯ͠Δͱ͜ΖΛɼจࣈ୯ҐͰͧΕʹ 1 Λ଍͠ 1, 6 ͱͯ͠ɼग़‫ݱ‬ՕॴॱϦετͰग़‫ݱ‬ՕN-gram Λ࡞੒͢ΔɽιʔείʔυશମΛ sethesethesesॴ 1 ͱग़‫ݱ‬Օॴ 6 ʹରԠ͢Δϋογϡ஋Λ‫ݟ‬ΔͱɼͲͪͱ͢Δɽ‫ݕ‬ग़͢ΔίʔυΫϩʔϯͷτʔΫϯ਺ͷ͖͍͠஋Β΋ (2342) ͰҰக͍ͯ͠Δɽͭ·Γग़‫ݱ‬Օॴ͕ 0 ͱ 5 ͷ

Λ 4 ͱ͠ɼ4 จࣈҎ্ͷҰகՕॴΛ‫͚ͭݟ‬Δɽ

4-gram ͕Ұக͠ɼ1 ͱ 6 ͷ 4-gram ͕Ұக͍ͯ͠ΔͷͰɼ͖͍͠஋͕ 4 ͷͨΊɼ4-gram Λ࢖༻͢Δɽ·ͣɼιʔεग़‫ݱ‬Օॴ 0 ͱ 5 ͔Β࢝·Δ 5 จࣈ͕ίʔυΫϩʔϯʹͳͬίʔυશମ͔Β 4-gram Λ࡞੒͢Δɽͦͷ 4-gram ΛϦε͍ͯΔͱ‫͑ݴ‬Δɽ͜͏ͯ͠ 4 จࣈͷΫϩʔϯηοτ {0, 5}

τԽ͢Δࡍʹ͸ɼग़‫ݱ‬Օॴɼจࣈྻɼϋογϡ஋Ληοτ͸ɼ5 จࣈͷΫϩʔϯηοτ {0, 5} ʹॻ͖‫͑׵‬ΒΕΔɽ͜

ͰϦετԽΛߦ͏ɽग़‫ݱ‬Օॴ͸ɼॳΊʹ‫ݱ‬ΕΔ 4-gram ΛͷΑ͏ʹΫϩʔϯηοτͷग़‫ݱ‬Օॴཁૉͷɼ࣍ͷग़‫ݱ‬Օॴ

0 ൪໨ͱͯ͠ɼ࣍ʹ‫ݱ‬ΕΔ 4-gram Λ 1 ൪໨ͱ͍͏൪Ͱઃఆ

ͷϋογϡ஋Λ‫ݟ‬Δ͜ͱͰ৽ͨͳΫϩʔϯηοτΛ୳ࡧ͢

͍ͯ͠Δɽϋογϡ஋͸ɼจࣈྻΛϋογϡԽͨ͠΋ͷͰΔɽ͜ͷ࡞‫ۀ‬ΛɼϢχʔΫϦετͷશͯͷϋογϡ஋ʹର͋Δɽ͜ͷϋογϡԽ͸ɼจࣈྻͷൺֱΑΓ਺஋ͷൺֱͷ࣮͠ߦ͢Δ͜ͱͰίʔυΫϩʔϯ୳ࡧΛऴ͑Δɽ

΄͏͕଎͍͜ͱΛར༻ͯ͠ɼߴ଎ԽΛਤͬͨ΋ͷͰ͋Δɽ͜ͷΑ͏ʹग़‫ݱ‬Օॴɼจࣈྻɼϋογϡ஋ΛηοτͰϦετԽͨ͠΋ͷΛग़‫ݱ‬ՕॴॱϦετͱ‫Ϳݺ‬ɽද 3 ʹɼग़‫ݱ‬Օc 2017 Information Processing Society of Japan

3.5 CCFinderSW ͷग़ྗ‫ࣜܗ‬
CCFinderSW ͸ Step4 ͱͯ͠ग़ྗ੔‫ॲܗ‬ཧΛߦ͍ɼϑΝ

5

Vol.2017-SE-196 No.20
2017/7/20

৘ใॲཧֶձ‫ڀݚ‬ใࠂ
IPSJ SIG Technical Reportද 3ग़‫ݱ‬ՕॴॱϦετͷྫ

Table 3 An Example of a List in Order of Appearanceϩάϥϛϯά‫͠૷࣮Ͱޠݴ‬ɼͦͷιʔείʔυΛ΢Σϒ্ʹެ։͍ͯ͠Δ΢ΟΩ‫ࣜܗ‬ͷ΢ΣϒϖʔδͰ͋Δ*2 ɽಉ͡{ ग़‫ݱ‬Օॴ }[จࣈྻ](ϋογϡ஋)λεΫ΁ͷ࣮૷Λ༷ʑͳ‫Ͱͱ͜͏ߦͰޠݴ‬ɼͦΕͧΕͷϓ{0}[seth](1234)

ϩάϥϛϯά‫Ͳ͕ޠݴ‬ͷΑ͏ʹྨࣅ͠ɼͲͷΑ͏ʹҟͳͬ

{1}[ethe](2342){2}

͍ͯΔ͔Λఏࣔ͢Δ͜ͱΛ໨తͱ͍ͯ͠Δ [6]ɽ[thes](3421){3}[hese](4212){4}[eset](2123){5}[seth](1234){6}[ethe](2342)

࢖༻͍ͯ͠Δ*3 ɽ·ͨɼͦͷϦϙδτϦͷ 2015 ೥ 11 ݄ 18

{7}[thes](3421)

೔ͷίϛοτʹΑΔ΋ͷͰ͋Δɽ

{8}[hese](4212){9}[eses](2121)·ͨຊ‫͢༻࢖Ͱڀݚ‬Δ RosettaCode ͷιʔείʔυ͸ɼRosettaCode ͷ΢ΣϒαΠτʹ‫͞ࡌܝ‬Ε͍ͯΔ΋ͷ͕

Github ্ͷϦϙδτϦʹϛϥʔϦϯά͞Ε͍ͯΔ΋ͷΛ

4.2 ίϝϯτআ‫ڈ‬ͷ༗༻ੑͷධՁຊ‫Ͱڀݚ‬ఏҊ͢Δίϝϯτϧʔϧͱͦͷআ‫ڈ‬ख๏ʹ͍ͭද 4ϢχʔΫϦετͷྫ

Table 4 An Example of a Unique Listͯͷɼ༗༻ੑΛࣔͨ͢Ίͷద༻࣮‫ݧ‬Λߦͬͨɽద༻ର৅ͱ
ͳΔͷ͸ɼRosettaCode ͷʠCommentsʡͱ͍͏໊લͷλ(ϋογϡ஋){ ग़‫ݱ‬Օॴ }εΫͰ͋Δɽ͜ͷλεΫ͸ιʔείʔυʹඞ֤ͣ‫ޠݴ‬ͷί

(1234)

{0,5}
{1,6}

ϝϯτ͕‫·ؚ‬Ε͍ͯΔͨΊɼࠓճͷ࣮‫ʹݧ‬ద͍ͯ͠Δɽ

(2342)(3421){2,7}(4212){3,8}࣮‫ݧ‬ͷखॱ͸ҎԼͷ௨ΓͰ͋Δɽ

(1)Comments λεΫͰ࢖༻͞Ε͍ͯΔɼఏҊख๏Ͱ࡟(2123){4}আՄೳͳ֤‫ޠݴ‬ͷίϝϯτϧʔϧΛྻ‫͢ڍ‬Δɽ

(2121){9}ɹ͜ͷத͔Βɼࣈ۟ղੳͷൣғͰՄೳͰ͕͋ͬͨɼࠓ
ճͷఏҊख๏Ͱ͸আ‫͍ͳ͖Ͱڈ‬ίϝϯτΛআ‫ڈ‬ෆՄೳͱͨ͠ɽΠϧʹग़ྗ͢Δࡍʹ CCFinderɼ·ͨ͸ CCFinderX ͱಉ͡‫Ͱࣜܗ‬ग़ྗ͢Δ͜ͱ͕Ͱ͖ΔɽCCFinder ͸ίʔυΫϩʔ

(2)ΦϓγϣϯΛ 26 छྨ࡞੒͢Δɽ͜ͷΦϓγϣϯʹ

ϯ෼ੳπʔϧͰ͋Δ Gemini[13] Ͱ࢖༻Ͱ͖Δ‫Ͱࣜܗ‬ग़ྗΑͬͯଟ͘ͷ‫ޠݴ‬ͷίϝϯτϧʔϧΛ໢ཏग़དྷΔΑ͏͞ΕɼCCFinderX ͸ಉ͘͡ίʔυΫϩʔϯ෼ੳπʔϧͰ͋ʹ͢Δɽ

Δ GemX ʹରԠͨ͠‫Ͱࣜܗ‬ग़ྗ͞Ε͍ͯΔ [11]ɽͭ·ΓɼɹϢʔβʹΑΔίϝϯτϧʔϧͷઃఆΛߋʹ༰қʹ͢

CCFinderSW Ͱग़ྗ͞ΕͨΫϩʔϯϖΞ৘ใ͸ Gemini ͱΔͨΊͷ࢓૊Έͱͯ͠ɼΞϧϑΝϕοτྻΛ༩͑Δͩ

GemX Ͱ෼ੳͰ͖Δɽ

͚ͰίϝϯτϧʔϧͷઃఆΛߦ͑Δͱ͍͏࢓૊ΈΛ࡞

ਤ 3 ͸ Python ͷιʔείʔυʹରͯ͠ CCFinderSW

੒ͨ͠ɽ26 छྨͱ͍͏‫ݶ‬ఆʹ͸ɼπʔϧΛ࣮ߦ͢Δࡍ

Λ࣮ߦͨ݁͠ՌΛɼGemini Ͱදࣔͨ͠΋ͷͰ͋ΔɽਤͷࠨʹɼҾ਺ͱͯ͠ίϝϯτϧʔϧΛ‫ه‬ड़͠΍͍͢ͱ͍͏

ଆʹදࣔ͞ΕͨϑΝΠϧͷ 62 ߦ໨͔Β 73 ߦ໨ͱɼӈଆʹ͜ͱΛ‫ࠐݟ‬ΜͰ͍Δɽίϝϯτͷϧʔϧ͸ՃࢉํࣜͰ

දࣔ͞ΕͨϑΝΠϧͷ 52 ߦ໨͔Β 63 ߦ໨͕ΫϩʔϯϖΞ͋Γɼྫ͑͹ʠadf”ͱ͍͏Ҿ਺Λ༩͑ͯπʔϧΛ࣮ߦʹͳ͍ͬͯͯɼͦΕͧΕ੺ͱ੨ͰϋΠϥΠτ͞Ε͍ͯΔɽ2

͢Δͱɼa ͱ d ͱ f ʹઃఆ͞ΕͨίϝϯτϧʔϧͰίͭͷΫϩʔϯϖΞ͸ؔ਺໊ͱม਺໊͕ҧ͓ͬͯΓɼ·ͨࠨϝϯτআ‫ڈ‬Λߦ͏ɽଆͷϑΝΠϧͷ 66 ߦ໨ʹ͸ίϝϯτ͕‫·ؚ‬Ε͍ͯΔͨΊɼ

ද 5 ʹɼ֤ΦϓγϣϯΛࣔͨ͠ɽz ʹରԠ͢Δϧʔϧ

λΠϓ 2 ͷίʔυΫϩʔϯ͕‫ݕ‬ग़͞Ε͍ͯΔ͜ͱ͕Θ͔Δɽ

͸ෳ਺ߦίϝϯτͷωετͷ‫ڐ‬ՄͷΦϓγϣϯΛׂΓ

CCFinder ͱ CCFinderX ͸ Python ͷίʔυΫϩʔϯ‫ݕ‬౰ͯΔͨΊɼදʹ͸ࡌ͍ͤͯͳ͍ɽࠨ͔Βೋྻ໨ͷ਺ग़ʹରԠ͍ͯ͠ͳ͍ͨΊɼຊ‫Ͱڀݚ‬։ൃͨ͠ CCFinderSWࣈ͸࢖༻ͨ͠ίϝϯτͷ෼ྨΛද͓ͯ͠Γɼ1 ͸ߦίͰ৽ͨʹରԠͰ͖ͨ͜ͱ͕ࣔ͞Ε͍ͯΔɽ

ϝϯτɼ2 ͸ෳ਺ߦίϝϯτɼ3 ͸ߦશମίϝϯτɼ4͸ෳ਺ߦશମίϝϯτɼ5 ͸จࣈɾจࣈྻϦςϥϧͷ

4.3 અͷద༻࣮‫Ͱݧ‬͸ɼ݁Ռͷ֬ೝΛ Gemini Ͱߦͬͨɽ

ϧʔϧͰ͋Δɽͳ͓ɼ֤‫ʹޠݴ‬ର͢ΔΦϓγϣϯͷద

4. ద༻࣮‫ݧ‬༻݁Ռͷৄࡉ͸ɼจ‫[ ݙ‬12] ʹ‫͍ͯ͠ࡌܝ‬Δɽ
͜ͷΦϓγϣϯΛ࢖༻ͯ͠ɼRosettaCode ͷʠCommentsʡຊষͰ͸ɼ։ൃͨ͠πʔϧͷ༗༻ੑΛࣔͨ͢Ίʹߦͬͨద༻࣮‫͍ͯͭʹݧ‬આ໌͢Δɽ

͕࣮૷͞Εͨ 175 ‫ʹޠݴ‬ରͯ͠ίϝϯτআ‫ڈ‬Λߦͬͨɽద
༻ग़དྷͳ͔ͬͨ‫ʹޠݴ‬͸ɼΦϓγϣϯͷྻʹෆՄͱ‫ه‬ड़

4.1 αϯϓϧίʔυू:RosettaCode
RosettaCode ͸ɼಉҰͷλεΫɼ·ͨ͸ྫ୊Λ༷ʑͳϓc 2017 Information Processing Society of Japan


*2
*3

http://rosettacode.org/wiki/Rosetta Codehttps://github.com/acmeism/RosettaCodeData

6Vol.2017-SE-196 No.20
2017/7/20

৘ใॲཧֶձ‫ڀݚ‬ใࠂ
IPSJ SIG Technical Reportਤ 3Gemini Λ༻͍ͨίʔυΫϩʔϯͷදࣔ

Fig. 3 Visualization of Code Clone Using Geminiද 5

26 छྨͷΦϓγϣϯ (z Λআ͘)

Table 5 26 Options (exclude z)

a

෼ྨ

։࢝

ऴྃ

5

’

’

o

෼ྨ

։࢝

ऴྃ

2

(*

*)

෼ྨ

։࢝

ऴྃ

1

c


2

(:

ͳ͠

:)

1

*>

ͳ͠

2
2

(#

#)

1

*|

ͳ͠

#|

|#

3

NOTE

ͳ͠

3

*

ͳ͠

3

*

ͳ͠

u

b

5

”

”

c

1

//

ͳ͠

2

/*

*/

2

/+

+/

2

#=

=#

2

<!–

–>

2

#{

}#

###

###

3

C

ͳ͠
ͳ͠

p

v

d

1

;

ͳ͠

2

e

1

#

ͳ͠

2

#cs

#ce

3

c

f

1

–

ͳ͠

2

#˜

˜#

3

:

ͳ͠

2

{|

|}

2

%REM

%END REM

2

{-

-}

1

REM

ͳ͠
ͳ͠

q

w

g

1

%

ͳ͠

h

1

!

ͳ͠

1

#!

ͳ͠

r

2

%{

%}

1

rem

1

’

ͳ͠

s

1

->

ͳ͠

1

Rem

ͳ͠

1

‘

ͳ͠

1

–##

ͳ͠

4

==comment

=cut

1

NB.

ͳ͠

2

–[[

]]

4

=begin=end

1

COMMENT

ͳ͠

4

=pod

=cut

3

#;

ͳ͠

3

NIL

ͳ͠

3

###

ͳ͠

3

end.

ͳ͠

i
j

t

k

2

{

}

l

2

[

]

1

IGNORELINE

ͳ͠

)

2

’COMMENT’

;

m
n

2
2

(
”

”

ͨ͠ɽ
͜ͷ࣮‫݁ݧ‬ՌΑΓɼ175 ‫ޠݴ‬ͷ͏ͪ 166 ‫͕ޠݴ‬ఏҊख๏ͰՄೳͰ͋Γɼ151 ‫ ͕ޠݴ‬26 छྨͷΦϓγϣϯʹΑͬͯରԠ͕ՄೳͰ͋Δ͜ͱ͕ࣔ͞ΕͨɽఏҊख๏ͰରԠෆՄೳͳxy͍‫ͯؔ͠ʹޠݴ‬͸ɼίϝϯτϧʔϧ͕ಠಛͰଞͷ‫ڞͱޠݴ‬௨͢Δ͜ͱ͕ͳ͍৔߹΍ɼίϝϯτϧʔϧ͕ଟ͗͢Δ৔߹
ͳͲ͕͋ͬͨɽ
ද 6 ͸ɼ166 ‫ޠݴ‬தͷ֤Φϓγϣϯͷ࢖༻‫͋Ͱ਺ޠݴ‬Δɽ

‫͕ޠݴ‬ੜ·ΕΔཧ༝ͱͯ͠͸ɼίϝϯτ͕ߏจղੳΛ͠ͳ͚Ε͹શ͘আ‫ڈ‬ग़དྷͳ͍৔߹΍ɼ‫͕ޠݴ‬೉ղͳจ๏Λ࣋ͬ
͍ͯΔ৔߹ͳͲ͕͋ͬͨɽ·ͨɼΦϓγϣϯͰରԠग़དྷͳ

c 2017 Information Processing Society of Japan

4.3 ίʔυΫϩʔϯ‫ݕ‬ग़ͷਫ਼౓ͷධՁ։ൃͨ͠πʔϧͷίʔυΫϩʔϯ‫ݕ‬ग़ͷਫ਼౓ʹ͍ͭͯධ7

Vol.2017-SE-196 No.20
2017/7/20

৘ใॲཧֶձ‫ڀݚ‬ใࠂ
IPSJ SIG Technical Reportද 6֤Φϓγϣϯͷ࢖༻‫਺ޠݴ‬

Table 6 The Numbers of Languages for Each Options
a

b

c

d

e

f

g

h

i

j

k

l

m

n

o

p

q

r

s

t

u

v

w

x

y

z

1

3

55

25

40

14

10

7

11

1

3

2

1

2

14

8

1

1

2

2

2

4

9

2

1

17

ද 7

ίʔυΫϩʔϯ‫ݕ‬ग़ͷ࠶‫཰ݱ‬

Table 7 Recall of the Code Clone Detection
‫໊ޠݴ‬

‫ݕ‬ग़਺

฼਺

࠶‫཰ݱ‬

C

9

9

1
1

C++

7

7

Go

16

16

1

Java

8

8

1

Python

3

3

1

Ruby

3

3

1

Ҏ֎ͷΦϓγϣϯͷઃఆΛߦΘͳ͔͕ͬͨɼτʔΫϯ෼ׂʹؔ͢ΔΦϓγϣϯΛ༻ҙ͢Δ͜ͱ͕ࠓ‫ޙ‬ͷ՝୊ͱ͍͑Δɽँࣙࢀߟจ‫ݙ‬[1][2]

Ձ͢Δ࣮‫ݧ‬ΛߦͬͨɽRosettaCode ͷ͋ΔλεΫͷιʔείʔυͷ͏ͪ਺‫ޠݴ‬ΛબΜͩɽͦͷ‫ޠݴ‬ͷιʔείʔυͷதΛಡΈɼλΠϓ 2 ·ͰͷίʔυΫϩʔϯͰ͋Δ΋ͷΛϦ

[3]ετΞοϓ͢Δɽͦͷ‫ޙ‬ίϝϯτϧʔϧͱ༧໿‫ޠ‬ͷೖྗΛͯ͠։ൃͨ͠πʔϧΛ࣮ߦ͠ɽ‫ݕ‬ग़͞ΕͨίʔυΫϩʔϯ[4]ͷதʹ͋Β͔͡ΊϦετΞοϓͨ͠ίʔυΫϩʔϯ͕‫·ؚ‬Ε͍ͯΔ͜ͱΛ֬ೝͨ͠ɽϦετΞοϓͨ͠ίʔυΫϩʔϯΛਖ਼ղू߹ͱͯ͠ɼ ‫ݕ‬ग़ͨ͠ίʔυΫϩʔϯͱͷ࠶‫཰ݱ‬[5]Λࢉग़ͨ͠ɽ
ࠓճ͸ɼద༻ର৅ͱͯ͠ RosettaCode ಺ͷʠSudokuʡͱ[6]͍͏໊લͷλεΫΛબ୒ͨ͠ɽ͜ͷλεΫΛબΜͩཧ༝͸ɼ

RosettaCode ͷଞͷλεΫͱൺ΂ͯҰͭҰͭͷιʔείʔυ͕௕͘ (ͦΕͧΕ 60 ͔Β 200 ߦఔ౓)ɼίʔυΫϩʔϯ[7]

͕‫·ؚ‬Ε͍ͯΔՄೳੑ͕ߴ͍͜ͱ͕‫͛ڍ‬ΒΕΔɽ·ͨɼύζϧΛղ͘ͱ͍͏಺༰ͷλεΫͰ͋Γɼύζϧͷߏ଄্ɼfor จͳͲͷ੍‫ޚ‬จ͕ଟ༻͞Ε͍ͯΔՄೳੑ͕ߴ͘ɼίʔυ

[8]Ϋϩʔϯ΋ଘࡏ͠΍͍͢ͱߟ͑ΒΕΔ͜ͱ΋‫͛ڍ‬ΒΕΔɽ
ࠓճ͸ର৅‫ޠݴ‬Λ 6 ‫ͯ͠ͱޠݴ‬ɼ࠷খτʔΫϯ਺Λ 15
ͱͯ͠πʔϧΛ࣮ߦͨ͠ɽʠSudokuʡ͕࣮૷͞Εͨ 40 ‫ݴ‬[9]‫ޠ‬ͷ͏ͪɼ༧໿‫ޠ‬Λ༻ҙͰ͖ͨ‫ޠݴ‬Λબͼɼ͞Βʹ࣮ࡍʹτʔΫϯ୯ҐͷίʔυΫϩʔϯ͕ଘࡏͨ͠΋ͷΛબΜͩɽ[10]࣮‫ݧ‬ͷ݁ՌΛɼද 7 ʹࣔ͢ɽ݁ՌɼϦετΞοϓͨ͠ίʔυΫϩʔϯΛશͯ‫ݕ‬ग़͢Δ͜ͱ͕ग़དྷͨɽϦετΞο
ϓͨ͠ίʔυΫϩʔϯΛ฼਺ɼͦͷ͏ͪͰπʔϧ͕‫ݕ‬ग़ͨ͠ίʔυΫϩʔϯ਺Λ‫ݕ‬ग़਺ͱͯ͠ɼ࠶‫཰ݱ‬Λࢉग़ͯ͠

[11]
[12]

͍Δɽ

5. ·ͱΊͱࠓ‫ޙ‬ͷ՝୊ຊ‫Ͱڀݚ‬։ൃͨ͠ίʔυΫϩʔϯ‫ݕ‬ग़πʔϧͰ͸ɼଟ͘ຊ ‫ ڀ ݚ‬͸ JSPS Պ ‫ ݚ‬අ 25220003ɼ16K16034ɼ

15H06344 ͷॿ੒Λड͚ͨɽ[13]ං‫ޙ‬๕थɼೇຊਅೋɼҪ্ࠀ࿠ɿίʔυΫϩʔϯ‫ݕ‬ग़ͱͦͷؔ࿈ٕज़ɼిࢠ৘ใ௨৴ֶձ࿦จࢽɼVol. J91-D, No. 6,
pp. 1465–1481 (2008).
Ҫ্ࠀ࿠ɼਆ୩೥༸ɼೇຊਅೋɿίʔυΫϩʔϯ‫ݕ‬ग़๏ɼίϯϐϡʔλιϑτ΢ΣΞɼVol. 18, No. 5, pp. 47–54(2001).
ං‫ޙ‬๕थɼਆ୩೥༸ɼೇຊਅೋɼҪ্ࠀ࿠ɿίʔυΫϩʔϯΛର৅ͱͨ͠ϦϑΝΫλϦϯάࢧԉ‫ڥ؀‬ɼిࢠ৘ใ௨৴
ֶձ࿦จࢽɼVol. Vol.J88-D-I, No. 2, pp. 186–195 (2005).
Kamiya, T., Kusumoto, S. and Inoue, K.: CCFinder: amultilinguistic token-based code clone detection system
for large scale source code, IEEE Transactions on Software Engineering, Vol. 28, No. 7, pp. 654–670 (2002).
Livieri, S.ɼGerman, D. M.ɼInoue, K.ɿA needle in thestack: eﬃcient clone detection for huge collections ofsource code, Technical report (2010).
Nanz, S. and Furia, C. A.: A Comparative Study of Programming Languages in Rosetta Code, Proceedings ofthe 37th International Conference on Software Engineering, pp. 778–788 (2015).
Roy, C. K., Cordy, J. R. and Koschke, R.: Comparison and evaluation of code clone detection techniques
and tools: A qualitative approach, Science of Computer
Programming, Vol. 74, No. 7, pp. 470–495 (2009).
২ాହ࢜ɼਆ୩೥༸ɼೇຊਅೋɼҪ্ࠀ࿠ɿΫϩʔϯ‫ݕ‬ग़πʔϧΛ༻͍ͨιʔείʔυ෼ੳπʔϧͷࢼ࡞ɼిࢠ৘
ใ௨৴ֶձٕज़‫ڀݚ‬ใࠂɼVol. 101, No. 240, pp. 17–24(2001).
ࢁ ా ਸ ਔɿN-gram Ϟ σ ϧ Λ ར ༻ ͠ ͨ ς Ω ε τ ෼ ੳ ɼ
http://www.shuiren.org/chuden/teach/n-gram/indexj.html.
ࢁాਸਔɿN-gram ํࣜΛར༻ͨ͠‫ࣈ׽‬จ‫ݙ‬ͷ෼ੳɼ໋ཱ‫ؗ‬ന઒ᯩ‫ه‬೦౦༸จࣈจԽ‫لॴڀݚ‬ཁɼNo. 1, pp. 1–23ʢΦϯϥΠϯʣɼೖखઌ http://ci.nii.ac.jp/naid/40016415257/
(2007).
ਆ ୩ ೥ ༸ɿthe archive of CCFinder Oﬃcial Site,
http://www.ccﬁnder.net/.
੉ଜ༤ҰɿॊೈʹมߋՄೳͳࣈ۟ղੳ‫ߏػ‬Λ࣋ͭίʔυΫϩʔϯ‫ݕ‬ग़πʔϧͷ։ൃɼେࡕେֶ‫ֶ޻ૅج‬෦৘ใՊֶՊ ଔ‫ۀ‬࿦จɼʢΦϯϥΠϯʣɼೖखઌ (http://sel.ist.osakau.ac.jp/lab-db/Bthesis/contents.ja/157.html) (2017).
ං‫ޙ‬๕थɼ٢ాଇ༟ɼೇຊਅೋɼҪ্ࠀ࿠ɿ࢈ֶ࿈‫جʹܞ‬͍ͮͨίʔυΫϩʔϯՄࢹԽख๏ͷվྑͱ࣮૷ɼ৘ใॲ
ཧֶձ࿦จࢽɼVol. 48, No. 2, pp. 811–822 (2007).

ͷ‫Ͱޠݴ‬ͷࣈ۟୯ҐͰͷίʔυΫϩʔϯΛ‫ݕ‬ग़Ͱ͖Δ͜ͱ
Λࣔͨ͠ɽ͔͠͠ɼࣈ۟ղੳ͕ग़དྷͳ͔ͬͨ‫ʹޠݴ‬ରͯ͠
ରԠ͢ΔΦϓγϣϯΛ࡞੒͢Δ͜ͱͰɼϢʔβʹΑΔઃఆ͕͞Βʹෳࡶʹͳͬͯ͠·͏Մೳੑ͕͋ΔɽτʔΫϯ෼ׂʹରͯ͠͸จࣈϦςϥϧͱจࣈྻϦςϥϧc 2017 Information Processing Society of Japan

8