8' 9 $-3 .2 #0%)

Scene identiﬁcation by character region segmentation in news videoì ú í 7†
3 0 6 ù‡† ÷5ý4ñõòü12ø ì û í‡0 ÿ ï 2‡
‡ 1óþñþñîöñôõòð
3Ichiro Ide†
Reiko Hamada‡Shuichi Sakai‡
Hidehiko Tanaka‡†National Institute of Informatics ‡Graduate School of Engineering, The University of Tokyo±å: ¾Ïe±ëãdpm?lnÍr[q`8ëãþðÌ¹«±»£mm`üãjîí£XwzléBv£YnV£8ÛzmaUW`qüãjÿÏ
7n²[À5T8Û`iëã mz<ú£®ý|a8ÛT2`¾Ïe±ëã¾HãrëuG|6Q£o`À5i¾H{å£Irz'9{åüã2z£o­| lax<ú`Irzm2`7%q 8%Å#Q±rA `8ÛTePÅr y a
Abstract:

We have been proposing an automatic news video indexing system that considers thecorrespondence between image and keyword. In this paper, we will introduce the scene identiﬁcation
method required for the image content analysis portion of the system, and also the result of its
application to actual video. The method is characteristic that it takes advantage of frequent humanﬁgure existence in news video, by segmenting character region from background when identifying ascene. As a result, the method showed 7% to 8% of improvment in identiﬁcation ability.

 1ëãFPs9dpm?lnÍr[q`Informedia ÅÕ°¨¬» [12] £{`ëãºe´Æe±£Vý|léBv|@%rÊ¤Y¢ mazqz`èç09
léBvT`ëãþðÌ¹«±»qÿy nO£Øô~}pmzp`üãjîí£Xwz`¾HYÙîí£XwzT [11] qAy ûtma8x nÍ`ó £Xwz`ëãþ¹«±»q ´Q£pmz`üãjîí£XwzléBvV£8ÛzmaléBvî°z`ªodpm?lmn
pmõùr[m¾Ïe±ëã£Úna0í9`¿1  |n`üã¹«±»5dq ´Qm`m¢ 4Whjmh When ikgàj̀xh Where ikg²[ j̀örh Who ikg¾Hj̀ô£h What ikgYàiK æÅVj
JZVOLPK
ide@nii.ac.jp
JM 101-8430 \R[WXYUQNS
2-1-2
JM 113-8656 \R[]RU^T 7-3-1
123

È 1 ªÐ×ãÑ«»ç¿©Üµº±åÿÏ£Ym`5dîí uléBv£Ynx
£_zzma¾Ïe±ëãîz`x æÅ?l£ß5|xÚl`xnI5ìlXoaUW`xVqüãjÿÏEm±w`¾Ïe±ëãþ¾Hãr?|x£Xwz`¾H{åIrjÿÏ`'9{åq²[À5TÈ 2 ªÐºÌä¹Ô¶£8Û|a`[4] pmQ]zh4C\£6ñz<õC`8ÛTePÅ£<õ|a¿ 2 ëãVÆmO5 £ |aëãÃÔeÈ 3 ÇÝé¨Þ²¯Ð×ã°ÍËK Î{üãqVÆy `üã9çÃÔeË4£¯Ð·»Ka`¯Ð·»Aç £ª·»Ka¿ 1  zV¯Ð·»÷ÝléBv£_:zma`üã9b '9{å£IrzJLÿÏ|7nrla`j9ç|¯Ð·»4£¯e×K`¾Ïe üãÛí£î°|i`x £JL$Ø|±ëãpm`j9¯Ð·»óîí|a x`È8jÿÏr÷#y¢ axU@%`¿ 3  |n`{åñsyQsrç5µ=£W|n¾H{åh2 $-3.2 '+1&{å£E{åí{åqirüãþ
2.1 7D%)F
èj|²\`¾H{å Ü4'9{å{å£I z`5dmjÿÏ£Ynao`{åI üãjÿÏY¢xnj̀¾Hh Who ikz`!|sr`ç0Åâ£Xwz`¸Y¢  @%rÊ¤Y¢mno`üãþ¾H
`/8{å579I ruaz {å£ÿz`ðÌ£¨sñz¹«±»h¬Õeqz¾Ïe±ëã`¾þÉ|° ²¼«ÎÅ¯Ð×iþ¾HYîíBv@%
|´Q8(øl`ç0¾Hãrñs Y¢ m [11]aj̀Yàh What ikz`uxrëma`¾Ïe±ëãüã Ù¡nñn$À5Tèj| [2, 7]axjÿÏ£Yni`¾Hãèj£Xw|xr `xxj²[h Whereik|üãj£X
nXo a`nÅq`¾H wzléBv£G|i7nüã¦qE
mn5Hèj£Ù8z¦q£ z #z`¾H{å£Irz'9{åüã`ç0Årñsué¢ xmaxn 2z u²[À5T®ý`i¾Ïe±
¾Hã0_z{åI `oüãÛí ëã£mmCePÅ<õ<ú£ |a
579/8{åI üãjÿÏi&©zs q`Ù9G|¾Hãüã2zMê,£&©z`È8ÿÏr 2.2 :BG&#A.G&%C*
÷#xrïy aÙÒ¡nüãjÿÏ£Yn`¾Hãèj`Ýü`¾Ã££z` 79¯Ð·»I}¾Ïe±ëãñn$jÀ5 ¾H{å'9{å£I |7nrlaTèjz [2, 7]`¿ 1 VjYàh What ik x`¾Ïe±üãþ¾Hh̀ 1 iyR­ZÀ5gmzmazqz`xnüãj µ=ñh̀ 2 iÈ[qmêy xrëm`mÿÏ`luüãÛírî°l`4W n nó5 s`Gj 2 9Grjßj̀¾Hh Who ikj²[h Where ik {åÿ£pmz`¿ 4  |n`{å£ÿÏ£Yn`üãþ¾H{åh¾Hãi £z¹×ÅÔe»¾H{å£;5|N

+I1- (<?=$,"!@E;>üã2zjBv£Yn@%z`§z`8j}£þºzè°Oüã2zîí£ºqCq:9$22üºe´Æe±|@% [6] `^UÅO£mmüã?l~ [5] rlaz
È 4 ³é¨´ÄÇÝé¨ÒÃ¥ qz`x ÅS pv~`¾Ïe±ü£ãn0í9°hY}i£î°|²\`ó ÿ;Tráao`¯hÍºÓ u0í9H!`î°Ia£-I5z2zj5£mm
T£ mza{åÿ`CMU pm3*y ¾ÏeÑÓ¿·»Öe¬3£ jÀ5rY¢ sût}`ç09
jÀ5£YnT¬mançmm Face Detector [10] £pmza9ù þüãÓZJq2z
zqz`G9kd¶+ñmêy üã z»`
5£7|T [9] £8Ûzmaxpm`±ó57}zWy }`{å Tj`}ç
09O)5÷O¬ÑÈzmÿ`È[qmê£Ù8z¹×Å ±´1£8Æzpua÷O¬Ñ±´1þÔe» mrs}`ÜñC¾¾ ÷O'r süãBÂ|ÓZJ}
H{å£Ðz£mmmaz`±´ £:rz` £-ëy~Výzüã2z1°©«Î±´I`CmmÛ~h 231 =i þ`ÓZJr}züã£¬Ñ±´ò|axm{åÿ¹×ÅÔe» mûAå`m\¢~üãzÓZJO.?
u ÐYo`9I z<ú£mma `lr÷#r`9ù rÚnî°rç09ût`yR?lÅ#r*y maçS`] Ð¬Ñ±´òz3~üã3 , '*( #0%)üã2z2 `±Èe¸ëã£ím~
z°Î×ÓÀ5T£8Ûzm [8]axT
3.1 5H$,"!(<@E;>`Ù3~£I}ze ¬Ñ±´8Æ£Ü±nÙ9¾H{å£Irz ' Yps`<õ~5¬Ñ±´}
9{åm`üãj`n² uI}£Ynaø_9}zmr`üãÛ[lq`£À5|a`'9{å íî|jÀ5l`88ÛTn
2zjh²[ i5|úr7n üãVÆjt À5Ymmalaç0üãîz`xnú7¡1zu`lazqz`¾Ïe±üãpm`9bþ95°| 4 #0%)!
ó£±w`ç5lmç5zÜ±ÜñCµ=#C¥`i¾Ïe
~r `?²[z`2z²[ ±ëãî|²[À5C£Ya5|úÆe±£Vý|xG9lhßy aU@%`xnhß`'9{å 4.1 6290
2z²[5|~ u²[À5£ 4.1.1 ä¯Ð
Yaxn÷¤~ uÀ5`UW
pvCn2 9«\~4£ß5z
C`2 áþ9üz 15lañ\~4£î°|i IÛ^/¾Ïe±ëã 20 UþçE`817 ¯Ð·»`OL#|:rz q`ð;Æ¬»ÓÑ ÔÃÔeËüã£mma²[ãmÅ
5êPs9À5T r7n #cá£`muq5?²[£¡naþºØ`<õiÈÿ+5hi`
3.2

Ú 1 ÎÔÅáßÀëÊ
²__[¾Hl ¾Hz \__:

h 1 i !Th 2 i ^þ!²h 3 i QþA²h 4 i T7
h 5 i ±´°©
ê\__:

221032311121652
17623123029231

124

332456425392817

È 5 ¡¢¤¡½§Ú 2 ¯Ð  ¾Æ¸
Z___
µ__=

1ÿã·ÿãÿãÃÔeËÙ
Áº©Ù`RGB 1£×8 64 I z·ÄÕ·¬V8s£$`64 DÆ¬»Óz;Gy a

î 320×  240 Â¬³ÓvS
24bith R, G, B 5· 8bit i
15 ÃÔeËV=zh 3ÙiJPEG

Ù_|Èÿ£5pma0í9`
1. !Th cabinet i2. ^þ!²h parliament i
3. QþA²h press conference i
4. T7h court i 4 }^Q5|?²[`5. ±´°©h studio i£Ñ5za; 1 5²[V~Ã£`¾HãeXIv |ap`üãþ¾Hr 3 ¾ÙQsñsum²\j¾Hlkz{åI î°z` Ü4j¾Hzkza`üãø6m¥½Õ­Áº©¹eÅÔ®eµ£mmû±%Râ£üz£`Áº©«ÎÅ¶ÎÇe¼£mme4I JPEG ÙÃÔeË÷Ýº¦°´Óòza«ÎÅ¶Îi¯hµ=£; 2  |a2.

·)?IDh®ÒÕ­ÑËi[1]üãþç5'rr Â¬³Ó·Ý\~G8sIDaUC`RGB 1£×8 16 I z·ÄÕ·¬Ý\~V'r 1f4
8s£$h̀ 16 × 16 × 4 = i1,024 DÆ¬»Óz;Gy ax¿ 5  |ncárl`À±»­ÑËüãÛíÉ¬Õ·f92£`®ÒÕ­ÑËÊ¬Õ2£-ë|ax 2z£gmz
qf`¾_|²[À5i`·f92rñsrqy¢ x`x2zÅcáÀ5Å#ê,£AlaÜñC`5d2z£tmm<õzmr`ëu²[ `²[VOL#|Q±`ê2z£`\9pmr7nlaêÁÕÛ¬ÂÕÜ± 2 2zÆ¬»Ó5dw£2 |
4.1.2 ¯ÐÖÓè}<õz`$ n`Æ¬kdüã2zrèj|q`UC`Ü »Ó F , F 9 θ iEhùåc0f1i£mmcñ 2 3·|:92z£mmc
F · F
1. ·G?IDhÀ±»­ÑËi(1)
F F } ≡ cos θ =| F || F |üãþÂ¬³Ó·G8sIDaUC
4.1.3

12

1

12

122

ÉÔËØÛ¬ÂÕ²[À5Å#<õl`Üñ5 |n}±Ý n Ýû,s£mmai²[À5i`e}zm²[q`±Ý n Ýþû,Ã£Õ²[£À5<ú|`xz:²[À5Å#r y a• }±Ý n Ýû,sh n = 1, 3, 5, 7, 9 ic5Ñ5²[æ|Û<õ~n`}±Ýn ÝþÈrû,Ã£Õ \a<ú
9 n = 1 ù`e}zm²[rÈ4.1.4lÕ \£ |aÈ 6 ÇÝé¨Þ²âàêÁÕÛ¬– With only segmentation¾H{åI £Ym`3~þ¾H{6278årèj|~}£<õz
Ü±µ=`Üñ¥C£Yc
²\h8ÛT`¾Hãèj
Xw ia
1. '9{åIrj¾Hlk~m`_|ç5• üã2zQs`ñsy¾H{å£Ð|a±´°©«Î±´`Face Detector ¿ 4 ¹×ÅÔe– ·À±»­ÑË»9Ir<ú£mma– ·®ÒÕ­ÑË2. 2zÿÐz¾H{å£Irz`'9{åÀ±» 4.3 62/'#34­ÑË®ÒÕ­ÑË£$a¾H{årèjzv `üãÛí£'9{å|a¿ 7 `<õÑ5z; 1 þ²[h 1 iqh 5 iÀ5<ú£á\z<õz­ÑÃla3. }<õ<õ~3~hÛ~q<õ~£©m x<úq`ÜñxrHoci\i}<õ£Yna• n r«yms`®ÒÕ­ÑËÀ±»­ÑË¢}qym<ú£ z`n rñsu
4. ²[À5 `"6QrA ax `®ÒÕ­
3. £Û~îzàM mz`Ñ5²[LÑË3³}züã}£[u |}±Ý n Ýû,s£:|a`¢}qM@l`À±»­Ñp`¿ 6  |n`2. q 4. ¦qpm`Ë"Årlx£ zmax¾H{åI Pú£Aj̀¾Hlk~Å`4.1.2 ¡x2zÊ¬Õbîz`¾H{åIr£Y¢}`üãÛíÉ¬ÕÅqÓZ|xrsa2 K~Ya• 8ÛTl¾H{åIr`2%q 3%xnz`Üñ 3 3}<õµ=`2
Å#Q±r ay`3~þ¾H
3üã2zmC£Ycãrèjzm£©m}<õ<ú`• }<õ7%q 8%Å#Q±rA aN<úq`¾Hãèjüã2zzX– Without segmentationw|xrnlxr by a¾H{åI £Y¢m²\hoT ia`üã2zzÀ±»­ÑËlm®ÒÕ– With segmentation¾H{åI £Ym`Û3~} ­ÑË£mm²\`n £ 1 lm 3 Ñ5z<õ£<õz²\h8ÛT iaz²\pvh̀ 1 i !Th cabinet ih̀ 2 i^þ!
4.2

100%
95%
90%
85%80%
75%
3Histo, w/o segmentation
Histo, with only segmentation
Correlo, with segmentation

15Top n9Histo, with segmentation
Correlo, w/o segmentation
Correlo, with only segmentation7È 7 ÅáÉÔ·­Ï¼Û¬n ¦®Ùæ

cárA a• ²[À5Å#ppq~ÃîÃ2~zma±Ý n =}¯Ð·»û,Ã£Õ
²[ mÀ5£Y`x6Q
 n = 3 ²\h¿ 10 #¿ 11 iB1la3~\rñsuëkl`ëu}~rèj|8sr[u}l`«\ëkÅ4m~4£mm`}±Ý n Ýû,Ã£Èzm²[Õx`lax q`¾H{åIrmmüã2zePÅ`î°|²[áxrIqaxx`²[V2`mêzmªÌÑ|ÄbhÑü÷#Ýübq~
9i `?²[pm`xnÄb²[Lekdüã92`üã9 7X|sm1Þç5M
£ |y¢ a• ·À±»­ÑË·®ÒÕ­ÑËd²[cárA a• á\<õ¡n`8ÛTePÅr
U9 y azqz`²[ePÅ²h parliamentih̀ 3 iQþA²h press conference i`
h 4 iT7h court ih̀ 5 i±´°©h studio i`5Ñ5²[LÀ5Å#£¿ 8 q¿ 11  |a`­ÑÃþ`5Ñ5²[V~Ã£×­ÑÃîÃ;zlax <úqÜñxrHoc

Identification Rate

È 8 ÅáÉÔ·­ 1 ¦®Ùæ¡

È 9 ÅáÉÔ·­ 1 ¦®Ùæ¢¤¡

È 10 ÅáÉÔ·­ 3 ¦®Ùæ
¡

È 11 ÅáÉÔ·­ 3 ¦®Ùæ¢¤¡

9

5UW`¾Ïe±ëãléBvVçEz`'9{åq¾H{å£Irz²[£À5|T£®ýza8ÛT`¾Ïe±ëãpm`5²[?|mn2£pmz`ÙléBvzpm~}<õÀ
5£Ynla<õCz`xT£i¾Ïe±ëãþ 817 ¯Ð·» mzx¡`á\<õm9Á£À5Å#£ z`{åI u`¾
Hèjíüã2zzK~Xw|xrnlxr by aCpm`¾H{åÐz¾Y`5yRµ=ñ{å££z÷¤¹×ÅÔe» mòr÷#l`ªoxIa
pvnÜ ½ `ç09ò÷#y¢ açS`ÙÑ5z²[V<õpm`Üñn_N?iûrsy  rÖmy c• Ñ5²[V2£yu;|üã2zrá`GÃ2z£BsÝ\¢~mm7nrla3~4qÙpuxrXo a• ²[À5Å#r~ÃîÃppq2~|6Qlxq`kd¶+îízI~£8P|`~Ã£zÃ\ä|7nrla`~Ã£ä|x²[OL#|4ñr>"y `ÙZ&wnBv£mmkdüã2z£ |
7nrla"tm£1õzñy`HenryRowley )w|a[2]äç`pUUu`>x`þì6`“¯Ð·»I} uëã9léBvT`”¹ h D-II i`Vol.J82-D-II`No.10`pp.1543–
1551, Oct. 1999.

[3] I. Ide, R. Hamada, S. Sakai, and H. Tanaka, “Semantic Analysis of Television News Captions Referring to Suﬃxes,” Proc. Fourth Intl. Workshop
on Information Retrieval with Asian Languages(IRAL’99), pp.37–42, Nov. 1999.

äç`>x`käç`þì6`“¾Ïe±ëãpv¾Hb'9{å£I z2zÿÏjÀ5`” ò 5 0¹ ú#´QÌº¦¥¯×È°§Ë J`pp.45–51, Dec. 1999.
[5] ^U Ç C `“ÅOüã?lË<õ`” ´¦ `Vol.40`No.3`pp.886–898,[4]March 1999.

[6]2ëC`ö¢ç`Fæ5`.Þl`“è°O2üºe´Æe±?l`” ´¦`Vol.33`No.11`pp.1373–1383, Nov. 1992.

[7] Y. Nakamura, and T. Kanade, “Semantic Analysis for Video Contents Extraction –Spottingby Association in News Video–,” Proc. Fourth
Intl. Multimedia Conf. (ACM Multimedia ’97),
pp.393–402, Nov. 1997.
[8]]_l`a¼ç`kÈC`“~üã£mm¯e×I}ëãléBvT`” ò 5 0¹
ú#´QÌº¦¥¯×È°§Ë J`pp.53–60,Dec. 1999.

[9]»_cì`[*g¹`ï_vçc“üã÷O1
(üºe´Æe± uüãqÿ~`”ò 4 0¹ ú#´QÌº¦¥¯×È°§Ë J`
pp.127–132, Dec. 1998.

Face detector[10]/[10] H. D. Rowley, S. Baluja, and T. Kanade,“Neural Network-Based Face Detection,” IEEETrans. Pattern Analysis and Machine Intelligence, Vol.20, No.1, pp.23–38, Jan. 1998.

[1] J. Huang, S. R. Kumar, and R. Zabih, “Image In-[11] S. Satoh, Y. Nakamura, and T. Kanade, “Name-dexing Using Color Correlograms,” Proc. IEEEConf. on Computer Vision and Pattern Recogni-It: Naming and Detecting Faces in NewsVideos,” IEEE Multimedia, Vol.6, No.1, pp.22–tion ’97, pp.762–768, June 1997.

35, March 1999.

[12] H. D. Wactler, A. G. Hauptmann, M. G.
Christel, R. A. Houghton, and A. M. Olligschlaeger, “Complementary video and audio
analysis for broadcast news archives,” Commun.
ACM, Vol.43, No.2, pp.42–47, Feb. 2000.