Source: http://www.google.de/patents/US8855796
Timestamp: 2017-10-17 22:31:33
Document Index: 514201631

Matched Legal Cases: ['art 5', 'art 5', 'art 6', 'art 6', 'art 6', 'Application No. 2006']

Patent US8855796 - Method and device for detecting music segment, and method and device for ... - Google Patentsuche
The present invention is to provide a method of efficiently detecting music scenes from television broadcasts. A music-segment detection method according to the present invention includes a data input step of inputting data including an audio and video signals; a plurality of candidate-music-segment...http://www.google.de/patents/US8855796?utm_source=gb-gplus-sharePatent US8855796 - Method and device for detecting music segment, and method and device for recording data
Veröffentlichungsnummer US8855796 B2
Anmeldenummer US 12/087,248
PCT-Nummer PCT/JP2006/325720
Eingetragen 25. Dez. 2006
Prioritätsdatum 27. Dez. 2005
Auch veröffentlicht unter CN101310327A, CN101310327B, EP1968043A1, EP1968043A4, EP1968043B1, US20090088878, WO2007074755A1
Veröffentlichungsnummer 087248, 12087248, PCT/2006/325720, PCT/JP/2006/325720, PCT/JP/6/325720, PCT/JP2006/325720, PCT/JP2006325720, PCT/JP6/325720, PCT/JP6325720, US 8855796 B2, US 8855796B2, US-B2-8855796, US8855796 B2, US8855796B2
Erfinder Isao Otsuka, Hidetsugu Suginohara
Patentzitate (50), Nichtpatentzitate (8), Referenziert von (13), Klassifizierungen (19), Juristische Ereignisse (1)
US 8855796 B2
1. A music-segment detection method comprising:
a plurality of candidate-music-segment detection steps of detecting, by utilizing a plurality of candidate-music-segment detection units, from the audio signal, respective candidate music-segments of the audio/video data, based on a plurality of criteria different from each other; and
a music-segment detection step of detecting a music segment based on the plurality of candidate music-segments each detected by the plurality of candidate-music-segment detection steps, wherein
the plurality of criteria includes a first criterion and a second criterion,
the first criterion has the detection characteristic configured to detect a music scene wherein the first criteria is set based on whether or not the characteristic of the audio signal corresponds to a class model determined to be a music scene among a plurality of predetermined audio class models,
the second criterion has the detection characteristic configured to detect a music segment wherein the second criterion is set based on the difference between the left and right channels of the audio signal,
wherein the music-segment detection step comprises:
searching for the start time of the candidate music-segment detected according to the second criterion around the start time of the candidate music-segment detected according to the first criterion;
adopting the start time found in the searching as the start time of the music segment;
searching for the end time of the candidate music-segment detected according to the second criterion around the end time of the candidate music-segment detected according to the first criterion; and
adopting the end time found in the searching as the end time of the music segment and wherein
the plurality of candidate-music-segment detection steps includes a plurality of respective correction steps each correcting the detected respective candidate music-segments: and
the music-segment detection step detects a music segment based on the respective candidate music-segments, instead of the plurality of candidate music-segments, each corrected by the plurality of correction steps.
5. A music-segment detection device comprising:
a music-segment detection unit for detecting a music segment based on the plurality of candidate music-segments each detected by the plurality of candidate-music-segment detection units, wherein
the second criterion has the detection characteristic configured to detect a music segment wherein the second criterion is set based on the difference between left and right channels of the audio signal,
wherein the music-segment detection unit is configured for:
adopting the end time found in the searching as the end time of the music segment, and wherein
the plurality of candidate-music-segment detection units includes a plurality of respective correction units each correcting the detected respective candidate music-segments; and
the music-segment detection unit detects a music segment based on the respective candidate music-segments, instead of the plurality of candidate music-segments, each corrected by the plurality of correction units.
9. The music-segment detection method of claim 1, wherein the music-segment detection step further comprises:
determining the candidate music-segment detected according to the first criterion as the music segment if there are no candidate music-segments detected according to the second criterion; and
eliminating the candidate music-segment detected according to the second criterion if there are no corresponding candidate music-segments detected according to the first criterion.
10. The music-segment detection device of claim 5, wherein the music-segment detection unit is further configured for:
D = ∑ n = 1 256 ( M l ( n ) - M r ( n ) ) 2 Equation 1
In the second correction unit 13, the differential amounts 42 are binarized using a predetermined threshold value 43, to assign a label indicating music to a segment exceeding the threshold value 43 and a label indicating non-music to a segment below the threshold value 43. Chart 5B in the figure is a graph 405 in which binarized labels 45 are plotted: segments assigned with the label indicating music and segments assigned with the label indicating non-music are expressed with the label axis 44 as “High” and “Low”, respectively. Chart 5C in the figure is a graph 407 in which corrected labels 47 are plotted that are obtained by correction-processing the labels 45.
Next, the processing (ST7) in the music detection unit 14 shown in FIG. 1 will be explained. FIG. 6 shows charts illustrating the process of detection processing of music segments by the music detection unit 14 of Embodiment 1. Chart 6A in the figure is a graph 500 in which labels 50 are plotted that are obtained by the correction-processing by the first correction unit 12, of labels calculated in the first music-candidate detection unit 10. Similarly, Chart 6B in the figure is a graph 502 in which labels 52 are plotted that are obtained by the correction-processing by the second correction unit 13, of differential amounts calculated in the second music-candidate detection unit 11. Chart 6C in the figure is a graph 504 in which labels 54 are plotted that are detection-processed based on the labels 50 and 52. In all the graphs 500, 502, and 504, segments assigned with labels indicating music and segments assigned with labels indicating non-music are expressed with the label axis 44 as “High” and “Low, respectively.
In a case of a music segment being detected from the candidate music-segments 51 a and 53 a both detected with them being time-wise overlapped partially, if their start times A1 and B1 are the same, the start time of the music segment 55 a after being detected is determined to be the time C1. The end time of the music segment 55 a after being detected, because the end time A2 is longer later than the time B2 (in the direction such that the music segment time-wise extends), is determined to be the time C2 by adopting the time A2.
Similarly, in a case of a music segment being detected from the candidate music-segments 51 b and 53 b, since the start time B3 is earlier than the start times A3, the start time of the music segment 55 b after being detected is determined to be the time C3 by adopting the start time B3. The end time of the music segment 55 b after being detected, because the end time B4 is later than the time A4, is determined to be the time C4 by adopting the time B4.
Lastly, in a case of a music segment being detected from the candidate music-segments 51 c and 53 d, since the start time A5 is longer than the start time B7, the start time of the music segment 55 d after being detected is determined to be the time C7 by adopting the time A5. The end time of the music segment 55 d after being detected, because the start time A6 is longer than the start time B8, is determined to be the time C8 by adopting the time A6. The music segments 55 a, 55 b, 55 c, and 55 d after being detected are thereby determined.
As explained above, while an example of a manner to write the start and end times of a music segment in meta-data is described in Embodiment 1, the meta-data is only required to have such a data structure that music information can be specified from a recorded program. For example, when a program is discretized at fixed time intervals such as one second, segments are binarized in such a way that a music scene segment is labeled with “High” (one) and a non-music scene segment is labeled with “Low” (zero); and then a data structure is formed in which the binarized result is enumerated; whereby the equivalent function can also be realized.
Next, a detection processing of a CM broadcast in the music detection unit 71 will be explained with reference to FIG. 12. As an example, when there are music segments 81 a and 81 b in the labels 80, which are obtained from candidate music-segments by a first and second music candidate detection means, and a CM broadcast segment 83 a detected from a CM broadcast by the CM detection unit 70, portions of segments overlapped with the CM broadcast segment 83 a, of the music segments 81 a and 81 b are correction-processed to non-music segments.
The above is specifically explained below with reference to FIG. 12. The start time D1 of the music segment 81 a is not overlapped with the CM broadcast, but the end time D2 is overlapped with the CM broadcast segment 83 a. Hence, the start time of the music segment 85 a after being CM-correction-processed is determined to be the time F1 by adopting the time D1, and the end time is determined to be the time F2 by adopting the start time E1 of the CM broadcast segment 83 a.
Similarly, the end time D4 of the music segment 81 b is not overlapped with the CM broadcast; but the start time D3 is overlapped with the CM broadcast segment 83 a. Hence, the start time of the music segment 85 b after being CM-correction-processed is determined to be the time F3 by adopting the end time E2 of the CM broadcast segment 83 a, and the end time is determined to be the time F4 by adopting the time D4. If a music segment after being detection-processed is entirely overlapped with a CM broadcast segment, which is not shown in the figure, the music segment is eliminated by the CM-correction processing.
US5298674 3. Dez. 1991 29. März 1994 Samsung Electronics Co., Ltd. Apparatus for discriminating an audio signal as an ordinary vocal sound or musical sound
US5375188 8. Juni 1992 20. Dez. 1994 Matsushita Electric Industrial Co., Ltd. Music/voice discriminating apparatus
US6525255 19. Nov. 1997 25. Febr. 2003 Yamaha Corporation Sound signal analyzing device
US7260439 * 1. Nov. 2001 21. Aug. 2007 Fuji Xerox Co., Ltd. Systems and methods for the automatic extraction of audio excerpts
US7336890 * 19. Febr. 2003 26. Febr. 2008 Microsoft Corporation Automatic detection and segmentation of music videos in an audio/video stream
US7386217 14. Dez. 2001 10. Juni 2008 Hewlett-Packard Development Company, L.P. Indexing video by detecting speech and music in audio
US7610205 * 12. Febr. 2002 27. Okt. 2009 Dolby Laboratories Licensing Corporation High quality time-scaling and pitch-scaling of audio signals
US20020152207 * 1. Mai 2001 17. Okt. 2002 Yevgenly Lyudovyk Method and apparatus for automatically processing a user's communication
US20030182105 * 21. Febr. 2003 25. Sept. 2003 Sall Mikhael A. Method and system for distinguishing speech from music in a digital audio signal in real time
US20030194210 * 15. Apr. 2003 16. Okt. 2003 Canon Kabushiki Kaisha Moving image playback apparatus, moving image playback method, and computer program thereof
US20040148164 * 23. Jan. 2003 29. Juli 2004 Aurilab, Llc Dual search acceleration technique for speech recognition
US20040167767 25. Febr. 2003 26. Aug. 2004 Ziyou Xiong Method and system for extracting sports highlights from audio signals
US20050075743 * 31. Juli 2001 7. Apr. 2005 Tetsujiro Kondo Digital signal processing method, learning method, apparatuses for them, and program storage medium
US20050109194 * 16. Nov. 2004 26. Mai 2005 Pioneer Corporation Automatic musical composition classification device and method
US20050154973 13. Febr. 2004 14. Juli 2005 Isao Otsuka System and method for recording and reproducing multimedia based on an audio signal
US20050154987 14. Jan. 2004 14. Juli 2005 Isao Otsuka System and method for recording and reproducing multimedia
US20050169114 30. Jan. 2003 4. Aug. 2005 Hosung Ahn Digital recorder for selectively storing only a music section out of radio broadcasting contents and method thereof
US20050198570 21. Jan. 2005 8. Sept. 2005 Isao Otsuka Apparatus and method for browsing videos
US20050211071 * 25. März 2004 29. Sept. 2005 Microsoft Corporation Automatic music mood detection
US20050216261 * 18. März 2005 29. Sept. 2005 Canon Kabushiki Kaisha Signal processing apparatus and method
US20050246169 * 22. Apr. 2005 3. Nov. 2005 Nokia Corporation Detection of the audio activity
US20060149693 4. Jan. 2005 6. Juli 2006 Isao Otsuka Enhanced classification using training data refinement and classifier updating
US20060212297 * 18. Aug. 2005 21. Sept. 2006 International Business Machines Corporation System and method using blind change detection for audio segmentation
US20060266200 3. Mai 2006 30. Nov. 2006 Goodwin Simon N Rhythm action game apparatus and method
US20070230899 1. Juni 2007 4. Okt. 2007 Canon Kabushiki Kaisha Moving image playback apparatus, moving image playback method, and computer program thereof
EP1150446A2 24. Apr. 2001 31. Okt. 2001 Sony Corporation Apparatus and method for processing signals and for recognising predetermined patterns in a signal, for example for detecting commercial messages in a broadcast signal
JP3156975B2 Titel nicht verfügbar
JP3475317B2 Titel nicht verfügbar
JP2000099069A Titel nicht verfügbar
JP2002344852A Titel nicht verfügbar
JP2003099083A Titel nicht verfügbar
JP2003298981A Titel nicht verfügbar
JP2003309814A Titel nicht verfügbar
JP2004120553A Titel nicht verfügbar
JP2005284308A Titel nicht verfügbar
JP2006301134A Titel nicht verfügbar
JPH099199A Titel nicht verfügbar
JPH0588695A Titel nicht verfügbar
JPH1155613A Titel nicht verfügbar
JPH08265660A Titel nicht verfügbar
JPH09284704A Titel nicht verfügbar
JPH10207455A Titel nicht verfügbar
JPH11266435A Titel nicht verfügbar
WO2003071537A1 30. Jan. 2003 28. Aug. 2003 Hosung Ahn Digital recorder for selectively storing only a music section out of radio broadcasting contents and method thereof
1 English-Language Translation of Chinese Office Action for corresponding Application No. CN 200680043035.7 dated Aug. 4, 2010.
2 Japanese Office Action for corresponding Application No. 2006-132898 dated Sep. 18, 2009 and English-language translation thereof.
3 Otsuka et al., "A highlight scene detection and Video Summarization System Using Audio Feature for a Personnal Video Recorder", 2005 Digest of Technical Papers, Intentional Conference on Consumer Electronics (IEEE Cat No. 05CH37619) IEEE Piscataway, NJ, Jan. 8, 2005, pp. 223-224; XP010796612.
4 Otsuka et al., "Detection of Music Segment Boundaries using Audio-Visual Features for a Personal Video Recorder" IEEE Transactions on Consumer Electronics, IEEE Service Center, New York, NY, vol. 53, No. 1, Feb. 1, 2007, pp. 150-154. XP011175935.
5 Otsuka et al., "Detection of Music Segment Boundaries using Augio-Visual Features for a Personal Video Recorder", Consumer Electronics, Jan. 1, 2007, pp. 1 and 2. XP031071464.
6 U.S. Notice of Allowance, dated Nov. 8, 2013, issued in copending U.S. Appl. No. 12/300,325.
7 USPTO Office Action for corresponding U.S. Appl. No. 12/300,325 dated Aug. 19, 2011.
8 USPTO Office Action for corresponding U.S. Appl. No. 12/300,325 dated May 10, 2012.
Internationale Klassifikation G06F17/00, H04N9/804, G11B20/10, H04N5/85, G10L25/78, H04N5/781, H04N9/82, G10H1/00, H04N9/806
Unternehmensklassifikation G11B2020/10546, H04N9/8205, H04N5/85, H04N9/8063, H04N5/781, G10H1/00, G10H2210/031, G10L25/78, H04N9/8042