Tempo detection device, tempo detection method and program

A tempo detection device includes: a basic feature amount extracting section which extracts a plurality of types of basic feature amounts from an input audio signal; a weighting and adding section which weights and adds the basic feature amounts of the plurality of types extracted in the basic feature amount extracting section to obtain an addition signal; and a tempo detecting section which detects BPM indicating the tempo on the basis of a periodic component included in the addition signal obtained in the weighting and adding section.

BACKGROUND

The present disclosure relates to a tempo detection device, a tempo detection method, and a program, and in particular, to a tempo detection device, a tempo detection method, and a program in which an audio signal of music is processed to detect the tempo of the music.

The music tempo represents the proceeding speed of music, and BPM (Beats Per Minute: the number of quarter notes per minute) is mainly used as an index representing the tempo of the music. In order to detect the BPM of music, there has been disclosed the following techniques in the related art.

Japanese Unexamined Patent Application Publication No. 2002-221240 discloses a technique which calculates autocorrelation of music waveform signals, analyzes a beat structure of music on the basis of the calculation result, and extracts the tempo of the music on the basis of the analysis result. Further, Japanese Unexamined Patent Application Publication No. 2007-033851 discloses a technique which divides an input audio signal into a plurality of frequency bands, detects peaks of the input audio signal for each frequency band, calculates a time interval in the peak locations, and detects the tempo on the basis of the time interval with a frequent peak generation.

SUMMARY

The technique disclosed in Japanese Unexamined Patent Application Publication No. 2002-221240 has a problem in that the calculation amount is excessive in consideration of a brief analysis on an embedded processor for a portable device. Further, the technique disclosed in Japanese Unexamined Patent Application Publication No. 2007-033851 is designed for a low calculation amount, but there are problems in that the time interval of the peaks does not correspond to BPM as it is in many cases and the detection efficiency is not sufficiently high. In particular, there are many cases where the BPM is mistakenly set to double or one half. For example, in a case where the correct BPM is 60, BPM=120 may be detected, or in a case where the correct BPM is 100, BPM=50 may be detected.

Accordingly, it is desirable to provide a technique which is capable of detecting the tempo of music in a low calculation amount with high efficiency.

According to an embodiment of the present disclosure, there is provided a tempo detection device including: a basic feature amount extracting section which extracts a plurality of types of basic feature amounts from an input audio signal; a weighting and adding section which weights and adds the basic feature amounts of the plurality of types extracted in the basic feature amount extracting section to obtain an addition signal; and a tempo detecting section which detects BPM indicating the tempo on the basis of a periodic component included in the addition signal obtained in the weighting and adding section.

According to the embodiment, the basic feature amount extracting section extracts the basic feature amounts of the plurality of types from the input audio signal. For example, the basic feature amount extracting section divides the input audio signal into frames including a predetermined number of pieces of sample data and extracts the basic feature amounts of the plurality of types for each frame. For example, in a case where a sampling frequency of the input audio signal is 22.050 kHz, the input audio signal is divided into frames including 1024 pieces of sample data.

For example, the basic feature amount extracting section includes a short-time Fourier transform section and a basic feature amount calculating section. The short-time Fourier transform section performs a short-time Fourier transform for each frame of the input audio signal. The basic feature amount calculating section calculates the basic feature amounts of the plurality of types, that is, “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off”, on the basis of a frequency spectrum for each frame output from the short-time Fourier transform section.

The weighting and adding section weights and adds the basic feature amounts of the plurality of types extracted in the basic feature amount extracting section to obtain the addition signal. Here, for example, weight coefficients are manually obtained, but may be automatically determined by learning. Further, the tempo detecting section detects the periodic component included in the addition signal obtained in the weighting and adding section, and detects the BPM indicating the tempo on the basis of the periodic component.

For example, the tempo detecting section includes a fast Fourier transform section, a score calculating section, and a BPM determining section. The fast Fourier transform section performs a fast Fourier transform for the addition signal for each frame, for a periodicity analysis.

The score calculating section divides respective samples in a frequency axis output from the fast Fourier transform section into a predetermined number of continuous frequency regions, which include a frequency region in which it is assumed that a correct BPM is present, and in which a frequency region adjacent to a low pass side becomes one half and a frequency region adjacent to a high pass side becomes double. Further, the score calculating section calculates a score corresponding to the level of each sample data for each frequency region and for each sample.

The BPM determining section includes a score adding section and a maximum value searching section. The score adding section matches the numbers of samples of the respective frequency regions, and adds the sample scores of the respective frequency regions for the corresponding samples on the basis of the score for each frequency region and for each sample calculated in the score calculating section. The maximum value searching section calculates a frequency corresponding to the samples having a maximum value among score addition values for each of the samples obtained by the addition in the score adding section, from the frequency region in which it is assumed that the correct BPM is present, and determines the BPM corresponding to the frequency as the BPM indicating the tempo.

In this way, according to the embodiment, the basic feature amounts of the plurality of types are extracted from the input audio signal; the basic feature amounts of the plurality of types are weighted and added to obtain the addition signal; and the BPM indicating the tempo is detected on the basis of the periodic component included in the addition signal. Accordingly, it is possible to detect the tempo of music in a low calculation amount with high efficiency.

According to the embodiment, for example, the tempo detection device further includes a tempo modifying section which modifies the BPM detected in the tempo detecting section on the basis of the basic feature amounts of the plurality of types extracted in the basic feature amount extracting section. The tempo modifying section may obtain a first sense of speed for determining whether the correct BPM is present on a high pass side with reference to the frequency region in which it is assumed that the correct BPM is present and obtain a second sense of speed for determining whether the correct BPM is present on a low pass side with reference to the frequency region in which it is assumed that the correct BPM is present, on the basis of the basic feature amounts of the plurality of types. Then, the tempo modifying section may double the BPM detected in the tempo detecting section, when it is determined that the correct BPM is present on the high pass side with reference to the frequency region in which it is assumed that the correct BPM is present through the first sense of speed, to output the BPM, may reduce the BPM detected in the tempo detecting section to one half, when it is determined that the correct BPM is present on the low pass side with reference to the frequency region in which it is assumed that the correct BPM is present through the second sense of speed, to output the BPM, and may output the BPM detected in the tempo detecting section as it is when it is determined that the correct BPM is not present on the high pass side with reference to the frequency region in which it is assumed that the correct BPM is present through the first sense of speed, and when it is determined that the correct BPM is not present on the low pass side with reference to the frequency region in which it is assumed that the correct BPM is present through the second sense of speed.

In this case, a modifying process of the BPM is performed by obtaining the first and second senses of speed for determining whether the correct BPM is present on the high pass side and the low pass side with reference to the frequency region in which it is assumed that the correct BPM is present, on the basis of the basic feature amounts of the plurality of types, and it is possible to appropriately modify the BPM in a case where the correct BPM is present on the high or low pass side with reference to the frequency region in which it is assumed that the correct BPM is present. Further, in this case, it is possible to use the basic feature amounts of the plurality of types extracted in the basic feature amount extracting section without performing extra basic feature amount calculation.

Further, according to the embodiment, for example, the basic feature amount extracting section divides the input audio signal into the frames including the predetermined number of pieces of sample data and extracts the basic feature amounts of the plurality of types for each frame, and the tempo modifying section is configured to obtain the first sense of speed and the second sense of speed for each block including a predetermined number of frames. Here, the tempo modifying section may obtain the first sense of speed by weighting averages and standard deviations of the basic feature amounts of the plurality of types in the predetermined number of frames by a first coefficient group obtained by learning in advance and by adding the weighted averages and standard deviations, and may obtain the second sense of speed by weighting the averages and the standard deviations of the basic feature amounts of the plurality of types in the predetermined number of frames by a second coefficient group obtained by learning in advance and by adding the weighted averages and standard deviations. For example, the basic feature amounts of the plurality of types include “ZCR”, “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off”.

According to the present disclosure, the basic feature amounts of the plurality of types are extracted from the input audio signal, the basic feature amounts of the plurality of types are weighted and added to obtain the addition signal, and the BPM indicating the tempo is detected on the basis of the periodic component included in the addition signal. Accordingly, it is possible to detect the tempo of music in a low calculation amount with high efficiency.

DETAILED DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments according to the present disclosure will be described in the following order;

1. First embodiment

2. Second embodiment

1. First Embodiment

[Configuration Example of Music Tempo Detection Device]

FIG. 1illustrates an example of a configuration of a music tempo detection device10according to a first embodiment. The music tempo detection device10detects the BPM (Beats Per Minute) representing the tempo of music per a predetermined time, for example, every 30 seconds, for an audio signal. The music tempo detection device10detects the BPM representing the music tempo, using values of various basic feature amounts obtained from data on an audio signal in the time axis and the frequency axis and periodicity thereof. The music tempo detection device10includes a basic feature amount extracting section100, a temporary BPM calculating section200, and a BPM calculating section300.

The basic feature amount extracting section100calculates a plurality of types of basic feature amounts, for each frame, from an input audio signal (PCM signal). In this embodiment, the basic feature amounts of the plurality of types correspond to “ZCR (Zero Crossing Rate)”, “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off”. These basic feature amounts are disclosed in “George Tzanetakis and Perry Cook, Musical genre classification of audio signals, IEEE Transactions of Speech and Audio Processing, 10(5):293-302, July 2002”.

The basic feature amounts of “ZCR”, “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” generally have the following implications. The “ZCR” is the number of times that a time waveform of an input audio signal intersects the transverse axis during unit time. The “Spectrum Flux” is power variation in a frequency spectrum for every frame. The “Spectrum Centroid” is the center of a frequency spectrum for every frame. The “Roll-Off” is a frequency reaching 85% of the total sum of the frequency spectrum for every frame.

The temporary BPM calculating section200considers the basic feature amounts of the plurality of types for every frame extracted by the basic feature amount extracting section100as time series data, and detects a periodic component (repetitive component) included in a weighted addition signal of the basic feature amount of the plurality of types, to thereby calculate the temporary BPM. The temporary BPM calculating section200uses the basic feature amounts of “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off”. The temporary BPM calculating section200forms a weighting and adding section and a tempo detecting section.

Here, the temporary BPM takes BPM0to BPM0×2, and approximately 75 is used as BPM0. Even in a case where a correct BPM is not present between BPM0to BPM0×2, the temporary BPM calculating section200outputs a value between BPM0to BPM0×2 as the temporary BPM. For example, in a case where the correct BPM is 180, the temporary BPM calculating section200outputs90as the temporary BPM. Further, for example, in a case where the correct BPM is 50, the temporary BPM calculating section200outputs100as the temporary BPM.

The BPM calculating section300calculates a sense of speed on the basis of the basic feature amounts extracted by the basic feature amount extracting section100, and determines whether the correct BPM is a BPM (high BPM) exceeding 150 or a BPM (low BPM) lower than BPM0(about 75). The BPM calculating section300uses the basic feature amounts of “ZCR (Zero Crossing Rate)”, “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off”, when calculating the sense of speed.

The BPM calculating section300doubles the temporary BPM calculated by the temporary BPM calculating section200to obtain BPM, when it is determined that the correct BPM is the high BPM. Further, the BPM calculating section300reduces the temporary BPM calculated by the temporary BPM calculating section200to one half to obtain the BPM, when it is determined that the correct BPM is the low BPM. Further, when it is determined that the correct BPM is neither the high BPM nor the low BPM, the BPM calculating section300uses the temporary BPM calculated by the temporary BPM calculating section200as BPM as it is. The BPM calculating section300forms a tempo modifying section.

An operation of the music tempo detection device10shown inFIG. 1will be described. The input audio signal (PCM signal) is supplied to the basic feature amount extracting section100. In the basic feature amount extracting section100, the basic feature amounts of “ZCR”, “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” are extracted from the input audio signal, for each frame.

The basic feature amounts of the “ZCR”, “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” for each frame which are extracted by the basic feature amount extracting section100are supplied to the temporary BPM calculating section200. In the temporary BPM calculating section200, each basic feature amount extracted for each frame by the basic feature amount extracting section100is considered as time series data, and is weighted and added. Further, in the temporary BPM calculating section200, the period component (repetitive component) included in the weighed addition signal is extracted, and the temporary BPM is calculated. The temporary BPM is a value between BPM0to BPM0×2 (BPM0is about 75).

The temporary BPM calculated by the temporary BPM calculating section200is supplied to the BPM calculating section300. The temporary BPM is a value between BPM0to BPM0×2 (BPM0is about 75). That is, in the temporary BPM calculating section200, even in a case where the correct BPM is not present between BPM0to BPM0×2, the value between BPM0to BPM0×2 is output as the temporary BPM. Further, the basic feature amounts of “ZCR”, “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” extracted by the basic feature amount extracting section100for each frame are supplied to the BPM calculating section300.

In the temporary BPM calculating section300, the sense of speed is calculated on the basis of the basic feature amounts of “ZCR”, “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” extracted by the basic feature amount extracting section100. In the BPM calculating section300, it is determined whether the correct BPM is the BPM (high BPM) exceeding BPM0×2 (BPM0is about 75), or the BPM (low BPM) lower than BPM0on the basis of the calculated speed sense.

Further, in the BPM calculating section300, when it is determined that the correct BPM is the high BPM, the temporary BPM calculated by the temporary BPM calculating section200is doubled to be output as the BPM. Further, in the BPM calculating section300, when it is determined that the correct BPM is the low BPM, the temporary BPM calculated by the temporary BPM calculating section200is reduced to one half to be output as the BPM. Further, in the BPM calculating section300, when it is determined that the BPM is neither the high BPM nor the low BPM, the temporary BPM calculated by the temporary BPM calculating section200is output as the BPM as it is.

[Description of Basic Feature Amount Calculating Section]

Details of the basic feature amount calculating section100will be described. As described above, the basic feature amount calculating section100calculates the basic feature amounts of the plurality of types used in the periodic component extraction process in the temporary BPM calculating section200and the speed sense calculation process in the BPM calculating section300. The basic feature amounts of the plurality of types correspond to “ZCR”, “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off”, as described above.

The basic feature extracting section100extracts “ZCR”, “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off”, from the input audio signal. The input audio signal is channel transformed and sampling frequency transformed in advance so that the input audio signal is monaural and has a sampling frequency of 22.050 kHz. The basic feature amount extracting section100divides the input audio signal into 1024 sample (about 46 msec) frames, calculates the basic feature amounts for each frame, and then stores the result in a buffer.

FIG. 2illustrates an example of a configuration of the basic feature amount extracting section100. The basic feature amount extracting section100includes a short-time Fourier transform section101, a flux calculating section102, a centroid calculating section103, a roll-off calculating section104, a ZCR calculating section105, and buffers106to109.

The ZCR calculating section105calculates “ZCR” according to the following formula (1), for each frame (1024 samples) using the input audio signal, that is, data in the time axis. Further, the ZCR calculating section105performs normalization so that the calculation result is changed into 1 from 0 in a normalization coefficient determined as the basic feature amount of “ZCR”, and stores the result in the buffer109. Here, “xt” represents sampling data of the input audio signal in a frame t, and “n” represents an index in a time axis direction. Further, “sign” is a function which determines the polarity of the signal. In a case where the signal is positive, “sign” is given “1”, and in a case where the signal is negative, “sign” is given “−1”. Here, “Zt” is “ZCR” in the frame t.

The short-time Fourier transform section101performs a short-time Fourier transform (STFT) for each frame, for the input audio signal, that is, the data in the time axis. The frequency spectrum for each frame output from the short-time Fourier transform section101is used for calculation of the basic feature amounts of “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” for each frame.

The flux calculating section102calculates “Spectrum Flux” by the following formula (2), for each frame, using the frequency spectrum for each frame obtained by the short-time Fourier transform section101. Further, the flux calculating section102performs normalization so that the calculation result is changed into 1 from 0 in a normalization coefficient determined as the basic feature amount of “Spectrum Flux”, and stores the result in the buffer106. Here, “N” represents the frequency spectrum (normalized as the total sum of power) of the input audio signal in the frame t, “M” represents the total number of spectrums, and “n” represents an index in a frequency axis direction. Further, “Ft” represents “Spectrum Flux” in the frame t.

The roll-off calculating section104calculates “Roll-Off” for each frame, using the frequency spectrum for each frame obtained by the short-time Fourier transform section101, and stores the calculation result in the buffer108. The roll-off calculating section104calculates the “Roll-Off” as a minimum Rt which satisfies the following formula (3). Further, the roll-off calculating section104performs normalization so that the calculation result is changed into 1 from 0 in a normalization coefficient determined as the basic feature amount of “Roll-Off”, and stores the result in the buffer (buffer4)108. Here, “X” represents the frequency spectrum of the input audio signal in the frame t, “M” represents the total number of spectrums, and “n” represents an index in a frequency axis direction.

The centroid calculating section103calculates “Spectrum Centroid” for each frame, using the frequency spectrum for each frame obtained by the short-time Fourier transform section101, according to the following formula (4). Further, the centroid calculating section103performs normalization so that the calculation result is changed into 1 from 0 in a normalization coefficient determined as the basic feature amount of “Spectrum Centroid”, and stores the result in the buffer106. Here, “X” represents the frequency spectrum of the input audio signal in the frame t, “M” represents the total number of spectrums, and “n” represents an index in a frequency axis direction. Further, “Ct” represents “Spectrum Centroid” in the frame t.

The operation of the basic feature amount extracting section100shown inFIG. 2will be briefly described. The input audio signal (PCM signal) is supplied to the short-time Fourier transform section101and the ZCR calculating section105. The input audio signal is channel transformed and sampling frequency transformed in advance so that the input audio signal is monaural and has a sampling frequency of 22.050 kHz.

The ZCR calculating section105calculates the basic feature amount of “ZCR” for each frame (1024 samples) using the input audio signal, that is, data in the time axis (see formula (1)). The ZCR calculating section105performs normalization so that the calculation result is changed into 1 from 0 in the normalization coefficient determined as the basic feature amount of “ZCR”, and stores the result in the buffer109which is a ZCR storing buffer.

Further, the short-time Fourier transform section101performs the short-time Fourier transform for each frame for the input audio signal, that is, data in the time axis. The frequency spectrum for each frame obtained by the short-time Fourier transform section101is supplied to the flux calculating section102, the centroid calculating section103, and the roll-off calculating section104.

The flux calculating section102calculates the basic feature amount of “Spectrum Flux” for each frame, using the frequency spectrum for each frame obtained by the short-time Fourier transform section101(refer to formula (2)). The flux calculating section102performs normalization so that the calculation result is changed into 1 from 0 in the normalization coefficient determined as the basic feature amount of “Spectrum Flux”, and stores the result in the buffer106which is a flux storing buffer.

The roll-off calculating section104calculates the basic feature amount of “Roll-Off” for each frame, using the frequency spectrum for each frame obtained by the short-time Fourier transform section101(refer to formula (3)). The roll-off calculating section104performs normalization so that the calculation result is changed into 1 from 0 in the normalization coefficient determined as the basic feature amount of “Roll-Off”, and stores the result in the buffer108which is a roll-off storing buffer.

The centroid calculating section103calculates the basic feature amount of “Spectrum Centroid” for each frame, using the frequency spectrum for each frame obtained by the short-time Fourier transform section101(refer to formula (4)). The centroid calculating section103performs normalization so that the calculation result is changed into 1 from 0 in the normalization coefficient determined as the basic feature amount of “Spectrum Centroid”, and stores the result in the buffer107which is a centroid storing buffer.

Details of the temporary BPM calculating section200will be described. As described above, the temporary BPM calculating section200considers the basic feature amounts of the plurality of types for each frame as time series data, and extracts the periodic component (repetitive component) included in the weighted addition signal of the basic feature amounts of the plurality of types, to thereby calculate the temporary BPM.

FIG. 3illustrates an example of a configuration of the temporary BPM calculating section200. The temporary BPM calculating section200includes a weighting and adding section210and a periodic component analyzing section220. The weighting and adding section210sequentially extracts the basic feature amounts of “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” for the respective frames from the buffers106,107, and108and performs weighting and addition, to thereby obtain a weighted addition signal.

The weighting and adding section210includes multipliers211to213and an adder214. The multiplier211multiplies “Spectrum Flux” extracted from the buffer106by a weight coefficient w1, to perform weighting. Further, the multiplier212multiplies “Spectrum Centroid” extracted from the buffer107by a weight coefficient w2, to perform weighting. Further, the multiplier213multiplies “Roll-Off” extracted from the buffer108by a weight coefficient w3, to perform weighting.

The adder214adds the basic feature amounts of “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” for the respective frames weighted by the multipliers211,212, and213, respectively, to sequentially output the weighted addition signals for the respective frames. The weight coefficients w1, w2, and w3are manually determined in advance or automatically determined by learning or the like so that the periodic component is desirably detected.

All the basic feature amounts of “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” tend to increase in locations where an attacking signal is generated. In view of an individual basic feature amount, since the basic feature amount increases in locations other than the focused periodic component, there are many cases where this becomes noise at the time of detection of the periodic component, which causes an error of the periodic component detection. In the weighted addition signal, since a location where all the basic feature amounts are changed at the same time is emphasized, it is possible to reduce noise, to thereby improve the detection performance of the periodic component.

The periodic component analyzing section220detects the periodic component (repetitive component) included in the weighted addition signal obtained by the weighting and adding section210, and detects a temporary BPM on the basis of the periodic component. The periodic component analyzing section220forms a tempo detecting section.FIG. 4illustrates an example of a configuration of the periodic component analyzing section220. The periodic component analyzing section220includes a fast Fourier transform section221, score calculating sections222to225, an adding section226, and a maximum value search section227.

The fast-Fourier transform section221performs the fast Fourier transform (FFT) for the weighted addition signals for the respective frames sequentially output from the weighting and adding section210. The size of the FFT corresponds to 1024 samples, for example. In this case, in the time series data, since the number of frames per second is 22050/1024, the sampling frequency when the time series data is fast Fourier transformed becomes 22050/1024 Hz. The Nyquist frequency at that time becomes 22050/(2×1024) Hz. In a case where 1024 samples are used as the size of FFT, frequency data of 1024 samples is obtained, and one sample corresponds to (22050/1024)/1024 Hz. Since the BPM corresponds to the number of repetitions per minute, one sample corresponds to 60×(22050/1024)/1024 BPM for each spectrum, in other words.

In a case where the periodic component is present in the weighted addition signal, the level of sampling data of a corresponding frequency location, among each sample data in the frequency axis obtained as a result of the fast Fourier transform, becomes the peak.FIG. 5illustrates an example of a result of the fast Fourier transform of the weighted addition signal. In this figure, the longitudinal axis represents the BPM (Beats Per Minute) corresponding to the frequency.

The score calculating sections222to225calculate scores for detection of the temporary BPM. As apparent from the result of the fast Fourier transform inFIG. 5, some peaks appear. The frequency location where the maximum value occurs is not necessarily limited to the correct BPM. For example, in a case where a sixteenth note component is strong, a strong peak appears in a location that is four times the correct BPM.

Before performing a correct BPM detection, the temporary BPM calculating section200detects the BPM when it is assumed that the correct BPM is BPM0to BPM0×2 (BPM0is about 75) as the temporary BPM. The score detecting sections222to225calculate a score indicating which BPM among BPM0to BPM0×2 looks most like the temporary BPM, from the result of the fast-Fourier transform, in order to calculate the temporary BPM.

In a case where a piece of music of BPM=100 is processed, a peak is generated to a frequency corresponding to BPM=100 and also peaks tend to be generated in frequency locations corresponding to BPM=50, BPM=200, and BPM=400. Thus, the periodic component analyzing section220divides the frequency region into the following four regions, and calculates scores in the respective regions. In the frequency division, the score is reduced to one half in a frequency region adjacent to a low pass side, and is doubled in a frequency region adjacent to a high pass side.

In a case where a lower limit value of the temporary BPM is set as BPM0, a frequency region1is a frequency region corresponding to BPM0/2<BPM≦BPM0, a frequency region2is a frequency region corresponding to BPM0<BPM≦BPM0×2, a frequency region3is a frequency region corresponding to BPM0×2<BPM≦BPM0×4, and a frequency region4is a frequency region corresponding to BPM0×4<BPM≦BPM0×8. If the range of the temporary BPM is set to about 75 to about 150, the BPM0becomes 60×(22050/1024)/1024×60.

The score calculating section222calculates the score of the frequency region1on the basis of each sample data existing in the frequency region1. The score calculating section223calculates the score of the frequency region2on the basis of each sample data existing in the frequency region2. The score calculating section224calculates the score of the frequency region3on the basis of each sample data existing in the frequency region3. The score calculating section225calculates the score of the frequency region4on the basis of each sample data existing in the frequency region4.

FIG. 6illustrates an example of the score calculations of each frequency region using the result (refer toFIG. 5) of the fast-Fourier transform. A signal of the frequency region1is considered as a component of one half of the temporary BPM corresponding to the location where the frequency is double. That is, the signal of the frequency region1becomes a half note component in a case where the temporary BPM is considered as a quarter note. Thus, the score calculating section222which calculates the score of the frequency region1uses its level as a sample score in the location where the frequency is double, for each sample data existing in the frequency region1. For example, the level of sample data existing in a location where BPM is 60 is used as a sample score corresponding to BPM=120.

The signal of the frequency region2is considered as the component of the temporary BPM. That is, the signal of the frequency region2becomes a quarter note component in a case where the temporary BPM is considered as the quarter note. Thus, the score calculating section223which calculates the score of the frequency region2uses its level as a sample score in a location where the frequency is the same, for each sample data existing in the frequency region2.

The signal of the frequency region3is considered as a component of double the temporary BPM corresponding to a location where the frequency is one half. That is, the signal of the frequency region3becomes an eighth note component in a case where the temporary BPM is considered as the quarter note. Thus, the score calculating section224which calculates the score of the frequency region3uses its level as a sample score in a location where the frequency is one half, for each sample data existing in the frequency region3. For example, the level of the sample data existing in a location where the BPM is 240 is used as a sample score corresponding to BPM=120.

The signal of the frequency region4is considered as a component quadruple the temporary BPM corresponding to a location where the frequency is 1/4. That is, the signal of the frequency region4becomes a sixteenth note component in a case where the temporary BPM is considered as the quarter note. Thus, the score calculating section225which calculates the score of the frequency region4uses its level as a sample score in a location where the frequency is 1/4, for each sample data existing in the frequency region4. For example, the level of the sample data existing in a location where the BPM is 480 is used as a sample score corresponding to BPM=120.

Returning toFIG. 4, the adding section226matches the sample numbers in the respective regions, and adds the scores in the respective regions calculated by the score calculating sections222to225for the corresponding samples. The adding section226forms a score adding section. The adding section226performs thinning out of the samples in the other frequency regions so that their sample numbers become the same as in the frequency region1in which the sample number is smallest, for example.

As described above, in a case where the frame frequency is 22.050/1024 kHz and the FFT size is 1024 samples, in the fast-Fourier transform section221, the sampling frequency is 22.050/1024 kHz and a frequency expression where the sample number (data number) is 1024 is obtained. In this case, the sample number of the frequency region1is 30, the sample number of the frequency region2is 60, the sample number of the frequency region3is 120, and the sample number of the frequency region4is 240 (refer toFIG. 5).

The thinning out of the samples in the frequency region2is performed as follows. While the sample number in the frequency region1is 30, the sample number in the frequency region2is 60. Thus, the adding section226divides the frequency region2into 30 blocks every two samples, and takes only maximum value of each block, to thereby thin out the samples to 30 samples.

Further, the sample thinning out in the frequency region3is performed as follows. While the sample number of the frequency region1is 30, the sample number of the frequency region3is 120. Thus, the adding section226divides the frequency region3into 30 blocks for every 4 samples, and takes only the maximum value of each block, to thereby thin out the samples to 30 samples.

Further, the sample thinning out in the frequency region4is performed as follows. While the sample number of the frequency region1is 30, the sample number of the frequency region4is 240. Thus, the adding section226divides the frequency region4into 30 blocks for every 8 samples, and takes only the maximum value of each block, to thereby thin out the samples to 30 samples.

The maximum value search section227searches for a maximum value from score addition values of the respective samples which are obtained by the addition in the adding section226, as shown inFIG. 6. Further, the BPM corresponding to the frequency in the frequency region2, which corresponds to the samples of the maximum score addition value, is used as the temporary BPM. Here, the frequency region2(frequency region corresponding to BPM0<BPM≦BPM0×2) is the frequency region where it is assumed that the correct BPM is present, as described above.

An operation of the temporary BPM calculating section200shown inFIG. 3will be briefly described. The basic feature amounts of “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” for the respective frames which are stored in the buffers106,107, and108are sequentially extracted, and then are supplied to the weighting and adding section210. The multiplier211multiplies “Spectrum Flux” extracted from the buffer106by the weight coefficient w1, to perform weighting. Further, the multiplier212multiplies “Spectrum Centroid” extracted from the buffer107by the weight coefficient w2, to perform weighting. Further, the multiplier213multiplies “Roll-Off” extracted from the buffer108by the weight coefficient w3, to perform weighting.

The output signals of the respective multipliers211to213are supplied to the adder214. The adder214adds the basic feature amounts of “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” for the respective frames weighted by the multipliers211to213, respectively, to sequentially obtain the weighted addition signals for the respective frames. The weighted addition signals are supplied to the periodic component analyzing section220.

The periodic component analyzing section220detects the periodic component (repetitive component) included in the weighted addition signal obtained by the weighting and adding section210, and detects the temporary BPM on the basis of the periodic component. That is, the Fourier transform section221of the periodic component analyzing section220performs the fast-Fourier transform for the weighted addition signals (time series data) of the respective frames which are sequentially output from the weighting and adding section210(refer toFIG. 4). The result of the fast-Fourier transform is supplied to the score calculating sections222to225(refer toFIG. 5).

The score calculating sections222to225calculate scores for detecting the temporary BPM (refer toFIG. 6). The score calculating section222calculates the score of the frequency region1on the basis of each sample data existing in the frequency region1(frequency region corresponding to BPM0/2<BPM≦BPM0). In this case, the level becomes a sample score in a location where the frequency is double, for each sample data existing in the frequency region1.

The score calculating sections223calculates the score of the frequency region2on the basis of each sample data existing in the frequency region2(frequency region corresponding to BPM0<BPM≦BPM0×2). The frequency region2is the frequency region where it is assumed that the correct BPM is present. In this case, the level becomes a sample score in a location where the frequency is the same, for each sample data existing in the frequency region2.

The score calculating sections224calculates the score of the frequency region3on the basis of each sample data existing in the frequency region3(frequency region corresponding to BPM0×2<BPM≦BPM0×4). In this case, the level becomes a sample score in a location where the frequency is one half, for each sample data existing in the frequency region3.

The score calculating sections225calculates the score of the frequency region4on the basis of each sample data existing in the frequency region4(frequency region corresponding to BPM0×4<BPM≦BPM0×8). In this case, the level becomes a sample score in a location where the frequency is 1/4, for each sample data existing in the frequency region4.

The scores of the respective frequency regions calculated by the score calculating sections222to225are supplied to the adding section226. The adding section226matches the sample numbers in the respective frequency regions, and adds the scores of the respective frequency regions for the corresponding samples, respectively. In this case, the adding section226performs thinning out of the samples in the other frequency regions so that their sample numbers become the same as in the frequency region1in which the sample number is smallest, for example.

The score addition value of the samples obtained by the adding section226is supplied to the maximum value search section227(seeFIG. 6). The maximum value search section227searches for the maximum value from score addition values of the respective samples. Further, in the maximum value search section227, the BPM corresponding to the frequency in the frequency region2, which corresponds to the samples of the maximum score addition value, is used as the temporary BPM.

Details of the BPM calculating section200will be described. The BPM calculating section200calculates the sense of speed on the basis of the basic feature amounts extracted by the basic feature amount extracting section100, and determines whether the temporary BPM calculated by the temporary BPM calculating section200should be modified. The temporary BPM calculating section200calculates the temporary BPM on the basis of the assumption that the BPM falls in BPM0to BPM0×2. The BPM calculating section300performs a high BPM determination (determine whether the BPM exceeds BPM0×2) and a low BPM determination (determine whether the BPM is lower than BPM0), to thereby obtain more accurate BPM.

As described above, the music tempo detection device10detects the BPM representing the tempo of music, for example, every 30 seconds for the audio signal. The BPM calculating section300further divides the signal for the 30 seconds into blocks for several 100 msec, and performs the high BPM determination and low BPM determination for each block. The BPM calculating section300uses the basic feature amounts of “ZCR”, “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” extracted by the basic feature amount extracting section100in the above-mentioned determinations.

As described above, the basic feature amount extracting section100extracts the basic feature amounts of “ZCR”, “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” for each frame, from the input audio signal (PCM signal). The BPM calculating section300calculates averages and standard deviations of the respective basic feature amounts for each block, and uses the result as feature amounts representing the block. Consequently, the BPM calculating section300obtains eight-dimensional feature vectors (f0, f1, f2, f3, f4, f5, f6, and f7) as the feature amounts. The BPM calculating section300calculates the inner product of the feature vectors and weight coefficients, to thereby perform the high BPM determination and low BPM determination.

Firstly, the BPM calculating section300performs the high BPM determination, that is, determines whether the BPM exceeds BPM0×2. The BPM calculating section300calculates a “speed sense1” for the high BPM determination, using the above-mentioned eight-dimensional feature vectors and the weight coefficients for the high BPM determination.

The weight coefficients for the high BPM determination are calculated by learning in advance. The learning is performed as follows, for example. That is, a group of music when a person feels that the BPM exceeds BPM0×2 and a group of music when a person feels that the BPM is lower than BPM0×2 are prepared, and the above-mentioned feature amounts (eight dimensional feature vectors) are calculated for all the music in each group. Further, Fisher's linear discriminant analysis is used, and an optimal projection for discriminating two groups is calculated. Coefficients obtained as a result are used as the weight coefficients for the high BPM determination.

The “speed sense1” corresponds to a degree where a person feels that the BPM exceeds BPM0×2. The BPM calculating section300calculates the “speed sense1” in a block k, by calculating the inner product of the feature amounts (eight-dimensional feature vectors) and the weight coefficients for the high BPM determination according to the following formula (5). Here, “a” represents the weight coefficients for the high BPM determination for the calculation of the “speed sense1”, and “f” represents the feature amounts in the block k.

The BPM calculating section300compares the calculated “speed sense1” with a predetermined threshold A. When the “speed sense1” is greater than the threshold A, the BPM calculating section300determines the BPM as double the temporary BPM, that is, “temporary BPM×2”. When the “speed sense1” is not greater than the threshold A, the BPM calculating section300moves to the low BPM determination. The threshold A is determined at the time of the learning of the weight coefficients for the high BPM determination.

In order to perform the low BPM determination, that is, to determine whether the BPM is lower than BPM0, the BPM calculating section300calculates a “speed sense2” using the above-mentioned eight-dimensional feature vectors and the weight coefficients for the low BPM determination.

The weight coefficients for the low BPM determination are determined by learning in advance. The learning is performed as follows, for example. That is, a group of music when a person feels that the BPM is lower than BPM0and a group of music when a person feels that the BPM is BPM0or higher, are prepared, and the above-mentioned feature amounts (eight-dimensional feature vectors) are calculated for all the music in each group. Further, the Fisher's linear discriminant analysis is used, and an optimal projection for discriminating two groups is calculated. Coefficients obtained as a result are used as the weight coefficients for the low BPM determination.

The “speed sense2” corresponds to a degree where the person feels that the BPM is lower than BPM0. The BPM calculating section300calculates the “speed sense2” in a block k, by calculating the inner product of the feature vectors (eight-dimensional feature vectors) and the weight coefficients for the low BPM determination according to the following formula (6). Here, “b” represents the weight coefficients for the low BPM determination for the calculation of the “speed sense2”, and “f” represents the feature amounts in the block k.

The BPM calculating section300compares the calculated “speed sense2” with a predetermined threshold B. When the “speed sense2” is greater than the threshold B, the BPM calculating section300determines the BPM as half the temporary BPM, that is, “temporary BPM/2”. When the “speed sense2” is not greater than the threshold B, the BPM calculating section300determines the BPM as the temporary BPM.

FIG. 7is a flowchart illustrating a procedure of the above-described BPM determination process for each block, in the BPM calculating section300. The BPM calculating section300starts the process in step ST1, and then proceeds to step ST2. In step ST2, the BPM calculating section300calculates the inner product of the feature amounts (eight-dimensional feature vectors) and the weight coefficients for the high BPM determination, to thereby calculate the “speed sense1” for the high BPM determination (refer to formula (5)).

Next, in step ST3, the BPM calculating section300determines whether the “speed sense1” is greater than the threshold A, that is, “speed sense1”>threshold value A. When the “speed sense1” is greater than the threshold value A, in step ST4, the BPM calculating section300determines the BPM as double the temporary BPM, that is, as “temporary BPM×2”, and then terminates the process in step ST5.

When the “speed sense1” is not greater than the threshold A in step ST3, the BPM calculating section300proceeds to the process of step ST6. In step ST6, the BPM calculating section300calculates the inner product of the feature amounts (eight-dimensional feature vectors) and the weight coefficients for the low BPM determination, to thereby calculate the “speed sense2” for the low BPM determination (refer to formula (6)).

Next, in step ST7, the BPM calculating section300determines whether the “speed sense2” is greater than the threshold B, that is, “speed sense2”>threshold value B. When the speed sense2is greater than the threshold value B, in step ST8, the BPM calculating section300determines the BPM as one half of the temporary BPM, that is, as “temporary BPM/2”, and then terminates the process in step ST5.

When the “speed sense2” is not greater than the threshold B in step ST7, the BPM calculating section300proceeds to the process of step ST9. In step ST9, the BPM calculating section300determines the BPM as the temporary BPM as it is, and then terminates the process in step ST5.

As described above, the BPM calculating section300divides the signal for 30 seconds into blocks for several 100 msec, and performs the high BPM determination and the low BPM determination for each block to determine the BPM. The BPM calculating section300outputs the most frequent block among all the blocks, as the BPM of the input audio signal for 30 seconds which is currently processed.

In the above-described high BPM determination and low BPM determination in the BPM calculating section300, it is possible to combine a plurality of determining devices. For example, a system which considers the BPM as BPM0×2 or higher and modifies the BPM into double in a case where a value which is equal to or higher than the threshold is obtained in any determining device, a system which considers the BPM as being less than BPM0and modifies the BPM into one half in a case where a value which is equal to or higher than the threshold is obtained in all the determining devices, or the like, are considered.

Further, as described above, the above-described music tempo detection device10detects the BPM representing the tempo of music per the predetermined time, for example, every 30 seconds, for the audio signal. Thus, in order to determine the BPM of the entire piece of music, it is necessary to combine the results for all 30 seconds. This process is realized by considering the BPM having the most frequent appearance from among the BPMs for all 30 seconds as the BPM of the entire piece of music, for example.

As described above, in the music tempo detection device10inFIG. 1, the basic feature amounts of “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” extracted from the input audio signal are weighted and added in the temporary BPM calculating section200. Further, the temporary BPM representing the tempo is calculated on the basis of the weighted addition signal. In the weighted addition signal, since a location where all the basic feature amounts are changed at the same time is emphasized, it is possible to reduce noise, to thereby enhance the detection performance of the periodic component. Accordingly, it is possible to calculate the temporary BPM in a low calculation amount with high efficiency by the temporary BPM calculating section200.

Further, in the music tempo detection device10inFIG. 1, the BPM calculating section300calculates the “speed sense1” and the “speed sense2” from the basic feature amounts of “ZCR”, “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off”. Further, the temporary BPM calculated by the temporary BPM calculating section200is appropriately modified on the basis of the “speed sense1” and the “speed sense2”. Further, the basic feature amounts of “ZCR”, “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” used in the BPM calculating section300are extracted by the basic feature amount extracting section100. Accordingly, it is possible to obtain the BPM in a low calculation amount with high efficiency by the BPM calculating section300.

Further, in the music tempo detection device10inFIG. 1, since the BPM can be detected in a low calculation amount with high efficiency, it is possible to detect the music tempo with high efficiency even on a portable device capable of being mounted with only a low resource processor. Accordingly, even in an environment in which it is difficult to use a PC application, it is possible to provide a function using the music tempo such as a music search based on the tempo.

2. Second Embodiment

FIG. 8illustrates an example of a configuration of a music analysis system5according to a second embodiment of the present disclosure. InFIG. 8, the same reference numerals are given to elements corresponding toFIG. 1.

The music analysis system5performs music classification and music tempo detection at the same time. In the music classification, the music analysis system5classifies music into classes including genres such as classical, rock, or jazz, and moods such as happy music or sad music on the basis of the input audio signal, and outputs a classification class “output class”. In the music tempo detection, the BPM representing the tempo of music is detected on the basis of the input audio signal to be output, in a similar way to the above-described first embodiment.

The music analysis system5includes a music classification device40and a music tempo detection device10A. The music classification device40will be firstly described. The music classification device40includes a basic feature amount extracting section510, a similarity estimating section520, and an output class determining section530.

The basic feature amount extracting section510calculates a plurality of types of basic feature amounts, for each frame, from an input audio signal (PCM signal). A detailed description of the basic feature amount extracting section510is omitted, but is configured in a similar way to the basic feature amount extracting section100of the music tempo detection device10inFIG. 1.

The similarity estimating section520calculates the similarity with a model indicating a classification class, using the basic feature amounts for each frame extracted by the basic feature amount extracting section510. Here, a likelihood calculation which uses GMM (Gaussian Mixture Model) is performed as the similarity calculation. In order to perform the likelihood calculation, a database including music which is to be classified into each class is created as learning data in advance.

After the feature amounts are calculated for the learning data in learning, modeling using the GMM is performed for each class. It is possible to use an EM algorithm for the modeling. The modeling may be performed offline, and parameters representing respective models are stored in the similarity estimating section520.

The similarity estimating section520calculates the log likelihoods for the models for the respective frames using the GMM parameters representing the respective classes. After the processes for all the frames are terminated, the total sum of the log likelihoods of all the frames is taken to be used as scores for the respective modes and genres. The output class determining section530outputs the class having the largest score as the process result, that is, a classification class “output class”.

Next, the music tempo detection device10A will be described. The music tempo detection device10A includes a temporary BPM calculating section200and a BPM calculating section300. Detailed description thereof is omitted, but the temporary BPM calculating section200and the BPM calculating section300are the same as the temporary BPM calculating section200and the BPM calculating section300in the music tempo detection device10inFIG. 1.

The temporary BPM calculating section200in the music tempo detection device10A weights and adds the basic feature amounts of “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” extracted by the basic feature amount extracting section510of the music classification device40. Further, the temporary BPM calculating section200calculates the temporary BPM representing the tempo on the basis of the weighted addition signal.

Further, the BPM calculating section300in the music tempo detection device10A calculates a “speed sense1” and a “speed sense2” on the basis of the basic feature amounts extracted by the basic feature amount extracting section510of the music classification device40. In this case, the basic feature amounts of the “ZCR”, “Spectrum Flux”, “Spectrum Centroid”, and “Roll-Off” are used. The BPM calculating section300appropriately modifies the temporary BPM calculated by the temporary BPM calculating section200on the basis of the “speed sense1” and the “speed sense2”, to output the BPM.

In the music analysis system5shown inFIG. 8, since the music tempo detection device10A has the same configuration as that of the music tempo detection device10shown inFIG. 1, it is possible to obtain the same effect. Further, in the music analysis system5, the basic feature amounts extracted by the basic feature amount extracting section510of the music classification device40can be effectively used in the music tempo detection device10A. Thus, it is possible to reduce the entire calculation amount.

Although not shown inFIG. 8, the music classification device40may use the BPM which is the analysis result of the music tempo detection device300as the feature amounts. For example, a lower limit and an upper limit of the BPM are determined for each class, and the output class determining section530may finally output the classification class “output class” only for music that falls in the range thereof.

The above-described music tempo detection device10and the music analysis system5may be configured by hardware, and may perform the same process using software.FIG. 9illustrates an example of a configuration of a computer device50which allows the process to be executed using software. The computer device50includes a CPU181, a ROM182, a RAM183and a data input/output section (data I/O)184.

The ROM182stores necessary data such as a process program of the CPU181, weight coefficients, and thresholds. The RAM183functions as a work area of the CPU181. The CPU181reads out the process program stored in the ROM182as necessary, transmits the read process program to the RAM183for expansion, and reads out the expanded process program to perform a process such as music tempo detection or music classification.

In the computer device50, a music audio signal (PCM signal) is input through the data I/O184, and is stored in the RAM183. A process such as music tempo detection or music classification is performed for the input audio signal stored in the RAM183by the CPU181. Further, the process result (BPM, output class) is output outside through the data I/O184, as necessary.

The above-described embodiments illustrate only the music tempo detection device10and the music analysis system5. The music tempo detection device10and the music analysis system5may be mounted and used in a portable device such as a mobile communication device or terminal or a mobile information device or terminal with a sound recording and reproducing function.

The present disclosure contains subject matter related to that disclosed in Japanese Priority Patent Application JP 2010-173253 filed in the Japan Patent Office on Aug. 2, 2010, the entire contents of which are hereby incorporated by reference.