Patent Description:
In the past, there have been suggested sound separation devices that eliminate noise and separate a speaker's voice (target sound) from an audio signal. For example, <CIT> discloses a method for eliminating noise by spectral subtraction.

However, when both the target sound and the noise are voices, it is difficult to separate the target sound only by spectral subtraction as disclosed in <CIT>.

International Patent Application Publication No. <CIT> discloses a sound separation device that can separate target sound even when both the target sound and the noise are voices. The sound separation device disclosed in International Patent Application Publication No. <CIT> calculates the respective power spectrums of a plurality of audio source signals and calculates the power spectrum difference. The sound separation device adjusts the level of each frequency component of each of the audio source signals with a gain based on the calculated power spectrum difference.

Patent Document <CIT> relates to a headset with two microphones attached to the ear cup which suppresses unwanted sounds using an adaptive spectral method and spectral subtraction.

Patent Document <CIT> relates to a microphone array with a noise-reducing function that has at least two microphones mounted on opposite sides of a device. This technique is designed to reduce wind-induced and other noise in microphone systems using diffraction filters and equalization filters.

Patent Document <CIT> relates to a voice control method in a multi-talker and multimedia environment. Echo caused by audio transducers can be suppressed by a non-linear attenuation, such as Wiener filtering or spectral subtraction.

In the method disclosed in International Patent Application Publication No. <CIT>, non-linear processing, in which different frequency components of each audio source signal are multiplied by different gains, is performed. This method disclosed in International Patent Application Publication No. <CIT> may cause a great distortion of the spectrum and may result in severe deterioration of the audio quality. In addition, the method disclosed in International Patent Application Publication No. <CIT> deteriorates the accuracy of signal processing based on the envelope (for example, voice recognition processing).

The above problems are solved by the subject-matter of the independent claims. Further preferred embodiments are given by the subject-matter of the dependent claims. Moreover, further examples are provided for facilitating the understanding of the invention.

An object of an embodiment of the present disclosure is to provide a filtering method that can separate target sound without causing a distortion of the spectrum.

A filtering method according to an embodiment of the present disclosure includes: receiving a first audio signal and a second audio signal that include sound emitted from a same sound source at different volumes generating a filter signal by convoluting adaptive filter coefficients into the second audio signal removing components of the filter signal from the first audio signal; and limiting a gain of the adaptive filter coefficients to <NUM> or less.

The embodiment of the present disclosure makes it possible to separate target sound without causing a distortion of the spectrum.

<FIG> is a schematic external view of a sound pickup device <NUM>. The sound pickup device <NUM> includes a microphone 15A and a microphone 15B. The microphones 15A and 15B individually pick up the surrounding sounds. In the example shown in <FIG>, the microphone 15A and the microphone 15B are to pick up the voice of a speaker V1 and the voice of another speaker V2, respectively.

<FIG> is a block diagram showing the configuration of the sound pickup device <NUM>. The sound pickup device <NUM> includes a communicator <NUM>, a processor <NUM>, a RAM <NUM>, a flash memory <NUM>, a microphone 15A, and a microphone 15B.

The sound pickup device <NUM> is an example of a filtering device according to the present disclosure. The sound pickup device <NUM> filters the audio signals caught by the microphones 15A and 15B. In <FIG>, the audio signal caught by the microphone 15A includes the voice of the speaker V1 and the voice of the speaker V2. Similarly, the audio signal caught by the microphone 15B includes the voice of the speaker V1 and the voice of the speaker V2.

The microphone 15A is placed near the speaker V1, and the microphone 15B is placed near the speaker V2. Accordingly, the microphone 15A picks up a greater volume (a higher level) of the voice of the speaker V1 than the voice of the speaker V2. The microphone 15B picks up a higher level of the voice of the speaker V2 than the level of the voice of the speaker V1. Thus, the audio signal caught by the microphone 15A and the audio signal caught by the microphone 15B include sounds from the same sound sources, but the volumes of the sounds included in the respective audio signals are different.

The sound pickup device <NUM> filters the audio signal caught by the microphone 15A to remove the components of the voice of the speaker V2, which are on low levels, from the audio signal. In addition, the sound pickup device <NUM> filters the audio signal caught by the microphone 15B to remove the components of the voice of the speaker V1, which are on low levels, from the audio signal.

The sound pickup device <NUM> sends the filtered audio signal to another device, for example, via the communicator <NUM>. The sound pickup device <NUM> may further include a speaker. In this case, the speaker emits sound in accordance with an audio signal that the speaker received from another device via the communicator <NUM>. In this case, for example, the sound pickup device <NUM> works as a telecommunication device that is connected to other devices located in distant places to send and receive audio data to and from the devices in distant places.

The processor <NUM> can perform various kinds of processing by reading programs from a flash memory <NUM> (a storage medium) and storing the programs temporarily on the RAM <NUM>. Such programs include a filtering program <NUM>. The flash memory <NUM> additionally stores operation programs, such as firmware, etc., for operating the processor <NUM>. The programs to be read by the processor <NUM> are not necessarily stored in the flash memory <NUM> in the sound pickup device <NUM> itself. For example, the programs may be stored in a storage medium in an external device such as a server or the like. In this case, the processor <NUM> reads a program from the server and stores the program on the RAM <NUM> to perform the processing when necessary.

The microphone 15A catches a first audio signal S1, and the microphone 15B catches a second audio signal S2. The microphones 15A and 15B convert the audio signals into digital signals and output the digital signals to the processor <NUM>.

The processor <NUM> filters the audio signal S1 caught by the microphone 15A and the audio signal S2 caught by the microphone 15B. <FIG> is a block diagram showing the functional configuration of the processor <NUM>. <FIG> is a flowchart showing a procedure for the filtering.

The processor <NUM> functionally includes a filter bank analyzer 121A, another filter bank analyzer 121B, a filter coefficient calculator <NUM>, a filter coefficient limiter <NUM>, a FIR (finite impulse response) filter <NUM>, an adder <NUM>, and a filter bank synthesizer <NUM>. These functions are implemented by the filtering program <NUM>.

First, the processor <NUM> receives the audio signal S1 caught by the microphone 15A and the audio signal S2 caught by the microphone 15B (S11). Thus, the processor <NUM> functions as a receiver. As described above, each of the audio signal S1 caught by the microphone 15A and the audio signal caught by the microphone 15B includes the voice of the speaker V1 and the voice of the speaker V2.

The filter bank analyzer 121A transforms the audio signal S1 caught by the microphone 15A into an audio signal F1 with respect to frequency and extracts various frequency components separately (S12). Similarly, the filter bank analyzer 121B transforms the audio signal S2 caught by the microphone 15B into an audio signal F2 with respect to frequency and extracts various frequency components separately (S12).

The filter coefficient calculator <NUM> calculates filter coefficients C2 to be used in the FIR filter <NUM> (S13). The filter coefficients C2 are coefficients expressed by complex numbers by which the frequency components of the audio signal F2 are to be multiplied respectively in the FIR filter <NUM>.

The filter coefficient limiter <NUM> limits the gains to be achieved by the filter coefficients C2 calculated by the filter coefficient calculator <NUM> to <NUM> or less (S14), thereby calculating corrected filter coefficients C2'. In this way, the filter coefficient limiter <NUM> prevents the frequency components of the audio signal F2 from being amplified.

The FIR filter <NUM> convolutes the corrected filter coefficient C2' into the audio signal F2, thereby generating a filter signal F2' (S15). The filter coefficients C2 are updated by the filter coefficient calculator <NUM>, and accordingly, the corrected filter coefficients C2' are updated. Thus, the FIR filter <NUM> functions as an adaptive filter.

The adder <NUM> subtracts the filter signal F2' from the audio signal F1, thereby removing the components of the filter signal F2' from the audio signal F1 (S16). The adder <NUM> outputs an audio signal F1' obtained by the subtraction. The adder <NUM> corresponds to an eliminator of the present disclosure.

The filter bank synthesizer <NUM> transforms the audio signal F1' into an audio signal S1' with respect to time (S17).

The filter coefficient calculator <NUM> receives the audio signal F2 and the audio signal F1. The filter coefficient calculator <NUM> updates the filter coefficients by using a specified algorithm, such as LMS (least mean squares) or the like. The filter coefficient calculator <NUM> generates an update filter signal by convoluting the filter coefficients C2 into the audio signal F2. The filter coefficient calculator <NUM> subtracts the update filter signal from the audio signal F1. The resultant signal is referred to as a reference signal. The filter coefficient calculator <NUM> updates the filter coefficients C2 to values that minimize the level of the reference signal. As the time passes, the calculated filter coefficients C2 are updated so that the update filter signal can remove the voice of the speaker V2 from the audio signal F1. Accordingly, the corrected filter coefficients C2' used in the FIR filter <NUM> are updated so that the filter signal F2' can remove the voice of the speaker V2 from the audio signal F1.

Both the audio signal F1 and the audio signal F2 include the voice of the speaker V1. The filter coefficient calculator <NUM> calculates filter coefficients C2 that minimize the level of the reference signal, and therefore, the filter coefficients C2 are updated to values that remove not only the voice of the speaker V2 but also the voice of the speaker V1 from the audio signal F1. However, the level of the voice of the speaker V1 in the audio signal F1 is higher than the level of the voice of the speaker V2 in the audio signal F1. Therefore, the filter coefficient calculator <NUM> calculates coefficients having gains of <NUM> or more, which will reduce the voice of the speaker V1 in the audio signal F1.

On the other hand, the filter coefficient limiter <NUM> limits the gains to be achieved by the filter coefficients to <NUM> or less. Accordingly, the level of the voice of the speaker V1 in the filter signal F2' is lower than the level of the voice of the speaker V1 in the audio signal F1. Therefore, the voice of the speaker V1 in the audio signal F1 is not completely removed by the filter signal F2' and remains in the audio signal F1'.

In this way, the processor <NUM> removes the voice of the speaker V2 as noise components and separates the voice of the speaker V1 as the target sound.

In order to remove the voice of the speaker V2 and not remove the voice of the speaker V1 from the audio signal F1, it is preferred that the filter coefficient calculator <NUM> updates the filter coefficients only when the level of the voice of the speaker V2 in the audio signal F2 is high. In other words, when the level of the voice of the speaker V2 in the audio signal F2 is low, it is preferred that the filter coefficients are not updated. Therefore, the filter coefficient calculator <NUM> may update the filter coefficients C2 for only the frequency components in which the volume ratio of the audio signal F2 to the audio signal F1 (F2/F1) excesses a predetermined threshold.

The threshold may be any value. For example, when the threshold is <NUM>, the filter coefficient calculator <NUM> updates the filter coefficients C2 for only the frequency components in which the level of the audio signal F2 is higher than the level of the audio signal F1. Accordingly, in the case where the threshold is <NUM>, only when the sound viewed as a noise component is picked up by the microphone 15B at a higher level than the level of the sound picked up by the microphone 15A, the processor <NUM> calculates a filter coefficient that removes the sound. In the case where the threshold is greater than <NUM> (for example, about <NUM>), only when the sound viewed as a noise component to be removed is at a still higher level, the processor <NUM> updates the filter coefficient. With this arrangement, updates of the filter coefficients are performed based on the audio signal F1 and the voice of the speaker V2, and the updates are unlikely to be affected by external noise. This heightens the accuracy of the filter coefficients and enhances the filtering effect. Thus, by setting the threshold to an arbitrary value greater than <NUM> depending on how much the voice of the speaker V2, which should be removed, is mixed in the audio signal, the effect of removing the sound viewed as noise components can be set arbitrarily.

The audio signal S2 picked up by the microphone 15B is subjected to the same processing. The processor <NUM> subtracts a filter signal generated from the audio signal S1 from the audio signal F2 obtained by frequency transformation of the audio signal S2. Thereby, the voice of the speaker V1 is removed, and the voice of the speaker V2 is separated as the target sound.

In this way, even when the speakers V1 and V2 speak at the same time, the sound pickup device <NUM> with the above-described structure can separate the picked-up sound into the voice of the speaker V1 and the voice of the speaker V2.

The sound pickup device <NUM> according to the present embodiment separates target sound from an audio signal by performing linear processing, specifically subtracting a filter signal outputted from an adaptive filter from the audio signal. Therefore, the sound pickup device <NUM> can separate target sound without distorting the spectrum of the audio signal.

The envelope of a spectrum relates to the characteristics of a person's vocal tract and is significantly important for voice recognition processing. Therefore, the separation of target sound according to the present embodiment is suited to be used for voice recognition processing.

<FIG> is a block diagram showing the functional configuration of the processor <NUM> when the processor <NUM> additionally performs voice recognition processing. The processor <NUM> shown in <FIG> further includes a voice recognizer <NUM>. There are no other differences between the functional configuration of the processor <NUM> shown in <FIG> and the functional configuration shown by the block diagram in <FIG>.

The voice recognizer <NUM> performs voice recognition based on an output signal from the adder <NUM>. In other words, the voice recognizer <NUM> performs voice recognition by using the audio signal F1' into which the voice of the speaker V1 is extracted. As described above, the sound pickup device <NUM> according to the present embodiment separates the voice of the speaker V1 by linear processing, and this processing does not cause a distortion of the envelope of the spectrum. Therefore, the accuracy of voice recognition performed by the voice recognizer <NUM> is improved.

However, in a case in which the filtered audio signal is sent to another device in a distant place so that a listener in the distant place can listen to the sound, the processor <NUM> may perform non-linear processing.

<FIG> is a block diagram showing the functional configuration of the processor <NUM> when the processor <NUM> performs non-linear processing. The processor <NUM> shown in <FIG> further includes a non-linear processor <NUM>. There are no other differences between the functional configuration of the processor <NUM> shown in <FIG> and the functional configuration shown by the block diagram in <FIG>.

The non-linear processor <NUM> performs non-linear processing of the output signal from the adder <NUM>. The non-linear processing may be any kind of processing. The non-linear processing is processing to remove the frequency components of a filter signal F2' from the audio signal F1' by spectrum subtraction or Wiener filtering. Alternatively, the non-linear processing may be processing to remove echo components from the audio signal F1' by spectrum subtraction or Wiener filtering. Alternatively, the non-linear processing may be processing to remove steady noise components from the audio signal F1' by spectrum subtraction or Wiener filtering. In this way, the non-linear processor <NUM> performs processing, for example, to emphasize the voice of the speaker V1.

<FIG> is a schematic external view of a sound pickup device 1A according to a modification. The sound pickup device 1A further includes another microphone 15C. The sound pickup device 1A has no other differences in structure from the sound pickup device <NUM>. In the example shown in <FIG>, the microphone 15A, the microphone 15B and the microphone 15C are to pick up the voice of the speaker V1, the voice of the speaker V2 and the voice of another speaker V3, respectively. The microphone 15C catches an audio signal S3. The audio signal S3 caught by the microphone 15C includes the voice of the speaker V1, the voice of the speaker V2 and the voice of the speaker V3. However, the audio signal S3 includes the voice of the speaker V3 at a higher level than the voice of the speaker V1 and the voice of the speaker V2.

<FIG> is a block diagram showing the functional configuration of the processor <NUM> when the processor <NUM> additionally receives the audio signal S3. The processor <NUM> further includes another filter bank analyzer 121C. The filter bank analyzer 121C transforms the audio signal S3 into an audio signal F3 with respect to frequency and extracts frequency components separately.

The filter coefficient calculator <NUM> calculates filter coefficients to be used in the FIR filter <NUM>, based on the audio signals F2 and F3. In this example, the filter coefficient calculator <NUM> calculates filter coefficients C2 and filter coefficients C3. The filter coefficient limiter <NUM> limits the gains to be achieved by the filter coefficients C2 and the filter coefficients C3 to <NUM> or less (S14), thereby calculating corrected filter coefficients C2' and corrected filter coefficients C3'. The FIR filter <NUM> convolutes the corrected filter coefficients C2' and the corrected filter coefficients C3' into the audio signal F2 and the audio signal F3, respectively, thereby generating a filter signal F'. More specifically, the filter signal F' is the sum of the result of the convolution of the corrected filter coefficients C2' into the audio signal F2 and the result of the convolution of the corrected filter coefficients C3' into the audio signal F3 (F2*C2' + F3'*C3').

The level of the voice of the speaker V1 in the audio signal S2 is lower than the level of the voice of the speaker V1 in the audio signal S1. Similarly, the level of the voice of the speaker V1 in the audio signal S3 is lower than the level of the voice of the speaker V1 in the audio signal S1. Therefore, the filter coefficient calculator <NUM> calculates filter coefficients C2 and C3 having gains of <NUM> or more, which will reduce the voice of the speaker V1 in the audio signal S1. However, the filter coefficient limiter <NUM> limits the gains to be achieved by the filter coefficients C2 and the filter coefficients C3 to <NUM> or less. Accordingly, the voice of the speaker V2 and the voice of the speaker V3 are removed from the audio signal S1, and the voice of the speaker V1 is separated.

The audio signal S2 and the audio signal S3 are subjected to the same processing. The voice of the speaker V1 and the voice of the speaker V3 are removed from the audio signal S2, and the voice of the speaker V2 is separated. The voice of the speaker V1 and the voice of the speaker V2 are removed from the audio signal S3, and the voice of the speaker V3 is separated.

Even when there are more audio signals, the audio signals are subjected to the same processing. When the second signal includes a number N of audio signals, a number N of kinds of filter coefficients are calculated. A filter signal is calculated by summing the results of respective convolutions of the number N of kinds of filter coefficients into the number N of audio signals (F1*C1' + F2*C2' +. From each of the number N of audio signals, the sound included therein at the highest level is separated.

In this way, when still more speakers speak at the same time and a plurality of microphones catch audio signals including the speakers' voices, the processor <NUM> can set the voice of the speaker who is the nearest to each microphone as target sound and separate the speaker's voice from the audio signal caught by the microphone.

It should be understood that the present embodiment has been described as an example and that the description is not limiting. The scope of the present disclosure is not limited to the embodiment and modifications above and is determined by the claims. Further, the scope of the disclosure shall be deemed to include equivalents of the scope of the claims.

Claim 1:
A filtering method for separating a target speaker's voice from voice components from one other speaker comprising:
receiving (S11) a first audio signal (S1) and a second audio signal (S2), wherein each of the audio signals include voice signals emitted by a target speaker and one other speaker, wherein volumes of the voice signals included in the respective audio signals are different;
transforming (S12) the first audio signal (S1) into a first transformed audio signal (F1) and the second audio signal (S2) into a second transformed audio signal (F2) in a frequency domain;
generating (S15) a filtered signal (F2') by applying an adaptive FIR filter to the second transformed audio signal (F2); and
subtracting the filtered signal (F2') from the first transformed audio signal (F1) to obtain a target voice signal (F1') corresponding to the voice of the target speaker separated from the voice of the other speaker;
characterized in that the method further comprises:
receiving by a filter coefficient calculator the transformed audio signals, wherein the filter coefficient calculator calculates (S13) adaptive filter coefficients (C2), wherein the adaptive filter coefficients (C2) are updated to minimize the level of a reference signal, the reference signal being generated by subtracting an update filter signal from the first transformed audio signal, the update filter signal being generated by convoluting the filter coefficients (C2) into the second transformed audio signal (F2);
calculating (S14) corrected adaptive filter coefficients (C2') by limiting (S14) a gain of the adaptive filter coefficients (C2) to <NUM> or less, wherein the limiting prevents the frequency components of the second audio transformed signal (F2) from being amplified by the FIR filtering; and
using the corrected adaptive filter coefficients as coefficients of the FIR filter.