Patent Description:
Patent Literatures <NUM> to <NUM> disclose a technique to obtain coherence of two microphones, and emphasize a target sound such as voice of a speaker.

For example, the technique of Patent Literature <NUM> obtains an average coherence of two signals by using two non-directional microphones and determines whether or not the sound is a target sound based on an obtained average coherence value. Patent Literature <NUM> discloses a sound pickup device according to the preamble part of appended claim <NUM>.

The conventional technique does not disclose that distant noise is reduced.

In view of the foregoing, an object of a preferred embodiment of the present invention is to provide a sound pickup device and a sound pickup method that are able to reduce distant noise with higher accuracy than conventionally. Solution to Problem.

A sound pickup device as defined in claim <NUM> is provided according to an aspect of the present invention. Advantageous embodiments can be configured according to any of the dependent claims. According to another aspect, the present invention provides a sound pickup method as defined in claim <NUM>.

According to a preferred embodiment of the present invention, distant noise is able to be reduced with higher accuracy than conventionally.

A sound pickup device of the present preferred embodiment includes a first microphone, a second microphone, and a level control portion. The level control portion obtains a correlation between a first sound pickup signal to be generated from the first microphone and a second sound pickup signal to be generated from the second microphone, and performs level control of the first sound pickup signal or the second sound pickup signal according to a ratio of a frequency component of which the correlation exceeds a threshold value.

Since nearby sound and distant sound include at least a reflected sound, coherence of a frequency may be extremely reduced. When a calculated value includes such an extremely low value, the average may be reduced. However, the ratio only affects how many frequency components that are equal to or greater than a threshold value are present, and whether the value itself of the coherence in a frequency that is less than a threshold value is a low value or a high value does not affect the level control at all. Accordingly, the sound pickup device, by performing the level control according to the ratio, a target sound is able to be emphasized with high accuracy and distant noise is able to be reduced.

<FIG> is an external schematic view showing a configuration of a sound pickup device 1A. In <FIG>, the main configuration according to sound pickup is described and other configurations are not described. The sound pickup device 1A includes a cylindrical housing <NUM>, a microphone 10A, and a microphone 10B.

The microphone 10A and the microphone 10B are disposed on an upper surface of the housing <NUM>. However, the shape of the housing <NUM> and the placement aspect of the microphones are merely examples and are not limited to these examples.

<FIG> is a plan view showing directivity of the microphone 10A and the microphone 10B. As an example, the microphone 10A is a directional microphone having the highest sensitivity in front (the left direction in the figure) of the device and having no sensitivity in back (the right direction in the figure) of the device. The microphone 10B is a non-directional microphone having uniform sensitivity in all directions. However, the directional aspect of the microphone 10A and the microphone 10B is not limited to this example. For example, both the microphone 10A and the microphone 10B may be non-directional microphones or may be both directional microphones. In addition, the number of microphones may not be limited to two, and, for example, three or more microphones may be provided.

<FIG> is a block diagram showing a configuration of the sound pickup device 1A. The sound pickup device 1A includes the microphone 10A, the microphone 10B, a level control portion <NUM>, and an interface (I/F) <NUM>. The level control portion <NUM> is achieved as a function of software when a CPU (Central Processing Unit) <NUM> reads out a program stored in a memory <NUM> being a storage medium. However, the level control portion <NUM> may be achieved by dedicated hardware such as an FPGA (Field-Programmable Gate Array). In addition, the level control portion <NUM> may be achieved by a DSP (Digital Signal Processor).

The level control portion <NUM> receives an input of a sound pickup signal S1 of the microphone 10A and a sound pickup signal S2 of the microphone 10B. The level control portion <NUM> performs level control of the sound pickup signal S1 of the microphone 10A or the sound pickup signal S2 of the microphone 10B, and outputs the signal to the I/F <NUM>. The I/F <NUM> is a communication interface such as a USB or a LAN. The sound pickup device 1A outputs a pickup signal to other devices through the I/F <NUM>.

<FIG> is a view showing an example of a functional configuration of the level control portion <NUM>. The level control portion <NUM> includes a coherence calculation portion <NUM>, a gain control portion <NUM>, and a gain adjustment portion <NUM>.

The coherence calculation portion <NUM> receives an input of the sound pickup signal S1 of the microphone 10A and the sound pickup signal S2 of the microphone 10B. The coherence calculation portion <NUM> calculates coherence of the sound pickup signal S1 and the sound pickup signal S2 as an example of the correlation.

The gain control portion <NUM> determines a gain of the gain adjustment portion <NUM>, based on a calculation result of the coherence calculation portion <NUM>. The gain adjustment portion <NUM> receives an input of the sound pickup signal S2. The gain adjustment portion <NUM> adjusts a gain of the sound pickup signal S2, and outputs the adjusted signal to the I/F <NUM>.

It is to be noted that, while this example shows an aspect in which the gain of the sound pickup signal S2 of the microphone 10B is adjusted and the signal is outputted to the I/F <NUM>, an aspect in which a gain of the sound pickup signal S1 of the microphone 10A is adjusted and the adjusted signal is outputted to the I/F <NUM> may be employed. However, the microphone 10B as a non-directional microphone is able to pick up sound of the whole surroundings. Therefore, it is preferable to adjust the gain of the sound pickup signal S2 of the microphone 10B, and to output the adjusted signal to the I/F <NUM>.

The coherence calculation portion <NUM> converts the signals into a signal X(f, k) and a signal Y(f, k) of a frequency axis (S11) by applying the Fourier transform to each of the sound pickup signal S1 and the sound pickup signal S2. The "f" represents a frequency and the "k" represents a frame number. The coherence calculation portion <NUM> calculates coherence (a time average value of the complex cross spectrum) according to the following Expression <NUM> (S12).

However, the Expression <NUM> is an example. For example, the coherence calculation portion <NUM> may calculate the coherence according to the following Expression <NUM> or Expression <NUM>. <MAT> <MAT>.

It is to be noted that the "m" represents a cycle number (an identification number that represents a group of signals including a predetermined number of frames) and the "T" represents the number of frames of <NUM> cycle.

The gain control portion <NUM> determines the gain of the gain adjustment portion <NUM>, based on the coherence. For example, the gain control portion <NUM> obtains a ratio R(k) of a frequency bin of which the amplitude of the coherence exceeds a predetermined threshold value γth, with respect to all frequencies (the number of frequency bins) (S13).

The threshold value γth is set to γth=<NUM>, for example. It is to be noted that f0 in the Expression <NUM> is a lower limit frequency bin, and f1 is an upper limit frequency bin.

The gain control portion <NUM> determines the gain of the gain adjustment portion <NUM> according to this ratio R(k) (S14). More specifically, the gain control portion <NUM> determines whether or not coherence exceeds a threshold value γth for each frequency bin, totals the number of frequency bins that exceed the threshold value, and determines a gain according to a total result. <FIG> is a view showing an example of a gain table. According to the gain table in the example shown in <FIG>, the gain control portion <NUM> does not attenuate the gain when the ratio R is equal to or greater than a predetermined value R1 (gain=<NUM>). The gain control portion <NUM> sets the gain to be attenuated as the ratio R is reduced when the ratio R is from the predetermined value R1 to a predetermined value R2. The gain control portion <NUM> maintains the minimum gain value when the ratio R is less than R2. The minimum gain value may be <NUM> or may be a value that is slightly greater than <NUM>, that is, a state in which sound is able to be heard very slightly. Accordingly, a user does not misunderstand that sound has been interrupted due to a failure or the like.

Coherence shows a high value when the correlation between two signals is high. Distant sound has a large number of reverberant sound components, and is a sound of which an arrival direction is not fixed. For example, in a case in which the microphone 10A has directivity and the microphone 10B is non-directivity, sound pickup capability to distant sound is greatly different. Therefore, coherence is reduced in a case in which sound from a distant sound source is inputted, and is increased in a case in which sound from a sound source near the device is inputted.

Therefore, the sound pickup device 1A does not pick up sound from a sound source far from the device, and is able to emphasize sound from a sound source near the device as a target sound.

The sound pickup device 1A of the present preferred embodiment has shown an example in which the gain control portion <NUM> obtains the ratio R(k) of a frequency of which the coherence exceeds a predetermined threshold value γth, with respect to all frequencies, and performs gain control according to the ratio. Since nearby sound and distant sound include a reflected sound, the coherence of a frequency may be extremely reduced. When such an extremely low value is included, the average may be reduced. However, the ratio R(k) only affects how many frequency components that are equal to or greater than a threshold value are present, and whether the value itself of the coherence that is less than a threshold value is a low value or a high value does not affect gain control at all, so that, by performing the gain control according to the ratio R(k), distant noise is able to be reduced and a target sound is able to be emphasized with high accuracy.

It is to be noted that, although the predetermined value R1 and the predetermined value R2 may be set to any value, the predetermined value R1 is preferably set according to the maximum range in which sound is desired to be picked up without being attenuated. For example, in a case in which the position of a sound source is farther than about <NUM> in radius and in a case in which a value of the ratio R of coherence is reduced, a value of the ratio R of coherence when a distance is about <NUM> is set to the predetermined value R1, so that sound is able to be picked up without being attenuated up to a distance of about <NUM> in radius. In addition, the predetermined value R2 is set according to the minimum range in which sound is desired to be attenuated. For example, a value of the ratio R when a distance is <NUM> is set to the predetermined value R2, so that sound is hardly picked up when a distance is <NUM> or more while sound is picked up as the gain is gradually increased when a distance is closer to <NUM>.

In addition, the predetermined value R1 and the predetermined value R2 may not be fixed values, and may dynamically be changed. For example, the level control portion <NUM> obtains an average value R0 (or the greatest value) of the ratio R obtained in the past within a predetermined time, and sets the predetermined value R1=R0+<NUM> and the predetermined value R2=R0-<NUM>. As a result, with reference to a position of the current sound source, sound in a range closer to the position of the sound source is picked up and sound in a range farther than the position of the sound source is not picked up.

It is to be noted that the example of <FIG> shows an aspect in which the gain is drastically reduced from a predetermined distance (<NUM>, for example) and sound from a sound source beyond a predetermined distance (<NUM>, for example) is hardly picked up, which is similar to the function of a limiter. However, the gain table, as shown in <FIG>, also shows various aspects. In the example of <FIG>, it is an aspect in which the gain is gradually reduced according to the ratio R, the reduction degree of the gain is increased from the predetermined value R1, and the gain is again gradually reduced at the predetermined value R2 or less, which is similar to the function of a compressor.

Subsequently, <FIG> is a view showing a configuration of a level control portion <NUM> according to Modification <NUM>. The level control portion <NUM> includes a directivity formation portion <NUM> and a directivity formation portion <NUM>. <FIG> is a flow chart showing an operation of the level control portion <NUM> according to Modification <NUM>. <FIG> is a block diagram showing a functional configuration of the directivity formation portion <NUM> and the directivity formation portion <NUM>.

The directivity formation portion <NUM> outputs an output signal M2 of the microphone 10B as the sound pickup signal S2 as it is. The directivity formation portion <NUM>, as shown in <FIG>, includes a subtraction portion <NUM> and a selection portion <NUM>.

The subtraction portion <NUM> obtains a difference between an output signal M1 of the microphone 10A and the output signal M2 of the microphone 10B, and inputs the difference into the selection portion <NUM>.

The selection portion <NUM> compares a level of the output signal M1 of the microphone 10A and a level of a difference signal obtained from the difference between the output signal M1 of the microphone 10A and the output signal M2 of the microphone 10B, and outputs a signal at a high level as the sound pickup signal S1 (S101). As shown in <FIG>, the difference signal obtained from the difference between the output signal M1 of the microphone 10A and the output signal M2 of the microphone 10B has the reverse directivity of the microphone 10B.

In this manner, the level control portion <NUM> according to Modification <NUM>, even when using a directional microphone (having no sensitivity to sound in a specific direction), is able to provide sensitivity to the whole surroundings of the device. Even in such a case, the sound pickup signal S1 has directivity, and the sound pickup signal S2 has non-directivity, which makes sound pickup capability to distant sound differ. Therefore, the level control portion <NUM> according to Modification <NUM>, while providing sensitivity to the whole surroundings of the device, does not pick up sound from a sound source far from the device, and is able to emphasize sound from a sound source near the device as a target sound.

The aspect of the directivity formation portion <NUM> and the directivity formation portion <NUM> is not limited to the example of <FIG>. In the pickup signal S1 and the pickup signal S2, in a case of an aspect in which the correlation with respect to a sound source near the housing <NUM> is high and the correlation with respect to a distant sound source is low, the configuration of the present preferred embodiment is able to be achieved.

For example, <FIG> is an external view of a sound pickup device 1B including three microphones (a microphone 10A, a microphone 10B, and a microphone 10C). <FIG> is a view showing a functional configuration of a directivity formation portion. <FIG> is a view showing an example of directivity.

As shown in <FIG>, in this example, all of the microphone 10A, the microphone 10B, and the microphone 10C are directional microphones. The microphone 10A, the microphone 10B, and the microphone 10C, in a plan view, have sensitivity in directions different from each other by <NUM> degrees.

The directivity formation portion <NUM> in <FIG> selects any one of signals of the microphone 10A, the microphone 10B, and the microphone 10C, and forms a directional first sound pickup signal. For example, the directivity formation portion <NUM> selects a signal at the highest level among the signals of the microphone 10A, the microphone 10B, and the microphone 10C.

The directivity formation portion <NUM> in <FIG> calculates the sum of the weights of the signals of the microphone 10A, the microphone 10B, and the microphone 10C, and forms a non-directional second sound pickup signal.

As a result, the sound pickup device 1B, even when including all directional (having no sensitivity in a specific direction) microphones, is able to provide sensitivity to the whole surroundings of the device. Even in such a case, the sound pickup signal S1 has directivity, and the sound pickup signal S2 has non-directivity, which makes sound pickup capability to distant sound differ. Therefore, the sound pickup device 1B, while providing sensitivity to the whole surroundings of the device, does not pick up sound from a sound source far from the device, and is able to emphasize sound from a sound source near the device as a target sound.

In addition, for example, even when all the microphones are non-directional microphones, for example, as shown in <FIG>, the directivity formation portion <NUM> calculates the sum of delays, so that, as shown in <FIG>, a pickup signal S1 having a strong sensitivity in a specific direction is also able to be generated. In such a case, although the example shows that three non-directional microphones are used, a pickup signal S1 having a strong sensitivity in a specific direction is also able to be generated by using two or four or more non-directional microphones.

Subsequently, <FIG> is a block diagram showing a functional configuration of an emphasis processing portion <NUM>.

Human voice has a harmonic structure having a peak component for each predetermined frequency. Therefore, the comb filter setting portion <NUM>, as shown in the following Expression <NUM>, passes the peak component of human voice, obtains a gain characteristic G(f, t) of reducing components except the peak component, and sets the obtained gain characteristic as a gain characteristic of the comb filter <NUM>.

In other words, the comb filter setting portion <NUM> applies the Fourier transform to the sound pickup signal S2, and further applies the Fourier transform to a logarithmic amplitude to obtain a cepstrum z c, t). The comb filter setting portion <NUM> extracts a c value cpeak(t)=argmaxc {z(c, t)} that maximizes this cepstrum z(c, t). The comb filter setting portion <NUM>, in a case in which the c value is other than cpeak (t) or approximate value of cpeak (t), extracts the peak component of the cepstrum as a cepstrum value z(c, t)=<NUM>. The comb filter setting portion <NUM> converts this peak component zpeak(c, t) back into a signal of the frequency axis, and sets the signal as the gain characteristic G(f, t) of the comb filter <NUM>. As a result, the comb filter <NUM> serves as a filter that emphasizes a harmonic component of human voice.

It is to be noted that the gain control portion <NUM> may adjust the intensity of the emphasis processing by the comb filter <NUM>, based on a calculation result of the coherence calculation portion <NUM>. For example, the gain control portion <NUM>, in a case in which the value of the ratio R(k) is equal to or greater than the predetermined value R1, turns on the emphasis processing by the comb filter <NUM>, and, in a case in which the value of the ratio R(k) is less than the predetermined value R1, turns off the emphasis processing by the comb filter <NUM>. In such a case, the emphasis processing by the comb filter <NUM> is also included in one aspect in which the level control of the sound pickup signal S2 (or the sound pickup signal S1) is performed according to the calculation result of the correlation. Therefore, the sound pickup device <NUM> may perform only emphasis processing on a target sound by the comb filter <NUM>.

It is to be noted that the level control portion <NUM>, for example, may estimate a noise component, and may perform processing to emphasize a target sound by reducing a noise component by the spectral subtraction method using the estimated noise component. Furthermore, the level control portion <NUM> may adjust the intensity of noise reduction processing based on the calculation result of the coherence calculation portion <NUM>. For example, the level control portion <NUM>, in a case in which the value of the ratio R(k) is equal to or greater than the predetermined value R1, turns on the emphasis processing by the noise reduction processing, and, in a case in which the value of the ratio R(k) is less than the predetermined value R1, turns off the emphasis processing by the noise reduction processing. In such a case, the emphasis processing by the noise reduction processing is also included in one aspect in which the level control of the sound pickup signal S2 (or the sound pickup signal S1) is performed according to the calculation result of the correlation.

<FIG> is a block diagram showing an example of a configuration of an external device (a PC: Personal Computer) <NUM> to be connected to the sound pickup device. The PC <NUM> includes an I/F <NUM>, a CPU <NUM>, an I/F <NUM>, and a memory <NUM>. The I/F <NUM> is a USB interface, for example, and is connected to the I/F <NUM> of the sound pickup device 1A, with a USB cable. The I/F <NUM> is a communication interface such as a LAN, and is connected to a network <NUM>. The CPU <NUM> receives an input of a pickup signal from the sound pickup device 1A through the I/F <NUM>. The CPU <NUM> reads out a program stored in the memory <NUM> and performs the function of a VoIP (Voice over Internet Protocol) <NUM> shown in <FIG>. The VoIP <NUM> converts the pickup signal into packet data. The CPU <NUM> outputs the packet data that has been converted by the VoIP <NUM> to the network <NUM> through the I/F <NUM>. As a result, the PC <NUM> is able to transmit and receive a pickup signal to and from another device to be connected through the network <NUM>. Therefore, the PC <NUM> is able to conduct an audio conference with a remote place, for example.

<FIG> is a block diagram showing a modification example of the sound pickup device 1A. In the sound pickup device 1A of this modification example, the CPU <NUM> reads out a program from the memory <NUM> and performs the function of a VoIP <NUM>. In such a case, the I/F <NUM> is a communication interface such as a LAN, and is connected to the network <NUM>. The CPU <NUM> outputs the packet data that has been converted by the VoIP <NUM> through I/F <NUM>, to the network <NUM> through the I/F <NUM>. Accordingly, the sound pickup device 1A is able to transmit and receive a pickup signal to and from another device to be connected through the network <NUM>. Therefore, the sound pickup device 1A is able to conduct an audio conference with a remote place, for example.

<FIG> is a block diagram showing an example of a configuration in a case in which the configuration of the level control portion <NUM> is provided in an external device (a server) <NUM>. The server <NUM> includes an I/F <NUM>, a CPU <NUM>, and a memory <NUM>. The I/F <NUM> is a USB interface, for example, and is connected to the I/F <NUM> of the sound pickup device 1A, with a USB cable.

In this example, the sound pickup device 1A does not include the level control portion <NUM>. The CPU <NUM> reads out a program from the memory <NUM> and performs the function of the VoIP <NUM>. In this example, the VoIP <NUM> converts the pickup signal S1 and the pickup signal S2 into packet data, respectively. Alternatively, the VoIP <NUM> converts the pickup signal S1 and the pickup signal S2 into one piece of packet data. Even when being converted into one piece of packet data, the pickup signal S1 and the pickup signal S2 are distinguished, respectively, and are stored in the packet data as different data.

In this example, the I/F <NUM> is a communication interface such as a LAN, and is connected to the network <NUM>. The CPU <NUM> outputs the packet data that has been converted by the VoIP <NUM> through I/F <NUM>, to the network <NUM> through the I/F <NUM>.

The I/F <NUM> of the server <NUM> is a communication interface such as a LAN, and is connected to the network <NUM>. The CPU <NUM> receives an input of the packet data from the sound pickup device 1A through the I/F <NUM>. The CPU <NUM> reads out a program stored in the memory <NUM> and performs the function of a VoIP <NUM>. The VoIP <NUM> converts the packet data into the pickup signal S1 and the pickup signal S2. In addition, the CPU <NUM> reads out a program from the memory <NUM> and performs the function of a level control portion <NUM>. The level control portion <NUM> has the same function as the level control portion <NUM>. The CPU <NUM> outputs again the pickup signal on which the level control has been performed by the level control portion <NUM>, to the VoIP <NUM>. The CPU <NUM> converts the pickup signal into packet data in the VoIP <NUM>. The CPU <NUM> outputs the packet data that has been converted by the VoIP <NUM> to the network <NUM> through the I/F <NUM>. For example, the CPU <NUM> transmits the packet data to a communication destination of the sound pickup device 1A. Therefore, the sound pickup device 1A is able to transmit the pickup signal on which the level control has been performed by the level control portion <NUM>, to the communication destination.

Claim 1:
A sound pickup device (1A) comprising a level control portion (<NUM>) that, according to a ratio of frequency bins of which a correlation between a first sound pickup signal to be generated from a first microphone (10A) and a second sound pickup signal to be generated from a second microphone (10B) exceeds a threshold value, performs level control of the first sound pickup signal or the second sound pickup signal,
wherein the level control portion (<NUM>) determines whether or not the correlation exceeds the threshold value for each frequency bin;
characterised in that the the level control portion further counts a number of frequency bins of which the correlation exceeds the threshold value among all frequency bins, and obtains the ratio by dividing the number of frequency bins of which the correlation exceeds the threshold value by a number of all frequency bins.