Source: http://www.freepatentsonline.com/6694028.html
Timestamp: 2020-01-26 06:07:23
Document Index: 348883201

Matched Legal Cases: ['art 50', 'art 60', 'art 50', 'art 60', 'art 70', 'art 140', 'art 50', 'art 60', 'art 70', 'art 60', 'art 70', 'art 50', 'art 60', 'art 70', 'art 50', 'art 60', 'art 70', 'art 50', 'art 60', 'art 70', 'art 50', 'art 60', 'art 70', 'art 180', 'art 190', 'art 190', 'art 50', 'art 50', 'art 50', 'art 50', 'art 190', 'art 50', 'art 50', 'art 50', 'art 60', 'art 60', 'art 60', 'art 130', 'art 140', 'art 61', 'art 130', 'art 140', 'art 60', 'art 60', 'art 60', 'art 70', 'art 50', 'art 60', 'art 50', 'art 60', 'art 60']

Microphone array system - Fujitsu Limited
United States Patent 6694028
Matsuo, Naoshi (Kanagawa, JP)
09/560355
381/122, 367/198, 381/92, 381/91
Download PDF 6694028 PDF help
6069961 Microphone system 2000-05-30 Nakazawa 381/92
JP10215497
JPH10215497A 1998-08-11
1. A microphone array system including a plurality of microphones and a signal processing unit, comprising: at least one microphone arranged along each axis direction on rectangular coordinates; and a received sound signal processing part for performing processing of sound signals received at the plurality of microphones, having a directional sound signal calculating function, which is essential, for estimating a directional sound signal to an arbitrary direction based on the received sound signal with a unidirectivity or bidirectivity pattern along each axis direction, and further having at least one function of other sound signal processing functions at the same time; wherein the plurality of microphones are non-directional microphones at least two non-directional microphones are arranged in a first axis direction, at least two non-directional microphones are arranged in a second axis direction that is orthogonal to the first axis, and the received sound signal processing part has a function for calculating a directional sound signal to an arbitrary direction based on a unidirectional estimated sound signal to a positive direction on the first axis and a bidirectional estimated sound signal to positive and negative directions on the second axis.
2. The microphone array system according to claim 1, comprising a movable camera, wherein an improvement for a directivity of a received sound signal to an image capturing direction of the movable camera and an improvement for a directivity of a received sound signal to a sound input from an operator of the movable camera are switched for implementation, using the directional sound signal calculating function and the sound source direction detecting function at the same time.
6. A method for performing sound processing using a microphone array system including a plurality of microphones and a signal processing unit, wherein at least one microphone is arranged along each axis direction on rectangular coordinates, the method comprising: an operation for performing processing of sound signals received at the plurality of microphones, wherein the received sound signal processing operation includes calculating a directional sound signal to an arbitrary direction based on the received sound signal with a unidirectivity or bidirectivity pattern along each axis, which is essential, and further performing at least one function of other sound signal processing functions at the same time; wherein the plurality of microphones are non-directional microphones, at least two non-directional microphones are arranged in a first axis direction and at least two non-directional microphones are arranged in a second axis direction that is orthogonal to the first axis, and the received sound signal processing operation includes calculating a directional sound signal to an arbitrary direction based on a unidirectional estimated sound signal to a positive direction on the first axis and a bidirectional estimated sound signal to positive and negative directions on the second axis.
FIG. 17 shows an example of the configuration of the directional sound signal calculating part 50c of Embodiment 4 of the present invention.
FIG. 18 shows an example of the configuration of the sound source direction detecting part 60c of Embodiment 4 of the present invention.
FIG. 2 is a diagram showing a microphone array with a configuration different from that of FIG. 1. In this example, a USB (universal serial bus) interface is used as the interface of the microphone. In this example as well, two axes of X and Y as shown in FIG. 1 are used as the axis. In the example shown in FIG. 2, the microphones 11 and 12 of the microphone array section 10 can be arranged in the same manner as in the example in FIG. 1. Each of the microphones 11 and 12 is connected to a bus 40 via a USB hub 90, a connector 20a and a USB interface 91 and connected to a directional sound signal calculating part 50, a sound source direction detecting part 60, and a noise suppressing part 70.
In a microphone array section 10a of the example shown in FIG. 3, four non-directional microphones 100a to 100d are arranged along the positive and negative directions of the X and Y axes, each microphone corresponding to one direction, to receive sound signals. The direction of the front of the microphone array system corresponds to the negative direction on the X axis. The microphones 100a to 100d are positioned close to each other. In this example, the distance between the microphones 100a and 100c and the distance between the microphones 100b and 100d are a value obtained by dividing the sound velocity by the sampling frequency. A delay unit 110 performs processing of delaying for one sampling period and is connected to the microphone 100c. Numerals 121 and 122 denote subtracters.
As shown in FIG. 3, the subtracter 121 subtracts the received sound signal of the microphone 100c that is delayed by one sampling period by the delay unit 110 from the received sound signal of the microphone 100a. As a result, a received sound signal having a unidirectivity pattern to the negative direction on the X axis as shown in FIG. 5A is generated. Furthermore, the subtracter 122 subtracts the received sound signal of the microphone 100d from the received sound signal of the microphone 100b. As a result, a received sound signal having a bidirectivity pattern to the negative and positive directions on the Y axis as shown in FIG. 5B is generated. In FIG. 5B, the positive direction on the Y axis corresponds to the plus directivity, and the negative direction on the Y axis corresponds to the minus directivity.
Next, the cross-correlation coefficient calculating part 140 calculates the cross-correlation coefficient between the received sound signal with a unidirectivity pattern processed by the subtracter 121 and the received sound signal with a bidirectivity pattern processed by the subtracter 122 in FIGS. 8A and 8B to 11A and 11B. The cross-correlation coefficient R can be calculated with the following equation. R=∑i=0I-1m(ti)n(ti)∑i=0I-1m(ti)2∑i=0I-1n(ti)2Equation 1
where m(ti) is a signal from the subtracter 121 and n(ti) is a signal from the subtracter 122, and 1 is the sampling number for calculation of the cross-correlation coefficient, and generally is a value more than several hundreds.
FIG. 13 is a diagram showing an example of the configuration of the microphone array system of Embodiment 2. A microphone array section 10b includes three unidirectional microphones 200a to 200c arranged in the negative direction on the X axis, and the positive and negative directions on the Y axis, namely, in the directions of 0°, 90° and 270°, respectively, so as to obtain received sound signals. The front direction of the microphone array system is set to be the negative direction on the X axis. In Embodiment 2, although a received sound signal with the unidirectivity pattern with respect to the direction of 0° is obtained, it is necessary to generate received sound signals with a bidirectivity pattern with respect to the positive and negative directions on the Y axis. The directional sound signal calculating part 50a, the sound source direction detecting part 60a, and the noise suppressing part 70a of Embodiment 2 have the following configurations. Numeral 122a denotes a subtracter.
The process in the first stage will be described. The sound signal received from a microphone having a bidirectional pattern to the positive and negative directions on the Y axis is generated in the following manner. The subtracter 122a subtracts the received sound signal of the microphone 200c from the received sound signal of the microphone 200b. As a result, the received sound signal having a bidirectivity pattern to the negative and positive directions on the Y axis as shown in FIG. 5B is generated.
The process for calculating the left (L) channel signal and the right (R) channel signal in the second stage is the same as that in Embodiment 1, except that the input signal from the subtracter 121 in FIG. 4 of Embodiment 1 is replaced by an input signal from the unidirectional microphone 200a, and the input signal from the subtracter 122 in FIG. 4 of Embodiment 1 is replaced by an input signal from the subtracter 122a. Similarly to Embodiment 1, the result of subtracting the received sound signal with the bidirectivity pattern from the received sound signal with the unidirectivity pattern by the subtracter 123 is used as the left channel signal. The result of adding the received sound signal with the unidirectivity pattern and the received sound signal with the bidirectivity pattern by the adder 124 is used as the right channel signal.
The process of the sound source direction detecting part 60a and the process of the noise suppressing part 70a are the same as those in Embodiment 1, and therefore is omitted, where appropriate.
As shown in FIG. 13, each of the functions of the directional sound signal calculating part 50a, the sound source direction detecting part 60a, and the noise suppressing part 70a can be utilized together with the directional sound signal calculating function or other functions at the same time.
FIG. 14 is a diagram showing an example of the configuration of the microphone array system of Embodiment 3. A microphone array section 10c includes a unidirectional microphone 200d having a directivity to the negative direction on the X axis (direction of 0°) and a bidirectional microphone 300a having directivities to the positive and negative directions on the Y axis (direction of 90° and 270°), so as to obtain received sound signals. In Embodiment 3, a received sound signal with the unidirectivity pattern with respect to the direction of 0° and a received sound signal with the bidirectivity pattern to the positive and negative directions on the Y axis are obtained from the microphones 200d and 300a. Therefore, there is no need of providing subtracters corresponding to the subtracters 121 and 122 in Embodiment 1 and the subtracter 222 in Embodiment 2. The directional sound signal calculating part 50b, the sound source direction detecting part 60b, and the noise suppressing part 70b are provided.
The process for calculating the left (L) channel signal and the right (R) channel signal by the directional sound signal calculating part 50b is the same as those in Embodiments 1 and 2, and also is the same as that of a conventional MS microphone, except the input signals as follows. The input signal from the subtracter 121 in FIG. 4 of Embodiment 1 is replaced by an input signal from the unidirectional microphone 200d, and the input signal from the subtracter 122 in FIG. 4 of Embodiment 1 is replaced by an input signal from the bidirectional microphone 300a. Similarly to Embodiment 1, the result of subtracting the received sound signal with the bidirectivity pattern from the received sound signal with the unidirectivity pattern by the subtracter 123 is used as the left channel signal. The result of adding the received sound signal with the unidirectivity pattern and the received sound signal with the bidirectivity pattern by the adder 124 is used as the right channel signal.
The process of the sound source direction detecting part 60b and the process of the noise suppressing part 70b are the same as those in Embodiment 1, and therefore is omitted, where appropriate.
Also in Embodiment 3, as shown in FIG. 14, each of the functions of the directional sound signal calculating part 50b, the sound source direction detecting part 60b, and the noise suppressing part 70b can be utilized together with the directional sound signal calculating function or other functions at the same time.
A microphone array section 10a includes non-directional microphones 100a to 100d having directivities to the negative direction on the X axis (0°), the positive direction on the Y axis (90°), the positive direction on the X axis (180°) and the negative direction on the Y axis (270°). The outputs of the microphones 100a to 100d are connected to delay units 110a to 110d, respectively. The outputs of the delay units 110a to 110d are connected to gain units 150a to 150d, respectively. A movable camera 160 is rotated at any angle from 0° to 360° so that the directions in which the camera takes an image (hereinafter, referred to as “camera image capturing direction”) can be changed. For convenience, the camera can be rotated at an angle of either one of 0°, 90°, 180° and 270°. A camera-orientation detector 170 detects the image capturing direction of the camera 160. For example, the orientation of the camera can be detected by presetting the reference direction of the axis of the housing of the camera with respect to the camera stand and detecting the amount of the rotation from the preset direction. A delay sampling number adjusting part 180 adjusts so that the delay sampling number of each of the delay units 110a to 110d corresponds to the delay sampling number shown in FIG. 16 based on the camera image capturing direction detected by the camera-orientation detector 170. A gain amount adjusting part 190 adjusts so that the amount of the gain of each of the gain units 150a to 150d corresponds to the amount of the gain shown in FIG. 16 based on the camera image capturing direction detected by the camera-orientation detector 170. Furthermore, as described later, the gain amount adjusting part 190 adjusts the gain amounts of gain units 150e and 150f in the directional sound signal calculating part 50c.
An adder 121c adds the output signal from the microphone 100a and the output signal from the microphone 100c that have been subjected to the delay and gain processes, and an adder 122c adds the output signal from the microphone 100b and the output signal from the microphone 100d that have been subjected to the delay and gain adjustment.
Next, FIG. 17 shows an example of the configuration of the directional sound signal calculating part 50c. The directional sound signal calculating part 50c includes gain units 10e to 150h so that adjustment of the gain amount of +1.0 or −1.0 is performed in accordance with the image capturing direction of the camera 160, unlike the directional sound signal calculating part 50 in FIG. 4. The gain units 150e to 150h are adjusted by the gain amount adjusting part 190 so that the gain amounts thereof corresponds to those shown in FIG. 16. Numerals 123c and 124c are adders, and are the same as the adder 124 in FIG. 4.
The output from the adder 123c is used as the left channel output signal, and the output from the adder 124c is used as the right channel output signal.
The delay sampling number of the delay units and the gain amount of the gain units with respect to the orientation of the camera provide the following advantages. Regarding the adjustment of the delay units, the delay sampling number of the delay unit connected to the non-directional microphone arranged farthest from the orientation of the camera (that is, the delay unit 150c in the case where the camera image capturing direction is 0°, and the delay unit 150d in the case where the camera image capturing direction is 90°) is set to be 1, and the delay sampling number of the other delay units is set to be 0. Therefore, regardless of the orientation of the camera, either 0°, 90°, 180° or 270°, this configuration is equivalent to that of Embodiment 1 in FIG. 3 from the aspects of the sound source direction and the arrangement of the non-directional microphones and the delays. Next, regarding the gain adjustment of the gain units 150a to 150d, the gain amount of the gain units 150a to 150d is +1.0 or −1.0, which is determined so that the functions of the adder 121c and 122c are equivalent to the subtraction process by the adders 121 and 122 in FIG. 3, regardless of the direction of the camera.
Furthermore, regarding the gain units 150e to 150h in the directional sound signal calculating part 50c, the gain amounts are adjusted so that the operations of the adders 123c and 124c are equivalent to the subtraction process by the subtracter 123 and the addition process by the adder 124 of Embodiment 1 in FIG. 4, regardless of the orientation of the camera, respectively.
Thus, regardless of the image capturing direction of the movable camera, either 0°, 90°, 180° or 270°, the directional sound signal calculating part 50c that functions in the same manner as the directional sound signal calculating part 50 of Embodiment 1 can be obtained by adjusting the delay sampling number of the delay units 110a to 110d and the gain amount of the gain units 150a to 150h.
Next, the configuration of the sound source direction detecting part 60c will be described. The sound source is detected in the same manner as in Embodiment 1, which utilizes the cross-correlation coefficient of the powers of the received sound signal with a unidirectivity pattern to the front direction of the camera and the received sound signal with a bidirectivity pattern to the positive and negative directions on the Y axis. However, in this embodiment, the delay sampling number and the gain amount of the delay units are adjusted.
FIG. 18 shows an example of the configuration of the sound source direction detecting part 60c.
The sound source direction detecting part 60c includes a power ratio calculating part 130c, a cross-correlation coefficient calculating part 140c, and a determining part 61c. As shown in FIG. 18, the output signals from the adders 121c and 122c are input to the power ratio calculating part 130c, and the output signals from the adders 121c and 122c are input to the cross-correlation coefficient calculating part 140c. The functions of the components of the sound source direction detecting part 60c have the functions of the corresponding components of the sound source direction detecting part 60 of Embodiment 1, and therefore will not be described further.
Thus, regardless of the image capturing direction of the movable camera, either 0°, 90°, 180° or 270°, the sound source direction detecting part 60c allows detection of whether or not the sound source is in the direction of the orientation of the camera.
The noise suppressing part 70c can have the same configuration as that of Embodiment 1 where the direction of the orientation of the camera is set to be the camera front by adjusting the delay sampling number and the gain amount in accordance with the orientation of the camera 160 in the same manner. The description thereof is omitted in this embodiment.
Non-directional microphones 100a to 100d in a microphone array 10d are the same as those in Embodiment 4, except that the outputs from microphones 100a and 100d are processed by two systems. Numerals 110e and 10f denote delay units. The delay unit 110e delays the received sound signal of a microphone 100c by the delay sampling number. The delay unit 110f delays the received sound signal of a microphone 100a by the delay sampling number. Thus, the received sound signal processings of the microphones 100a and 100c are performed by two systems in parallel so as to generate received sound signals with two patterns of the unidirectivity pattern to the 0° direction and the unidirectivity pattern to the 180° direction. Subtracters 121d and 122d are the same as the subtracters 121 and 122 of Embodiment 1, and the results are input to a directional sound signal calculating part 50d. On the other hand, a subtracter 121e subtracts the received sound signal of the microphone 100a that is delayed by one sampling from the received sound signal of the microphone 100c so as to generate a received sound signal with a unidirectivity pattern to the 180° direction, and the result is input to a sound source direction detecting part 60d.
The directional sound signal calculating part 50d is the same as that in FIG. 4 of Embodiment 1, except that the input signal from the subtracter 121 in FIG. 4 of Embodiment 1 is replaced by an input signal from the subtracter 121d, and the input signal from the subtracter 122 in FIG. 4 of Embodiment 1 is replaced by an input signal from the subtracter 122d. Similarly to Embodiment 1, the result of subtracting the received sound signal with the bidirectivity pattern from the received sound signal with the unidirectivity pattern by the subtracter 123 is used as the left channel signal. The result of adding the received sound signal with the unidirectivity pattern and the received sound signal with the bidirectivity pattern by the adder 124 is used as the right channel signal.
The sound source detecting part 60d is the same as that in FIG. 7 of Embodiment 1, except that the input signal from the subtracter 121 in FIG. 7 is replaced by a signal from the subtracter 121e, and the input signal from the subtracter 122 in FIG. 7 is replaced by a signal from the subtracter 122d.
The sound source detecting part 60d detects whether or not the spoken sound is in the direction of the camera operator, namely, whether or not the sound source is in the 180° direction. In the case where the sound source is detected in that direction, a voice memo switch 400 is turned on, the signal from the subtracter 121d is delivered to a recording part for recording. The signal from the subtracter 121d has a directivity pattern to the camera operator, and therefore is recorded as a speech memorandum.
<- Previous Patent (Discrete multi-chann...) | Next Patent (Unobtrusive removal ...) ->