Source: https://patents.google.com/patent/US9330675B2/en
Timestamp: 2019-12-15 01:18:39
Document Index: 271433694

Matched Legal Cases: ['art 600', 'art 600', 'art 600', 'art 600', 'art 900', 'art 900', 'art 900', 'art 1300', 'art 1300']

US9330675B2 - Method and apparatus for wind noise detection and suppression using multiple microphones - Google Patents
US9330675B2
US9330675B2 US13/250,355 US201113250355A US9330675B2 US 9330675 B2 US9330675 B2 US 9330675B2 US 201113250355 A US201113250355 A US 201113250355A US 9330675 B2 US9330675 B2 US 9330675B2
US13/250,355
US20120121100A1 (en
2011-09-30 Priority to US13/250,355 priority patent/US9330675B2/en
2012-05-17 Publication of US20120121100A1 publication Critical patent/US20120121100A1/en
2016-05-03 Publication of US9330675B2 publication Critical patent/US9330675B2/en
230000001629 suppression Effects 0 claims description title 227
Unlike sound based pressure waves that go everywhere, air turbulence caused by wind is usually a fairly local event. Therefore, in a system that utilizes two or more spatially separated microphones to pick up sound signals (e.g., speech), wind noise picked up by one of the microphones often will not be picked up (or at least not to the same extent) by the other microphone(s). Embodiments of methods and apparatuses that utilize this tact and others to effectively detect and suppress wind noise using multiple microphones that are spatially separated are described.
Acoustic noise suppression module 310 is configured to exploit this difference in magnitude to filter the wind noise suppressed primary signal {circumflex over (P)}(f) using wind noise suppressed reference signal {circumflex over (R)}(f) to provide, as output, speech signal Ŝ1(f), which represents the acoustic and wind noise suppressed speech signal. As illustrated, acoustic noise suppression module 310 specifically includes a time-varying blocking matrix (BM) 315 and a time-varying active noise canceller (ANC) 320.
The method of flowchart 600 begins at step 605 and transitions to step 610. At step 610, the maximum normalized correlation of primary signal P(f) in the pitch period range is calculated (labeled as prim. mic. single channel correlation (SCC) in FIG. 6), the maximum normalized correlation of reference signal R(f) in the pitch period range is calculated (labeled as ref. mic. SCC in FIG. 6), and the cross-channel normalized correlation between primary signal P(f) and reference signal R(f) is calculated (labeled as cross-channel correlation (CCC) in FIG. 6).
During decision step 625, if CCC is above the defined threshold and primary microphone SCC and reference microphone SCC are below the defined threshold, primary microphone C-WND signal is set to a value that indicates wind noise is not present in primary signal P(f) and reference microphone C-WND signal is set to a value that indicates wind noise is not present in reference signal R(t) as shown in step 620. In general, if CCC is above the defined threshold and primary microphone SCC and reference microphone SCC are below the defined threshold, it is assumed that primary signal P(f) and reference signal R(f) include unvoiced speech and/or background noise and wind noise is not present in either primary signal P(f) or reference signal R(f). On the other hand, if the three conditions in step 625 are not all true, flowchart 600 proceeds to step 630.
It should be noted, in regard to flowchart 600, that different defined thresholds can be used for comparison against each correlation value (i.e., CCC, prim. mic. SCC, and ref. mic. SCC). It should be further noted, in regard to flowchart 600, that the relative differences between the three correlation values can be further used to detect whether wind noise is present or absent in primary signal P(f) and reference signal R(f). For example, in addition to requiring all three correlation values be above some defined threshold in step 615 in order to assume that wind noise is not present in either primary signal P(f) or reference signal R(f), it can be further required that the relative difference in value between one or more of the correlation values be within some defined range. In addition, it should be further noted that the three correlation values calculated in step 610 can be non-normalized in other embodiments.
TABLE 1 Example Sub-band Groupings Sub-band # component cosine wave # 1 3-4 2 5-6 3 7-8 4 9-10 5 11-12 6 13-14 7 15-17 8 18-20 9 21-23 10 24-27 11 28-31 12 32-36 13 37-42 14 43-49 15 50-56 16 57-64
In one embodiment, the cosine wave components are grouped into each sub-band by adding their corresponding squared magnitudes together. For example, the 3rd and 4th cosine wave components are grouped into the first sub-band, as indicated by table 1 above, by adding their corresponding squared magnitudes together. The resulting sum represents an estimated energy of the first sub-band. Extending the exemplary sub-band grouping provided in table 1 to the illustration of FIG. 7A, sub-band analysis module 705 provides the resulting squared sum of the 3rd and 4th cosine wave component magnitudes as output Y1(k,1), where Y1(k,i) is a two dimensional array indexed by frame number (k) and sub-band number (i). Thus, Y1(k,1) represents the estimated energy of the first sub-band in the kth frame of primary signal P(f), Y1(k,2) represents the estimated energy of the second sub-band in the kth frame of primary signal P(f), etc.
In another embodiment, energy ratio calculation module 715 is configured to subtract the sub-band energies of primary signal P(f), provided by sub-band analysis module 705, from corresponding sub-band energies of reference signal R(f), provided by sub-band analysis module 710, to determine differences in energy. The resulting values of each subtraction are provided as output R(k,i), in this embodiment, energy ratio calculation module 714 may be more aptly referred to as an energy difference calculation module 714.
T old(k+1,i)=T new(k,i)
where Told(k,i) represents the threshold value calculated for the ith sub-band of the previous frame (i.e., frame k−1) and α is a smoothing factor with a value between 0 and 1. As illustrated in FIG. 7A, threshold calculation module 720 provides, as output, the calculated threshold values (Tnew(k,i)) and the differences in energy between corresponding sub-bands of primary signal P(f) and reference signal R(f)(R(k,i)).
It should be noted that, in FIG. 8, 0 dB and −20 dB are provided by way of example and not limitation. Persons skilled in the relevant art(s) will recognize that other suppression gain values can be used for differences in sub-band energy that fall below T1 or above T2. In addition, it should be noted that the linearly increasing function of suppression gain between T1 and T2 is provided by way of example and not limitation. Persons skilled in the relevant art(s) will recognize that other increasing functions of suppression gain, such as an exponentially increasing function of suppression gain, can be used to describe the suppression gains between T1 and T2.
Referring now to FIG. 9, a flowchart 900 of an example method for multi-microphone wind noise detection and suppression in accordance with embodiments of present invention is illustrated. The method of flowchart 900 can be implemented by wind noise detection and suppression module 305 as described above and illustrated in FIG. 4. However, it should be noted that the method can be implemented by other systems and components as well. It should be further noted that some of the steps of flowchart 900 do not have to occur in the order shown in FIG. 9.
A general, overlap-add operation of two signals can be defined by:
s(n)=s out(n)·w out(n)+s in(n)·w in(n);n=0 . . . N−1
Referring now to FIG. 12, an example implementation of primary microphone wind noise suppression module 1015 is illustrated in accordance with embodiments of the present invention. Primary microphone wind noise suppression module 1015 is configured to perform wind noise suppression on primary signal P(f) on a frame-by-frame basis to provide, as output, wind noise suppressed primary signal PT. As illustrated in FIG. 12, wind noise suppression module 1015 includes a control module 1205 and a waveform substitution module 1210. Waveform substitution module 1210 is configured to replace (at least a portion of) a wind noise corrupted frame of primary signal P(f) with (at least a portion of) a comparatively cleaner frame of adjusted reference signal 1040 as described above in FIG. 10. In addition, wind noise suppression module 1015 optionally further includes one or more of packet loss concealment (PLC) module 1215, weighted sum module 1220, and single-channel noise suppression module 1225.
In operation, control module 1205 is configured to receive primary microphone wind noise detection signal 1025 and reference microphone wind noise detection signal 1030 that respectively indicate whether wind noise is present or absent in primary signal P(f) and reference signal R(f) (and, thereby, whether wind noise is present or absent in adjusted reference signal 1040). Based on these two signals, control module 1205 controls the operation of waveform substitution module 1210 and, if included, PLC module 1215, weighted sum module 1220, and single channel noise suppression module 1225. More specifically, based on different wind noise scenarios indicated by primary microphone wind noise detection signal 1025 and reference microphone wind noise detection signal 1030, control module 1205 use a different one of waveform substitution module 1210, PLC module 1215 weighted sum module 1220, and single channel noise suppression module 1225 to suppress wind noise in primary signal P(f) or make primary signal P(f) more consistent across time. The resulting signals from these modules are provided as output via wind noise suppressed primary signal {circumflex over (P)}(f).
w 1 = 1 / r r + 1 / r = 1 r 2 + 1
w 2 = r r + 1 / r = r 2 r 2 + 1 .
At step 1325, a determination is made as to whether wind noise is present or above a threshold in reference signal R(f) and absent or below a threshold in primary signal P(f), as indicated by the wind noise detection signals produced at step 1310. If wind noise is present or above a threshold in reference signal R(f) and absent or below a threshold in primary signal P(f), flowchart 1300 proceeds to step 1330 where single channel noise suppression is performed using single channel noise suppression module 1225 as discussed above in regard to FIG. 12. Otherwise, flowchart 1300 proceeds to step 1335.
Computer system 1400 also includes a main memory 1406, preferably random access memory (RAM), and may also include a secondary memory 1408. Secondary memory 1408 may include, for example, a hard disk drive 1410 and/or a removable storage drive 1412, representing a floppy disk drive, a magnetic tape drive, an optical disk drive, or the like. Removable storage drive 1412 reads from and/or writes to a removable storage unit 1416 in a well-known manner Removable storage unit 1416 represents a floppy disk, magnetic tape, optical disk, or the like, which is read by and written to by removable storage drive 1412. As will be appreciated by persons skilled in the relevant art(s), removable storage unit 1416 includes a computer usable storage medium having stored therein computer software and/or data.
a multi-method wind noise detection module configured to generate first and second wind noise detection signals that indicate whether wind noise is present or absent in the primary signal;
a wind noise detection signal combining module configured to combine the first and second wind noise detection signals to provide a primary microphone wind noise detection signal that indicates whether wind noise is present or absent in the primary signal; and
a primary microphone wind noise suppression module configured to determine a suppression gain for a sub-band of the primary signal based on:
the primary microphone wind noise detection signal, and
a comparison, performed by a suppression gain calculation module, of a difference in energy between the sub-band of the primary signal and a corresponding sub-band of a reference signal received by a reference microphone to a speech threshold and a wind threshold, wherein the speech threshold or the wind threshold is calculated based on the difference in energy and a previously calculated threshold value.
2. The apparatus of claim 1, wherein the primary microphone wind noise suppression module comprises:
an energy ratio calculation module configured to divide an estimate of the energy of the sub-band of the primary signal by an estimate of the energy of the corresponding sub-band of the reference signal to determine the difference in energy.
3. The apparatus of claim 1, wherein the primary microphone wind noise suppression module is further configured to determine the suppression gain such that the suppression gain is a constant value if the difference in energy is below the speech threshold or above the wind threshold.
4. The apparatus of claim 1, wherein the primary microphone wind noise suppression module is further configured to determine the suppression gain such that the suppression gain increases as the difference in energy increases from the speech threshold to the wind threshold.
5. The apparatus of claim 1, wherein the primary microphone wind noise suppression module is further configured to smooth the suppression gain over time.
6. The apparatus of claim 1, wherein the primary microphone wind noise suppression module further comprises:
a suppression gain mapping module configured to determine suppression gains for a group of frequency components in the sub-band of the primary signal by interpolating between the suppression gain and a suppression gain for an additional sub-band of the primary signal.
7. The apparatus of claim 1, wherein the first wind noise detection signal is generated based on the primary signal and the reference signal.
8. The apparatus of claim 1, wherein the multi-method wind noise detection module comprises:
a correlation based wind noise detection module configured to generate the first wind noise detection signal; and
a spectral deviation based wind noise detection module configured to generate the second wind noise detection signal.
9. The apparatus of claim 8, wherein the correlation based wind noise detection module is configured to generate the first wind noise detection signal based on:
the correlation of a first set of samples of the primary signal with a first set of samples of the reference signal,
the correlation of the first set of samples of the primary signal with a second set of samples of the primary signal, wherein the second set of samples of the primary signal are in a pitch period range of the first set of samples of the primary signal, and
the correlation of the first set of samples of the reference signal with a second set of samples of the reference signal, wherein the second set of samples of the reference signal are in a pitch period range of the first set of samples of the reference signal.
10. The apparatus of claim 9, wherein the spectral deviation based wind noise detection module is configured to generate the second wind noise detection signal by comparing a frequency spectrum of the primary microphone with a frequency spectrum associated with wind noise.
11. The apparatus of claim 1, further comprising a reference microphone wind noise suppression module.
12. A method for detecting and suppressing wind noise in a primary signal received by a primary microphone, the apparatus comprising:
generating first and second wind noise detection signals that indicate whether wind noise is present or absent in the primary signal;
combining the first and second wind noise detection signals to provide a primary microphone wind noise detection signal that indicates whether wind noise is present or absent in the primary signal; and
determining a suppression gain for a sub-band of the primary signal based on:
13. The method of claim 12, where the determining the suppression gain further comprises:
dividing an estimate of the energy of the sub-band of the primary signal by an estimate of the energy of the corresponding sub-band of the reference signal to determine the difference in energy.
14. The method of claim 12, wherein the determining the suppression gain further comprises:
determining the suppression gain such that the suppression gain is a constant value if the difference in energy is below the speech threshold or above the wind threshold.
15. The method of claim 12, wherein the determining the suppression gain further comprises:
determining the suppression gain such that the suppression gain decreases as the difference in energy increases from the speech threshold to the wind threshold.
smoothing the suppression gain over time.
determining suppression gains for a group of frequency components in the sub band of the primary signal by interpolating between the suppression gain and a suppression gain for an additional sub-band of the primary signal.
18. The method of claim 12, wherein the generating the first wind noise detection signal comprises:
generating the first wind noise detection signal based on the primary signal and the reference signal.
19. The method of claim 12, wherein the generating the first wind noise detection signal comprises:
correlating a first set of samples of the primary signal with a first set of samples of the reference signal;
correlating the first set of samples of the primary signal with a second set of samples of the primary signal, wherein the second set of samples of the primary signal are in a pitch period range of the first set of samples of the primary signal; and
correlating, the first set of samples of the reference signal with a second set of samples of the reference signal, wherein the second set of samples of the reference signal are in a pitch period range of the first set of samples of the reference signal.
20. The method of claim 12, wherein generating the second wind noise detection signal comprises:
comparing a frequency spectrum of the primary microphone with a frequency spectrum associated with wind noise.
US13/250,355 2010-11-12 2011-09-30 Method and apparatus for wind noise detection and suppression using multiple microphones Active 2034-08-23 US9330675B2 (en)
US13/250,355 US9330675B2 (en) 2010-11-12 2011-09-30 Method and apparatus for wind noise detection and suppression using multiple microphones
US20120121100A1 US20120121100A1 (en) 2012-05-17
US9330675B2 true US9330675B2 (en) 2016-05-03
US20120123771A1 (en) 2010-11-12 2012-05-17 Broadcom Corporation Method and Apparatus For Wind Noise Detection and Suppression Using Multiple Microphones
JP2011522294A (en) 2011-07-28 System, method, apparatus and computer program product for spectral contrast enhancement
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ZHANG, XIANXIAN;CHEN, JUIN-HWEY;ZENG, HUAIYU;AND OTHERS;REEL/FRAME:027001/0680